We Keep Coding

Representing data between Javascript and server-side language

I have Javascript using Ajax to get a list of notes from Java (using the Play Framework). The notes have many properties, such as text, timestamp, attachments, users, etc. The notes are displayed truncated in li items. Click on them to see them in full.
I can't depend on their li representation for all the data the app will need. So my only solution is to pass full Note parameters from Java and 'recreate' the class in Javascript to cache these notes in an array. When li is clicked it spits out data from the Javascript array at the associated index. I would rather do this than make another call to the server to get the specific contents of the note.
Is this bad practice? It seems like there should be a more reliable, less 'code-reusing' way.

There are many ways to approach the problem. I almost always opt to sending the raw data and let the client handle adding elements on the page. There are entire frameworks for this (Angular, Backbone, etc.) if what you're doing is more than trivial.
For the actual data serialization, I recommend JSON. Nearly everything supports it, and in my opinion it is much easier to deal with than XML.

create html elements on the serverside VS get data as JSON and create tags with javascript

I want to create a AJAX search to find and list topics in a forum (just topic link and subject).
The question is: Which one of the methods is better and faster?
GET threads list as a JSON string and convert it to an object, then loop over items and create a <li/> or <tr>, write data (link, subject) and append it to threads list. (jQuery Powered)
GET threads list which it wrapped in HTML tags and print it (or use innerHTML and $(e).html())
Thanks...

I prefer the second method.
I figure server-side you have to either convert your data to JSON or html format so why not go directly to the one the browser understands and avoid having to reprocess it client-side. Also you can easily adapt the second method to degrade gracefully for users who have disabled JavaScript (such that they still see the results via standard non-JS links.)

I'm not sure which way is better (I assume the second method is better as it would seem to touch the data less) but a definitive way to found out is try both ways and measure which one does better.

'Faster' is probably the second method.
'Better' is probably subjective.
For example, I've been in situations (as a front end dev) where I couldn't alter the html the server was returning and i wished they would have just delivered a json object so i could design the page how i wanted.
Also, (perhaps not specific to your use case), serving up all the html on initial page load could increase the page size and load time.

Server generated HTML is certainly faster if the javascript takes long time to process the JSON and populate the html.
However, for maintainability, JS is better. You can change HTML generation just by changing JS, not having to update server side code, making a delta release etc etc.
Best is to measure how slow it really is. Sometimes we think it is slow, but then you try it out in real world and you don't really see a big difference. You might have the major delay in transmitting the JSON object. That delay will still be there and infact increase if you send an html representation from the server.
So, if you bottleneck really is parsing JSON and generating html, not the transmission from server, then sending html from server makes sense.
However, you can do a lot of optimization in producing the html and parsing JSON. There are so many tricks to make that faster. Best if you show me the code and I can help you make a fast JS based implementation or can tell you to do it on the server.

Passing data between elements in UI

I'm new to web applications and am trying to understand the best way to work with data in HTML. I'm using Appengine (Python) and have managed to load a bunch of data to the view. In the simplest form, it's a set of movie names and each name has associated details with it (e.g. year, rating etc). Now how do I pass data between the movie link and then a div where all the details will be displayed? I'll be using jQuery for some controls in my application so I'm wondering if there's a way to do data binding to controls with that?
Additionally, can anyone tell me what're the standards around this i.e. if I load all this data to the UI in one call (assuming it's not a lot of movie titles), wouldn't it make it easy for people to screen scrape this information? Or is there some obfuscation that's typically used here?
Sorry if I'm not very clear but I really am an absolute beginner with web development!
Update1:
I found the jQuery data() api. It seems like this'll work. Comments?
Update2:
Some testing later and it turns out that data() actually attaches the data to the elements rather than showing it in a div itself.

There's a few ways to do it but the basic idea is to put the data in the HTML in a way that is not visibly rendered, then use Javascript to parse the HTML and pull the data out when you need it.
The easiest way on modern browsers is to use data- attributes. These are any attribute that start with data-, and you can name the rest yourself. For example:
Czar Wars
In this case, the user will only see a link called "Tsar Wars" but your javascript can easily access the data- attributes to get the data it needs. The other benefit of this approach is that jQuery will automatically make data- attributes accessible by the data() api.
Another way to do it is to have a hidden HTML list element with all your data elements in the list, but you'll have to parse this all yourself.
There's no standard obfuscation. You'll need to obfuscate yourself on the server side, and unobfuscate in your JS. It's not too difficult to figure out any obfuscation algorighm in js, so this is not worth your while.
If the data really is private, then you would have to architect it as to do all the processing on the server. For example, only show tokens (like 1234), and use AJAX calls to pass the token to the server so the server can do the data processing and spit back publicly safe results to the script.

Use .js files for caching large dropdown lists

I would like to keep the contents of large UI lists cached on the client, and updated according to criterial or regularly. Client side code can then just fill the dropdowns locally, avoiding long page download times.
These lists can be close to 4k items, and dynamically filtering them without caching would result in several rather large round trips.
How can I go about this? I mean, what patterns and strategies would be suitable for this?

Aggressive caching of JSON would work for this, you just hash the JS file and throw it on the end of it's URL to update it when it changes. One revision might look like this:
/media/js/ac.js?1234ABCD
And when the file changes, the hash changes.
/media/js/ac.js?4321DCBA
This way, when a client loads the page, your server-side code links to the hashed URL, and the client will get a 304 Not Modified response on their next page load (assuming you have this enabled on your server). If you use this method you should set the files to never expire, as the "expiring" portion will be dealt with by the hash, i.e., when the JS file does expire, the hash will change and the client won't get a 304, but rather a 200.
ac.js might contain a list or other iterable that your autocomplete code can parse as it's completion pool and you'd access it just like any other JS variable.
Practically speaking, though, this shouldn't be necessary for most projects. Using something like memcached server-side and gzip compression will make the file both small and amazingly fast to load. If the list is HUGE (say thousands of thousands of items) you might want to consider this.

Combres is a good solution for this - it will track changes and have the browser cache the js forever until a change is made, in which case it changes the URL of the item.
http://combres.codeplex.com/

You might consider rather than storing the data locally using jQuery and AJAX to dynamically update the dropdown lists. Calls can be made whenever needed and the downloads would be pretty quick.
Just a thought.
This might be helpful:
http://think2loud.com/using-jquery-and-xml-to-populate-a-drop-down-box/

If its just textual data, you have compression enabled on the web server, and there are less than 100 items, then there may be no need to maintain lists in the client script.
Its usually best to put all your data (list items are data) in one place so you dont have to worry about synchronization.

Best practices for returning and displaying data from AJAX calls

I have some jquery/php interaction set up on a page. It submits some data to the server and gets back records of data which are then to be aligned on the page for comparison and possible action beyond that.
My question is what is the best practice for returning the info and then displaying it?
Return JSON object and then create
html on the fly with js and display
the data?
Return JSON object and then
put that data in already created
containers for the data on the page?
Return pure html from the server and
just put that on the page?
I was rolling these through my head last night and couldn't really figure out if one way would be better for any particular reason.
I'm not a js guru so wasn't really sure the pros/cons and caveats to these different methods.

I think it ends up depending on your app.
Pure HTML is the easiest, you just drop in in place and viola. JQuery makes it a breeze to add the proper events and what not.
However, whenever I've used AJAX it's always evolved into returning JSON and building elements on the fly. If you are populating a tree for example, it may become messy to get the proper nesting. This forces you to have to do client side code anyway at which point simply using JSON from the start is cleaner.
Also, If you plan to use the data calls from other pages then using JSON is the way to go because the HTML will bee fixed.

This completely depends on the way you have things set up in your application. I, for one, prefer to return JSON (second approach in your list), because I have an 'error code' parameter that I check in onSuccess function before updating the content on the page, and if it's not zero then I notify the user of the error, including the error message that came back from the server (server-side validation, database timeout, etc.).

I think you're missing a perfectly valid option, one which I use often. This is my typical schema and it has yet to fail me... :-)
Here's the basic jQuery template I use:
$(function() {
$.getJSON('/some/page',{foo:bar,bar:foo},function(json) {
if(json.outcome == 'success') {
$('body').prepend(json.html);
} else {
// Somehow let the user know why it didn't work
alert(json.error);
}
});
});
Here's the basic backend (PHP in my case) structure I use:
<?php // Page: '/some/page'
/* Blah Blah Blah... do whatever needs to be done... */
// If everything turns out okay (assuming '$output' is the HTML
// you want to display...
echo json_encode(array('outcome'=>'success','html'=>$output));
// If something goes wrong... just do:
echo json_encode(array('outcome'=>'error','error'=>'Uh oh... something is broken'));
Naturally, you'll want to be more specific with your error by putting them into some variable or something. But, you should get the idea. Also, of course you can add more information to the json output. You can have some pre-made HTML and also some other information like a 'success notice' or a new class name for some element, I dunno... whatever... the possibilities are endless.
Anyways, the reason I choose this route is because it's usually faster (based on my experience) to append pre-made HTML to the DOM rather than looping over JSON and inserting the stuff unless it's just, like, a bit of text to replace into an element. But, the method I've shown is, IMO, the best of both worlds. You can attach HTML as a string to one of the JSON properties.
Happy jQuerying :-)

The "possible action beyond that" part of your question makes a big difference. If you need to do other things with the data besides display it, returning as JSON is a clearly better option because you can work with the data as a native JavaScript object instead of having to traverse the HTML DOM. If all you ever intend to do is display it, I don't see any reason to go through the trouble of building that display in JavaScript; just build the HTML in your presentation layer on the server.

This came up recently and possible a dupe: The AJAX response: Data (JSON, XML) or HTML snippet?.
If you are going to be creating HTML then you may as well be returning HTML directly and inject it into the DOM. However, there are times you need to work with objects which is where JSON comes in handy.
If you return a Person object for example then you can greet Person.Name and show Person.Preferences which is really handy. It depends on your design but the general considerations should be to keep HTML out of Javascript unless you're building RIA.

I have used all three and have come to the conclusion that returning HTML is better when introducing new elements to a page.
My experiance is that when building HTML with javascript I am usually replicating work that will have already have been done for the non javascript user journey.
I still prefer parsing json for updating existing elements or creating javascript only functionality. I tell myself this for bandwidth, but I think it just because I love javascript.
As a forth option, I read a great post about how Flickr deal with vast quantities of data with string concatination. Basically just parse a big o' string down the pipe and chop it up on the client. This significantly reduces the on the server, with only a marginal increase on the client.

Returning pure HTML is the best solution. For the most part gzip should neutralize any difference in bandwidth, and rendering via javascript on the client can be slow if the client is a crappy machine. Finally, writing javascript to render HTML is hard to work with compared to using something nice like a view if you use MVC.