We Keep Coding

Dynamically / iteratively build up page content

I am planning to create a website which will let you iteratively construct an SQL query.
The idea is the following:
while(user wants more where clauses)
{
show selection (html select) for table columns
let user choose one column
upon selection, show distinct values of that column
let user choose one/multiple value(s)
}
I know how to handle the SQL part, but I am not sure how to tackle the iterative building of the page.
So my questions are:
What is the best method to build the page iteratively with the idea sketched above?
What do I do, if the user changes one of the previous selections?
The website will be build with Perl and I am thinking of utilizing Ajax for the dynamic part.
Any help is much appreciated.

If I were to do this, I'd use SQL::Abstract.
UPDATE:
If you don't want to redraw the whole page, you're going to be using AJAX. So find yourself a JavaScript library that you feel comfortable with that includes ajax calls. Jquery has this, a bunch of others do too. People have differing opinions on various libraries.
Anyway, your workflow looks like this:
user submits form
javascript performs client-side validation
javascript submits AJAX-style to the server
Server performs server-side validation, data manipulation, etc.
Server responds with data paylod
client updated the screen based on the contents of the payload.
So let's concentrate on 5 and 6.
Data Payload: The X in AJAX means XML, but many apps send back JSON, or HTML.
Update the Screen:
You can apply HTML directly to the existing page by setting a tag's innerHTML or outerHTML attribute. But that doesn't update the DOM. If you don't dig around the DOM in your clcinet code, then this can suffice. If you dig around, then you need to build nodes and add them to you page's DOM, so you might want to consider sending back JSON or XML.
So let's say that you have a div with id='generatedSQL' when your AJAX call retruns, it will fire a callback method (let's call it updateSQL()) and inside the callback you'll have code that looks approximately like:
$(#generatedSQL).innerHTML = theVariableHoldingTheHtml;
Your other option is to parse the JSONXML/etc. and using createNode(),etc, build new DOM bits and plug them into your page. I don't have the jquery chops to write this for you. I look it up every time I need to do something like it.
If the query text is only ever display-only, and you never try to dig around in it on the client side, just use the innerHTML method, whether you pass HTML or pass JSON and generate HTML from it. If the query text is important to the inner workings of the client, then you'll need to write bunch of DOM manipulation code.

No sure what you would be using but, if uses clicks on something then use the Onload event of the element with ajax call to script which brings back the data/content and on readystate just update the innerHTML of the container DIV.
Hope following link will help you understand the concept will give you a code to start with as well.
http://www.w3schools.com/php/php_ajax_database.asp

create html elements on the serverside VS get data as JSON and create tags with javascript

I want to create a AJAX search to find and list topics in a forum (just topic link and subject).
The question is: Which one of the methods is better and faster?
GET threads list as a JSON string and convert it to an object, then loop over items and create a <li/> or <tr>, write data (link, subject) and append it to threads list. (jQuery Powered)
GET threads list which it wrapped in HTML tags and print it (or use innerHTML and $(e).html())
Thanks...

I prefer the second method.
I figure server-side you have to either convert your data to JSON or html format so why not go directly to the one the browser understands and avoid having to reprocess it client-side. Also you can easily adapt the second method to degrade gracefully for users who have disabled JavaScript (such that they still see the results via standard non-JS links.)

I'm not sure which way is better (I assume the second method is better as it would seem to touch the data less) but a definitive way to found out is try both ways and measure which one does better.

'Faster' is probably the second method.
'Better' is probably subjective.
For example, I've been in situations (as a front end dev) where I couldn't alter the html the server was returning and i wished they would have just delivered a json object so i could design the page how i wanted.
Also, (perhaps not specific to your use case), serving up all the html on initial page load could increase the page size and load time.

Server generated HTML is certainly faster if the javascript takes long time to process the JSON and populate the html.
However, for maintainability, JS is better. You can change HTML generation just by changing JS, not having to update server side code, making a delta release etc etc.
Best is to measure how slow it really is. Sometimes we think it is slow, but then you try it out in real world and you don't really see a big difference. You might have the major delay in transmitting the JSON object. That delay will still be there and infact increase if you send an html representation from the server.
So, if you bottleneck really is parsing JSON and generating html, not the transmission from server, then sending html from server makes sense.
However, you can do a lot of optimization in producing the html and parsing JSON. There are so many tricks to make that faster. Best if you show me the code and I can help you make a fast JS based implementation or can tell you to do it on the server.

Optimal way to pass system values to javascript

What is the most effective way to pass object and category ids or other system variables which shouldn't be presented to the user, from server to the browser?
Let's say I have a list of items where I can do something with each of them by javascript, for example show tooltip html or add to favorites by ajax, or display on a map. Where is it best to save that tooltip html, or database id, or geoposition?
Some options I can think of are:
some dictionary within <script></script> tag for each item,
microformats,
inline xml,
rel attributes,
css class names with specific information, e.g. class="lat-12345 uid-45678",
one <script></script> with a dictionary of html ids mapping dictionaries with system values in the template,
javascript generated from the database and included via <script src="..."></script> with a dictionary of html ids mapping dictionaries with system values in the template,
ajax requests for all cases when I need more information than just id,
event handlers with parameters within html tags, e.g. onmouseover="tooltip(this, 123, 'Hello world!')".
P.S. I prefer unobtrusive solutions and also the loading/execution time is important.

Perhaps I am missing something... why not just JSON?
How you "send" it (either in the initial page load as "javascript" or via AJAX or whatnot) is really just a trivial detail determined mostly by when the data is available. (JSON is a subset of legal JavaScript syntax.)
Then it's just a matter of the correct transformation. Of course, by pushing this to JSON/JS, you may render some non-JS clients non-interoperable, if that's a consideration for you. If such is indeed the case, why not just perform the transformation server-side using well, any number of the techniques you put at top?
You can also use arbitrary attributes in HTML (the HTML5 spec may include "data-*" which is formally legalized) -- while not technically "correct", all major web-browsers will accept unknown attributes which can be accessed through the DOM API.

I'd prefer a single AJAX call to fetch whatever data you know you need at the outset, so you can have a simple JSON object available in your script. You can, of course, supplement that with additional calls should you find you need more information.
If that's impractical, then "hardcoding" a JavaScript object in a <script>...</script> tag is the next best option. Of course, "hardcoding" is from the browser's perspective. The actual content would surely be written by server-side script from your database.

One method you can use is custom attributes. I think you refer to this as micro-formats, but I am not entirely sure if they are the same thing so I have written a description below.
Having hit the same question before, I basically use something like the following:
<div data-pid="1234">
<a href="#" class="add-to-favourites">
<img src="addToFavourites.png" />
</a>
</div>
$("a.add-to-favourites").click(function() {
$.load("/Favourites/Add.php?prodID="+$(this).parent().attr("data-pid"));
});
This should do exactly what you want to do. The reason I have put the pid in the div, not the a tag, is that you can then place all the other product information within the div with other actions the user can take, for example displaying a tooltip on mouseover using data-description, or displaying on a map using data-geo-x and data-geo-y. Of course you can name these anything you want.
Support / Acceptance
This is becoming a perfectly accepted way to do what you want to do. HTML 5 supports this for precisely the kind of thing you are trying to achieve.
So it is supported by HTML 5, but what about HTML 4?
It may make HTML 4 invalid, but the world is moving on to bigger and better things. Older browsers (IE6 and before, FF1 / 2, Opera 7 / 8 / 9) are becoming less common so it shouldnt be a problem. It wont actually break older browsers - the functionality will still work.
Important validity note
Make sure you prepend the data- onto the attribute name. This will make the attribute perfectly valid in HTML 5.
A few extra hints
In jQuery 1.5, I have heard from an answer to my question that you can simply specify attr("pid") to return the value of data-pid. If this is the case then I would be careful when naming the second part of the attribute name after the name of an actual attribute (for example, instead of data-id, use data-pid - especially if the id attribute is specified. I am not sure what effect it would have if you didn't do this, but its better to avoid the problem in the first place than have issues with the site at a later date due to this.
Hope this is what you were looking for.

ASP.NET offers a very convenient way to do this. You can simply write a JavaScript object. I am sure other templating engines offer similar ways to do this.
var person = {
Name : <%= _person.Name %>,
Age : <%= _person.Age %>
};

I would implement a Javascript singleton AppCacheManager that initializes in the document.ready event. A bit JS oop and you have a fully fledged OOP datastore.
Whenever information is needed, you load it through Ajax / RESTful Webservice and cache it in the AppCache Manager. So you have 2 caches: 1. Browser Cache, possible due to RESTful webservice URL caching, and 2: the JS Cache Manager.
You access all requests to the AppCacheManager which transparently fetches the new data or returns the cached data, so that the client doesnt need to know anything of the caching.
in short:
write a JS CacheManager
don't fetch the whole data at once but in small parts when needed and cache them
define a convenient interface for the cachemanager
Example usage:
linktext
Unobtrusiveness is a very difficult thing in JS and i'd be eager to know something about that, too.
hope that helped.

Best practices for returning and displaying data from AJAX calls

I have some jquery/php interaction set up on a page. It submits some data to the server and gets back records of data which are then to be aligned on the page for comparison and possible action beyond that.
My question is what is the best practice for returning the info and then displaying it?
Return JSON object and then create
html on the fly with js and display
the data?
Return JSON object and then
put that data in already created
containers for the data on the page?
Return pure html from the server and
just put that on the page?
I was rolling these through my head last night and couldn't really figure out if one way would be better for any particular reason.
I'm not a js guru so wasn't really sure the pros/cons and caveats to these different methods.

I think it ends up depending on your app.
Pure HTML is the easiest, you just drop in in place and viola. JQuery makes it a breeze to add the proper events and what not.
However, whenever I've used AJAX it's always evolved into returning JSON and building elements on the fly. If you are populating a tree for example, it may become messy to get the proper nesting. This forces you to have to do client side code anyway at which point simply using JSON from the start is cleaner.
Also, If you plan to use the data calls from other pages then using JSON is the way to go because the HTML will bee fixed.

This completely depends on the way you have things set up in your application. I, for one, prefer to return JSON (second approach in your list), because I have an 'error code' parameter that I check in onSuccess function before updating the content on the page, and if it's not zero then I notify the user of the error, including the error message that came back from the server (server-side validation, database timeout, etc.).

I think you're missing a perfectly valid option, one which I use often. This is my typical schema and it has yet to fail me... :-)
Here's the basic jQuery template I use:
$(function() {
$.getJSON('/some/page',{foo:bar,bar:foo},function(json) {
if(json.outcome == 'success') {
$('body').prepend(json.html);
} else {
// Somehow let the user know why it didn't work
alert(json.error);
}
});
});
Here's the basic backend (PHP in my case) structure I use:
<?php // Page: '/some/page'
/* Blah Blah Blah... do whatever needs to be done... */
// If everything turns out okay (assuming '$output' is the HTML
// you want to display...
echo json_encode(array('outcome'=>'success','html'=>$output));
// If something goes wrong... just do:
echo json_encode(array('outcome'=>'error','error'=>'Uh oh... something is broken'));
Naturally, you'll want to be more specific with your error by putting them into some variable or something. But, you should get the idea. Also, of course you can add more information to the json output. You can have some pre-made HTML and also some other information like a 'success notice' or a new class name for some element, I dunno... whatever... the possibilities are endless.
Anyways, the reason I choose this route is because it's usually faster (based on my experience) to append pre-made HTML to the DOM rather than looping over JSON and inserting the stuff unless it's just, like, a bit of text to replace into an element. But, the method I've shown is, IMO, the best of both worlds. You can attach HTML as a string to one of the JSON properties.
Happy jQuerying :-)

The "possible action beyond that" part of your question makes a big difference. If you need to do other things with the data besides display it, returning as JSON is a clearly better option because you can work with the data as a native JavaScript object instead of having to traverse the HTML DOM. If all you ever intend to do is display it, I don't see any reason to go through the trouble of building that display in JavaScript; just build the HTML in your presentation layer on the server.

This came up recently and possible a dupe: The AJAX response: Data (JSON, XML) or HTML snippet?.
If you are going to be creating HTML then you may as well be returning HTML directly and inject it into the DOM. However, there are times you need to work with objects which is where JSON comes in handy.
If you return a Person object for example then you can greet Person.Name and show Person.Preferences which is really handy. It depends on your design but the general considerations should be to keep HTML out of Javascript unless you're building RIA.

I have used all three and have come to the conclusion that returning HTML is better when introducing new elements to a page.
My experiance is that when building HTML with javascript I am usually replicating work that will have already have been done for the non javascript user journey.
I still prefer parsing json for updating existing elements or creating javascript only functionality. I tell myself this for bandwidth, but I think it just because I love javascript.
As a forth option, I read a great post about how Flickr deal with vast quantities of data with string concatination. Basically just parse a big o' string down the pipe and chop it up on the client. This significantly reduces the on the server, with only a marginal increase on the client.

Returning pure HTML is the best solution. For the most part gzip should neutralize any difference in bandwidth, and rendering via javascript on the client can be slow if the client is a crappy machine. Finally, writing javascript to render HTML is hard to work with compared to using something nice like a view if you use MVC.

What is the best way to include data in an HTML page?

What is the best way to include data in an HTML page? The data is not human-readable and will be processed by a script once the page has loaded. The options I can think of are:
Using class and title attributes on hidden/empty <div> or <span> elements within the page
JSON in a <script> element at the bottom of the page
Load the data via an XMLHttpRequest after the page has loaded
XML Data Islands
All of these methods seem to come with drawbacks so I would like to know what your thoughts are.

I would use JSON in a <script> element. Actually, I would make it an actual script. The browser is going to parse and evaluate the JSON, so take the opportunity to store it in some variable.
Hiding elements using CSS is kind of fragile, as some clients (not necessarily browsers, think search engines) may still see them as part of the page data.
Loading the data through XHR after page load is fine, but it's not strictly an answer to the question. Its also a bit slower as it incurs an additional server round-trip (think of your antipodean users, low latency is very important to them).
XML Data Islands: I am not sure what you are referring to, but it sound like something that would cause much validator complaints, and which might be fragile in that string node could be rendered by the browser.
So, storing the data in a <script> element sounds like the simplest, safest, most appropriate way to answer the question.

I'd go with JSON. Or some other JavaScript code if there is reason for it. That will be easiest for the script to consume, and it is limited only to what the script itself is limited to.