We Keep Coding

Is there any difference between making DOM on the server/client side? (speed perspective) [duplicate]

I've done some web-based projects, and most of the difficulties I've met with (questions, confusions) could be figured out with help. But I still have an important question, even after asking some experienced developers: When functionality can be implemented with both server-side code and client-side scripting (JavaScript), which one should be preferred?
A simple example:
To render a dynamic html page, I can format the page in server-side code (PHP, python) and use Ajax to fetch the formatted page and render it directly (more logic on server-side, less on client-side).
I can also use Ajax to fetch the data (not formatted, JSON) and use client-side scripting to format the page and render it with more processing (the server gets the data from a DB or other source, and returns it to the client with JSON or XML. More logic on client-side and less on server).
So how can I decide which one is better? Which one offers better performance? Why? Which one is more user-friendly?
With browsers' JS engines evolving, JS can be interpreted in less time, so should I prefer client-side scripting?
On the other hand, with hardware evolving, server performance is growing and the cost of sever-side logic will decrease, so should I prefer server-side scripting?
EDIT:
With the answers, I want to give a brief summary.
Pros of client-side logic:
Better user experience (faster).
Less network bandwidth (lower cost).
Increased scalability (reduced server load).
Pros of server-side logic:
Security issues.
Better availability and accessibility (mobile devices and old browsers).
Better SEO.
Easily expandable (can add more servers, but can't make the browser faster).
It seems that we need to balance these two approaches when facing a specific scenario. But how? What's the best practice?
I will use client-side logic except in the following conditions:
Security critical.
Special groups (JavaScript disabled, mobile devices, and others).

In many cases, I'm afraid the best answer is both.
As Ricebowl stated, never trust the client. However, I feel that it's almost always a problem if you do trust the client. If your application is worth writing, it's worth properly securing. If anyone can break it by writing their own client and passing data you don't expect, that's a bad thing. For that reason, you need to validate on the server.
Unfortunately if you validate everything on the server, that often leaves the user with a poor user experience. They may fill out a form only to find that a number of things they entered are incorrect. This may have worked for "Internet 1.0", but people's expectations are higher on today's Internet.
This potentially leaves you writing quite a bit of redundant code, and maintaining it in two or more places (some of the definitions such as maximum lengths also need to be maintained in the data tier). For reasonably large applications, I tend to solve this issue using code generation. Personally I use a UML modeling tool (Sparx System's Enterprise Architect) to model the "input rules" of the system, then make use of partial classes (I'm usually working in .NET) to code generate the validation logic. You can achieve a similar thing by coding your rules in a format such as XML and deriving a number of checks from that XML file (input length, input mask, etc.) on both the client and server tier.
Probably not what you wanted to hear, but if you want to do it right, you need to enforce rules on both tiers.

I tend to prefer server-side logic. My reasons are fairly simple:
I don't trust the client; this may or not be a true problem, but it's habitual
Server-side reduces the volume per transaction (though it does increase the number of transactions)
Server-side means that I can be fairly sure about what logic is taking place (I don't have to worry about the Javascript engine available to the client's browser)
There are probably more -and better- reasons, but these are the ones at the top of my mind right now. If I think of more I'll add them, or up-vote those that come up with them before I do.
Edited, valya comments that using client-side logic (using Ajax/JSON) allows for the (easier) creation of an API. This may well be true, but I can only half-agree (which is why I've not up-voted that answer yet).
My notion of server-side logic is to that which retrieves the data, and organises it; if I've got this right the logic is the 'controller' (C in MVC). And this is then passed to the 'view.' I tend to use the controller to get the data, and then the 'view' deals with presenting it to the user/client. So I don't see that client/server distinctions are necessarily relevant to the argument of creating an API, basically: horses for courses. :)
...also, as a hobbyist, I recognise that I may have a slightly twisted usage of MVC, so I'm willing to stand corrected on that point. But I still keep the presentation separate from the logic. And that separation is the plus point so far as APIs go.

I generally implement as much as reasonable client-side. The only exceptions that would make me go server-side would be to resolve the following:
Trust issues
Anyone is capable of debugging JavaScript and reading password's, etc. No-brainer here.
Performance issues
JavaScript engines are evolving fast so this is becoming less of an issue, but we're still in an IE-dominated world, so things will slow down when you deal with large sets of data.
Language issues
JavaScript is weakly-typed language and it makes a lot of assumptions of your code. This can cause you to employ spooky workarounds in order to get things working the way they should on certain browsers. I avoid this type of thing like the plague.
From your question, it sounds like you're simply trying to load values into a form. Barring any of the issues above, you have 3 options:
Pure client-side
The disadvantage is that your users' loading time would double (one load for the blank form, another load for the data). However, subsequent updates to the form would not require a refresh of the page. Users will like this if there will be a lot of data fetching from the server loading into the same form.
Pure server-side
The advantage is that your page would load with the data. However, subsequent updates to the data would require refreshes to all/significant portions of the page.
Server-client hybrid
You would have the best of both worlds, however you would need to create two data extraction points, causing your code to bloat slightly.
There are trade-offs with each option so you will have to weigh them and decide which one offers you the most benefit.

One consideration I have not heard mentioned was network bandwidth. To give a specific example, an app I was involved with was all done server-side and resulted in 200Mb web page being sent to the client (it was impossible to do less without major major re-design of a bunch of apps); resulting in 2-5 minute page load time.
When we re-implemented this by sending the JSON-encoded data from the server and have local JS generate the page, the main benefit was that the data sent shrunk to 20Mb, resulting in:
HTTP response size: 200Mb+ => 20Mb+ (with corresponding bandwidth savings!)
Time to load the page: 2-5mins => 20 secs (10-15 of which are taken up by DB query that was optimized to hell an further).
IE process size: 200MB+ => 80MB+
Mind you, the last 2 points were mainly due to the fact that server side had to use crappy tables-within-tables tree implementation, whereas going to client side allowed us to redesign the view layer to use much more lightweight page. But my main point was network bandwidth savings.

I'd like to give my two cents on this subject.
I'm generally in favor of the server-side approach, and here is why.
More SEO friendly. Google cannot execute Javascript, therefor all that content will be invisible to search engines
Performance is more controllable. User experience is always variable with SOA due to the fact that you're relying almost entirely on the users browser and machine to render things. Even though your server might be performing well, a user with a slow machine will think your site is the culprit.
Arguably, the server-side approach is more easily maintained and readable.
I've written several systems using both approaches, and in my experience, server-side is the way. However, that's not to say I don't use AJAX. All of the modern systems I've built incorporate both components.
Hope this helps.

I built a RESTful web application where all CRUD functionalities are available in the absence of JavaScript, in other words, all AJAX effects are strictly progressive enhancements.
I believe with enough dedication, most web applications can be designed this way, thus eroding many of the server logic vs client logic "differences", such as security, expandability, raised in your question because in both cases, the request is routed to the same controller, of which the business logic is all the same until the last mile, where JSON/XML, instead of the full page HTML, is returned for those XHR.
Only in few cases where the AJAXified application is so vastly more advanced than its static counterpart, GMail being the best example coming to my mind, then one needs to create two versions and separate them completely (Kudos to Google!).

I know this post is old, but I wanted to comment.
In my experience, the best approach is using a combination of client-side and server-side. Yes, Angular JS and similar frameworks are popular now and they've made it easier to develop web applications that are light weight, have improved performance, and work on most web servers. BUT, the major requirement in enterprise applications is displaying report data which can encompass 500+ records on one page. With pages that return large lists of data, Users often want functionality that will make this huge list easy to filter, search, and perform other interactive features. Because IE 11 and earlier IE browsers are are the "browser of choice"at most companies, you have to be aware that these browsers still have compatibility issues using modern JavaScript, HTML5, and CSS3. Often, the requirement is to make a site or application compatible on all browsers. This requires adding shivs or using prototypes which, with the code included to create a client-side application, adds to page load on the browser.
All of this will reduce performance and can cause the dreaded IE error "A script on this page is causing Internet Explorer to run slowly" forcing the User to choose if they want to continue running the script or not...creating bad User experiences.
Determine the complexity of the application and what the user wants now and could want in the future based on their preferences in their existing applications. If this is a simple site or app with little-to-medium data, use JavaScript Framework. But, if they want to incorporate accessibility; SEO; or need to display large amounts of data, use server-side code to render data and client-side code sparingly. In both cases, use a tool like Fiddler or Chrome Developer tools to check page load and response times and use best practices to optimize code.
Checkout MVC apps developed with ASP.NET Core.

At this stage the client side technology is leading the way, with the advent of many client side libraries like Backbone, Knockout, Spine and then with addition of client side templates like JSrender , mustache etc, client side development has become much easy.
so, If my requirement is to go for interactive app, I will surely go for client side.
In case you have more static html content then yes go for server side.
I did some experiments using both, I must say Server side is comparatively easier to implement then client side.
As far as performance is concerned. Read this you will understand server side performance scores.
http://engineering.twitter.com/2012/05/improving-performance-on-twittercom.html

I think the second variant is better. For example, If you implement something like 'skins' later, you will thank yourself for not formatting html on server :)
It also keeps a difference between view and controller. Ajax data is often produced by controller, so let it just return data, not html.
If you're going to create an API later, you'll need to make a very few changes in your code
Also, 'Naked' data is more cachable than HTML, i think. For example, if you add some style to links, you'll need to reformat all html.. or add one line to your js. And it isn't as big as html (in bytes).
But If many heavy scripts are needed to format data, It isn't to cool ask users' browsers to format it.

As long as you don't need to send a lot of data to the client to allow it to do the work, client side will give you a more scalable system, as you are distrubuting the load to the clients rather than hammering your server to do everything.
On the flip side, if you need to process a lot of data to produce a tiny amount of html to send to the client, or if optimisations can be made to use the server's work to support many clients at once (e.g. process the data once and send the resulting html to all the clients), then it may be more efficient use of resources to do the work on ther server.

If you do it in Ajax :
You'll have to consider accessibility issues (search about web accessibility in google) for disabled people, but also for old browsers, those who doesn't have JavaScript, bots (like google bot), etc.
You'll have to flirt with "progressive enhancement" wich is not simple to do if you never worked a lot with JavaScript. In short, you'll have to make your app work with old browsers and those that doesn't have JavaScript (some mobile for example) or if it's disable.
But if time and money is not an issue, I'd go with progressive enhancement.
But also consider the "Back button". I hate it when I'm browsing a 100% AJAX website that renders your back button useless.
Good luck!

2018 answer, with the existence of Node.js
Since Node.js allows you to deploy Javascript logic on the server, you can now re-use the validation on both server and client side.
Make sure you setup or restructure the data so that you can re-use the validation without changing any code.

Pros and cons of fully generating html using javascript

Recently, I have come up with some idea about how to improve the overall performance for
a web application in that instead of generating a ready-to-show html page from the web server, why not let it be fully generated in the client side.
Doing it this way, only pure data (in my case is data in JSON format) needs to be sent to the browser.
This will offload the work of html generation from the server to the client's browser and
reduce the size of the response packet sent back to users.
After some research, I have found that this framework (http://beebole-apps.com/pure/)
does the same thing as mine.
What I want to know is the pros and cons of doing it this way.
It is surely faster and better for web servers and with modern browser, Javascript code can run fast so page generation can be done fast.
What should be a downside for this method is probably for SEO.
I am not sure if search engines like Google will appropriately index my website.
Could you tell me what the downside for this method is?
Ps: I also attached sample code to help describe the method as follows:
In the head, simple javascript code will be written like this:
<script type='javascript' src='html_generator.js'/>
<script>
function onPageLoad(){
htmlGenerate($('#data').val());
}
</script>
In the body, only one element exist, used merely as a data container as follows:
<input type='hidden' id='data' value='{"a": 1, "b": 2, "c": 3}'/>
When the browser renders the file, htmlGenerate function which is in html_generator.js will be called and the whole html page will be generated in this function. Note that the html_generator.js file can be large since it contains a lot of html templates but since it can be cached in the browser, it will be downloaded once and only once.

Downsides
Search Engines will not be able to index your page. If they do, you're very lucky.
Users with JavaScript disabled, or on mobile devices, will very likely not be able to use it.
The speed advantages might turn out to be minimal, especially if the user's using a slow JavaScript engine like in IE.
Maintainability: Unless you automate the generation of your javascript, it's going to be a nightmare!
All in all
This method is not recommended if you're doing it only for speed increase. However, it is often done in web applications, where users stay on the same page, but then you would probably be better off using one of the existing frameworks for this, such as backbone.js, and make sure it remains crawlable by following Google's hashbang guidelines or HTML5 PushState (as suggested by #rohk).
If it's just performance you're looking for, and your app doesn't strictly need to work like this, don't do it. But if you do go this way, then make sure you do it in the standardized way so that it remains fast and indexable.

Client-side templating is often used in Single Page Applications.
You have to weight the pros and cons:
Pros :
More responsive interface
Load reduced on your web server
Cons :
SEO will be harder than for a classic website (unless you use HTML5 PushState)
Accessibility : you are relying heavily on javascript...
If you are going this way I recommend that you look at Backbone.js.
You can find tutorials and examples here : http://www.quora.com/What-are-some-good-resources-for-Backbone-js
Examples of SPA :
Document Cloud
Grooveshark
Gmail
Also look at this answer : https://stackoverflow.com/a/8372246/467003

Yikes.
jQuery templates are closer to what you are thinking. Sanity says you should have some HTML templates that you can easily modify at will. You want MAINTAINABILITY not string concatenations that keep you tossing and turning in your worst nightmares.
You can continue with this madness but you will end up learning the hard way. I employ you to first try some protypes using jQuery templates and your pure code way. Sanity will surely overcome you my friend and I say that coming from trying your way for a time. :)
Its possible to load content in dynamically of the template you need as well using AJAX. It doesn't make sense that you will have a panacea approach where every single conceivable template is needed to be downloaded in a single request.

The pros? I really can't think of any. You say that it will be easier in the webserver, but serving HTML is what web servers are designed to do.
The cons? It goes against just about every best practise when it comes to building websites:
search engines will not be able to index your site well, if at all
reduced maintainability
no matter how fast JS engines are, the DOM is slow, and will never be as fast as building HTML on the server
one JS error and your entire site doesn't render. Oops. Browsers however are extremely tolerant of HTML errors.
Ultimately HTML is a great tool for displaying content. JavaScript is a great way of adding interaction. Mixing them up like you suggest is just nuts and will only cause problems.

Cons
SEO
Excluding user without javascript
Pros
Faster workflow / Fluider interface
Small load reduce
If SEO is not important for you and most of your user have Javascript you could create a Single Page Applications. It will not reduce your load very much but it can create a faster workflow on the page.
Btw: I would not pack all templates in one file. It would be too big for big projects.

Json only web application. What are the cons? (or Pros)

I want to design a web application whose only interface is json i.e. all the http requests receive responses only in json format and dont render any html on the server side. All the form posts convert the form data into a json object and then post it as a string. All the rendering is done by client side javascript.
The one downside of this approach I know is that browsers without javascript wont be able to do much with this architecture but the interaction on the site is rich enough to be useless to non-javascript browsers anyway.
Are there any other downsides of this approach of designing a web application?

It's an increasingly-common pattern, with tools such as GWT and ext-js. Complex web apps such as GMail have been over 90% JS-created DOM for some time. If you are developing a traditional 'journal' type website with mainly written content to be read this approach will be overkill. But for a complex app that wishes to avoid page refreshes it may well be appropriate.
One downside is that not only does it require a browser that supports JavaScript, it is also easy for the computing resources required by the app to creep up to the point where it needs quite a powerful browser. If you develop in Chrome on a top-end PC you might come to run the app on a less powerful machine such as a netbook or mobile device and find it has become quite sluggish.
Another downside is you lose the opportunity to use HTML tools when working on your pages, and that viewing your application's pages' DOM trees under Firebug or Chrome Developer Tools may be hard work because the relationship between the elements and your code may not be clear.
Edit: another thing to consider is that it is more work to make pages more accessible, as keyboard shortcuts may have to be added (you may not be able to use the browser built in behavior here) and users with special needs may find it more difficult to vary the appearance of the app, for instance by increasing font size.
Another edit: it's unlikely now text content on your website will be successfully crawled by search engines. For this reason you sometimes see server created text-only pages representing the same content, that refer browsers to the JS-enabled version of the page.

Other than the issue you point out, there's another: Speed. But it's not necessarily a big issue, and in fact using JSON rather than HTML may (over slower connections) improve rather than hamper speed.
Web browsers are highly optimised to render HTML, both whole pages (e.g., normally) and fragments (e.g., innerHTML and the various wrappers for it, like jQuery's html or Prototype's update). There's a lot you can do to minimize the speed impact of working through your returned data and rendering the result, but nothing is going to be quite as fast as grabbing some HTML markup from the server and dumping it into the browser for display.
Now, that said, it's not necessarily going to be a big problem at all. If you use efficient techniques (some notes in this article), and if you primarily render the results by building up HTML strings you're then going to hand to the brower (again, via innerHTML or wrappers for it), or if you're rending only a few elements at a time, it's unlikely that there will be any perceptible speed difference.
If, instead, you build up substantial trees by creating and appending individual elements via the DOM API or wrappers for it, you're very likely to notice a performance impact. That seems like the right thing to do, but it involves lots of trips across the DOM/JavaScript boundary and means the browser has to present the DOM version of things to your code at all intermediate steps; in contrast, when you hand it an HTML string, it can do its thing and romp through it full-speed-ahead. You can see the difference in this performance test. It's substantial.
Over slower connections, the speed impact may be made up for or even overcome if the JSON data is more compact than the HTML would have been, because of the smaller size on the wire.

You've got to be more mindful of high-latency, low-bandwidth connections when you're building something like this. The likelihood is, you're going to be making a lot of Ajax calls to sync data and grab new data from the server, and the lag can be noticeable if there's a lot of latency. You need a strategy in place to keep the user informed about the progress of any communication between client and server.
In development, it's something that can be overlooked, especially if you're working with a local web server, but it can be killer in production. It means looking into prefetching and caching strategies.
You also need an effective way to manage HTML fragments/templates. Obviously, there are some good modules out there for rendering templates - Mustache.js, Underscore template, etc. - but keeping on top of the HTML fragments can cause some maintenance headaches. I tend to store the HTML templates in separate files, and load them dynamically via Ajax calls (plus caching to minimise HTTP requests).
Edit - another con:
Data syncing - if you use a server database as your data "authority" then it's important to keep data in sync between the server and client. This is even more relevant if changes to data on one client affects multiple clients. You then get into the realms of dealing with realtime, asynchronous updates, which can cause some interesting conceptual challenges. (Fortunately, using frameworks and libraries such as Socket.IO and Backbone.js can really make things easier).
Edit - pros:
There are some huge advantages to this type of application - it's far more responsive, and can really enhance the user experience. Trivial actions that would normally require a round-trip to the server and incur network overhead can now be performed quickly and seamlessly.
Also, it allows you to more effectively couple data to your views. Chances are, if you're handling the data on the client-side, you will have a framework in place that allows you to organise the data and make use of an ORM - whether its Backbone.js, Knockout.js or something similar. You no longer have to worry about storing data in html attributes or in temporary variables. Everything becomes a lot more manageable, and it opens the door for some really sophisticated UI development.
Also also, JavaScript opens up the possibility for event-driven interaction, which is the perfect paradigm for highly interactive applications. By making use of the event loop, you can hook your data directly to user-initiated and custom events, which opens up great possibilities. By hooking your data models directly to user-driven events, you can robustly handle updates and changes to data and render the appropriate output with minimal fuss. And it all happens at high speed.

I think the most important thing is what is your requirement, if you want to build a interactive application, giving desktop like feel then go for client side development. Using Javascript framework like backbone.js or knockout.js will really help in organizing and maintaining the code. The advantages are already detailed out in previous answers. As respect to the performance in rendering with respect to server side rendering is concerned here is a nice blog post which made thinking.
http://openmymind.net/2012/5/30/Client-Side-vs-Server-Side-Rendering/

Why don't I just build the whole web app in Javascript and Javascript HTML Templates?

I'm getting to the point on an app where I need to start caching things, and it got me thinking...
In some parts of the app, I render table rows (jqGrid, slickgrid, etc.) or fancy div rows (like in the New Twitter) by grabbing pure JSON and running it through something like Mustache, jquery.tmpl, etc.
In other parts of the app, I just render the info in pure HTML (server-side HAML templates), and if there's searching/paginating, I just go to a new URL and load a new HTML page.
Now the problem is in caching and maintainability.
On one hand I'm thinking, if everything was built using Javascript HTML Templates, then my app would serve just an HTML layout/shell, and a bunch of JSON. If you look at the Facebook and Twitter HTML source, that's basically what they're doing (95% json/javascript, 5% html). This would make it so my app only needed to cache JSON (pages, actions, and/or records). Which means you'd hit the cache no matter if you were some remote api developer accessing a JSON api, or the strait web app. That is, I don't need 2 caches, one for the JSON, one for the HTML. That seems like it'd cut my cache store down in half, and streamline things a little bit.
On the other hand, I'm thinking, from what I've seen/experienced, generating static HTML server-side, and caching that, seems to be much better performance wise cross-browser; you get the graphics instantly and don't have to wait that split-second for javascript to render it. StackOverflow seems to do everything in plain HTML, so does Google, and you can tell... everything appears at once. Notice how though on twitter.com, the page is blank for .5-1 seconds, and the page chunks in: the javascript has to render the json. The downside with this is that, for anything dynamic (like endless scrolling, or grids), I'd have to create javascript templates anyway... so now I have server-side HAML templates, client-side javascript templates, and a lot more to cache.
My question is, is there any consensus on how to approach this? What are the benefits and drawbacks from your experience of mixing the two versus going 100% with one over the other?
Update:
Some reasons that factor into why I haven't yet made the decision to go with 100% javascript templating are:
Performance. Haven't formally tested, but from what I've seen, raw html renders faster and more fluidly than javascript-generated html cross-browser. Plus, I'm not sure how mobile devices handle dynamic html performance-wise.
Testing. I have a lot of integration tests that work well with static HTML, so switching to javascript-only would require 1) more focused pure-javascript testing (jasmine), and 2) integrating javascript into capybara integration tests. This is just a matter of time and work, but it's probably significant.
Maintenance. Getting rid of HAML. I love HAML, it's so easy to write, it prints pretty HTML... It makes code clean, it makes maintenance easy. Going with javascript, there's nothing as concise.
SEO. I know google handles the ajax /#!/path, but haven't grasped how this will affect other search engines and how older browsers handle it. Seems like it'd require a significant setup.

Persistant private data storage.
You need a server to store data with various levels of public/private access. You also need a server for secure closed source information. You need a server to do heavy lifting that you don't want to do on the client. Complex data querying is best left upto your database engine. Indexing and searching is not yet optimised for javascript.
Also you have the issues of older browsers being far slower. If your not running FF4/Chrome or IE9 then there is a big difference between data manipulation and page construction on the client and the server.
I myself am going to be trying to build a web application made entirely using a MVC framework and template's but still using the server to connect to secure and optimised database.
But in general the application can indeed be build entirely in javascript and using templates. The various constructs and javascript engines have advanced enough to do this. There are enough popular frameworks out there to do this. The Pure javascript web applications are no longer experiments and prototypes.
Oh, and if were recommending frameworks for this, then take a look at backbone.js.
Security
Let's not forget that we do not trust the client. We need serverside validation. JavaScript is interpreted, dynamic and can be manipulated at run time. We never trust client input.

I used to mix these two approaches but then switched to client side rendering entirely because it was really hard to handle heavy JavaScript properly the otherwise. As a complete solution can recommend the approach of the JavaScriptMVC framework.
It has a view rendering engine called EJS which can compress your views into plain JavaScript for a production build of your application. That makes it extremely fast (all your HTML gets preloaded with your single compressed JavaScript file so it is rendered as soon as you receive your models JSON data). I personally couldn't notice a performance difference between EJS rendering and transferring plain HTML.
Then for your API, following the REST principles, caching is one of the key constraints. So if your application supports HTTP caching properly for JSON data and using your compressed client side templates you might even see a performance improvement.

I could be way off here, but...
Have you ever looked at CouchDB? (I have no affiliation w/ them BTW) I could be way wrong, but your situation sounds like it may be a perfect fit for the use of the Apache CouchDB I haven't really used it yet myself but I took a good look at it a short while back and it is a very interesting database.
It is a document based database that uses a REST api for connections (very versatile and easy to use). It is also very JSON centric, very fast and a tiny footprint; they say it can reside on phones and other embedded uses too but at the same time is supposed to be extremely scalable (upwards that is). If your a big JS user (which you sound like you are) then you may be right at home with it.
I was just thinking that it may come in handy in any number of ways that have been proposed here and thought I'd chime in just to give you an idea for storage options :)

What is a good book or resource for writing large ajax applications? [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
I am very experienced in engineering large-scale systems, but I am still relatively new to ajax-based design. I know how to use the apis, and I am fairly comfortable using jquery and javascript as a whole, but I often find myself thinking way too hard about the overall architecture.
Right now, my current application just has javascript files sprinkled all over the place, all in a /js directory. Most of them use jQuery, but some use YUI or a combination between the two because the features weren't available in jQuery.
Some of my REST server methods accept normal GET methods with request parameters, but others needed much more complex POSTs to handle the incoming data (lists of lists of objects). The handling of all of my ajax stuff is a mix and mash of different methods as a result of the complexity of the data I'm dealing with.
What I'd really like is to read about how to design an ajax-based system that is very clean and elegant architecturally, and is consistent from the simplest to the most complex of cases. Does such a resource exist?
Also suggestions on naming conventions of javascript files and conventions for ajax endpoint directory/method names?
Also how to do with entering form data? Should you use get or post to do this?
Also about validation of form data when all the constraints are already on the server? How to make this very trivial to do so you're not doing it for each form?
What are the best ways to generate new page content when people click things and settings this up so that it's easy to do over and over.
How to deal with application-specific javascript files depending on each other and managing this nicely.
I am also using Spring and Spring-MVC, but I don't expect this to make much difference. My questions are purely browser related.

There's a TL;DR summary at the end.
I can't really point you to a good resource for this as I haven't found one myself. However, all is not lost. You already have experience in developing large-scale applications and taking this knowledge into the browser-space doesn't require a lot of re-thinking.
First of all, unless your application is really trivial, I wouldn't start refactoring the entire codebase straight away because there are bound to be endless cases you haven't thought of yet.
Design the core architecture of the system you want first. In your case you probably want all your AJAX requests to go through one point. Select the XHR interface from either jQuery or YUI and write a wrapper around it that takes an option hash. All the XHR calls you write for new code go through there. This allows you to switch out the framework performing the XHR calls at any time with another framework or your own.
Next up, harmonize the wire protocol. I'd recommend using JSON and POST requests (POST requests have the additional benefit for FORM submissions of not being cached). Make a list of the different types of request/responses you need. For each of these responses, make a JS object to encapsulate them. (E.g. the form submission response is returned to the caller as a FormReponse object which has accessor functions for the validation errors, etc). The JS overhead for this is totally trivial and makes it easy to change the JSON protocol itself without going through your widget code to change the access of the raw JSON.
If you're dealing with a lot of forms, make sure they all have the same structure so you can use a JS object to serialize them. Most frameworks seem to have various native functions to do this, but I'd recommend rolling your own so you don't have to deal with shortcomings.
The added business value at this point is of course zero because all you have is the start of a sane way of doing things and even more JS code to load into your app.
If you have new code to write, write it on the APIs you've just implemented. That's a good way to see if you're not doing anything really stupid. Keep the other JS as it is for now but once you have to fix a bug or add a feature there, refactor that code to use your new APIs. Over time you'll find that the important code is all running on your APIs and a lot of the other stuff will slowly become obsolete.
Don't go overboard with re-inventing the wheel, though. Keep this new structure limited to data interaction and the HTTP wire and use your primary JS framework for handling anything related to the DOM, browser quirks, etc.
Also set up a global logger object and don't use console directly. Have your logger object use console or a custom DOM logger or whatever you need in different environments. That makes it easy to build in custom log levels, log filters, etc. Obviously you have to set up your build environment to scrub that code out for production builds (you do have a build process for this, right?)
My personal favorite for relatively sane JS source-code layout and namespacing is the Dojo framework. Object definitions relative to their namespace have obvious relations to their location on disk, there's a build system in place for custom builds, third-party modules, etc. The Dojo dependency/build system depends on dojo.require and dojo.provide statements in the code. When running on source a dojo.require statement will trigger the blocking load of the resource in question. For production the build system follows these statements and inserts the resource into the final bundle at that location. The documentation is a bit sparse, but it's definitely a good start for inspiration.
The TL;DR answer is,
Have all XHR calls go through a single interface
Don't pass raw response data back to higher levels
Do gradual refactoring of existing code
Harmonize the wire protocol
Use the dynamic power of JS for building light-weight proxy code
Sane code structure and call graphs look the same in JS as they do in other languages

Ajax Patterns is a pretty awesome book/site: http://ajaxpatterns.org/
Otherwise you may want to check out Advanced Ajax Architecture Best Practices
How you go about designing your site should be based on the features and size of your app. I would keep your focus there as opposed to looking for a architecture that works for all cases.