I'm working on a project and there is some battle between how some JS filtering should be implemented and I would like to ask you guys some input on this.
Today we have this site that displays a long list of repeated entries of data and some JS filtering would be nice for the users to navigate through. The usual stuff: keyword, order, date, price, etc. The question is not the use of JS, which is obvious, but the origin of the data. One person defends that the HTML itself should be used and that the JS should parse through it making the user's desired filtering. Another person defends that we should use a JSON generated in the server, and that JSON should be the data's origin.
What you guys think on this? What are the pros and cons?
As a final request, I would like you to be the most informative as possible since your answers will be used and referenced for all us in the company. (Yes, that is how we trust you!:)
The right action is matter of taste and system architecture as well as utility.
I would go with dynamically generated pages with JS and JSON -- These days I think you can safely assume that most browsers has Javascript enabled -- however you may need to make provisions for crawler (GoogleBot, Bing, Ask etc) as they may not fully execute all JS and hence may not index the page if you do figure out some kind of exception for supporting those.
Using JS+JSON also means that you make your code work so that support for mobile diveces is done client side, without the webserver having to create anything special.
Doing DOM manipulation as the alternative would not be my best friend, as the logic of the page control and layout is split-up in two places -- partly in the View controller on the webserver, and partly in the JavaScript -- it is in my opinion better to have it in one place and have the view controller only generate JSON and server the root pages etc.
However this is a matter of taste, and im not sure that I would be able to say that there is one correct and best solution.
I think it's a lot cleaner if the data is delivered in JSON and then the presentation HTML or view of that data is generated from that JSON with javascript.
This fits the more classic style of keeping core data structures separate from views. In this manner you can generate all types of views without having to constantly munge/revise the way you store, access and manipulate the data. You can even build classes and methods to develop a clean interface on your data that is entirely independent of how that data is displayed.
The only issue I see with that is if the browser doesn't support javascript and that browser is a desired viewer. In that case, you have to include a default HTML version from the server that will obviously not be manipulated and the JSON will be ignored.
The middle ground is that you include both JSON and the "default", initial HTML view of that data in rendered HTML. The view comes up quickly and non-JS browsers can see something useful. But, then any future manipulation of the view (sorting, for example) uses the JSON data and generates a new clean view from the JSON data. No data is then ever "parsed" from the HTML view.
In larger projects, this also can facilitate the separation of presentation from data manipulation so different people may work on creating HTML views vs. manipulate the data (like sorting).
I would make the multiple ajax calls to the server and have it return the sorted/filtered data. If you server backend is fast than it won't be very taxing and you could even cache the data between requests.
If you only have 50-100 items than it would be reasonable to send it all to the client and have javascript sort and filter it.
Some considerations to help make the decision
Is the information sensitive and unique? (this voids and benefit to caching in my first point)
What is the most common request that will happen and are you optimizing for that?
How much data is there? (tens of rows, hundreds, thousands, millions)?
Does you site have to work with JavaScript turned off? (supporting older browsers?)
Is your development team more comfortable doing this in the front-end or back-end?
The answer is that it depends on your situation.
Related
As I can't orientate freely in the topic of building dynamic sites, it is quite hard to me to google this. So I'll try to explain the problem to you.
I'm developing a simple social network. I've built a basic PHP API represented by the files like "get_profile.php", "add_post.php", etc. with the POST method that is used to pass some data. Then I try to get the data using JS AJAX (php functions return it by JSON), which means I get all the data that I need to show on a page after the page is loaded. That causes decreasing of a page loading speed and I feel like this structure is really wrong.
I hope you'll explain me how to build a proper structure or at least give me some links to read. Thanks.
Populate the HTML with the (minimum) required data on the server side and load all other necessary data on the client side using AJAX (as you already do).
In any case, I would profile your application to find the most important bottle necks. Do you parallelize AJAX requests?
Facebook, for example, doesn't populate its HTML with the actual data on the server side, but provides the rough structure, which is later filled using AJAX requests.
If I understood your architecture right, it sounds ok.
Advices
Making your architecture similar to this allows you to deliver templates for the page structure that you then populate with data from your ajax request. This makes your server faster also since it doesn't have to render the HTML also.
Be careful with the amount of requests you make though, if each client makes a lot of them you will have a problem.
Try and break your application into different major pieces and treat each one in turn. This will allow you to separate them into modules later on. This practice is also referred as micro-services architecture.
After you broke them down try and figure user interaction and patterns. This will help you design your database and model in a way in which you can easily optimise for most frequest use-cases.
The way of the pros.
You should study how facebook is doing things. They are quite open about it.
For example, the BigPipe method is the fastest I have seen for loading a page.
Also, I think you should read a bit about RESTful applications and SOA type architectures.
This is something I've been thinking about a lot. What are the advantages and disadvantages of a design where the pages are un-mapped pure HTML and JavaScript and the data is all JSON pulled from the Struts 2? No action mappings means relative page references so we don't have much use for the Struts 2 URL tags or any other tags for that matter. Literally ANY information we need from the model can be retrieved through JSON or handled with a custom Interceptor. Session management for example is usually dealt with in an Interceptor anyways. I'm assuming that JavaScript can get information from the headers here which may not be correct. Please let me know if I'm wrong. Even if I am wrong, one might still use a hidden control or some other means to stash this type of information in. Regardless, I feel certain that it can be done although I am fuzzy on the specifics.
In a nutshell the end result would seem to be a pure HTML / JQuery UI which a UI-only developer, an expert on page design and usability can work with without ever needing to know Java. A person who's sole purpose is to manage the user experience from the UI side. With the use of tabs, menus pop-ups etc., pages might never need anything except JSON data which would enhance browser caching. This idea also greatly reduces the need for Web Services, since you already have a cleanly de-coupled model and view. We won't need OGNL or JSTL or have to mess with the value stack or type converters. If you are using Hibernate, I've already seen libraries out there that will convert Hibernate entities directly to JSON.
Am I completely off my rocker here? Is it possible to get what Servlets and JSP promised without all the cruft?
I'm not sure what your final sentence means at all. Not sure what you mean by "reducing the need for web services", because what you describe are web services returning JSON.
That aside; of course you can do everything client-side via JavaScript and get only JSON from S2. If your client-side framework supports it, I don't really think there are any drawbacks, as long as your app doesn't require data-specific bookmarking or you're using a JS library that supports it.
The level of caching may be slightly higher, but a well-designed, non-JSON-only app can do quite a bit of caching anyway, so I'm not sure how much of a boost you'd get from that--but certainly some.
Depending on the browser you may run into a variety of issues regarding memory usage, leaks, etc. which can be difficult (or impossible) to track down or resolve, but depending on your app's usage patterns, and your user's browser preferences, this may never become an issue.
With modern browsers, and modern client-side JS frameworks, this kind of app is completely possible, and is being done more and more--it's not a unique idea.
This is a design question. I find myself going back and forth between two UI design styles.
The UI on a prototype I developed recently relied heavily on loading the elements of the UI as partial views via AJAX. I did not like that approach because I had to do a lot of requests to the server while my page was loading. However, the plus of this design was that I could easily edit the partial view templates, so the code was easier to manage.
In the next iteration I decided to package all the information at once and then use JavaScript to generate partial views and plug this information into it (and only use Ajax when on-demand up-to-date information was actually needed). My page loads faster, but I find myself generating a lot of HTML snippets in JavaScript, which is getting harder to manage.
As I see it, using Ajax you get:
Easier to maintain (+)
Longer UI response times (-)
And with JavaScript only, you get
Faster UI response (+)
Easier to handle server-side errors (+)
Harder to maintain (-)
So, There are a few things on which I would like to hear your comments:
I do not like using Ajax unless I don't have a need for actual on-demand data. Am I wrong?
Are there frameworks/libraries that would make managing HTML-generating JavaScript code easier?
Are there any other pros/cons of the two approaches that I have missed?
Thanks,
There are templating libraries for JavaScript, if you want to get that involved. In general I would avoid manually sticking together HTML strings from JS unless there's a particular need to (eg in some cases for performance when dealing with very large tables).
HTML-hacking is difficult to read and prone to HTML-injection security holes when you don't get escaping right.
Start instead with the DOM methods, and use DOM-like content manipulation libraries to make it easier. For example, if using jQuery, do this:
$('<a>', {href: somelink, text: sometext})
and not this:
$(''+sometext+'') // insecure mess
There doesn't need to be a big difference between fetching data through XMLHttpRequest vs including it in the HTML document itself. You can direct a bunch of JSON data to the same function that will build page parts whether it was just fetched by XMLHttpRequest in an update operation, or included in function calls at document load time.
When including data in the page you will generally need to include a timestamp they were generated in any case, so that if the browser goes back to the page a while later without reloading it, the browser can detect that the information is now out-of-date, and cue up an XMLHttpRequest to update it.
The usual question when you're creating your page content from data with client-side JavaScript is, are you going to fill in the initial static HTML version from the server too? If so, you're going to be duplicating a lot of your content generation work in both client-side JS and the server-side language(*). If not, then you're making the content invisible to non-JS user agents which includes search engines. Whether or not that matters to you typically depends on what the app is doing (ie does it need to be searchable and accessible?)
(*: unless you can use server-side JavaScript from something like node.js and re-use your content generation code. This is still a somewhat rare approach.)
Why not look at integrating require.js into your workflow? I'm not entirely sure how it would work with templates, but included in their pipeline is the ability to package all required script files into a single .js to be served by a single server request/response.
I have no personal experience about it, but Closure looks promising. The thing about its templates being usable on both server and client side might be of interest to you. Here is what was said about using it in Google+:
The cool thing about Closure templates is they can be compiled into both Java and JavaScript. So we use Java server-side to turn the templates into HTML, but we can also do the same in JavaScript client-side for dynamic rendering. For instance, if you type in a profile page URL directly, we'll render it server-side, but if you go to the stream say and navigate to someone's profile page, we do it with AJAX and render it client-side using the same exact template.
When working with remote data after a page is loaded, for small datasets I prefer returning data only and adding to the UI with templates.
For large datasets, I recommend using your partial views to render the html on the server to reduce overhead in the client as #bobince mentioned.
For client-side state tracking, check out Knockout at http://www.knockoutjs.com. It uses an MVVM approach with data models bound to UI elements and makes it very simple to send the data back to the server via AJAX. It works with the jquery.tmpl template library out of the box or you can integrate another library of preference with a little more effort.
As far as managing templates, it's easy enough to store common templates in an object, either on the server to be retrieved with your remote data, or in a javascript object on the client.
I'm getting to the point on an app where I need to start caching things, and it got me thinking...
In some parts of the app, I render table rows (jqGrid, slickgrid, etc.) or fancy div rows (like in the New Twitter) by grabbing pure JSON and running it through something like Mustache, jquery.tmpl, etc.
In other parts of the app, I just render the info in pure HTML (server-side HAML templates), and if there's searching/paginating, I just go to a new URL and load a new HTML page.
Now the problem is in caching and maintainability.
On one hand I'm thinking, if everything was built using Javascript HTML Templates, then my app would serve just an HTML layout/shell, and a bunch of JSON. If you look at the Facebook and Twitter HTML source, that's basically what they're doing (95% json/javascript, 5% html). This would make it so my app only needed to cache JSON (pages, actions, and/or records). Which means you'd hit the cache no matter if you were some remote api developer accessing a JSON api, or the strait web app. That is, I don't need 2 caches, one for the JSON, one for the HTML. That seems like it'd cut my cache store down in half, and streamline things a little bit.
On the other hand, I'm thinking, from what I've seen/experienced, generating static HTML server-side, and caching that, seems to be much better performance wise cross-browser; you get the graphics instantly and don't have to wait that split-second for javascript to render it. StackOverflow seems to do everything in plain HTML, so does Google, and you can tell... everything appears at once. Notice how though on twitter.com, the page is blank for .5-1 seconds, and the page chunks in: the javascript has to render the json. The downside with this is that, for anything dynamic (like endless scrolling, or grids), I'd have to create javascript templates anyway... so now I have server-side HAML templates, client-side javascript templates, and a lot more to cache.
My question is, is there any consensus on how to approach this? What are the benefits and drawbacks from your experience of mixing the two versus going 100% with one over the other?
Update:
Some reasons that factor into why I haven't yet made the decision to go with 100% javascript templating are:
Performance. Haven't formally tested, but from what I've seen, raw html renders faster and more fluidly than javascript-generated html cross-browser. Plus, I'm not sure how mobile devices handle dynamic html performance-wise.
Testing. I have a lot of integration tests that work well with static HTML, so switching to javascript-only would require 1) more focused pure-javascript testing (jasmine), and 2) integrating javascript into capybara integration tests. This is just a matter of time and work, but it's probably significant.
Maintenance. Getting rid of HAML. I love HAML, it's so easy to write, it prints pretty HTML... It makes code clean, it makes maintenance easy. Going with javascript, there's nothing as concise.
SEO. I know google handles the ajax /#!/path, but haven't grasped how this will affect other search engines and how older browsers handle it. Seems like it'd require a significant setup.
Persistant private data storage.
You need a server to store data with various levels of public/private access. You also need a server for secure closed source information. You need a server to do heavy lifting that you don't want to do on the client. Complex data querying is best left upto your database engine. Indexing and searching is not yet optimised for javascript.
Also you have the issues of older browsers being far slower. If your not running FF4/Chrome or IE9 then there is a big difference between data manipulation and page construction on the client and the server.
I myself am going to be trying to build a web application made entirely using a MVC framework and template's but still using the server to connect to secure and optimised database.
But in general the application can indeed be build entirely in javascript and using templates. The various constructs and javascript engines have advanced enough to do this. There are enough popular frameworks out there to do this. The Pure javascript web applications are no longer experiments and prototypes.
Oh, and if were recommending frameworks for this, then take a look at backbone.js.
Security
Let's not forget that we do not trust the client. We need serverside validation. JavaScript is interpreted, dynamic and can be manipulated at run time. We never trust client input.
I used to mix these two approaches but then switched to client side rendering entirely because it was really hard to handle heavy JavaScript properly the otherwise. As a complete solution can recommend the approach of the JavaScriptMVC framework.
It has a view rendering engine called EJS which can compress your views into plain JavaScript for a production build of your application. That makes it extremely fast (all your HTML gets preloaded with your single compressed JavaScript file so it is rendered as soon as you receive your models JSON data). I personally couldn't notice a performance difference between EJS rendering and transferring plain HTML.
Then for your API, following the REST principles, caching is one of the key constraints. So if your application supports HTTP caching properly for JSON data and using your compressed client side templates you might even see a performance improvement.
I could be way off here, but...
Have you ever looked at CouchDB? (I have no affiliation w/ them BTW) I could be way wrong, but your situation sounds like it may be a perfect fit for the use of the Apache CouchDB I haven't really used it yet myself but I took a good look at it a short while back and it is a very interesting database.
It is a document based database that uses a REST api for connections (very versatile and easy to use). It is also very JSON centric, very fast and a tiny footprint; they say it can reside on phones and other embedded uses too but at the same time is supposed to be extremely scalable (upwards that is). If your a big JS user (which you sound like you are) then you may be right at home with it.
I was just thinking that it may come in handy in any number of ways that have been proposed here and thought I'd chime in just to give you an idea for storage options :)
Hey everyone, I've been thinking about how the majority of web apps works at the moment. If as an example the backend is written in java/php/python what you probably see is that the backend is "echoing / printing" the html ready to the browser, right.
For web apps that works using almost exclusively ajax, is there a reason to not simply communicate without html, as an example just by using JSON objects passing back and fourth between the server and client, and instead of "printing or echoing" html in our script/app backend we simply echo the json string, ajax fetches it and converts the JSON string to an object with all of our attributes/arrays and so on.
Surely this way we have less characters to send, no html tags and so on, and on the client side we simply use frameworks such as jQuery etc and create/format our html there instead of printing and echoing the html in the server scripts?
Perhaps people already do this but I have not really seen a lot of apps work this way?
The reason is that I want todo this is because I would like to separate the layer of presentation and logic more than it currently is, so instead of "echoing" html in my java/php I just "echo" json objects, and javascript takes care of the whole presentation layer, is there something fundamentally wrong with this, what are your opinions?
Thanks again Stackoverflow.
There are quite a few apps that work this way (simply communicating via AJAX using JSON objects rather than sending markup).
I've worked on a few and it has its advantages.
In some cases though (like when working with large result sets) it makes more sense to render the markup on the server side and send it to the browser. That way, you're not relying on JavaScript/DOM Manipulation to create a large document (which, depending on the browser, would perform poorly).
This is a very sensible approach, and is actually used in some of our applications in production.
The main weakness of the approach is that it increases the load on the browser resource-wise and therefore might - in light of browsers often already-sluggish JS performance - lead to worse user experience unless the presentation layer mechanics is very well tuned.
Now a days many webapps use this approach like gmail and other big apps even facebook this
the main advantage of this approach is user dont need to refresh all the pages and he gets what we want to show him or what he desired.
but we have to make a both version like ajax and normal page refresh what if the user refresh the page.
we can use jquery template which generates html and also google's closer which is used by a gmail and other google products.