This is more a question about best practices. When trying to apply MVC-like design pattern in a web application, I often find myself wonder how I should go about updating the View.
For example, if I am to update View_1 who has X number of elements. Is it better to:
A: iterate through each of the X elements, figuring out which ones need to be updated and apply the DOM change at a very fine granularity.
or
B: using the data supplied by Model or some other data structure to regenerate the markup for this entire View and all its enclosing elements, and replace the root element of View_1 in one DOM manipulation?
Correct me if I am mistaken. I heard that rendering engines are usually more efficient at replacing at large chunk of the DOM in one go, than multiple smaller DOM operations. If that is the case then approach B is superior. However, even using template engines, I still sometimes find it difficult to avoid rewrite markups for parts of the view that aren't changed.
I looked into the source code for project Bespin before the renamed it. I distinctly remember that they implemented some sort of rendering loop mechanism where DOM operations are queued and applied in fixed time intervals, much like how games manage their frames. This is similar to Approach A. I can also see the rationales behind this approach. The small DOM operations applied in such manner keeps the UI responsive (especially important for a web text editor). Also this way, the application can be made more efficient by only update the elements that needs to be changed. Static text and aesthetic elements can remain untouched.
Those are my arguments for both sides. What do you guys think? Are we looking for a happy medium somewhere, or one approach is by and large superior?
Also, are there any good books/papers/sites on this particular topic?
(let's assume the web app in question is interaction heavy with many dynamic updates)
It's true that rendering engines usually handle change in large chunks faster than multiple small changes.
tl;dr: The bespin way would be ideal, and if you can, do it in a worker.
Depending on the size of the amount of changes you might want to try to start a worker and do the calculation of the changes inside the worker since long running JS lock up the UI. You might consider using the following flow:
Create a object with a part of the dom tree and also the parent id.
Stringify the object to JSON
Start a worker
Pass in the stringified object into the worker
Receive and parse the string.
Work on changing all the necessary parts of the dom tree that you passed in.
Stringify the object again.
Pass the object back to the main thread.
Parse and extract new dom tree.
Insert into the dom again.
This will be faster if there are many changes and a small tree. If it's a big tree and few changes just doing the changes locally in a copy of the real DOM tree will be faster and then updating the DOM in one go.
Also read googles sites about page speed:
https://code.google.com/speed/articles/
And especially this article:
https://code.google.com/speed/articles/javascript-dom.html
After 3 years of working with various web technologies, I think I finally found a great balance between the two approach: virtual DOM
Libraries like vdom, Elm, mithril.js, and to some extend Facebook React track a thin abstraction of the actual DOM, figures out what needs to be changed, and tries to apply the smallest possible DOM change.
e.g.
https://github.com/Matt-Esch/virtual-dom/tree/master/vdom
Related
I would like to refresh a Bootstrap v4 table (https://getbootstrap.com/docs/4.0/content/tables) based on the values fetched from the server side, for example a REST API resource.
I am currently struggling with the different approaches below:
Websockets
Webworker
Recursive setTimeout + ajax polling
Any other and better solution?
My main requirement would be something that is refreshed every 500 ms or less (e.g. stock prices).
Also I am wondering how to handle the most efficiently possible the DOM rendering of the table.
Web Workers can't directly interact with the DOM and your task is not that intensive.
I'd say WebSockets + DOM manipulation via (data) attributes and separate node insertion instead of a huge chunk of nodes inserted at once. It may be a bit slower, but there's not much of a difference and you might not even notice it. See Fastest DOM insertion
I'd update things separately because:
1) It's tidier and maintainable,
2) You don't need to worry about event delegation or reinitialization of particular stuff
3) The flow feels more natural instead of just getting a huge chunk of mark-up and "pasting" it in the DOM.
If you won't be adding new nodes and you will just listen for data changes on existing nodes, then I'd clearly suggest going for attribute-based changes.
I recently ported a heavy page to React. I've kept the html almost identical. The main difference being that, earlier, the server rendered html was directly given to the browser and now, the react rewrite pulls json via a server side API and uses React to manage the DOM.
I've seen heap snapshots for the earlier implementation going up to 55 MBs. For the same data, the heap snapshot for the React.js implementation comes to around 100+ MBs(almost double)
I understand that the json data held in memory will contribute to some increase in the memory consumed. But, when I examined the heap snapshot, I see that around 60% of the retained size is due to some objects whose retaining path contain deleteAllListeners > .... > unmountComponentAtNode . I am trying to understand what that means in terms of reducing the memory used.
Also, could the "data-reactid" attributes added by React to the DOM contribute to a non-negligible increase in memory consumption ?
This question has some more details that might help.
React is using something called Virtual DOM. Basically it constructs alternative DOM tree in memory, in addition to the existing browser DOM tree, but to perform efficient updates it has to keep last displayed Virtual DOM tree in memory, so it can generate fast and efficient updates to the browser DOM tree.
From details of second question, I understand that you have an infinite scroll, where you basically add new components (without removing new ones) when user scrolls down the page. So this should be the source of increased memory usage (since now you have data + virtual dom in memory, compared to the previous solution)
The way it fix it is to render only components which are actually visible to the user, you can try to use react-list, or implement your own component for this.
Context:
I have a web application that processes and shows huge log files. They're usually only about 100k lines long, but it can be up to 4 million lines or more. To be able to scroll through that log file (both user initiated and via JavaScript) and filter the lines with decent performance I create a DOM element for each line as soon as the data arrives (in JSON via ajax). I found this better for performance then constructing the HTML at the back-end. Afterwards I save the elements in an array and I only show the lines that are visible.
For max 100k lines this takes only about a few seconds, but anything more takes up to one minute for 500k lines (not including the download). I wanted to improve the performance even more, so I tried using HTML5 Web Workers. The problem now is that I can't create elements in a Web Worker, not even outside the DOM. So I ended up doing only the json to HTML conversion in the Web Workers and send the result to the main thread. There it is created and stored in an array. Unfortunately this worsened the performance and now it takes at least 30 seconds more.
Question: So is there any way, that I'm not aware of, to create DOM elements, outside the DOM tree, in a Web Worker? If not, why not? It seems to me that this can't create concurrency problems, as creating the elements could happen in parallel without problems.
Alright, I did some more research with the information #Bergi provided and found the following discussion on W3C mailing list:
http://w3-org.9356.n7.nabble.com/Limited-DOM-in-Web-Workers-td44284.html
And the excerpt that answers why there is no access to the XML parser or DOM parser in the Web Worker:
You're assuming that none of the DOM implementation code uses any sort
of non-DOM objects, ever, or that if it does those objects are fully
threadsafe. That's just not not the case, at least in Gecko.
The issue in this case is not the same DOM object being touched on
multiple threads. The issue is two DOM objects on different threads
both touching some global third object.
For example, the XML parser has to do some things that in Gecko can
only be done on the main thread (DTD loading, offhand; there are a
few others that I've seen before but don't recall offhand).
There is however also a workaround mentioned, which is using a third-party implementation of the parsers, of which jsdom is an example. With this you even have access to your own separate Document.
So is there any way, that I'm not aware of, to create DOM elements, outside the DOM tree, in a Web Worker?
No.
Why not? It seems to me that this can't create concurrency problems, as creating the elements could happen in parallel without problems.
Not for creating them, you're right. But for appending them to the main document - they would need to be sent to a different memory (like it's possible for blobs) so that they're inaccessible from the worker thereafter. However, there's absolutely no Document handling available in WebWorkers.
I create a DOM element for each line as soon as the data arrives (in JSON via ajax). Afterwards I save the elements in an array and I only show the lines that are visible.
Constructing over 500k DOM elements is the heavy task. Try to create DOM elements only for the lines that are visible. To improve performance and showing the first few lines faster, you also might chunk their processing into smaller units and use timeouts in between. See How to stop intense Javascript loop from freezing the browser
You have to understand the nature of a webworker. Programming with threads is hard, especially if you're sharing memory; weird things can happen. JavaScript is not equipped to deal with any kind of thread-like interleaving.
The approach of webworkers is that there is no shared memory. This obviously leads to the conclusion that you can't access the DOM.
There is no direct way to access the DOM through Web Workers.
I recently released #cycle/sandbox, it is still WIP, but it proves with the Cycle JS architecture it is fairly straight forward to declare UI behaviour in the Web Worker. The actual DOM is only touched in the main thread, but event listeners, and DOM updates are indirectly declared in the worker, and a synthesized event object is sent when something happens on those listeners. Furthermore it is straight forward to mount these sandboxed Cycle Components side-by-side regular Cycle Components.
http://github.com/aronallen/-cycle-sandbox/
I don't see any reason why you can't construct html strings using web-workers. But I also don't think there would be much of a performance boost.
This isn't related to Web-Workers, but it relates to the problem you're trying to solve. Here are some thing that might help speed things up:
Use DocumentFragments. Add elements to them as the data comes in, and add the fragments to the DOM at an interval (like once a second). This way you don't have to touch the DOM (and incur a redraw) every time a line of text is loaded.
Do loading in the background, and only parse the lines as the user hits the bottom of the scroll area.
According to https://developer.mozilla.org/en-US/docs/Web/Guide/Performance/Using_web_workers there's no access to the DOM from a web worker unfortunately.
You have a couple of anti-patterns in your design:
Creating a DOM object has considerable overhead, and you are
creating potentially millions of them at once.
Trying to get a web worker to manage the DOM is exactly what web
workers are not for. They do everything else so the DOM event loop stays responsive.
You can use a cursor pattern to scroll through arbitrarily large sets of data.
DOM posts a message to worker with start position and number of lines requested (cursor).
Web worker random accesses logs, posts back the fetched lines (cursor data).
DOM updates an element with the async cursor response event.
This way, the heavy lifting is done by the worker, whose event loop is blocked during the fetch instead of the DOM, resulting in happy non-blocked users marvelling at how smooth all your animations are.
Update for 2022 (actually available in chrome since 2018):
If you are ok with displaying your logs in a canvas element, you could use the new OffscreenCanvas api.
The OffscreenCanvas interface provides a canvas that can be rendered off screen. It is available in both the window and worker contexts.
You could then asynchronously display frames produced in the Worker back to a canvas element on the main thread.
More examples here.
So you can't directly create DOM in a webworker - however, there may be another option to do a fair bit of your processing outside the main thread.
Check out this jsPerf I just created: http://jsperf.com/dom-construction-obj-vs-str
Essentially, you could be emitting POJSO's that have all the same values you get from a DOM, and convert it to DOM objects after receiving the message (this is what you're doing when you get HTML back, after all; POJSOs are just lower overhead, by virtue of not requiring further string processing). In this way you could even do things like emit event listeners and such (by, say, prefixing the event name with '!', and having the value map to some template-supplied view argument).
Meanwhile, without the DOM parser available, you'll need your own thing to convert a template as-needed, or to compile one to a format that's fast.
No you can't create DOM elements in a web worker, but you can create a function that accepts the post message from that web worker, that does create the DOM elements. I think the deign that your looking for is called array chucking. And you would need to mix that with the web worker design pattern.
Is it a bad practice to have backbone views which do not depend on any kind of templating system?
Of course, this does not mean that any kind of DOM code will be generated by hand, using hardcoded strings within the views. No no no. I can of course cache the basic layout for each view in DOM elements, set to display:none. However, I want to reduce any kind of value setting within the templates themselves. I'd rather do that using jquery, or any other kind of DOM modifier form within the view itself. This way I save myself the constant discrepancies and the countless hours of efforts that I've exposed my apps and myself to, using Mustache, Handelbars, the Underscore templating system, etc. Having all the view logic in one place makes everything much cleaner, at least in my view. It gives a lot of benefits, such as proper partial rendering, value binding, etc, that I'd need tons of days to spend on if I wanted to implement them with mustache or something.
The only problem that I see might occur is whether the constant checks that I'd do with jQuery will be performing fast enough, but I guess that it shouldn't be such a problem at the end.
What do you think? Good? Bad? Practical?
IMHO not using a template engine is not, per-se, a bad design decision.
Template engines are meant to produce cleaner and more maintainable code, if you think not using them produces cleaner and more maintainable code them you can run without them.
I my opinion is better to use template engines but this is just a matter of taste. I also combine the templates with manual DOM modifications through this.$el.find(".my-element").html( "new value" ); for partial updates.
Templates are good. They save you a lot of time coding and allow you to loop over elements and create multiple elements with very little code.
That said if the render is called too much or you destroy and recreate DOM elements/templates, it costs a lot in performance.
I have a PerfView for backbone that can display 1,000,000 models in a scrollview at 120FPS in chrome. The trick is I only render the template once with all the possible nodes I need then change the content in the DOM when the model changes. Also using an object pool for the DOM elements will let you reuse them. Check out the modelMap property on line 237 (https://github.com/puppybits/BackboneJS-PerfView/blob/master/index.html) on how to semi-automate updating an element so you don't overuse a template.
I'm building an AJAX 'web application' where, once the UI is loaded, calls to the server are for 'data exchange' only. As a result a lot of UI manipulation will be done using Javascript. Lets say the Javascript retrieves some data consisting of multiple fields from the server using AJAX. To put it on the screen I can think of multiple approaches -
Call methods like createElement() and appendChild() to build an interface to display the retrieved data
Populate the .innerHTML for the container element then for every data field lookup a container in the newly added HTML and populate it. The data that goes into .innerHTML could be stored in a JS variable or contained in a hidden node, or fetched using a seperate AJAX call.
Have the interface stored in a hidden node. Clone it (using cloneNode()) and put it in the actual container (using appendChild()) and then populate it with fields like in method 2.
and there are probably more ways.
Could you share pros, cons and possible gotachas in these approaches from cross-browser support, performance and coding complexity point of views?
Somewhat related question: Is client-side UI rendering via Javascript a good idea?
Thanks.
ok, let's start off:
Use toolkits
First you would want to invest time learning a JS toolkit. While others suggest native JS is good (which it really is), but you would not want to waste time on building apps that don't work cross-browser or spend too much time testing it. People in the community have invested their time in doing that for you. Show some love to the open community and use their product. I personally use jQuery, but there are others like Dojo and YUI.
But still use native JS whenever possible. It's still faster.
Structure your code
After a toolkit, you need some structure. BackboneJS will take care of that. It's to structure your code so that your code is reusable and well.. won't end up as spaghetti on your screen. Other tools like RequireJS are also useful for those scripts that need other scripts to run.
Templates: From strings to elements
Then, with that, you now have a toolkit but you still need to build the interface. It's better if you use a templating engine like Mustache or Handlebars. These render templates for your UI from strings (yes, plain strings of HTML). Just send over your template data from the server, store it in your app (in a variable or browser localstorage), and reuse it as necessary. No need for cloning hidden nodes!
Don't touch that DOM
As for approaching the DOM, you should touch the DOM only when necessary. DOM is slow, manipulating it is he** slow! that means you should avoid unnecessary animations, avoid excessive element append and remove, as well as changing styles. Check this article about avoiding too much reflow and repaints. Face it, the user won't event notice the round corners of your boxes, or the gradient background and don't even care if you did a slide animation or a fade-out. What they want is to get the job done and not adore the fireworks display.
Also, remove anything that isn't on screen. You might end up having 20% content on screen, and 80% off screen - a waste of memory.
Cache: fetch once, store, use infinitely for later
Now, your app is getting heavy and you want to shave off some HTTP requests. You can do this by caching. I usually use caching mostly on the templates so that every new UI, you don't need to load again from the server. You can do this by storing stuff in an object. you can go a little further and use the browser's localStorage when available.
Caching isn't all for the network. Let's say you have some complex calculations you want to use later, or some text from an unfinished form, use the cache for that too.
Avoid HTTP requests (or at least lighten them up)
While lightening up your app by using AJAX, you will inevitably be tempted to use AJAX just about anywhere - don't abuse it. Often times i see people who aggressively poll the server (every half-second or less). This not only strains the server (too many requests), but also the browser (wasting processing cycles) and the network (bandwidth). There are several practices done these days to avoid added HTTP requests:
Image Spriting - The art of placing a lot of images into one image and using background-position to change the image. Surely beats 100 individual HTTP requests
Use JSON for textual data - AJAX was meant to use XML.. then came along JSON that was a fat-free, platform-independent format of strucured data.
Do not return HTML-formatted data - With exemption of templates, which are HTML strings, you should never return HTML-formatted data over the wire. Have JS do JSON+templates on the client-side instead.
I'm building a similar app (Lightweight CMS)
In my view the approach you take will be dependent on the complexity of data that you are sending from the server > manipulating on the client side > and returning back to the server and db.
The cms that I'm working on is very basic and does not require heavy-duty manupulation on the client side. TinyMCE is as far as it will go.
Initially I was building the client admin area by echoing <input> from php and then collecting the data by parsing the DOM, converting to JSON and AJAX it back to the server (I found this code very helpful)
That of course required the user to hit "save" after editing or adding new data.
I later on decided that I needed something even more responsive and simpler than that so I'm now re-implementing everything for Jeditable which AJAXes the data as soon as the client hits "OK" on that particular field. No "main save" required.
To conclude it's really an area that is pretty uncharted. I mean, it appears to me that people in the industry do not like to blur that line between back-end and front-end and find "one solution" that will do the entire DB>SERVER>CLIENT>SERVER>DB operation.
I'd love to see how you solved your problem.
Just my 2 cents.