So we have all heard that v8 uses the thing called hidden classes where when many objects have the same shape, they just store a pointer to the shape struct which stores fixed offsets. I have heard this a million time, and I very much get how this reduces memory usage by A LOT (not having to store a map for each one is amazing) and potentially because of that a bit faster performance.
However I still don't understand how it avoids dynamic lookup. The only thing I have heard is storing a cache between a string (field name) and a fixed offset, and checking it every time, but if there's a cache miss (which is likely to happen) there will still be a dyanmic lookup.
Everyone says that this is almost as fast as C++ field access (which are just a mov instruction usually), however this 1 field access cache isn't even close.
Look at the following function:
function getx(o)
{
return o.x;
}
How will v8 make the access of the x field so fast and avoid dynamic lookup?
(V8 developer here.)
The key is that hidden classes allow caching. So, certainly, a few dynamic lookups will still be required. But thanks to caching, they don't have to be repeated every single time a property access like o.x is executed.
Essentially, the first time your example function getx is called, it'll have to do a full property lookup, and it'll cache both the hidden class that was queried and the result of the lookup. On subsequent calls, it'll simply check the incoming o's hidden class against the cached data, and if it matches, it uses the cached information about how to access the property. Of course, on mismatch, it has to fall back to a dynamic lookup again. (In reality it's a bit more complicated because of additional tradeoffs that are involved, but that's the basic idea.)
So if things go well, a single dynamic lookup is sufficient for an arbitrary number of calls to such a function. In practice, things go well often enough that if you started with a very simple JS engine that didn't do any such tricks, and you added caching of dynamic lookup results, you could expect approximately a 10x performance improvement across the board. Adding an optimizing compiler can then take that idea one step further and embed cached results right into the optimized code; that can easily give another 10x improvement and approach C-like performance: code still needs to contain hidden class checks, but on successful check can read properties with a single instruction. (This is also why JS engines can't just "optimize everything immediately" even if they wanted to: optimization crucially depends on well-populated caches.)
Related
I was happily using Map for indexed accessed everywhere in my JavaScript codebase, but I've just stumbled upon this benchmark: https://stackoverflow.com/a/54385459/365104
I've re-created it here as well: https://jsben.ch/HOU3g
What benchmark is doing is basically filling a map with 1M elements, then iterating over them.
I'd expect the results for Map and Object to be on par, but they differ drastically - in favor of Object.
Is this expected behavior? Can it be explained? Is it because of the ordering requirement? Or because map is doing some key hashing? Or just because Map allows any object as key (I'd expect it using pointer address for key then, which does not need any hashing)? What is the difference between the Map and Object indexing algorithms?
This is quite unexpected and discouraging - basically I'll have to revert back to the old-school "object as map" coding style.
Update #1
As suggested in comments, the Object might be optimized to Array (since it is indexed by integer, starting from the zero).
Changing the iteration order from size to 0 - Object is still 2x faster. When using strings as index, Map performs 2x better.
(V8 developer here.)
I'll have to revert back to the old-school "object as map" coding style.
If you do that, you will have fallen victim to a misleading microbenchmark.
In the very special case of using consecutive integers as keys, a plain Object will be faster, yes. Nothing beats a contiguous array in that scenario. So if the "indexed accesses everywhere in your codebase" that you mentioned are indeed using index sets like the integers from 0 to 1M, then using an Object or Array is a good idea. But that's a special case. If the index space is sparse, things will already look different.
In the general case of using arbitrary strings in random order, a Map will perform significantly better than an Object. Even more importantly, the way such object property accesses are handled (in V8, and quite possibly in other engines too) has non-local effects: if one function puts excessive stress on the slow path of the object property lookup handling system, then that will likely slow down some other functions relying on that same slow path for their property accesses.
The fundamental reason is that engines optimize different things for different usage patterns. An engine could implement Objects and Maps pretty much the same under the hood; but that wouldn't be the ideal behavior, because different usage patterns benefit from different internal representations and implementation choices. So engines allow you to provide them with a hint: if you use a Map, the engine will know that you're planning to use the thing as a map (duh!), where random keys will come and go. If you use an Object, then the engine will (at least at first) assume that you want the set of optimizations that work best for your average object, where the set of properties is fairly small and static. If you use an Array (or Object with only integer properties, which is nearly the same thing in JS), then you're making it easy for the engine to give you fast integer-indexed accesses.
Using "x" + i as key is a good suggestion to demonstrate how quickly a microbenchmark can be changed so it appears to produce opposite results. But here's a spoiler: if you do (only) this modification, then a large part of what you're measuring will be number-to-string conversion and string internalization, not Map/Object access performance itself.
Beware of microbenchmarks; they are misleading. You really have to analyze them quite deeply (by profiling, and/or by inspecting generated code, and/or by tracing other engine internals) to be sure that they're measuring what you think they're measuring, and hence are producing results that are telling you what you think they're telling you.
In general, it is strongly recommended to use representative test cases for performance measurements. Ideally, your app itself; or by extracting a realistic part of it into a testcase operating on realistic data. And if you can't measure a difference between two implementation choices with a stress test for your entire production app, then it's not a difference worth worrying about it. With a microbenchmark (i.e. a couple of artificially crafted lines), I can "prove" almost anything that doesn't apply to the general case.
I just stumbled upon this jsperf result: http://jsperf.com/delet-is-slow
It shows that using delete is slow in javascript but I am not sure I get why. What is the javascript engine doing behind the scene to make things slow?
I think the question is not why delete is slow... The speed of a simple delete operation is not worth measuring...
The JS perf link that you show does the following:
Create two arrays of 6 elements each.
Delete at one of the indexes of one array.
Iterate through all the indexes of each array.
The script shows that iterating through an array o which delete was applied is slower than iterating though a normal array.
You should ask yourself, why delete makes an array slow?
The engine internally stores array elements in contiguous memory space, and access them using an numeric indexer.
That's what they call a fast access array.
If you delete one of the elements in this ordered and contiguous index, you force the array to mutate into dictionary mode... thus, what before was the exact location of the item in the array (the indexer) becomes the key in the dictionary under which the array has to search for the element.
So iterating becomes slow, because don't move into the next space in memory anymore, but you perform over and over again a hash search.
You'll get a lot of answers here about micro-optimisation but delete really does sometimes have supreme problems where it becomes incredibly slow in certain scenarios that people must be aware of in JS. These are to my knowledge edge cases and may or may not apply to you.
I recommend to profile and benchmark in different browsers to detect these anomalies.
I honestly don't know the reasons as I tend to workaround it this but I would guess combinations of quirks in the GC (it is might be getting invoked too often), brutal rehashing, optimisations for other cases and weird object structure/bad time complexity.
The cases usually involve moderate to large numbers of keys, for example:
Delete from objects with many keys:
(function() {
var o={},s,i,c=console;
s=new Date();for(i=0;i<1000000;i+=10)o[i]=i;c.log('Set: '+(new Date()-s));
s=new Date();for(i=0;i<50000;i+=10)delete(o[i]);c.log('Delete: '+(new Date()-s));})();
Chrome:
Set: 21
Delete: 2084
Firefox:
Set: 74
Delete: 2
I have encountered a few variations of this and they are not always easy to reproduce. A signature is that it usually seems to degrade exponentially. In one case in Firefox delete inside a for in loop would degrade to around 3-6 operations a second where as deleting when iterating Object.keys would be fine.
I personally tend to think that these cases can be considered bugs. You get massive asymptotic and disproportionate performance degradation that you can work around in ways that shouldn't change the time or space complexity or that might even normally make performance moderately worse. This means that when considered as a declarative language JS gets the implementation/optimisations wrong. Map does not have the same problem with delete so far that I have seen.
Reasons:
To be sure, you would have to look into the source code or run some profiling.
delete in various scenarios can change speed arbitrarily based on how engines are written and this can change from version to version.
JavaScript objects tend to not be used with large amounts of properties and delete is called relatively infrequently in every day usage. They're also used as part of the language heavily (they're actually just associative arrays). Virtually everything relies on an implementation of an object. IF you create a function, that makes an object, if you put in a numeric literal it's actually an object.
It's quite possible for it to be slow purely because it hasn't been well optimised (either neglect or other priorities) or due to mistakes in implementation.
There are some common possible causes aside from optimisation deficit and mistakes.
Garbage Collection:
A poorly implemented delete function may inadvertently trigger garbage collection excessively.
Garbage collection has to iterate everything in memory to find out of there are any orphans, traversing variables and references as a graph.
The need to detect circular references can make GC especially expensive. GC without circular reference detection can be done using reference counters and a simple check. Circular reference checking requires traversing the reference graph or another structure for managing references and in either case that's a lot slower than using reference counters.
In normal implementations, rather than freeing and rearranging memory every time something is deleted, things are usually marked as possible to delete with GC deferred to perform in bulk or batches intermittently.
Mistakes can potentially lead to the entire GC being triggered too frequently. That may involve someone having put an instruction to run the GC every delete or a mistake in its statistical heuristics.
Resizing:
It's quite possible for large objects as well the memory remapping to shrink them might not be well optimised. When you have dynamically sized structures internally in the engine it can be expensive to resize them. Engines will also have varying levels of their own memory management on top of that of that provides by the operating system which will significantly complicate things.
Where an engine manages its own memory efficiently, even a delete that deletes an object efficiently without the need for a full GC run (has no circular references) this can trigger the need to rearrange memory internally to fill the gap.
That may involve reallocating data of the new size, then copying everything from the old memory to the new memory before freeing the old memory. It can also require updating all of the pointers to the old memory in some cases (IE, where a pointer pointer [all roads lead to Rome] wasn't used).
Rehashing:
It may also rehash (needs as the object is an associative array) on deletes. Often people only rehash on demand, when there are hash collisions but some engines may also rehash on deletes.
Rehashing on deletes prevents a problem where you can add 10 items to an object, then add a million objects, then remove those million objects and the object left with the 10 items will both take up more memory and be slower.
A hash with ten items needs ten slots with an optimum hash, though it'll actually require 16 slots. That times the size of a pointer will be 16 * 8 bytes or 128bytes. When you add the million items then it needs a million slots, 20 bit keys or 8 megabytes. If you delete the million keys without rehashing it then the object you have with ten items is taking up 8 megabyte when it only needs 128 bytes. That makes it important to rehash on item removal.
The problem with this is that when you add you know if you need to rehash or not because there will be a collision. When deleting, you don't know if you need to rehash or not. It's easy to make a performance mistake with that and rehash every time.
There are a number of strategies for reasonable downhashing intervals although without making things complicated it can be easy to make a mistake. Simple approaches tend to work moderately well (IE, time since last rehash, size of key versus minimum size, history of collision pairs, tombstone keys and delete in bulk as part of GC, etc) on average but can also easily get stuck in corner cases. Some engines might switch to a different hash implementation for large objects such as nested where as others might try to implement one for all.
Rehashing tends to work the same as resizing for simply implementations, make an entirely new one then insert the old one into it. However for rehashing a lot more needs to be done beforehand.
No Bulk:
It doesn't let you delete a bunch of things at once from a hash. It's usually much more efficient to delete items from a hash in bulk (most operations on the same thing work better in bulk) but the delete keyword only allows you to delete one by one. That makes it slow by design for cases with multiple deletes on the same object.
Due to this, only a handful of implementation of delete would be comparable to creating a new object and inserting the items you want to keep for speed (though this doesn't explain why with some engines delete is slow in its own right for a single call).
V8:
Slow delete of object properties in JS in V8
Apparently this was caused due to switching out implementations and a problem similar to but different to the downsizing problem seen with hashes (downsizing to flat array / runthrough implementation rather than hash size). At least I was close.
Downsizing is a fairly common problem in programming which results in what is somewhat akin to a game of Jenga.
Javascript arrays used to be just objects with some special syntax, but JIT engines can optimize the array to behave like a true array with O(1) access rather than O(log n) access. I am currently writing a cpu intensive loop where I need to a certain I get O(1) access rather than O(log n). But the problem is that I may need to do array [5231] = obj1 right after its created but the array will eventually be filled. I am worried that such behaviour may trick the JIT to thinking that I am using it as a sparse array and thus I wouldnt get O(1) access time I require. My question is that is there ways I can tell or at least hint the javascript engine that I want a true array?
In particular, do I need to initialize the array with values first? Like fill all of it with reference to a dummy object (my array will only contain references to objects of same prototype or undefined). Would just set the array.length = 6000 be enough?
EDIT: based on http://jsperf.com/sparse-array-v-true-array, it seems filling the array beforehand is a good idea.
I realized I have made a big derp on my profiling test because it is an animation program with varying frame rate and my profiling was focused on detecting where the cpu intensive parts are. It was not suitable in comparing between the two methods as one method would simply run slower but the relative cpu time between different function calls would be the same.
Anyhow, I wrote a test on jspref: http://jsperf.com/sparse-array-v-true-array and (unless I made an error , which so, please do correct me!) A randomly accessed array without filling it first runs twice slower in Chrome and Firefox compared to filled version. So it seems that it is a good idea to fill it in first if you are writing things like a spatial hashmap.
The Object.observe() JavaScript API allows any piece of code to receive change notifications for all property changes of any JavaScript object.
Doesn't this severely affect the code generation and performance optimizations that can be performed by the JavaScript Engine (i.e. V8)? It seems like the generated native code now has to check for every single write to the object if a change notification must be generated. It is not possible to statically determine if a given object has notifications set up or not. So the checks cannot be optimized out.
It seems like any conforming JavaScript engine is now locked in to a permanent and severe loss in performance due to this API.
Modern JavaScript engines utilize inline caching and adaptive recompilation techniques to minimize impact of the dynamic dispatch on the generated code.
If we are speaking about V8 then the fact whether object is observed or not is encoded in its hidden class. Both inline caches stubs and optimized code already check hidden class against some expected value to determine whether an object has an expected shape or not. The very same check gives information about the fact whether the object is observed or not. So nothing changes on the code paths that work with non-observed objects. Starting to observe the object is treated the same way as changing it shape: object's hidden class is switched to a different one, with an observed bit set: you can read Runtime_SetIsObserved to see this.
Similar reasoning applies to the parts of the system that omit guards in the optimized code and instead deoptimize code dependant on "shape" assumptions: once an object becomes observed all optimized code depending on the assumption that such object was not observed will be deoptimized. Thus again no price is paid for unobserved objects.
That said, current implementation of Object.observe in V8 makes observed objects pay a high price because it normalizes them (turns them into dictionary representation) and requires round trips through runtime system for observation recording. But there are no inherent technical difficulties in significantly reducing this cost later.
Doesn't this severely affect the code generation and performance optimizations that can be performed by the JavaScript Engine (i.e. V8)?
Yes. Just the same as Proxies, Getters/Setters and maybe even prototype objects - all of them are dynamic in JavaScript.
However, due to their asynchronity new (and better) optimisations will be possible; and they could make other, more inefficient code obsolete. Citing the Goals from the harmony draft:
No wrapper or proxy objects needed, providing memory efficiency and object identity
Change notifications on add/delete of a property on an object
Change notifications on modifications to property descriptor of properties on an object
The ability for an object to manually indicate when an accessor property has changed
Efficiently implementable in engines
Simple, targeted, extension to current ES
Asynchronous notification of changes, but allow synchronous fetching of changes pending delivery
I'm building one of my first web apps using HTML5, specifically targeting iPhones.
Since I'm pretty new to this, I'm trying to develop some good coding habits, follow best practices, optimize performance, and minimize the load on the resource-constrained iPhone.
One of the things I need to do frequently... I have numerous divs (each of which has a unique id) that I'm frequently updating (e.g., with innerHTML), or modifying (e.g., style attributes with webkit transitions and transforms).
In general - am I better off using getElementByID each time I need a handle to a div, or should I store references to each div I access in "global" variables at the start?
(I use "global" in quotes because I've really just got one truly global variable - it's an object that stores all my "global" variables as properties).
I assume using getElementByID each time must have some overhead, since the function needs to traverse the DOM to find the div. But, I'm not sure how taxing or efficient this function is.
Using global variables to store handles to each element must consume some memory, but I don't know if these references require just a trivial amount of RAM, or more than that.
So - which is better? Or, do both options consume such a trivial amount of resources that I should just worry about which produces more readable, maintainable code?
Many thanks in advance!
"In general - am I better off using getElementByID each time I need a handle to a div, or should I store references to each div"
When you're calling getElementById, you're asking it to perform a task. If you don't expect a different result when calling the same method with the same argument, then it would seem to make sense to cache the result.
"I assume using getElementByID each time must have some overhead, since the function needs to traverse the DOM to find the div. But, I'm not sure how taxing or efficient this function is."
In modern browsers especially, it's very fast, but not as fast as looking up a property on your global object.
"Using global variables to store handles to each element must consume some memory, but I don't know if these references require just a trivial amount of RAM, or more than that."
Trivial. It's just a pointer to an object that already exists. If you remove the element from the DOM with no intention to use it again, then of course you'll want to release your hold on it.
"So - which is better? Or, do both options consume such a trivial amount of resources that I should just worry about which produces more readable, maintainable code?"
Depends entirely on the situation. If you're only fetching it a couple times, then you may not find it worthwhile to add to your global object. The more you need to fetch the same element, the more sense it makes to cache it.
Here's a jsPerf test to compare. Of course size of your DOM as well as length of variable scope traversal and the number/depth of properties in your global object will play some role.
Using a local variable or even an object property is much faster than getElementById(). However, both are so fast that their performance is generally irrelevant compared to any other operation you might do once you have the element. Event setting a single property on the element is orders of magnitude slower than retrieving it by either method.
So the main reason to cache an element is to avoid the rather long-winded document.getElementById(... syntax, or to avoid having element ID strings scattered all over your code.