I've got this problem I have been working on and found some interesting behavior. Basically, if I benchmark the same code multiple times in a row, the code execution gets significantly faster.
Here's the code:
http://codepen.io/kirkouimet/pen/xOXLPv?editors=0010
Here's a screenshot from Chrome:
Anybody know what's going on?
I'm checking performance with:
var benchmarkStartTimeInMilliseconds = performance.now();
...
var benchmarkEndTimeInMilliseconds = performance.now() - benchmarkStartTimeInMilliseconds;
Chrome's V8 optimizing compiler initially compiles your code without optimizations. If a certain part of your code is executed very often (e.g. a function or a loop body), V8 will replace it with an optimized version (so called "on-stack replacement").
According to https://wingolog.org/archives/2011/06/08/what-does-v8-do-with-that-loop:
V8 always compiles JavaScript to native code. The first time V8 sees a
piece of code, it compiles it quickly but without optimizing it. The
initial unoptimized code is fully general, handling all of the various
cases that one might see, and also includes some type-feedback code,
recording what types are being seen at various points in the
procedure.
At startup, V8 spawns off a profiling thread. If it notices that a
particular unoptimized procedure is hot, it collects the recorded type
feedback data for that procedure and uses it to compile an optimized
version of the procedure. The old unoptimized code is then replaced
with the new optimized code, and the process continues
Other modern JS engines identify such hotspots and optimize them as well, in a similar fashion.
Related
I've got this problem I have been working on and found some interesting behavior. Basically, if I benchmark the same code multiple times in a row, the code execution gets significantly faster.
Here's the code:
http://codepen.io/kirkouimet/pen/xOXLPv?editors=0010
Here's a screenshot from Chrome:
Anybody know what's going on?
I'm checking performance with:
var benchmarkStartTimeInMilliseconds = performance.now();
...
var benchmarkEndTimeInMilliseconds = performance.now() - benchmarkStartTimeInMilliseconds;
Chrome's V8 optimizing compiler initially compiles your code without optimizations. If a certain part of your code is executed very often (e.g. a function or a loop body), V8 will replace it with an optimized version (so called "on-stack replacement").
According to https://wingolog.org/archives/2011/06/08/what-does-v8-do-with-that-loop:
V8 always compiles JavaScript to native code. The first time V8 sees a
piece of code, it compiles it quickly but without optimizing it. The
initial unoptimized code is fully general, handling all of the various
cases that one might see, and also includes some type-feedback code,
recording what types are being seen at various points in the
procedure.
At startup, V8 spawns off a profiling thread. If it notices that a
particular unoptimized procedure is hot, it collects the recorded type
feedback data for that procedure and uses it to compile an optimized
version of the procedure. The old unoptimized code is then replaced
with the new optimized code, and the process continues
Other modern JS engines identify such hotspots and optimize them as well, in a similar fashion.
how does the jit compiler work in JAVASCRIPT ???
when we only have a 1 compilation phase which declares all of our variables and functions (during the creation of the glbal execution context ?o
...when we only have a 1 compilation phase which declares all of our variables and functions...
That's where you're going wrong. :-) Modern JavaScript engines don't have a single compilation phase. Instead, they do an initial pass through the code using something really fast, and then for code that gets reused enough to make it worthwhile, they apply an optimizer to rewrite the code in place with faster code.
In Chrome's V8, the first phase used to be a compiler (called Full-codegen) and the second phase (when needed) was an optimizing compiler called Crankshaft, but they've switched to an interpreter called Ignition for the first phase that parses the code, generates bytecode, and executes that, and then for code that makes it worthwhile, they apply an optimizing compiler called TurboFan to the bytecode. (See that blog post for the details.) This was originally to minimize the memory impact of one-off setup code, but it turned out that generating bytecode was faster than generating machine code and startup performance was actually improved as well.
Executing this snippet in the Chrome console:
function foo() {
return typeof null === 'undefined';
}
for(var i = 0; i < 1000; i++) console.log(foo());
should print 1000 times false, but on some machines will print false for a number of iterations, then true for the rest.
Why is this happening? Is it just a bug?
There is a chromium bug open for this:
Issue 604033 - JIT compiler not preserving method behavior
So yes It's just a bug!
It's actually a V8 JavaScript engine (Wiki) bug.
This engine is used in Chromium, Maxthron, Android OS, Node.js etc.
Relatively simple bug description you can find in this Reddit topic:
Modern JavaScript engines compile JS code into optimized machine code
when it is executed (Just In Time compilation) to make it run faster.
However, the optimization step has some initial performance cost in
exchange for a long term speedup, so the engine dynamically decides
whether a method is worth it depending on how commonly it is used.
In this case there appears to be a bug only in the optimized path,
while the unoptimized path works fine. So at first the method works as
intended, but if it's called in a loop often enough at some point the
engine will decide to optimize it and replaces it with the buggy
version.
This bug seems to have been fixed in V8 itself (commit), aswell as in Chromium (bug report) and NodeJS (commit).
To answer the direct question of why it changes, the bug is in the "JIT" optimisation routine of the V8 JS engine used by Chrome. At first, the code is run exactly as written, but the more you run it, the more potential there is for the benefits of optimisation to outweigh the costs of analysis.
In this case, after repeated execution in the loop, the JIT compiler analyses the function, and replaces it with an optimised version. Unfortunately, the analysis makes an incorrect assumption, and the optimised version doesn't actually produce the correct result.
Specifically, Reddit user RainHappens suggests that it is an error in type propagation:
It also does some type propagation (as in what types a variable etc can be). There's a special "undetectable" type for when a variable is undefined or null. In this case the optimizer goes "null is undetectable, so it can be replaced with the "undefined" string for the comparison.
This is one of the hard problems with optimising code: how to guarantee that code which has been rearranged for performance will still have the same effect as the original.
This was fixed two month ago and will land in Chrome soon (already in Canary).
V8 Issue 1912553002 - Fix 'typeof null' canonicalization in crankshaft
Chromium Issue 604033 - JIT compiler not preserving method behavior
Executing this snippet in the Chrome console:
function foo() {
return typeof null === 'undefined';
}
for(var i = 0; i < 1000; i++) console.log(foo());
should print 1000 times false, but on some machines will print false for a number of iterations, then true for the rest.
Why is this happening? Is it just a bug?
There is a chromium bug open for this:
Issue 604033 - JIT compiler not preserving method behavior
So yes It's just a bug!
It's actually a V8 JavaScript engine (Wiki) bug.
This engine is used in Chromium, Maxthron, Android OS, Node.js etc.
Relatively simple bug description you can find in this Reddit topic:
Modern JavaScript engines compile JS code into optimized machine code
when it is executed (Just In Time compilation) to make it run faster.
However, the optimization step has some initial performance cost in
exchange for a long term speedup, so the engine dynamically decides
whether a method is worth it depending on how commonly it is used.
In this case there appears to be a bug only in the optimized path,
while the unoptimized path works fine. So at first the method works as
intended, but if it's called in a loop often enough at some point the
engine will decide to optimize it and replaces it with the buggy
version.
This bug seems to have been fixed in V8 itself (commit), aswell as in Chromium (bug report) and NodeJS (commit).
To answer the direct question of why it changes, the bug is in the "JIT" optimisation routine of the V8 JS engine used by Chrome. At first, the code is run exactly as written, but the more you run it, the more potential there is for the benefits of optimisation to outweigh the costs of analysis.
In this case, after repeated execution in the loop, the JIT compiler analyses the function, and replaces it with an optimised version. Unfortunately, the analysis makes an incorrect assumption, and the optimised version doesn't actually produce the correct result.
Specifically, Reddit user RainHappens suggests that it is an error in type propagation:
It also does some type propagation (as in what types a variable etc can be). There's a special "undetectable" type for when a variable is undefined or null. In this case the optimizer goes "null is undetectable, so it can be replaced with the "undefined" string for the comparison.
This is one of the hard problems with optimising code: how to guarantee that code which has been rearranged for performance will still have the same effect as the original.
This was fixed two month ago and will land in Chrome soon (already in Canary).
V8 Issue 1912553002 - Fix 'typeof null' canonicalization in crankshaft
Chromium Issue 604033 - JIT compiler not preserving method behavior
In the bluebird docs, they have this as an anti-pattern that stops optimization.. They call it argument leaking,
function leaksArguments2() {
var args = [].slice.call(arguments);
}
I do this all the time in Node.js. Is this really a problem. And, if so, why?
Assume only the latest version of Node.js.
Disclaimer: I am the author of the wiki page
It's a problem if the containing function is called a lot (being hot). Functions that leak arguments are not supported by the optimizing compiler (crankshaft).
Normally when a function is hot, it will be optimized. However if the function contains unsupported features like leaking arguments, being a hot function doesn't help and it will continue running slow generic code.
The performance of an optimized function compared to an unoptimized one is huge. For example consider a function that adds 3 doubles together: http://jsperf.com/213213213 21x difference.
What if it added 6 doubles together? 29x difference Generally the more code the function has, the more severe the punishment is for that function to run in unoptimized mode.
For node.js stuff like this in general is actually a huge problem due to the fact that any cpu time completely blocks the server. Just by optimizing the url parser that is included in node core (my module is 30x faster in node's own benchmarks), improves the requests per second of mysql-express from 70K rps to 100K rps in a benchmark that queries a database.
Good news is that node core is aware of this
Is this really a problem
For application code, no. For almost any module/library code, no. For a library such as bluebird that is intended to be used pervasively throughout an entire codebase, yes. If you did this in a very hot function in your application, then maybe yes.
I don't know the details but I trust the bluebird authors as credible that accessing arguments in the ways described in the docs causes v8 to refuse to optimize the function, and thus it's something that the bluebird authors consider worth using a build-time macro to get the optimized version.
Just keep in mind the latency numbers that gave rise to node in the first place. If your application does useful things like talking to a database or the filesystem, then I/O will be your bottleneck and optimizing/caching/parallelizing those will pay vastly higher dividends than v8-level in-memory micro-optimizations such as above.