The new JavaScript engine takes advantage of multiple CPU cores through Windows to interpret, compile, and run code in parallel. - http://technet.microsoft.com/en-us/library/gg699435.aspx
and
The Chakra engine interprets, compiles, and executes code in parallel and takes advantage of multiple CPU cores, when available. - http://msdn.microsoft.com/en-us/ie/ff468705.aspx
Wait, what?!? Does this mean we've got multi-threaded parallel JavaScript code execution (outside of web-workers) in IE9?
I'm thinking this is just a bad marketing gimmick but would like to see some more info on this. Maybe they mean different browser windows/tabs/processes can utilize multiple CPUs?
Conclusions, based largely on the comments and hence provided as a community wiki answer so that this question ends up with an actual answer:
It's likely that Microsoft mean that the separate tasks of (i) interpreting and/or running; and (ii) compiling occur in parallel. It's probable that they've applied technology like Sun's old HotSpot JVM so that the Javascript virtual machine interprets code at the first instance because it can start doing that instantly. It also JIT compiles any code that appears to be used sufficiently frequently for doing so to be a benefit. It may even have different levels of compiler optimisation that it slowly dials up. In that case it may be using multiple cores to interpret or run one fragment of code while also compiling arbitrarily many others, or even while recompiling and better optimising the same piece of code that is being run.
However, it's also possible on a technical level that you could perform static analysis to determine when callbacks are mutually independent in terms of state, and allow those callbacks to execute in parallel if the triggering events prompted them to do so. In that way a Javascript virtual machine could actually interpret/run code in parallel without affecting the semantically serial nature of the language. Such a system would be logically similar to the operation of superscalar CPUs, albeit at a much greater remove and with significantly greater complexity.
Related
I keep reading about JavaScript implementations being single threaded, so I was wondering if it means that it takes a single logical thread of a CPU? or are software threads completely different from hardware threads?
(V8 developer here.)
JavaScript implementations being single threaded
That's not quite correct. JavaScript as a language is single-threaded. That means it doesn't give you, the programmer, a way to say "please start a new thread running this code here in parallel to my other code". Only one of your functions will execute at a time. A new function (e.g., a scheduled callback) can only start executing when the currently executing function has returned.
(Web workers are not a contradiction to this. They do execute concurrently with your main code, but that's not a JavaScript language feature. Instead, the browser environment gives you a way to spawn an independent second instance of a single-threaded JavaScript execution environment.)
JavaScript implementations, i.e. engines, are free to use as many threads as they want for background tasks. For example, when you use asynchronous DOM features (like the fetch API), then typically another thread takes care of doing the work (in this case, the fetching) in the background (typically on another CPU core). Once the result is available, a callback is scheduled, which has to wait until the single main thread is free to execute it. For V8 specifically, I can tell you that it also uses background threads for parsing and compiling JavaScript code, as well as for garbage collection. (Other engines probably do that too, but I don't know them as well.)
are software threads completely different from hardware threads?
Well, a software thread is a "thread"/strand/sequence of execution that wants to run, and a "hardware thread" is the hardware's ability to execute it. Personally, I think "hardware thread" is a confusing misnomer, and it would make more sense to call it "(logical) CPU core" instead; at any rate it amounts to the same thing.
So yes, a single-threaded program will run on one hardware thread (or not run at all). JavaScript programs themselves are single-threaded (this is defined by the language), but engines running the program typically use several threads (i.e., several software threads running on several "hardware threads"/CPU cores).
A software thread, at any given time, is executing at most in only a single CPU core. I have included the "at most" because a software thread can be in a bunch of states other than execution in which it isn't running (it can be waiting, sleeping, dead...).
You have no guarantees that a software thread executes always on the same CPU core. Although, for performance reasons (mostly related to caches), your operating system thread scheduler will probably try to execute a thread always on the same core.
Hardware threads (or logical cores, because the operating system sees them as if they were real hardware cores) are a feature of some processors that allow multiplexing the execution of several threads on the same core under some condition. For example, you can execute two threads in a single core if you have twice the registers and both are using different parts of the core or waiting (e.g. one is adding numbers while the other is doing logical operations) at any instant of time.
In summary, your operating system is, probably, using the same core for your JavaScript engine most of the time it is executing, but you really shouldn't rely on this assumption for anything.
Not enough that JavaScript isn't multithreaded, apparently JavaScript doesn't even get its own but shares a thread with a load of other stuff. Even in most modern browsers JavaScript is typically in the same queue as painting, updating styles, and handling user actions.
Why is that?
From my experience an immensely improved user experience could be gained if JavaScript ran on its own thread, alone by JS not blocking UI rendering or the liberation of intricate or limited message queue optimization boilerplate (yes, also you, webworkers!) which the developer has to write themselves to keep the UI responsive all over the place when it really comes down to it.
I'm interested in understanding the motivation which governs such a seemingly unfortunate design decision, is there a convincing reason from a software architecture perspective?
User Actions Require Participation from JS Event Handlers
User actions can trigger Javascript events (clicks, focus events, key events, etc...) that participate and potentially influence the user action so clearly the single JS thread can't be executing while user actions are being processed because, if so, then the JS thread couldn't participate in the user actions because it is already doing something else. So, the browser doesn't process the default user actions until the JS thread is available to participate in that process.
Rendering
Rendering is more complicated. A typical DOM modification sequence goes like this: 1) DOM modified by JS, layout marked dirty, 2) JS thread finishes executing so the browser now knows that JS is done modifying the DOM, 3) Browser does layout to relayout changed DOM, 4) Browser paints screen as needed.
Step 2) is important here. If the browser did a new layout and screen painting after every single JS DOM modification, the whole process could be incredibly inefficient if the JS was actually going to make a bunch of DOM modifications. Plus, there would be thread synchronization issues because if you had JS modifying the DOM at the same time as the browser was trying to do a relayout and repaint, you'd have to synchronize that activity (e.g. block somebody so an operation could complete without the underlying data being changed by another thread).
FYI, there are some work-arounds that can be used to force a relayout or to force a repaint from within your JS code (not exactly what you were asking, but useful in some circumstances).
Multiple Threads Accessing DOM Really Complex
The DOM is essentially a big shared data structure. The browser constructs it when the page is parsed. Then loading scripts and various JS events have a chance to modify it.
If you suddenly had multiple JS threads with access to the DOM running concurrently, you'd have a really complicated problem. How would you synchronize access? You couldn't even write the most basic DOM operation that would involve finding a DOM object in the page and then modifying it because that wouldn't be an atomic operation. The DOM could get changed between the time you found the DOM object and when you made your modification. Instead, you'd probably have to acquire a lock on at least a sub-tree in the DOM preventing it from being changed by some other thread while you were manipulating or searching it. Then, after making the modifications, you'd have to release the lock and release any knowledge of the state of the DOM from your code (because as soon as you release the lock, some other thread could be changing it). And, if you didn't do things correctly, you could end up with deadlocks or all sorts of nasty bugs. In reality, you'd have to treat the DOM like a concurrent, multi-user datastore. This would be a significantly more complex programming model.
Avoid Complexity
There is one unifying theme among the "single threaded JS" design decision. Keep things simple. Don't require an understanding of a multiple-threaded environment and thread synchronization tools and debugging of multiple threads in order to write solid, reliable browser Javascript.
One reason browser Javascript is a successful platform is because it is very accessible to all levels of developers and it relatively easy to learn and to write solid code. While browser JS may get more advanced features over time (like we got with WebWorkers), you can be absolutely sure that these will be done in a way that simple things stay simple while more advanced things can be done by more advanced developers, but without breaking any of the things that keep things simple now.
FYI, I've written a multi-user web server application in node.js and I am constantly amazed at how much less complicated much of the server design is because of single threaded nature of nodejs Javascript. Yes, there are a few things that are more of a pain to write (learn promises for writing lots of async code), but wow the simplifying assumption that your JS code is never interrupted by another request drastically simplifies the design, testing and reduces the hard to find and fix bugs that concurrency design and coding is always fraught with.
Discussion
Certainly the first issue could be solved by allowing user action event handlers to run in their own thread so they could occur any time. But, then you immediately have multi-threaded Javascript and now require a whole new JS infrastructure for thread synchronization and whole new classes of bugs. The designers of browser Javascript have consistently decided not to open that box.
The Rendering issue could be improved if desired, but at a significant complication to the browser code. You'd have to invent some way to guess when the running JS code seems like it is no longer changing the DOM (perhaps some number of ms go by with no more changes) because you have to avoid doing a relayout and screen paint immediately on every DOM change. If the browser did that, some JS operations would become 100x slower than they are today (the 100x is a wild guess, but the point is they'd be a lot slower). And, you'd have to implement thread synchronization between layout, painting and JS DOM modifications which is doable, but complicated, a lot of work and a fertile ground for browser implementation bugs. And, you have to decide what to do when you're part-way through a relayout or repaint and the JS thread makes a DOM modification (none of the answers are great).
I'm new to JavaScript so forgive me for being a n00b.
When there's intensive calculation required, it more than likely involves loops that are recursive or otherwise. Sometimes this may mean having am recursive loop that runs four functions and maybe each of those functions walks the entire DOM tree, read positions and do some math for collision detection or whatever.
While the first function is walking the DOM tree, the next one will have to wait its for the first one to finish, and so forth. Instead of doing this, why not launch those loops-within-loops separately, outside the programs, and act on their calculations in another loop that runs slower because it isn't doing those calculations itself?
Retarded or clever?
Thanks in advance!
Long-term computations are exactly what Web Workers are for. What you describe is the common pattern of producer and/or consumer threads. While you could do this using Web Workers, the synchronization overhead would likely trump any gains even on highly parallel systems.
JavaScript is not the ideal language for computationally demanding applications. Also, processing power of web browser machines can vary wildly (think a low-end smartphone vs. a 16core workstation). Therefore, consider calculating complex stuff on the server and sending the result to the client to display.
For your everyday web application, you should take a single-threaded approach and analyze performance once it becomes a problem. Heck, why not ask for help about your performance problem here?
JavaScript was never meant to do perform such computationally intensive tasks, and even though this is changing, the fact remains that JavaScript is inherently single-threaded. The recent web workers technology provides a limited form of multi-threading but these worker threads can't access the DOM directly; they can only send/receive messages to the main thread which can then access it on their behalf.
Currently, the only way to have real parallel processing in JS is to use Web Workers, but it is only supported by very recent browsers. And if your program requires such a thing, it could mean that you are not using the right tools (for example, walking the DOM tree is generally done by using DOM selectors like querySelectorAll).
By now, most mainstream browsers have started integrating optimizing JIT compilers to their JavaScript interpreters/virtual machines. That's good for everyone. Now, I'd be hard-pressed to know exactly which optimizations they do perform and how to best take advantage of them. What are references on optimizations in each of the major JavaScript engines?
Background:
I'm working on a compiler that generates JavaScript from a higher-level & safer language (shameless plug: it's called OPA and it's very cool) and, given the size of applications I'm generating, I'd like my JavaScript code to be as fast and as memory-efficient as possible. I can handle high-level optimizations, but I need to know more about which runtime transformations are performed, so as to know which low-level code will produce best results.
One example, from the top of my mind: the language I'm compiling will soon integrate support for laziness. Do JIT engines behave well with lazy function definitions?
This article series discusses the optimisations of V8. In summary:
It generates native machine code - not bytecode (V8 Design Elements)
Precise garbage collection (Wikipedia)
Inline caching of called methods (Wikipedia)
Storing class transition information so that objects with the same properties are grouped together (V8 Design Elements)
The first two points might not help you very much in this situation. The third might show insight into getting things cached together. The last might help you create objects with same properties so they use the same hidden classes.
This blog post discusses some of the optimisations of SquirrelFish Extreme:
Bytecode optimizations
Polymorphic inline cache (like V8)
Context threaded JIT (introduction of native machine code generation, like V8)
Regular expression JIT
TraceMonkey is optimised via tracing. I don't know much about it but it looks like it detects the type of a variable in some "hot code" (code run in loops often) and creates optimised code based on what the type of that variable is. If the type of the variable changes, it must recompile the code - based off of this, I'd say you should stay away from changing the type of a variable within a loop.
I found an additional resource:
What does V8 do with that loop?
I was wondering, if there is any generalities (among all the javascript engines out there) in the cost related to execute a given instruction vs another.
For instance, eval() is slower than calling a function that already has been declared.
I would like to get a table with several instructions/function calls vs an absolute cost, maybe a cost per engine.
Does such a document exists?
There's a page here (by one of our illustrious hosts, no less) that gives a breakdown by browser and by general class of instruction:
http://www.codinghorror.com/blog/archives/001023.html
The above page links to a more detailed breakdown here:
http://www.codinghorror.com/blog/files/sunspider-09-benchmark-results.txt
Neither of those pages breaks down performance to the level of individual function calls or arithmetic operations or what have you. Still, there is quite a bit of potentially useful information.
There is also a link to the benchmark itself:
http://www2.webkit.org/perf/sunspider-0.9/sunspider.html
By viewing the benchmark source you can get a better idea of what specific function calls are being tested.
It also seems like it might be a simple matter to create your own custom version of the benchmark that collects the more specific data you are interested in. You could then run the modified benchmark on a variety of browsers, perhaps even taking advantage of a service such as browsershots.org to test a wide spectrum of browsers. Not sure how well that would work, but it might be fun to try....
It is of course possible that the same operation executed in the same browser might take significantly different amounts of time depending on the context in which it's being used, in ways that might not be immediately obvious. For example, I could imagine a Javascript engine spending more time optimizing code that is executed frequently, with the result that the code executed in a tight loop might run faster than identical code executed infrequently. Of course, that might not matter much in practice. Still, I imagine that a good table of the sort you are looking for might also summarize any such effects if they turned out to be important.