Is 'require' synchronous in AMD (asynchronous module definition)? If so, what makes this specification asynchronous? What if I have require() (and it hasn't been loaded yet) in the middle of my code, will it stall execution? Talking browser-side.
There are two different synchronous concepts here.
The first is "Will it stop my entire webpage, and sit and wait for the file.".
The answer is no. RequireJS doesn't do that if you've got a script with dependencies.
If you use it appropriately, it uses a promise-system.
What that means is that if you send in your callback and define your requirements for that file, the callback won't be run until all of the required files are loaded.
If there's a require inside of one of those required files, then THAT callback won't be run until ITS dependencies have loaded.
The outermost callback (the one that would be at the bottom of your script, normally), won't run until everything inside has.
This works on a promise system.
It's worth understanding how promise systems work (similar to an observer-pattern, in a way).
They're meant to be passed around or chained, based on an event, rather than having multiple people listen in any order.
var widget = new Widget(),
widgetLoaded = widget.load(url); // return a promise to let the program use the widget
widgetLoaded.then(function () { widget.move(35); })
.then(function () { widget.setColour("Blue"); })
.then(function () { widget.show(); });
This is like returning this so that you can chain function calls, except that the calls don't actually happen until widget.load() completes.
The widget will actually control when this happens, by keeping its promise if the widget loads and everything is fine, or by breaking its promise if something went wrong.
In most promise systems, .then or whatever they call it, either takes two functions (kept and broken -- in my systems, brokens are always optional), or they take an object with success and failure -- $.ajax does this, and then lets you predetermine what you want to do with the data when it's loaded, or if it fails -- promises.
So your page still work 100% asynchronously (without interrupting the UI), but it's 100% synchronous in that all of the modules will fire in the right order.
One thing you MUST REMEMBER:
If you have these dependencies in your code, you can not have any dependencies lying around at the bottom of your script, waiting to run, inline.
They must all be locked away inside of your callback, or locked inside a function waiting to be called by your callback.
This is simply because it is an asynchronous process, in terms of actual processing, and will not block the browser from running events/JS, rendering the page, et cetera.
For requireJS:
You have to pass a callback method alongside the required modules to .require(), that will get fired when the resources were loaded successfully. So, of course you should/can only access loaded AMD or CommonJS modules just within that callback.
for NodeJS:
Yes, .require() does work synchronously. NodeJS uses the CommonJS module system, not AMD.
Related
I'm a NodeJs developer that happens to have some web applications.
I'm very familiar with asynchronous code and how the event loop works. However I been facing a problem that I was unable to fix until I realized that asynchronous code may behave different when it is splitted across several script tags.
The situation was as follows
I had one script tag at the head section with some asynchronous code. Such code expected some function to exist. That function should be declared on the second script tag in a synchronous fashion, so at the time the async stuff completes that function should exist.
Te second script tag is 100% synchronous and at the middle of the code it created a function called boot.
Let me ilustrate it with some simplified code:
// <script>
somePromise()
.then( ()=> window.boot())
// </script>
// < ... some html code and body >
// <script>
window.boot = function (){}
// </script>
Since the promise callback should be executed asynchronously I expected it to go to the event loop and allow the rest of the synchronous code to execute.
On some browsers, this worked as I expect. When I say some browsers I mean browsers of different users on different computers, because the behavior varies even using the same browser brand and version. However, there were situations when the promise callback was executed before the second second script tag had a chance to start and raising an error.
How is this supposed to work on browsers ? Is each script tag executed until all its code is executed, even the asynchronous one ? If that is the case, why does it work for some other browsers?
Thanks in advance.
The HTML5 parser is not synchronous.
This is my own reworded extract of parts of HTML Parser Threading on MDN. Please refer to the original for formal treatment and heavy reading.
HTML between the script tag [pairs] is parsed by a nsHtml5StreamParser which executes off the main thread.
The parser runs its own event loop, and responds to requests from the main thread to perform onDataAvailable and OnStopRequest processing. Processing calls DoDataAvailable and doStopRequst respectively.
DoDataAvailable outputs small DOM tree operations, such as element creation into a queue processed in the main thread. The tree op queue is flushed to the main thread from time to time ** using a timer.**
After flushing the parser dispatches a runnable to the main thread to process tree additions. The runner (or executor) in the main thread calls nsHtml5TreeOpExecutor::RunFlushLoop().
A comment from #KarelG in this question says that network data is usually processed in 8Kb chunks. The question is well worth reading.
So the JavaScript Event Loop sometimes gets an opportunity to fulfill a promise and execute an onFulfilled handler before the second script element is parsed and executed - as you already discovered!
In summary it apppear that the vagaries of network retrieval of source code, asynchronous HTML parsing that uses a timer to instigate processing of tree building operations, and any further asynchronous operations that the tree builder may or may not invoke, all combine together to create a race condition.
The unpredictability of the failure branch of the race condition, when window.boot is not defined when called, is most likely due to the combined effects of browser brand, network speed, device utilization, length and type of HTML content, and HTML parser timing and timers.
The obvious conclusion is that code should not set up this kind of race condition in the first place. It is unsafe coding practice. Thankfully you can work around it.
Actually, promise is not suported in IE, so what you could do is to use some external libs, like bluebirdJS, or you could use a transpiler (babel or another one)
If you use plain script tags on an HTML page, rendering is blocked until the script has been downloaded and parsed. To avoid that, for faster page display, you can add the 'async' attribute, which tells the browser to continue processing down the page without waiting for that script. However, that inherently means that other javascript that refers to anything in that script will probably crash, because the objects it requires don't exist yet.
As far as I know, there's no allScriptsLoaded event you can tie into, so I'm looking for ways to simulate one.
I'm aware of the following strategies to defer running other code until an async script is available:
For a single script, use their 'onload' event or attribute. However, there's no built-in way I know of to tell when ALL scripts have loaded if there's more than one.
Run all dependent code in onload event handlers attached to the window. However, those wait for all images too, not just all scripts, so the run later than would be ideal.
Use a loader library to load all scripts; those typically provide for a callback to run when everything has loaded. Downside (besides needing a library to do this, which has to load early), is that all code has to wrapped in a (typically anonymous) function that you pass into the loader library. That's as opposed to just creating a function that runs when my mythical allScriptsLoaded fires.
Am I missing something, or is that the state of the art?
The best you could hope for would be to know if there are any outstanding async calls (XMLHttpRequest, setTimeout, setInterval, SetImmediate, process.nextTick, Promise), and wait for there to not be one. However, that is an implementation detail that is lost to the underlying native code--javascript only has its own event loop, and async calls are passed off to the native code, if I understand it correctly. On top of that, you don't have access to the event loop. You can only insert, you can't read or control flow (unless you're in io.js and feeling frisky).
The way to simulate one would be to track your script calls yourself, and call after all script are complete. (i.e., track every time you insert a relevant script into the event loop.)
But yeah, the DOM doesn't provide a NoAsyncPending global or something, which is what you'd really require.
I am working on a library that will expose 3 methods. All three of these methods depend on another library (lets call it libA) having been loaded and that library loads asynchronously.
I could write the code to expose the methods as soon as the JS file has finished loading but then the user would have to defer execution until libA has finished.
I was hoping to, rather, expose "a version" of those 3 methods immediately while libA continues to load asynchronously in the background. Any calls to those 3 methods would get queued up until libA is done loading. And after libA is done loading those 3 methods would get replaced with the real one and the queue would be processed.
Let's say I have this:
var myLib = (function(){
// expose 3 functions for the user to use
return {
"func1" : function(opts){},
"func2" : function(opts){},
"func3" : function(opts){}
}
})();
This library will be loaded by the user. At the same time libA will also be loaded. It may take longer to load then mine or it may finish before mine starts.
So if the user runs myLib.func1({}) and libA has not finished loading then it should get queued up and then when libA is done it should execute. If, on the other hand, libA has finished loading then it would execute immediately.
My initial thought is to do something like this:
var myLib = (function(){
// queue the calls
var queue {
"func1" : [],
"func2" : [],
"func3" : []
};
// the "temp" functions that just add them to the queue
var ret = {
"func1" : function(opts){ queue.func1.push(opts); },
"func2" : function(opts){ queue.func2.push(opts); },
"func3" : function(opts){ queue.func3.push(opts); }
}
// this may happen before the user users my library or after
// if it happens before then the functions will be replaced before the user calls them and they won't need to be queued
// if it happens after then the functions will get replaced and the queued up ones will be executed
waitForLibAToFinish(function(){
ret.funct1 = function(opts){alert("this is the real func1");},
ret.funct2 = function(opts){alert("this is the real func2");},
ret.funct3 = function(opts){alert("this is the real func3");},
// process the queue
for(var i = 0; i < queue.func1.length; ++i)
{
ret.func1(queue.func1[i]);
}
// repeat for func2 and func3 queue
});
return ret;
})();
But this just seems like a bad way to do it. Plus I'll have to have a queue for each function and call each one. There must be a way to abstract that part out so its more generic for all the methods my library exposes.
I'm open to any ideas/suggestions. I just cannot (for numerous reasons out of my control) use any 3rd party libraries like Promise or JavaScript. And, my code has to work on IE8. Trust me, I hate it more than you do.
===
I'm adding some more context/details.
I know I mention SharePoint but this is not an SP specific question.
I am building a library that offers convenience methods to simplify otherwise complicated tasks in SharePoint. All my library does is call SP libraries. The relevant SP libraries are loaded asynchronously. So one of two things needs to happen:
Either the user needs to wait for the relevant SP libraries to finish loading before calling any of the methods in my library
Or my library needs to wait for the relevant SP libraries to finish
The function SP offers to let you "queue" up your function until a relevant SP library is loaded is ExecuteOrDelayUntillScriptLoaded.
My library is currently built on the first option. I wanted to simplify the experience for the user so they don't have to check (by way of ExecuteOrDelayUntillScriptLoaded). I could change it to the 2nd way but there are reasons I don't want to.
Instead, what I was hoping to do is include 1 check in my library with ExecuteOrDelayUntillScriptLoaded that would "modify" what my library does. Before the relevant libraries are done loading my library would just queue up the calls. And after it is done loading and ExecuteOrDelayUntillScriptLoaded fires, my code would:
Execute everything in the queue
Modify each function my library exposes to directly execute and not queue
Don't do the queuing yourself. Leave the library loading and dependency resolution to the user. Just expose a function that instantiates your library, the user may call it when library A is ready and get the module with usable methods back.
There even already is a standard on how to do that: Asynchronous Module Definition. (You don't need require.js as a third-party library, you can implement the bare essentials yourself)
If the user wants (needs) to call your library methods before everything is loaded, they can always queue the premature invocations themselves in any manner they want. But usually this should not be necessary.
So I've been using require.js for while now, but I realized that I actually don't know how it works under the hood. It says that it's an AMD loader.
I do understand that CommonJS is synchronous, which means that it blocks execution of other codes while it's being loaded. On the other hand, AMD is asynchronous. This is where I get confused.
When I define a module, it has to load a,b,c in order to execute the callback. How does asynchronous work here?
Isn't it synchronous when it has to load those three dependencies first?
Does it mean that AMD loads a,b,c asynchronously then checks to see if those files are loaded (doesn't care about the order) then execute the callback?
define("name",["a","b","c"], function(a,b,c){
});
As you know, "AMD" (Asynchronous Module Definition (AMD)) is a specific API. There are many AMD-compatible "loaders", including RequireJS, curl.js and Dojo (among others).
Just as frameworks like JQuery and Dojo give you an API over raw Javascript; a program that uses AMD:
1) requires you an AMD-compatible .js library,
2) demands certain programming "rules" and "conventions", and
3) Ultimately sits "on top" of Javascript, which runs on your "Javascript engine" (be it IE, Chrome, Firefox - whatever).
Here are a couple of links I found useful:
https://www.ibm.com/developerworks/mydeveloperworks/blogs/94e7fded-7162-445e-8ceb-97a2140866a9/entry/loading_jquery_with_dojo_1_7_amd_loader2?lang=en
http://dojotoolkit.org/reference-guide/1.8/loader/amd.html
http://blog.millermedeiros.com/amd-is-better-for-the-web-than-commonjs-modules/
http://addyosmani.com/writing-modular-js/
PS:
To answer your immediate question, the latter link has a bit of discussion about "require()" and "dynamically_loaded dependencies".
Since I wrote an AMD loader, I'll try to answer the questions directly:
Isn't it synchronous when it has to load those three dependencies first?
Javascript, by definition, is single threaded. That means that anything you run in it always runs sequentially. The only thing that you can do in a browser is include scripts using the "async" parameter on the script tag, which will make the order in which scripts are being loaded undefined (asynchronous). Once a script executes it will be the only one executing at that point in time.
Does it mean that AMD loads a,b,c asynchronously then checks to see if those files are loaded (doesn't care about the order) then execute the callback?
Correct. AMD-define() allows you to load all scripts in any order you wish (i.e. ultimately you let the browser roll the dice and load them in any order it sees fit at any time it sees fit).
Then any time a define() is called, the AMD-loader will check if the current list of dependencies for this define has already been satisfied. If it is, it will call the current callback immediately, and after that, it will check if any of the previously registered define-callbacks can be called too (because all of their dependencies have been satisfied). If the dependencies for this callback have not all been satisfied yet, the callback is added to the queue to be resolved later.
This eventually results in all callbacks being called in the correct dependency order, regardless of the order in which the scripts have been loaded/executed in the first place.
I'm testing some javascript libraries I've built by using ruby's harmony. Everything is working perfectly except AJAX calls — does anyone have any ideas how to achieve this?
My code looks something like this (I'm using RightJS):
my.js
function makerequest(argument) {
new Xhr('http://mysite.com/some/jsonp'),{
jsonp: true,
params: {something: argument},
onSuccess: function() {
// Calls a function defined outside my library
parse_response(this.json)
}
}).send()
}
test_makerequest.rb
require 'rubygems'
require 'harmony'
require 'test/unit'
require 'shoulda'
class RequestTest < Test::Unit::TestCase
context "The requesty thing" do
setup do
#page = Harmony::Page.new
#page.load(File.expand_path('js/my.js'))
end
should "get stuff from mysite.com" do
# Here I need to define a 'parse_response' which this
# should section will wait for and then do an 'assert'
# on the results.
results = callback_to_get_results_from__make_request
#page.execute('make_request("an argument")')
assert results == {'foo' => 'bar'}
end
end
end
So yeah, my question is, how should I assign results above such that I can get the results of the async callback?
Async calls are generally problematic with JS unit testing. I've solved it in the past by making the XHR calls effectively synchronous by blocking the main thread until the XHR call succeeds. How to do this varies depending on your framework, and I'm unfamiliar with Harmony and RightJS. However, the logic would look something like this:
Override your XHR method so that it maintains a lock/mutex. The lock can be a simple boolean. When any XHR is in process, the lock will be true, when no XHR is in process the lock will be false.
In your test, before you run your assertion, add a loop that executes your XHR and then waits until the lock clears.
Once the loop finishes, you'll be sure the XHR has completed, and you can run your assertion.
The intent of this approach is to avoid modifying your code, which would invalidate your tests. Unfortunately, this also means you can't test the return value itself, but I think this will be a caveat of any strategy that doesn't modify your code.
Another approach would be to actually run your tests asynchronously--ie. attach your assertion to the onSuccess callback using AOP. However, many testing suites don't play nicely with asynchronous tests, since the setup of one test may be starting before the teardown of the previous test (ie. because the previous test is still waiting for the async call to return).
One last approach would be to mock all asynchronous calls. Ie. just override XHR and have your test assert that it was called with the right arguments. I might favor this approach if I have integration tests that do go through the whole async stack.