We Keep Coding

Dynamically loaded js, mutation observer and promises for nested js loading

I am working on a mechanism to dynamically load a set of js files that have some dependencies between them (e.g., my code after bootstrap after jquery). I defined a nested json type structure with the dependency relationships and then invoke a Promise based loadscript mechanism (thanks to some of the posts here) that makes async calls to load js files that are at the same level and invokes a recursive loadscript for js files that have dependencies between them.
The files are all correctly appended to HEAD in the order that i expect them to be. For the dependencies, i invoke loading a file for which there is a dependency in the onload handler of its predecessor. However, this does not seem to guarantee that the loaded js code is available to use (i.e., did not seem that the append had completed and the code been executed - e.g., var/const definitions were not available at the time of onload being invoked based on break points, etc... in debugger - i guess this seems correct).
So, added a mutationobserver that observed that the files were added to the DOM (e.g., childList change to HEAD). However, it appears that the mutation is invoked before the code is actually available for use.
Why do i think this? i put break points, console logs with timing info, etc.. at the point where the mutationObserver indicated that a change corresponding to the insertion of the newly loaded script had occurred (e.g., the observer firing because of the script added) but at that point, the code (e.g., some var definition) was not available in the globals in the debugger.
So, the question is not specific about my code but more about whether it is possible to know that the dynamically included js code has run and is available to use. For my own code i can handle this through other mechanisms but for third party js files i do not want to touch them.
So, more specifically, it does not seem that the actual appending of code to the DOM HEAD at the point detected by the mutationObserver actually means that the code has been executed and that it seems from other posts (e.g., Execution of dynamically loaded JS files or load and execute order of scripts ) that the actual execution of the code is somewhat asynchronous depending on what the browser is actually doing.
Any thoughts on the overall process understanding would be greatly appreciated.
Thanks,

Script File Executing After Inline Script With CDN or External Domain On HTML injection

I am having an issue with HTML injection into an already loaded DOM where the inline javascript is being loaded after the script file is downloaded. From what I know this should not be async and the inline script should execute after the script file. This works if the domain name is the same as the calling page, but using a CDN or even a subdomain does the same thing. Is there something I should do to rework how I am calling these? I swear this worked before as I had the CDN on for over a week but maybe I never caught this issue.
Console
Loading Inline Script
VM1400:3 Uncaught TypeError: Cannot read property 'init' of undefined(anonymous function)
app.members.event.js?v=204&_=1453644424985:5 Loading Script File
app.members.event.js?v=204&_=1453644424985:71 Finished Script File
Javascript
<script type="text/javascript" src="https://test.azureedge.net/Areas/Directors/scripts/app.members.event.js?v=204"></script>
<script type="text/javascript">
console.log('Loading Inline Script');
app.viewModel.members.event.init();
console.log('Finished Inline Script');

One way is to use jquery's getScript() function.
But preferably, you may use native javascript to load the script file and then run the inline script.
Maybe i have not understood the question clearly.
Edit:
This is a quote from the HTML5 spec regarding script elements.
If the element has a src content attribute, run these substeps:
Let src be the value of the element's src attribute.
If src is the empty string, queue a task to fire a simple event named
error at the element, and abort these steps.
Resolve src relative to the element.
If the previous step failed, queue a task to fire a simple event named
error at the element, and abort these steps.
Do a potentially CORS-enabled fetch of the resulting absolute URL,
with the mode being the current state of the element's crossorigin
content attribute, the origin being the origin of the script element's
Document, and the default origin behaviour set to taint.
The resource obtained in this fashion can be either CORS-same-origin
or CORS-cross-origin. This only affects how error reporting happens.
For performance reasons, user agents may start fetching the script (as
defined above) as soon as the src attribute is set, instead, in the
hope that the element will be inserted into the document (and that the
crossorigin attribute won't change value in the meantime). Either way,
once the element is inserted into the document, the load must have
started as described in this step. If the UA performs such
prefetching, but the element is never inserted in the document, or the
src attribute is dynamically changed, or the crossorigin attribute is
dynamically changed, then the user agent will not execute the script
so obtained, and the fetching process will have been effectively
wasted.
Then, the first of the following options that describes the situation
must be followed:
If the element has a src attribute, and the element has a defer
attribute, and the element has been flagged as "parser-inserted", and
the element does not have an async attribute The element must be added
to the end of the list of scripts that will execute when the document
has finished parsing associated with the Document of the parser that
created the element.
The task that the networking task source places on the task queue once
the fetching algorithm has completed must set the element's "ready to
be parser-executed" flag. The parser will handle executing the script.
If the element has a src attribute, and the element has been flagged
as "parser-inserted", and the element does not have an async attribute
The element is the pending parsing-blocking script of the Document of
the parser that created the element. (There can only be one such
script per Document at a time.)
The task that the networking task source places on the task queue once
the fetching algorithm has completed must set the element's "ready to
be parser-executed" flag. The parser will handle executing the script.
If the element does not have a src attribute, and the element has been
flagged as "parser-inserted", and either the parser that created the
script is an XML parser or it's an HTML parser whose script nesting
level is not greater than one, and the Document of the HTML parser or
XML parser that created the script element has a style sheet that is
blocking scripts The element is the pending parsing-blocking script of
the Document of the parser that created the element. (There can only
be one such script per Document at a time.)
Set the element's "ready to be parser-executed" flag. The parser will
handle executing the script.
If the element has a src attribute, does not have an async attribute,
and does not have the "force-async" flag set The element must be added
to the end of the list of scripts that will execute in order as soon
as possible associated with the Document of the script element at the
time the prepare a script algorithm started.
The task that the networking task source places on the task queue once
the fetching algorithm has completed must run the following steps:
If the element is not now the first element in the list of scripts
that will execute in order as soon as possible to which it was added
above, then mark the element as ready but abort these steps without
executing the script yet.
Execution: Execute the script block corresponding to the first script
element in this list of scripts that will execute in order as soon as
possible.
Remove the first element from this list of scripts that will execute
in order as soon as possible.
If this list of scripts that will execute in order as soon as possible
is still not empty and the first entry has already been marked as
ready, then jump back to the step labeled execution.
If the element has a src attribute The element must be added to the
set of scripts that will execute as soon as possible of the Document
of the script element at the time the prepare a script algorithm
started.
The task that the networking task source places on the task queue once
the fetching algorithm has completed must execute the script block and
then remove the element from the set of scripts that will execute as
soon as possible.
Otherwise The user agent must immediately execute the script block,
even if other scripts are already executing. Fetching an external
script must delay the load event of the element's document until the
task that is queued by the networking task source once the resource
has been fetched (defined above) has been run.
From this I think that your "external" file is loaded after the inline script block. I would therefore use the "getScript()" function from jquery to make sure that the script is loaded before the inline script block.

This is a common problem in the injection scenerio. It occurs due to the variable delays in script availability, as well as due to the parallel and differing implementations on various browsers.
There are 3 options, depending on whether the source code is available for editing or not, and if more than 2 dependencies exist between the script files.
Option 1. Using defer attribute in the script tag
This option can be used, if both scripts are remote (ie, not inline)
"defer" indicates to the browser that the script has to execute after the document has been parsed (quoted from MDN). This is applicable only for remote (not inline) scripts, that have the "src" attribute.
https://html.spec.whatwg.org/multipage/scripting.html#attr-script-defer
You can use it like below.
Defer is supported on major browsers, and I validated on Chrome, Firefox, Webkit on Tizen, and Safari:
https://developer.mozilla.org/en/docs/Web/HTML/Element/script#Browser_compatibility
To provide concrete example of the above cases, refer below. Note that the below have been validated on Firefox, Chrome, IE11, Safari on iPhone, and Webkit on Tizen.
Case 1:
Many Javascript files - All independent:
If there is no dependency, the "defer" attribute allows the HTML to be loaded quickly. The script takes over after downloading, without issues (assuming onload etc are taken care).
Case 2:
Two javascript files test1.js and test2.js - One dependent on the other:
If test2.js is dependent on the loading of test1.js, then the script tag for test2.js "only" should have the defer attribute.
This usage is shown in
http://www.gpupowered.org/loadtest/2_defer.html
Incorrect usage is shown in
http://www.gpupowered.org/loadtest/no_defer.html (Both scripts do not have defer tag - this fails)
http://www.gpupowered.org/loadtest/all_defer.html (Both scripts having defer tag - this also fails)
Async usage that does not work is at,
http://gpupowered.org/loadtest/2_async.html (this fails)
Where does "defer" not meet the needs ?
If the functionality is split across several JS files (say n), and all "n-1" need to be downloaded before "n"th file can start processing some variables, even though the "defer" attribute might be present on all the script source tags, it is rendered irrelevant because the order in which they are received are indeterminate.
More background on defer and the various options for deferred loading (doesnot cover the multi defer case)
http://www.html5rocks.com/en/tutorials/speed/script-loading/
https://developer.mozilla.org/en/docs/Web/HTML/Element/script
Option 2: Using state variables
This option can be used if some additional state variables can be added to both javascript source files.
The approach relies on a named variable in the dependent js file, and a named function in the js file that uses the dependent file. If the dependent file is not loaded at the time the user file is trying to access its functionality, it exits and will be called back when it is really loaded.
This is demonstrated in the below html file.
http://gpupowered.org/loadtest/variable.html (works correctly)
This option does not work if the loading needs to happen repetitively (ie, loading of multiple files of same name etc).
Option 3: Native script loader
In this case, there are multiple javascript files having dependencies between each other.
There is no solution for this case using defer or async or other specification provided tags. For the use-case I had in the remote labs in gpupowered.org, I had to implement my own native script loader using XMLHttpRequest, and the source for this is provided in the link below. This uses worker threads as some of the textures I have are fairly big. The callback function can be used to implement the dependency logic as per the application needs. For example, keep count of all loaded scripts and then trigger full execution etc.
https://github.com/prabindh/gpupowered.gl/blob/master/worker/worker_object_loader.js
The jquery script loader uses the HTTP request as well, though I have not checked if it uses a worker for loading.
https://api.jquery.com/jquery.getscript/

I have two theories:
Might be that there's something in the external script that's delaying the creation of the app.viewModel.members object (either a timeout or an event handler that takes a while to fulfil). This can be easily tested by setting a long timeout in your inline script (f.i. 5000+ms) and then checking if the models object exists.
There's something funky going on when loading a same origin script.
In this case you could try is delay the execution of your inline script by doing the following:
<script type="text/javascript">
document.addEventListener("DOMContentLoaded", function(event) {
app.viewModel.members.event.init();
});
</script>
or just put your inline code in an external .js file and call it with the 'deferred' flag:
<script type="text/javascript" src="https://test.azureedge.net/Areas/Directors/scripts/app.members.event.js?v=204"></script>
<script type="text/javascript" src="{link-to-external-js-file}" defer></script>

function onloadCallback(){
app.viewModel.members.event.init();
}

Use this:
<script type="text/javascript" src="https://test.azureedge.net/Areas/Directors/scripts/app.members.event.js?v=204&onload=onloadCallback"></script
<script type="text/javascript">function onloadCallback(){
app.viewModel.members.event.init();}</script>

What does "async" do when added in a javascript script tag? [duplicate]

There are so many different ways to include JavaScript in a html page. I know about the following options:
inline code or loaded from external URI
included in <head> or <body> tag [1,2]
having none, defer or async attribute (only external scripts)
included in static source or added dynamically by other scripts (at different parse states, with different methods)
Not counting browserscripts from the harddisk, javascript:URIs and onEvent-attributes [3], there are already 16 alternatives to get JS executed and I'm sure I forgot something.
I'm not so concerned with fast (parallel) loading, I'm more curious about the execution order (which may depend on loading order and document order). Is there a good (cross-browser) reference that covers really all cases? E.g. http://www.websiteoptimization.com/speed/tweak/defer/ only deals with 6 of them, and tests mostly old browsers.
As I fear there's not, here is my specific question: I've got some (external) head scripts for initialisation and script loading. Then I've got two static, inline scripts in the end of the body. The first one lets the script loader dynamically append another script element (referencing external js) to the body. The second of the static, inline scripts wants to use js from the added, external script. Can it rely on the other having been executed (and why :-)?

If you aren't dynamically loading scripts or marking them as defer or async, then scripts are loaded in the order encountered in the page. It doesn't matter whether it's an external script or an inline script - they are executed in the order they are encountered in the page. Inline scripts that come after external scripts are held until all external scripts that came before them have loaded and run.
Async scripts (regardless of how they are specified as async) load and run in an unpredictable order. The browser loads them in parallel and it is free to run them in whatever order it wants.
There is no predictable order among multiple async things. If one needed a predictable order, then it would have to be coded in by registering for load notifications from the async scripts and manually sequencing javascript calls when the appropriate things are loaded.
When a script tag is inserted dynamically, how the execution order behaves will depend upon the browser. You can see how Firefox behaves in this reference article. In a nutshell, the newer versions of Firefox default a dynamically added script tag to async unless the script tag has been set otherwise.
A script tag with async may be run as soon as it is loaded. In fact, the browser may pause the parser from whatever else it was doing and run that script. So, it really can run at almost any time. If the script was cached, it might run almost immediately. If the script takes awhile to load, it might run after the parser is done. The one thing to remember with async is that it can run anytime and that time is not predictable.
A script tag with defer waits until the entire parser is done and then runs all scripts marked with defer in the order they were encountered. This allows you to mark several scripts that depend upon one another as defer. They will all get postponed until after the document parser is done, but they will execute in the order they were encountered preserving their dependencies. I think of defer like the scripts are dropped into a queue that will be processed after the parser is done. Technically, the browser may be downloading the scripts in the background at any time, but they won't execute or block the parser until after the parser is done parsing the page and parsing and running any inline scripts that are not marked defer or async.
Here's a quote from that article:
script-inserted scripts execute asynchronously in IE and WebKit, but
synchronously in Opera and pre-4.0 Firefox.
The relevant part of the HTML5 spec (for newer compliant browsers) is here. There is a lot written in there about async behavior. Obviously, this spec doesn't apply to older browsers (or mal-conforming browsers) whose behavior you would probably have to test to determine.
A quote from the HTML5 spec:
Then, the first of the following options that describes the situation
must be followed:
If the element has a src attribute, and the element has a defer
attribute, and the element has been flagged as "parser-inserted", and
the element does not have an async attribute
The element must be added
to the end of the list of scripts that will execute when the document
has finished parsing associated with the Document of the parser that
created the element.
The task that the networking task source places on the task queue once
the fetching algorithm has completed must set the element's "ready to
be parser-executed" flag. The parser will handle executing the script.
If the element has a src attribute, and the element has been flagged
as "parser-inserted", and the element does not have an async attribute
The element is the pending parsing-blocking script of the Document of
the parser that created the element. (There can only be one such
script per Document at a time.)
The task that the networking task source places on the task queue once
the fetching algorithm has completed must set the element's "ready to
be parser-executed" flag. The parser will handle executing the script.
If the element does not have a src attribute, and the element has been
flagged as "parser-inserted", and the Document of the HTML parser or
XML parser that created the script element has a style sheet that is
blocking scripts The element is the pending parsing-blocking script of
the Document of the parser that created the element. (There can only
be one such script per Document at a time.)
Set the element's "ready to be parser-executed" flag. The parser will
handle executing the script.
If the element has a src attribute, does not have an async attribute,
and does not have the "force-async" flag set The element must be added
to the end of the list of scripts that will execute in order as soon
as possible associated with the Document of the script element at the
time the prepare a script algorithm started.
The task that the networking task source places on the task queue once
the fetching algorithm has completed must run the following steps:
If the element is not now the first element in the list of scripts
that will execute in order as soon as possible to which it was added
above, then mark the element as ready but abort these steps without
executing the script yet.
Execution: Execute the script block corresponding to the first script
element in this list of scripts that will execute in order as soon as
possible.
Remove the first element from this list of scripts that will execute
in order as soon as possible.
If this list of scripts that will execute in order as soon as possible
is still not empty and the first entry has already been marked as
ready, then jump back to the step labeled execution.
If the element has a src attribute The element must be added to the
set of scripts that will execute as soon as possible of the Document
of the script element at the time the prepare a script algorithm
started.
The task that the networking task source places on the task queue once
the fetching algorithm has completed must execute the script block and
then remove the element from the set of scripts that will execute as
soon as possible.
Otherwise The user agent must immediately execute the script block,
even if other scripts are already executing.
What about Javascript module scripts, type="module"?
Javascript now has support for module loading with syntax like this:
<script type="module">
import {addTextToBody} from './utils.mjs';
addTextToBody('Modules are pretty cool.');
</script>
Or, with src attribute:
<script type="module" src="http://somedomain.com/somescript.mjs">
</script>
All scripts with type="module" are automatically given the defer attribute. This downloads them in parallel (if not inline) with other loading of the page and then runs them in order, but after the parser is done.
Module scripts can also be given the async attribute which will run inline module scripts as soon as possible, not waiting until the parser is done and not waiting to run the async script in any particular order relative to other scripts.
There's a pretty useful timeline chart that shows fetch and execution of different combinations of scripts, including module scripts here in this article: Javascript Module Loading.

A great summary by #addyosmani
Shamelessly copied from https://addyosmani.com/blog/script-priorities/

The browser will execute the scripts in the order it finds them. If you call an external script, it will block the page until the script has been loaded and executed.
To test this fact:
// file: test.php
sleep(10);
die("alert('Done!');");
// HTML file:
<script type="text/javascript" src="test.php"></script>
Dynamically added scripts are executed as soon as they are appended to the document.
To test this fact:
<!DOCTYPE HTML>
<html>
<head>
<title>Test</title>
</head>
<body>
<script type="text/javascript">
var s = document.createElement('script');
s.type = "text/javascript";
s.src = "link.js"; // file contains alert("hello!");
document.body.appendChild(s);
alert("appended");
</script>
<script type="text/javascript">
alert("final");
</script>
</body>
</html>
Order of alerts is "appended" -> "hello!" -> "final"
If in a script you attempt to access an element that hasn't been reached yet (example: <script>do something with #blah</script><div id="blah"></div>) then you will get an error.
Overall, yes you can include external scripts and then access their functions and variables, but only if you exit the current <script> tag and start a new one.

After testing many options I've found that the following simple solution is loading the dynamically loaded scripts in the order in which they are added in all modern browsers
loadScripts(sources) {
sources.forEach(src => {
var script = document.createElement('script');
script.src = src;
script.async = false; //<-- the important part
document.body.appendChild( script ); //<-- make sure to append to body instead of head
});
}
loadScripts(['/scr/script1.js','src/script2.js'])

I had trouble understanding how to get an embedded module-script to execute before the onload event happens. The answers above helped a lot but let me add a partial answer about what fixed my particular problem of misunderstanding the "Load and execute order of scripts".
I first used ... which caused an odd problem that it worked when loading the page normally, but not when running it in debugger on FireFox. That made debugging very difficult.
Note: Scripts whose type is "module" always have an implicit "deferred" attribute which means they don't stop the parsing of html, which means the onload-event can happen before the script gets executed. I did not want that. But I did want to use type="module" to make my un-exported JavaScript functions and variables invisible to other scripts on the same page.
I tried different options but thanks to the above answers I gained the insight that if you add the async -attribute to a script of type module it means that the script loads asynchronously BUT once it is loaded it executes immediately.
But in my case this was a script embedded in an HTML page. THEREFORE it meant nothing needed to load "asynchronously". It was already loaded with the page, since it was embedded in it. Therefore it with this change did get executed immediately -- which is what I wanted.
So I think it is worthwhile to point out this specific case because it is somewhat counter-intuitive: To get an embedded script executed IMMEDIATELY you must add the ASYNC attribute to its tag.
Ordinarily one might think that "async" means something happens asynchronously, in indeterminate order, not immediately. But the thing to realize is that "async" causes asynchronous LOADING, but immediate EXECUTION after the loading is complete. And when the script is embedded, no loading needs to be done, and therefore you get immediate execution.
Summary: Use
<script type="module" async> ... </script>
to get a module-script embedded to an HTML-page to execute immediately.

Perfect match for your query!
If none of the solutions worked for you then please refer to the below solution of mine which I have developed from my side.
I was also looking for the solution but after searching a lot, I summarized my code as below which is working perfectly for me!!
This is useful when you want functionality such that after previous script has fully loaded then and then only load next script!
just create a file named jsLoadScripts.js and insert it into the head or at the bottom of the body.
//From Shree Aum Software Solutions
//aumsoftwaresolutions#gmail.com
//script incrementor for array
scriptIncrementor = 0;
//define your script urls here
let scripts = [
"https://ajax.googleapis.com/ajax/libs/jquery/3.5.1/jquery.min.js",
"jsGlobalFunctions.js",
"jsDateParser.js",
"jsErrorLogger.js",
"jsGlobalVariables.js",
"jsAjaxCalls.js",
"jsFieldsValidator.js",
"jsTableClickEvent.js",
"index.js",
"jsOnDocumentReady.js",
];
//it starts with the first script and then adds event listener to it. which will load another script on load of it. then this chain goes on and on by adding dynamic event listeners to the next scripts!
function initializeScripts() {
var script = document.createElement("script");
script.type = "text/javascript";
script.src = scripts[scriptIncrementor];
document.head.appendChild(script);
script.addEventListener("load", function () {
loadNextScript();
scriptIncrementor++;
});
}
// this function adds event listener to the scripts passed to it and does not allow next script load until previous one has been loaded!
function loadNextScript() {
if (scriptIncrementor != scripts.length - 1) {
var script = document.createElement("script");
script.type = "text/javascript";
script.src = scripts[scriptIncrementor + 1];
document.head.appendChild(script);
script.addEventListener("load", function () {
loadNextScript();
scriptIncrementor++;
});
}
}
// start fetching your scripts
window.onload = function () {
initializeScripts();
};
This may cause you some speed related issues so, you can call function initializeScripts() with your custom needs!

catch every script load-complete event with javascript or jQuery

I need to know when a script is loaded and it's code is executed so that I can make sure the variables are not undefined. I searched and did not find any question like this one... again I need loaded-executed event not just download-complete.
and I will use 'id' on each script so I can find which is loaded.
<script src="utils.js" id="js_utils"></script>

If you're not loading scripts dynamically, you can rely on their order of running (which would be the order in which they appear in the document), see here:
load and execute order of scripts
Otherwise, you can either:
Add a line of code to the end of each script file loaded dynamically that runs your block of code.
or
If you don't have the ability to edit those files, start an interval as soon as you inject them into the code, tracking the object you want set before running your block of code.

Is there any way, other than to use eval/handleAs: "javascript" to dynamically fetch scripts (via XMLHttpRequest)

Here I have asked a question pertaining to exceptions raised when dynamically loading scripts via XMLHttpRequest (in other words when executed via eval)
In a related question, I wanted to know whether loading scripts dynamically as such is considered bad practice to begin with. In my particular case I have an HTML Canvas element, and rather than load all possible shapes, I want to fetch them dynamically, without reloading the page, and execute them on return. The problem I am having there is that if the code associated with that shape is incorrect, the error message displayed is not very useful (indicates location of eval statement, not incorrect statement). Is there another way to dynamically fetch code from the server and execute it, while better informing the location of the exception when it occurs.

If you want to load a script use a <script> element. If you want to dynamically load a script, create the <script> element dynamically.
var script = document.createElement('SCRIPT');
script.src = "<url to load>";
document.getElementsByTagName("HEAD")[0].appendChild(script);
It's not guaranteed to be synchronous the way eval with synchronous XHR is, but ideally you'd structure your code to take advantage of asynchony.

Adding to Mike's answer, if you want good debugger support including a script tag is probably the way to go, since that is what debuggers are used to working on. The main differences from eval that you need to be aware of are:
Eval runs in the scope of where its called while included script tags run in the global scope.
Eval is synchronous while included tags run asynchronously. (You will need to use something like JSONP and dojo.io.script if you need to run code after the script tag finishes).
If the scripts are fixed you can also consider debugging them by including their script tags and deploying them as you see fit.