We Keep Coding

How exactly are the function calls ordered in an asynchronous JavaScript program?

I am learning the concept of asynchronous programming in JavaScript (JS). But, I am having a hard time understanding the same. For the last few days, I had been reading various articles on the internet to understand it, but I am unable to grasp the idea.
So, here are the doubts I have:
setTimeout(function(){ alert("Hello 1"); }, 3000); // .....(i)
console.log("Hi!"); // .....(ii)
setTimeout(function(){ alert("Hello 2"); }, 2000); // .....(iii)
Consider the above code. I learnt that JS uses a call-stack and an event-queue to order the execution of instructions. In the above code, when the JS interpreter sees the (i) line, it will enqueue that setTimeout into the event-queue, then moves to (ii), puts it in the call-stack, executes it, then moves to (iii), where it again enqueues the setTimeout into the event-queue (and this queue is not empty), right?
If what I had written in the above question is correct, then once we get to the end of the code since the call-stack is empty the setTimeouts enqueued into the event-queue get executed one by one, right? - That means if we assume it took (say) 10ms to come to the end of the code, then since the event-queue has the setTimeout (i) in the front, it waits for 3s, then pops the alert: "Hello 1", at the time = 3010ms, the dequeues it, and similarly the setTimeout (iii) gets executed after 2 more seconds and then the alert: "Hello 2" pops at the time = 5010ms, right?
Let's suppose that instead of setTimeouts at (i) and (iii), we had addEventListener()'s with some call-back functions. Even in this case, will the call-back functions of the event listeners be enqueued in the event-queue? I feel they don't get enqueued because we could have triggered the call-back of (iii), before the call-back of (i). So, what exactly happens in this case? Is there anything else other than the call-stack and event-queue that somehow stores the information about them and triggers their call-backs accordingly?
In a nut-shell how exactly are the instructions ordered? What exactly happens in the background?
I would be really thankful for a comprehensive answer. It would be great if you can also provide links to some comprehensive materials on this topic.
Thank you for the help!

As you might be aware by now JavaScript engine executes on a single thread, so how are asynchronous operations handled? You are partially true in the below statement, but there is more to it :
Consider the above code. I learnt that JS uses a call-stack and an
event-queue to order the execution of instructions.
True, we do have a call stack and an event loop. But we also have a WEB APIs environment, Call-back Queue and a Micro-task Queue.
Whenever there is any asynchronous task, it moves to the WEB API Environment, for example, when you have an tag with a very large image in the "src" attribute, this image is not downloaded synchronously, because that would block the thread, instead it is moved into the WEB API Environment where the image is loaded.
<img src="largeimg.jpg">
Now, if you want to do something once the image is loaded, you will need to listen to the image's 'load' event.
document.querySelector('img').addEventListener('load', imgLoadCallback);
Now once the image has been loaded, this callback function is still not executed, instead now it is moved into the callback queue. The callback function waits in the callback queue, the event loop will check for synchronous code, and wait until the call stack is empty. Once the call stack is empty, the event loop will push in a first in callback function into the call stack in one event loop tick. And that is when that call back function is executed.
However, this changes when there are micro-tasks such as Promises. When there is a promise, it is sent to the microtask queue. Microtasks will always have priority over the callbacks and they can and will halt the callbacks until they are executed, event loop will always prioritize microtasks.
This is how the JavaScript Call Stack, Event Loop, Call Back Queue, Microtasks Queue and WEB API Environments work.
Now Run this below code, before running try to guess the outcome. It will be exactly as per what I have written above :
//Synchronous Code - Always prioritized over async code
console.log('Asynchronous TEST start');
//It is a 0 Second Timer, But a timer is not a microtask
setTimeout(() => console.log('0 sec timer'), 0);
//Promise is a microtask
Promise.resolve('Resolved promise 1').then(res => console.log(res));
//2nd promise is a microtask too
Promise.resolve('Resolved promise 2').then(res => {
for (let i = 0; i < 1000000000; i++) {} //very large loop
console.log(res);
});
//Synchronous Code - Always prioritized over async code
console.log('Test end');
SPOILER ALERT for above snippet:
As you can see, the timer runs in the end although it is a 0 second timer, it does not actually execute at 0 seconds. Why is that? Because Settimeout uses a callback, and promises are microtasks, Microtask Priority is always greater than Callback Priority

You are correct up until this point:
That means if we assume it took (say) 10ms to come to the end of the code, then since the event-queue has the setTimeout (i) in the front, it waits for 3s, then pops the alert: "Hello 1", at the time = 3010ms
setTimeout will queue the callback to run after a certain time from the moment the setTimeout is called. For example, if setTimeout(fn, 3000) is run, and then 5 seconds of expensive blocking code runs, fn will run immediately after those 5 seconds. If 1 second of blocking code runs instead, fn will run 2 seconds after that blocking code finishes. For example:
console.log('script start');
// Putting the below in a setTimeout so that the above log gets rendered
setTimeout(() => {
setTimeout(() => {
console.log('setTimeout callback');
}, 1000);
const t0 = Date.now();
while (Date.now() - t0 < 700);
console.log('loop done');
}, 30);
Above, you can see that the for loop takes some time to finish, but once it does, the setTimeout callback runs nearly immediately afterwards.
You can think of it like: when setTimeout is called, at Date.now() + delay, a new task gets pushed to the macrotask queue. Other code may be running at the time the task gets pushed, or it may have taken some time before the code after the setTimeout finished, but regardless, the callback will run as soon as it can after Date.now() + delay.
This process is described precisely in the specification:
(After waiting is finished...) Queue a global task on the timer task source given method context to run task.
The task does not exist in the queue (or in the stack) until the time elapses, and the function call only goes into the stack once the task starts running - which may occur as soon as the time elapses, or it may take some additional time if a different task is running at that time.
we had addEventListener()'s with some call-back functions. Even in this case, will the call-back functions of the event listeners be enqueued in the event-queue?
No - their handlers will only get put into the queue once the listener fires.

Why callback never gets called/executed inside while loop? [duplicate]

The following example is given in a Node.js book:
var open = false;
setTimeout(function() {
open = true
}, 1000)
while (!open) {
console.log('wait');
}
console.log('open sesame');
Explaining why the while loop blocks execution, the author says:
Node will never execute the timeout callback because the event loop is
stuck on this while loop started on line 7, never giving it a chance
to process the timeout event!
However, the author doesn't explain why this happens in the context of the event loop or what is really going on under the hood.
Can someone elaborate on this? Why does node get stuck? And how would one change the above code, whilst retaining the while control structure so that the event loop is not blocked and the code will behave as one might reasonably expect; wait
will be logged for only 1 second before the setTimeout fires and the process then exits after logging 'open sesame'.
Generic explanations such as the answers to this question about IO and event loops and callbacks do not really help me rationalise this. I'm hoping an answer which directly references the above code will help.

It's fairly simple really. Internally, node.js consists of this type of loop:
Get something from the event queue
Run whatever task is indicated and run it until it returns
When the above task is done, get the next item from the event queue
Run whatever task is indicated and run it until it returns
Rinse, lather, repeat - over and over
If at some point, there is nothing in the event queue, then go to sleep until something is placed in the event queue or until it's time for a timer to fire.
So, if a piece of Javascript is sitting in a while() loop, then that task is not finishing and per the above sequence, nothing new will be picked out of the event queue until that prior task is completely done. So, a very long or forever running while() loop just gums up the works. Because Javascript only runs one task at a time (single threaded for JS execution), if that one task is spinning in a while loop, then nothing else can ever execute.
Here's a simple example that might help explain it:
var done = false;
// set a timer for 1 second from now to set done to true
setTimeout(function() {
done = true;
}, 1000);
// spin wait for the done value to change
while (!done) { /* do nothing */}
console.log("finally, the done value changed!");
Some might logically think that the while loop will spin until the timer fires and then the timer will change the value of done to true and then the while loop will finish and the console.log() at the end will execute. That is NOT what will happen. This will actually be an infinite loop and the console.log() statement will never be executed.
The issue is that once you go into the spin wait in the while() loop, NO other Javascript can execute. So, the timer that wants to change the value of the done variable cannot execute. Thus, the while loop condition can never change and thus it is an infinite loop.
Here's what happens internally inside the JS engine:
done variable initialized to false
setTimeout() schedules a timer event for 1 second from now
The while loop starts spinning
1 second into the while loop spinning, the timer is ready to fire, but it won't be able to actually do anything until the interpreter gets back to the event loop
The while loop keeps spinning because the done variable never changes. Because it continues to spin, the JS engine never finishes this thread of execution and never gets to pull the next item from the event queue or run the pending timer.
node.js is an event driven environment. To solve this problem in a real world application, the done flag would get changed on some future event. So, rather than a spinning while loop, you would register an event handler for some relevant event in the future and do your work there. In the absolute worst case, you could set a recurring timer and "poll" to check the flag ever so often, but in nearly every single case, you can register an event handler for the actual event that will cause the done flag to change and do your work in that. Properly designed code that knows other code wants to know when something has changed may even offer its own event listener and its own notification events that one can register an interest in or even just a simple callback.

This is a great question but I found a fix!
var sleep = require('system-sleep')
var done = false
setTimeout(function() {
done = true
}, 1000)
while (!done) {
sleep(100)
console.log('sleeping')
}
console.log('finally, the done value changed!')
I think it works because system-sleep is not a spin wait.

There is another solution. You can get access to event loop almost every cycle.
let done = false;
setTimeout(() => {
done = true
}, 5);
const eventLoopQueue = () => {
return new Promise(resolve =>
setImmediate(() => {
console.log('event loop');
resolve();
})
);
}
const run = async () => {
while (!done) {
console.log('loop');
await eventLoopQueue();
}
}
run().then(() => console.log('Done'));

Node is a single serial task. There is no parallelism, and its concurrency is IO bound. Think of it like this: Everything is running on a single thread, when you make an IO call that is blocking/synchronous your process halts until the data is returned; however say we have a single thread that instead of waiting on IO(reading disk, grabbing a url, etc) your task continues on to the next task, and after that task is complete it checks that IO. This is basically what node does, its an "event-loop" its polling IO for completion(or progress) on a loop. So when a task does not complete(your loop) the event loop does not progress. To put it simply.

because timer needs to comeback and is waiting loop to finish to add to the queue, so although the timeout is in a separate thread, and may indeed finsihed the timer, but the "task" to set done = true is waiting on that infinite loop to finish

var open = false;
const EventEmitter = require("events");
const eventEmitter = new EventEmitter();
setTimeout(function () {
open = true;
eventEmitter.emit("open_var_changed");
}, 1000);
let wait_interval = setInterval(() => {
console.log("waiting");
}, 100);
eventEmitter.on("open_var_changed", () => {
clearInterval(wait_interval);
console.log("open var changed to ", open);
});
this exemple works and you can do setInterval and check if the open value changed inside it and it will work

Why are some JavaScript developers using setTimeout for one millisecond? [duplicate]

This question already has answers here:
Why is setTimeout(fn, 0) sometimes useful?
(19 answers)
Closed 9 years ago.
I have problem when using jQuery plugin tablesorter and I can't call trigger twice.
For example this won't work:
this._$table.trigger('update');
this._$table.trigger('sorton', [[[1,1]]]);
But this works:
this._$table.trigger('update');
setTimeout($.proxy(function() {
this._$table.trigger('sorton', [[[1,1]]]);
}, this), 1);
And then I see that problem was in trigger 'update', it call method with body:
function () {
var me = this;
setTimeout(function () {
// rebuild parsers.
me.config.parsers = buildParserCache(
me, $headers);
// rebuild the cache map
cache = buildCache(me);
}, 1);
}
Why did the tablesorter developer use setTimeout with one millisecond?

Short asnwer: Function execution queueing
This is the short answer to your question. setTimeout with either 0 or 1 millisecond is used for function execution queueing. Read on to find out why and how.
Javascript has single threaded execution
Javascript engine is a single threaded process. So whenever developers wanted to defer some function execution to get executed right after the current one that's just being executed, a setTimeout is being used to actually queue the next function... It doesn't have anything to do directly with events although functions may be event handlers. The only event in this equation is the timeout event that setTimeout creates.
This is an example of two functions where the first function during its execution queues a second function to be executed right after it.
function first()
{
// does whatever it needs to
// something else needs to be executed right afterwards
setTimeout(second, 1);
// do some final processing and exit
return;
}
function second()
{
// whatever needs to be done
}
So to javascript engine thread the execution queue looks like this:
first()
second()
Mind that this has nothing to do with function call stack.
Why 1ms?
1ms is a very short amount of time, which (almost) assures that your second function will get executed right after your first function returns. You may see sometimes even 0ms which actually executes it right after first function returns.
If one would on the other hand use longer time i.e. 100ms this could result in a different function getting executed in the meantime and that could have an undesired effect on the whole UI process.
Why function queueing in the first place?
Browsers nowadays prevent client side functionality to hang current browser session by observing long running functions. If a particular function runs long enough, browser Javascript execution engine will pause it and ask the user whether they want to terminate it (kill it) or wait for it to complete.
This is usually undesired effect when you actually do have a long running function. For instance imagine you have a function that has to loop through a large number of items processing each one during the process. You definitely don't want the user to terminate the process because the loop needs to execute.
What's the solution in this case? In such case instead of having a single function with loop and executing it, you'd rather have the loop (queueing) function that would then queue function calls for processing each item. This is just an outer skeleton of such functionality.
function queueItems(items) {
for(var i = 0; i < items.length, i++)
{
setTimeout((function(item) {
return function() {
processItem(item);
};
})(items[i]), 0);
}
}
function processItem(item) {
// process individual item
}
This way you'd prevent your functions to run too long and after each executed function control would get back to Javascript engine resetting its function-hang timer. But be aware that while your functions are being executed your UI will likely be unresponsive or at most unpredictable. It may be better to queue your function with some time space in between so UI stays responsive if that's desired.

It's an old hack. If an event needs to be triggered after another event you can use setTimeout with 1ms to make sure the event is triggered after the other event.

I think that since trigger('update') internally has a setTimeout, only by setting another setTimeout you can achieve the desired order of statement execution. If you don't call 'sorton' through setTimeout it will be executed before 'update'.
On the other hand I guess 'update' uses setTimeout for preventing 'update' from being a blocking function when it may take a long time to be executed.

What are the proper use cases for process.nextTick in Node.js?

I have seen process.nextTick used in a few places and can't quite tell what it's being used for.
https://github.com/andrewvc/node-paperboy/blob/master/lib/paperboy.js#L24
https://github.com/substack/node-browserify/blob/master/index.js#L95
What are the main/proper use cases of process.nextTick in Node.js? The docs basically say it's a more optimized way of doing setTimeout, but that doesn't help much.
I used to do a lot of ActionScript, so the idea of "waiting until the next frame" to execute code makes sense on some level - if you're running an animation you can have it update every frame rather than every millisecond for example. It also makes sense when you want to coordinate setting a bunch of variables - you change the variables in frame 1, and apply the changes in frame 2. Flex implemented something like this in their component lifecycle.
My question is, what should I be using this for in server-side JavaScript? I don't see any places right off the bat where you'd need this kind of fine-tuned performance/flow control. Just looking for a point in the right direction.

process.nextTick puts a callback into a queue. Every callback in this queue will get executed at the very beginning of the next tick of the event loop. It's basically used as a way to clear your call stack. When the documentation says it's like setTimeout, it means to say it's like using setTimeout(function() { ... }, 1) in the browser. It has the same use cases.
One example use case would be, you create a constructor for some object that needs events bound to it. However, you can't start emitting events right away, because the code instantiating it hasn't had time to bind to events yet. Your constructor call is above them in the call stack, and if you continue to do synchronous things, it will stay that way. In this case, you could use a process.nextTick before proceeding to whatever you were about to do. It guarantees that the person using your constructor will have time enough to bind events.
Example:
var MyConstructor = function() {
...
process.nextTick(function() {
self._continue();
});
};
MyConstructor.prototype.__proto__ = EventEmitter.prototype;
MyConstructor.prototype._continue = function() {
// without the process.nextTick
// these events would be emitted immediately
// with no listeners. they would be lost.
this.emit('data', 'hello');
this.emit('data', 'world');
this.emit('end');
};
Example Middleware using this constructor
function(req, res, next) {
var c = new MyConstructor(...);
c.on('data', function(data) {
console.log(data);
});
c.on('end', next);
}

It simply runs your function at the end of the current operation before the next I/O callbacks. Per documentation you can use it run your code after the callers synchronous code has executed, potentially if you can use this to give your API/library user an opportunity to register event handlers which need to be emitted ASAP. Another use case is to ensure that you always call the callbacks with asynchronously to have consistent behaviours in different cases.
In the past process.nextTick would be have been used provide an opportunities for I/O events to be executed however this is not the behaviour anymore and setImmediate was instead created for that behaviour. I explained a use case in the answer of this question.

"Every callback in this queue will get executed at the very beginning of the next tick of the event loop" is not correct. Actually, nextTick() runs right after completing the current phase and before starting the next phase. Minute details are important!
A function passed to process.nextTick() is going to be executed on the current iteration of the event loop, after the current operation ends. This means it will always execute before setTimeout and setImmediate.
Understanding setImmediate()

Is looping events in JavaScript using document.dispatchEvent possible?

I'd like to create an event loop mechanism in JavaScript/DOM using only dispatchEvent calls.
For example:
document.addEventListener("LoopingEvent", loop, true);
var loop = function() {
doSomeWork();
updateUI();
document.dispatchEvent(loopEvent);
};
var loopEvent = document.createEvent('Events');
loopEvent.initEvent("LoopingEvent", true, true);
document.dispatchEvent(loopEvent);
When run, a call stack OutOfRange error is thrown. If I change the loop handler's dispatch call to use a window.setTimeout delay it loops without error.
Just wondering if there is a way to use dispatchEvent looping infinitely without resorting to setInterval or setTimeout? The advantage in a dispatchEvent looping pattern is that the calls occur when the work is done rather than at set time intervals.
Thanks in advance for any insights...

dispatchEvent sends the event synchronously to the target, so when you use dispatchEvent the event handler frames accumulate on the stack and eventually overflow.
If you want to simply "loop forever" you have a few choices. Which choice is correct depends on how you want your code to interact with other events. I notice that your code suggests that it will updateUI(). Well, your event handler needs to return to the browser's event loop periodically so that it can paint your updated UI. Using setTimeout(loop, 0); is a good way to achieve this:
function loop() {
doSomeWork();
updateUI();
setTimeout(loop, 0);
}
loop(); // get things started
The call to setTimeout will return before loop is invoked again; then the browser will invoke loop again. In between calls to loop the browser may run other code, such as painting the changes in the UI, or invoking other event handlers in response to clicks and other events.
If you want you can make your loop run more slowly by increasing the delay from 0 msec to something larger. This might be useful if your loop is doing animation.
If you want your loop to stop, then don't call setTimeout and it won't be called again.
Now here is an alternative technique:
If you are using a relatively recent version of Firefox, Chrome or Safari you can use a new feature called workers to run your loop. Workers have their own event loop, so it is OK to write code like this:
// worker code
while (true) {
doSomeWork();
// post messages to update the UI
}
Workers run separately from other scripts; to push results into the page itself you need to use a function called postMessage. Here is the relevant spec, however you might want to also search for tutorials or post a follow-up question because working off the spec can be difficult at first.

It looks like you're inducing an infinite loop that will continue to run indefinitely. A timer delay between execution is necessary to let other functions queue on the thread.
The advantage in a dispatchEvent looping pattern is that the calls occur when the work is done rather than at set time intervals.
setTimeout with a delay of 0ms would achieve that effect, although a looping setTimeout or setInterval would cause another infinite loop to be created, so at least some delay is necessary, as I pointed out above.

I can't comment about dispatchEvent() but what's wrong with this pattern:
function DoSomeWork()
{
// do work
if (moreWorkNeedsDoing)
{
setTimeout(DoSomeWork, 0);
}
}
The function will iterate 'immediately' as long as there is work to do.