Understanding the asyncronous nature of fetch().then() - javascript

This is a predominantly theoretical question as I guess I would not use the fetched data outside of the .then() method.
I was just checking the behaviour of fetch.then() out of curiosity as I like to test and understand the principles of the languages I learn.
As expected
setTimeout(()=>{
console.log("myGlobVar in setTimeOut");
console.log(myGlobVar)}, 1000);
returns the fetched object, provided I use a delay long enough in setTimeOut (about 40ms in this case)
But I'm curious why the
console.log("myGlobVar after for loop");
console.log(myGlobVar); //returns undefined
does not work.
My rationale here was that running the loop long enough, would give enough time to fetch() to fetch the data and assign it to myGlobVar, which doesn't happen no matter how long it runs.
Actually -counterintuitively- for extremely high numbers in the loop (1000000000) even the
setTimeout(()=>{
console.log("myGlobVar in setTimeOut");
console.log(myGlobVar)}, 1000);
returns undefined.
<script>
"use strict"
let myGlobVar;
function loadPeaks() {
console.log("loadPeaks");
fetch('soundofChengdu.json')
.then(response => {
if (!response.ok) {
throw new Error("HTTP error: " + response.status);
}
return response.json();
})
.then(peaks => {
// wavesurfer.load('soundofChengdu.mp3', peaks.data);
myGlobVar= peaks;
})
.catch((e) => {
console.log({errorIs: e})
// console.error('error', e);
});
}
loadPeaks();
console.log("myGlobVar without waiting");
console.log(myGlobVar); //returns undefined
setTimeout(()=>{console.log("myGlobVar in setTimeOut"); console.log(myGlobVar)}, 1000); //returns undefined under +-40 ms, works above.
let b;
console.log("loop running");
for (let a=0; a<100000000; a++) {
b=b+a;
}
console.log("myGlobVar after for loop");
console.log(myGlobVar); //returns undefined
</script>

The first part is quite usual, you blocked the event-loop synchronously, no other task or even microtask can be executed in between so your variable is not set.
The second part about setTimeout() firing before the request's Promise gets resolved is a bit more interesting though, what you've discovered here is the task prioritization system.
All tasks don't have the same priority, and browsers are free to execute one kind of task before another.
For instance here, your browser gives more importance to timer tasks than to network tasks. And for the ones wondering "but fetch().then() is a microtask and should have higher priority", the returned Promise is resolved in a "normal" task (see this answer of mine), and that task is subject to the task prioritization system.

The code after loadPeaks() is synchronously called, meaning that the code will not wait for loadPeaks() to finish running and will run completely before loadPeaks() finishes.
This is the same for the for loop. Regardless of the iteration count, the for loop will finish before the request finishes. You can test this by adding a console.log() statement in your second then() call and see when it is printed in relation to the rest of your log statements.
Promise.then() on the other hand, waits for the request to finish. In your case, fetch() will run first and the then() calls will run sequentially, waiting for the previous call to finish before running.

My rationale here was that running the loop long enough, would give enough time to fetch() to fetch the data and assign it to myGlobVar, which doesn't happen no matter how long it runs.
JavaScript is actually strictly single-threaded, unlike most other languages you're probably familiar with. Instead of having threads running in parallel, it uses an Event Loop pattern where each operation runs, pushes follow-up operations onto the queue, and then yields back to the loop. I'll quote from the article I just linked to explain how this affects your code (emphasis mine):
"Run-to-Completion"
Each message is processed completely before any other message is processed.
This offers some nice properties when reasoning about your program, including the fact that whenever a function runs, it cannot be preempted and will run entirely before any other code runs (and can modify data the function manipulates). This differs from C, for instance, where if a function runs in a thread, it may be stopped at any point by the runtime system to run some other code in another thread.
This explains why the code after the for loop never has access to that global variable no matter how high the iteration count is: the event loop is still executing your original function, so none of those promise handlers have had any chance to execute.
#Kaiido's answer for your second question is much better than what I was going to write, so I won't bother. I'll leave the above up just in case a more basic explanation helps.

Related

How does concurrency work while using async/await?

I couldn't understand the execution order of the two codes below.
async function main(test)
{
while(1) console.log(test);
}
main("a");
main("b");
This code above logs infinite "a".
async function main(test)
{
while(1) await console.log(test);
}
main("a");
main("b");
While this code above logs infinite "a" and "b".
I wanted to understand async/await better but this behaviour above made me confused. Like how are those log functions handled by event loop?
An async function runs synchronously until the first await, return, or implicit return (the code falling off the end of the function). That's so it can start the asynchronous process that it will later report the completion of (by settling the promise it returns). So your first example logs a forever because that's done by the synchronous part of the first call to main; execution never reaches the second call to main.
Your second example introduces asynchronousness by adding await (it doesn't matter that what await is awaiting isn't a promise [console.log returns undefined]; it gets wrapped in one). That means that the first call to main runs synchronously until after the first console.log, then stops and returns a promise while it waits for the promise wrapped around undefined to settle.¹ That allows the second call to main to go forward, doing the same thing. At that point, the code making the calls to main is complete and any pending promise callbacks can be executed. The first pending promise callback is for the await in the first call to main, so that call does another loop iteration; the second pending promise is for the await in the second call to main, so it does another loop iteration; and so on, and so on, and so on.
You mentioned "concurrency." The two calls to main in your second example do run concurrently but are not multi-threaded; instead, they each alternately get access to the one main thread briefly until their next await. JavaScript works on a model of one thread per "realm" (loosely, per global environment). So you don't have to worry about threading issues like reordered operations or stale caches like you would in a multi-threaded environment (unless you're using shared memory, but you aren't in that code); there's only one thread.
¹ Technically, the promise is already settled in this case before await starts awaiting it (because it's just a promise wrapped around undefined), but when you attach to a promise, it never triggers your code synchronously, even if the promise is already settled.
Using Asynchronous functions within Javascript usually resolves data that is not readily available. Examples include calling from a database and scraping web pages for data.
The code provided endlessly loops because there is no end to the said loop (while). In the context of the code provided, while(1) will always remain true, as there is no condition to state otherwise. In practical use, loops are designed to cease when a condition is met.
Here's a basic example of using a while loop sourced from MDN Docs covering break
let n = 0;
while (n < 3) {
n++;
}
console.log(n);
// expected output: 3
To better understand asynchronous programming within Javascript, I highly recommend the MDN Docs on async functions.
To answer your question regarding console logging's interactions with await: JS will 'freeze' the script when it reads an await and work on completing its defined actions, then returns to read the following line. In the context of the code, Javascript is doing its job. It circles back to a lack of a break or condition for the loop.

Is non-parallel access to a method in node JS guaranteed?

Javascript is single threaded and - Node.js uses an asynchronous event-driven design pattern, which means that multiple actions are taken at the same time while executing a program.
With this in mind, I have a pseudo code:
myFunction() // main flow
var httpCallMade = false // a global variable
async myFunction() {
const someData = await callDB() // LINE 1 network call
renderMethod() // LINE 2 flow1
}
redisPubSubEventHandler() { // a method that is called from redis subscription asynchronously somewhere from a background task in the program
renderMethod() // LINE 3 flow2
}
renderMethod(){
if(!httpCallMade) {
httpCallMade = true //set a global flag
const res = makeHTTPCall() // an asynchronous network call. returns a promise.
} // I want to ensure that this block is "synchronized" and is not acessible by flow1 and flow2 simultaneously!
}
myFunction() is called in the main thread - while redisPubSubEventHandler() is called asynchronously from a background task in the program. Both flows would end in calling renderMethod(). The idea is to ensure makeHTTPCall() (inside renderMethod) is only allowed to be called once
Is it guaranteed that renderMethod() would never be executed in parallel by LINE2 and LINE3 at the same time? My understanding is that as soon as renderMethod() is executed - event loop will not allow anything else to happen in server - which guarantees that it is only executed once at a given time (even if it had a network call without await).
Is this understanding correct?
If not, how do I make synchronize/lock entry to renderMethod?
Javascript is single-threaded. Therefore, unless you are deliberately using threads (eg. worker_threads in node.js) no function in the current thread can be executed by two parallel threads at the same time.
This explains why javascript has no mutex or semaphore capability - because generally it is not needed (note: you can still have race conditions because asynchronous code may be executed in a sequence you did not expect).
There is a general confusion that asynchronous code means parallel code execution (multi-threaded). It can but most of the time when a system is labeled as asynchronous or non-blocking or event-oriented INSTEAD of multi-threaded it often means that the system is single-threaded.
In this case asynchronous means parallel WAIT. Not parallel code execution. Code is always executed sequentially - only, due to the ability of waiting in parallel you may not always know the sequence the code is executed in.
There are parts of javascript that execute in a separate thread. Modern browsers execute each tab and iframe in its own thread (but each tab or iframe are themselves single-threaded). But script cannot cross tabs, windows or iframes. So this is a non-issue. Script may access objects inside iframes but this is done via an API and the script itself cannot execute in the foreign iframe.
Node.js and some browsers also do DNS queries in a separate thread because there is no standardized cross-platform non-blocking API for DNS queries. But this is C code and not your javascript code. Your only interaction with this kind of multi-threading is when you pass a URL to fetch() or XMLHttpRequest().
Node.js also implement file I/O, zip compression and cryptographic functions in separate threads but again this is C code, not your javascript code. All results from these separate threads are returned to you asynchronously via the event loop so by the time your javascript code process the result we are back to executing sequentially in the main thread.
Finally both node.js and browsers have worker APIs (web workers for browsers and worker threads for node.js). However, both these API use message passing to transfer data (in node only a pointer is passed in the message thus the underlying memory is shared) and it still protects functions from having their variables overwritten by another thread.
In your code, both myFunction() and redisPubSubEventHandler() run in the main thread. It works like this:
myFunction() is called, it returns immediately when it encounters the await.
a bunch of functions are declared and compiled.
we reach the end of your script:
// I want to ensure that this method is "synchronized" and is not called by flow1 and flow2 simultaneously!
}
<----- we reach here
now that we have reached the end of script we enter the event loop...
either the callDB or the redis event completes, our process gets woken up
the event loop figures out which handler to call based on what event happened
either the await returns and call renderMethod() or redisPubSubEventHandler() gets executed and call renderMethod()
In either case both your renderMethod() calls will execute on the main thread. Thus it is impossible for renderMethod() to run in parallel.
It is possible for renderMethod() to be half executed and another call to renderMethod() happens IF it contains the await keyword. This is because the first call is suspended at the await allowing the interpreter to call renderMethod() again before the first call completes. But note that even in this case you are only in trouble if you have an await between if.. and httpCallMade = true.
You need to differentiate between synchronous and asynchronous, and single- and multi-threaded.
JavaScript is single-threaded so no two lines of the same execution context can run at the same time.
But JavaScript allows asynchronous code execution (await/async), so the code in the execution context does not need to be in the order it appears in the code but that different parts of the code can be executed interleaved (not overlapped) - which could be called "running in parallel", even so, I think this is misleading.
event-driven design pattern, which means that multiple actions are taken at the same time while executing a program.
There are certain actions that can happen at the same time, like IO, multiprocessing (WebWorkers), but that is (with respect to JavaScript Code execution) not multi-threaded.
Is it guaranteed that renderMethod() would never be executed in parallel by LINE2 and LINE3 at the same time?
Depends on what you define as parallel at the same time.
Parts of logic you describe in renderMethod() will (as you do the request asynchronously) run interleaved, so renderMethod(){ if(!httpCallMade) { could be executed multiple times before you get the response (not the Promise) back from makeHTTPCall but the code lines will never executed at the same time.
My understanding is that as soon as renderMethod() is executed - event loop will not allow anything else to happen in server - which guarantees that it is only executed once at a given time (even if it had a network call without await).
The problem here is, that you somehow need to get the data from your async response.
Therefore you either need to mark your function as async and use const res = await makeHTTPCall() this would allow code interleaving at the point of await. Or use .then(…) with a callback, which will be executed asynchronously at a later point (after you left the function)
But from the beginning of the function to the first await other the .then not interleaving could take place.
So your httpCallMade = true would prevent that another makeHTTPCall could take place, before the currently running is finished, under the assumption that you set httpCallMade to false only when the request is finished (in .then callback, or after the await)
// I want to ensure that this method is "synchronized" and is not called by flow1 and flow2 simultaneously!
As soon as a get a result in an asynchronous way, you can't go back to synchronous code execution. So you need to have a guard like httpCallMade to prevent that the logic described in renderMethod can run multiple times interleaved.
Your question really comes down to:
Given this code:
var flag = false;
function f() {
if (!flag) {
flag = true;
console.log("hello");
}
}
and considering that flag is not modified anywhere else, and many different, asynchronous events may call this function f...:
Can "hello" be printed twice?
The answer is no: if this runs on an ECMAScript compliant JS engine, then the call stack must be empty first before the next job is pulled from an event/job queue. Asynchronous tasks/reactions are pushed on an event queue. They don't execute before the currently executing JavaScript has run to completion, i.e. up until the call stack is empty. So they never interrupt running JavaScript code pre-emptively.
This is true even if these asynchronous tasks/events/jobs are scheduled by other threads, lower-level non-JS code,...etc. They all must wait their turn to be consumed by the JS engine. And this will happen one after the other.
For more information, see the ECMAScript specification on "Job". For instance 8.4 Jobs and Host Operations to Enqueue Jobs:
A Job is an abstract closure with no parameters that initiates an ECMAScript computation when no other ECMAScript computation is currently in progress.
[...]
Only one Job may be actively undergoing evaluation at any point in time.
Once evaluation of a Job starts, it must run to completion before evaluation of any other Job starts.
For example, promises generate such jobs -- See 25.6.1.3.2 Promise Resolve Functions:
When a promise resolve function is called with argument resolution, the following steps are taken:
[...]
Perform HostEnqueuePromiseJob(job.[[Job]], job.[[Realm]]).
It sounds like you want to do something like a 'debounce', where any event will cause makeHttpCall() execute, but it should only be executing once at a time, and should execute again after the last call if another event has occurred while it was executing. So like this:
DB Call is made, and makeHttpCall() should execute
While makeHttpCall() is executing, you get a redis pub/sub event that should execute makeHttpCall() again, but that is delayed because it is already executing
Still before the first call is done, another DB call is made and requires makeHttpCall() to execute again. But even though you have received two events, you only need to have it called one time to update something with the most recent information you have.
The first call to makeHttpCall() finishes, but since there have been two events, you need to make a call again.
const makeHttpCall = () => new Promise(resolve => {
// resolve after 2 seconds
setTimeout(resolve, 2000);
});
// returns a function to call that will call your function
const createDebouncer = (fn) => {
let eventCounter = 0;
let inProgress = false;
const execute = () => {
if (inProgress) {
eventCounter++;
console.log('execute() called, but call is in progress.');
console.log(`There are now ${eventCounter} events since last call.`);
return;
}
console.log(`Executing... There have been ${eventCounter} events.`);
eventCounter = 0;
inProgress = true;
fn()
.then(() => {
console.log('async function call completed!');
inProgress = false;
if (eventCounter > 0) {
// make another call if there are pending events since the last call
execute();
}
});
}
return execute;
}
let debouncer = createDebouncer(makeHttpCall);
document.getElementById('buttonDoEvent').addEventListener('click', () => {
debouncer();
});
<button id="buttonDoEvent">Do Event</button>

How do Promises change the use of functions

I am having trouble finding a use for Promises. Wouldn't these 2 approaches below work the same exact way? Since the while loop in loopTest() is synchronous, logStatement() function wouldn't run until it's complete anyways so how would the the 2nd approach be any different ..wouldn't it be pointless in waiting for it to resolve() ?
1st approach:
function loopTest() {
while ( i < 10000 ) {
console.log(i)
i++
})
}
function logStatement() {
console.log("Logging test")
}
loopTest();
logStatement();
2nd approach:
function loopTest() {
return new Promise((resolve, reject) => {
while ( i < 10000 ) {
console.log(i)
i++
if (i === 999) {
resolve('I AM DONE')
}
})
});
}
function logStatement() {
console.log("Logging test")
}
loopTest().then(logStatement());
Promises don't make anything asynchronous,¹ so you're right, there's no point to using a promise in the code you've shown.
The purpose of promises is to provide a standard, composable means of observing the result of things that are already asynchronous (like ajax calls).
There are at least three massive benefits to having a standardized way to observe the results of asynchronous operations:
We can have standard semantics for consuming individual promises, rather than every API defining its own signature for callback functions. (Does it signal error with an initial parameter that's null on success, like Node.js? Does it call the callback with an object with a success flag? Or...)
We can have standard ways of composing/combining them, such as Promise.all, Promise.race, Promise.allSettled, etc.
We can have syntax to consume them with our usual control structures, which we have now in the form of async functions and await.
But again, throwing a promise at a synchronous process almost never does anything useful.²
¹ One very small caveat there: The handler functions to attach to a promise are always triggered asynchronously, whether the promise is already settled or not.
² Another small caveat: Sometimes, you have a synchronous result you want to include in a composition operation (Promise.all, etc.) with various asynchronous operations. In that case, wrapping the value in a promise that's instantly fulfilled is useful — and in fact, all the standard promise combinators (Promise.all, etc.) do that for you, as does await.
There's no point in what you are doing, because your function body is just a blocking loop.
To get a benefit from Promises, use it with APIs that do something with IO, such as a HTTP request, or reading a file from disk.
These APIs all traditionally used callbacks, and are now mostly Promise based.
Anything function that uses a Promise-based function, should itself also be Promise-based. This is why you see a lot of promises in modern code, as a promise only has to be used at 1 level in a stack for the entire stack to be asynchronous in nature.
Is this a better example of how Promises are used? This is all I can think of to make it show use to me:
Version 1
function getData() {
fetch('https://jsonplaceholder.typicode.com/todos/1')
.then(data => data.json())
.then(json => console.log(json))
}
function logInfo() {
console.log("i am a logger")
}
getData()
logInfo()
// "I am a logger"
// {"test": "json"}
Version 2
function getData() {
return fetch('https://jsonplaceholder.typicode.com/todos/1')
.then(data => data.json())
.then(json => console.log(json))
}
function logInfo() {
console.log("i am a logger")
}
getData().then(logInfo);
// "{"test": "json"}
// "I am a logger"
// waits for API result to log _then_ logInfo is run , which makes a log statement
There's definitely benefits to using Promises but that's only in certain scenarios where their usage would seem viable.
Your example could represent what would happen when you retrieve data from an external source synchronously, it would block the thread preventing further code from executing until the loop terminates (I explain below why exactly that happens) - wrapping it in a promise gives no different output in that the thread is still being blocked and when the next message in the queue has to be processed, it gets processed like normal right after it ends.
However an implementation similar to this could achieve a while loop running in a non-blocking manner, just an idea (don't mean to derail this topic with setInterval's implementation):
let f = () => {
let tick = Date.now;
let t = tick();
let interval = setInterval(() => {
if (tick() - t >= 3000) {
console.log("stop");
clearInterval(interval);
}
}, 0);
};
f()
console.log("start");
Basically the time is checked/handled in a separate thread in the browser and the callback is executed every time the time specified runs out while the interval hasn't been cleared, after the call stack becomes empty (so UI function isn't affected) and the current executing function terminates/ends or after other functions above it in the stack finish running. I don't know about the performance implications of doing something like this but I feel like this should only be used when necessary, since the callback would have to be executed very frequently (with 0 timeout, although it's not guaranteed to be 0 anyway).
why it happens
I mainly want to clarify that while the handler functions will be scheduled to be executed asynchronously, every message in the queue has to be processed completely before the next one and for the duration your while loop executes, no new message can be processed in the event queue so it would be pointless to involve Promises where the same thing would happen without them.
So basically the answer to:
wouldn't it be pointless in waiting for it to resolve() ?
is yes, it would be pointless in this case.

How does the JS event loop behave when a Promise's resolution depends on setTimeout?

console.log('1')
setTimeout(() => {
console.log('2')
}, 0)
function three() {
return new Promise(resolve => {
setTimeout(() => {
return new Promise(resolve => resolve('3'))
},0)
})
}
three().then(result => console.log(result))
console.log('4')
This code snippet outputs 1 4 2
This is the behavior I would expect based on my understanding of javascript's event loop and concurrency model. But it leaves me with some lingering questions.
Before getting to those questions, I'll first break down my understanding of this code snippet.
Why code outputs 1
no explanation needed
Why code outputs 4
the callback that outputs 2 gets loaded into the event queue (aka macro task queue) after 0ms, but doesn't get executed until the main call stack is emptied.
even if three was a promise that was immediately resolved, its code is loaded into the job queue (aka microtask queue) and wouldn't be executed until the main call stack is emptied (regardless of the contents of the event queue)
Why code outputs 2
after console.log(4) the main call stack is empty and javascript looks for the next callback to load on the main stack. It's pretty safe to assume that at this point, some "worker thread" had already put the callback function that outputs 2 onto the macro task queue. This gets loaded onto the stack and 2 is output.
Why code does NOT output 3
This is where its a little blurry for me. The function three returns a promise that is then-ed in the main thread. Callback functions passed through then are loaded onto microtask queue and executed before the next task in the macrotask queue. So while you might think it'll run before the callback that logs 2, its actually theoretically impossible for it to run at all. That's because the Promise is only resolved via the callback function of its setTimeout, and that callback function (because of setTimeout) would only run if the main execution thread (the same thread that's waiting for the promise to resolve) is empty.
Why does this bother me
I'm trying to build a complete theoretical mental model of how javascript handles concurrency. One of the missing pieces in that model is the relationship between network requests, promises, and the event loop. Take the above code snippet, and suppose I replace three's setTimeout with some sort of network request (a very common thing in async web development). Assuming that the network request behaves similarly to setTimeout, in that when the "worker thread" is done, a callback is pushed to the macro task queue, it's hard for me to understand how that callback even gets executed. But this is something that happens literally all the time.
Can someone help me understand? Do I have any missing gaps in my current understanding of js concurrency? Have I made an incorrect assumption? Does any of this actually make any sense? lol
Why code does NOT output 3
In this code:
function three() {
return new Promise(resolve => {
setTimeout(() => {
return new Promise(resolve => resolve('3'))
},0)
})
}
three().then(result => console.log(result))
You never resolve the first Promise that three() creates. Since that's the one that is returned form three(), then the .then() handler in three().then(...) is never called.
You do resolve the promise created inside the timer, but you're returning that promise only to the timer callback which does nothing.
If you change your code to this:
function three() {
return new Promise(resolve => {
setTimeout(() => {
resolve('3');
},0)
})
}
three().then(result => console.log(result))
Then, you would see the 3get output.
So, this doesn't have anything to do with the event loop or how it works. It has to do with not resolving the promise that three() returns so the .then() handler on that promise never gets called.
I'm trying to build a complete theoretical mental model of how javascript handles concurrency. One of the missing pieces in that model is the relationship between network requests, promises, and the event loop. Take the above code snippet, and suppose I replace three's setTimeout with some sort of network request (a very common thing in async web development). Assuming that the network request behaves similarly to setTimeout, in that when the "worker thread" is done, a callback is pushed to the macro task queue, it's hard for me to understand how that callback even gets executed. But this is something that happens literally all the time.
Network requests and promises and timers all go through the event loop. There are very complicated rules about how multiple events in queue at the same time are prioritized relative to one another. .then() handlers are generally prioritized first.
Think of the Javascript interpreter as this simplistic sequence.
Get event from event queue
If nothing in the event queue, sleep until something is in the event queue
Run callback function associated with the event you pull from the event queue
Run that callback function until it returns
Note, it may not be completely done with its work because it may
have started other asynchronous operations and set up its own
callbacks or promises for those. But, it has returned from the
original callback that started it
When that callback returns, go back to the first step above and get the next event
Remember that network requests, promises, timers and literally ALL asynchronous operations in node.js go through the event queue in this manner.
Why you assume that network request would behave as setTimeout?
It is a Promise which resolved() would go in microtasks

Unexpected behavior mixing process.nextTick with async/await. How does the event loop work here?

My code:
async function run(){
process.nextTick(()=>{
console.log(1);
});
await (new Promise(resolve=>resolve()).then(()=>{console.log(2)}));
console.log(3);
process.nextTick(()=>{
console.log(4);
});
new Promise(resolve=>resolve()).then(()=>{console.log(5)});
}
run();
My expected output is the numbers to print 1,2,3,4,5 in order, but instead I get:
1
2
3
5
4
As expected, the first nextTick is evaluated before the first .then callback, because process.nextTick and .then are both deferred to future ticks, and process.nextTick is declared before .then. So 1 and 2 are outputted in order as expected.
The code should not reach what is after await until after .then is resolved, and this works as expected, as 3 is outputted in the expected place.
Then essentially we have a repeat of the first part of the code, but this time .then is called before process.nextTick.
This seems like inconsistent behavior. Why does process.nextTick get called before the .then callback the first time around but not the second?
The node.js event queue is not a single queue. It is actually a bunch of different queues and things like process.nextTick() and promise .then() handlers are not handled in the same queues. So, events of different types are not necessarily FIFO.
As such, if you have multiple things that go in the event queue around the same time and you want them served in a specific order, the simplest way to guarantee that order is to write your code to force the order you want, not to try to guess exactly how two things are going to get sequenced that went into the queue around the same time.
It is true that two operations of the exact same type like two process.nextTick() operations or two resolved promise operations will be processed in the order they were put into the event queue. But, operations of different types may not be processed in the order relative to each other that they were put in the event queue because different types of events are processed at different times in the cycle the event loop makes through all the different types of events.
It is probably possible to fully understand exactly how the event loop in node.js works for every type of event and predict exactly how two events that enter the event queue at about the same time will be processed relative to one another, but it is not easy. It is further complicated by the fact that it also depends upon where the event loop is in its current processing when the new events are added to the event queue.
As in my delivery example in my earlier comments, when exactly a new delivery will be processed relative to other deliveries depends upon where the delivery driver is when the new order arrives in the queue. The same can be true of the node.js event system. If a new event is inserted while node.js is processing a timer event, it may have a different relative order to other types of events than if it node.js was processing a file I/O completion event when it was inserted. So, because of this significant complication, I don't recommend trying to predict the execution order of asynchronous events of different types that are inserted into the event queue at about the same time.
And, I should add that native promises are plugged directly into the event loop implementation (as their own type of micro task) so a native promise implementation may behave differently in your original code than a non-native promise implementation. Again a reason not to try to forecast exactly how the event loop will schedule different types of events relative to one another.
If the order of processing is important to your code, then use code to enforce a specific completion processing order.
As an example of how it matters what the event queue is doing when events are inserted into the event queue, your code simplified to this:
async function run(){
process.nextTick(()=>{
console.log(1);
});
await Promise.resolve().then(()=>{console.log(2)});
console.log(3);
process.nextTick(()=>{
console.log(4);
});
Promise.resolve().then(()=>{console.log(5)});
}
run();
Generates this output:
1
2
3
5
4
But, simply change when the run() is called to be Promise.resolve().then(run) and the order is suddenly different:
async function run(){
process.nextTick(()=>{
console.log(1);
});
await Promise.resolve().then(()=>{console.log(2)});
console.log(3);
process.nextTick(()=>{
console.log(4);
});
Promise.resolve().then(()=>{console.log(5)});
}
Promise.resolve().then(run);
Generates this output which is quite different:
2
3
5
1
4
You can see that when the code is started from a resolved promise, then other resolved promises that happen in that code get processed before .nextTick() events which wasn't the case when the code was started from a different point in the event queue processing. This is the part that makes the event queue system very difficult to forecast.
So, if you're trying to guarantee a specific execution order, you have to either use all the same type of events and then they will execute in FIFO order relative to each other or you have to make your code enforce the execution order you want. So, if you really wanted to see this order:
1
2
3
4
5
You could use all promises which would essentially map to this:
async function run(){
Promise.resolve().then(() => {
console.log(1);
})
Promise.resolve().then(() => {
console.log(2)
});
await Promise.resolve().then(()=>{});
console.log(3);
Promise.resolve().then(() => {
console.log(4)
});
Promise.resolve().then(()=>{console.log(5)});
}
run();
Or, you change the structure of your code so the code makes it always process things in the desired order:
async function run(){
process.nextTick(async ()=>{
console.log(1);
await Promise.resolve().then(()=>{console.log(2)});
console.log(3);
process.nextTick(()=>{
console.log(4);
Promise.resolve().then(()=>{console.log(5)});
});
});
}
run();
Either of these last two scenarios will generate the output:
1
2
3
4
5
Thanks to the helpful comments that made me realize that not all javascript ways of deferring to a later tick are created equal. I had figured that (new Promise()).then, process.nextTick, even setTimeout(callback,0) would all be exactly the same but it turns out I can't assume that.
For now I'll leave here that the solution to my problem is simply to not use process.nextTick if it does not work in the expected order.
So I can change my code to (Disclaimer this is not actually good async code in general):
async function run(){
process.nextTick(()=>{
console.log(1);
});
await (new Promise(resolve=>resolve()).then(()=>{console.log(2)}));
console.log(3);
(new Promise(resolve=>resolve())).then(()=>{console.log(4)});
(new Promise(resolve=>resolve())).then(()=>{console.log(5)});
}
run();
Now I'm ensuring that 4 is logged before 5, by making both of them the same type of async call. Making both of them use process.nextTick would also ensure that 4 logs before 5.
async function run(){
process.nextTick(()=>{
console.log(1);
});
await (new Promise(resolve=>resolve()).then(()=>{console.log(2)}));
console.log(3);
process.nextTick(()=>{
console.log(4)
});
process.nextTick(()=>{
console.log(5)
});
}
run();
This is a solution to my problem, but if anyone wants to provide a more direct answer to my question I'll be happy to accept.

Categories