We Keep Coding

How does concurrency work while using async/await?

I couldn't understand the execution order of the two codes below.
async function main(test)
{
while(1) console.log(test);
}
main("a");
main("b");
This code above logs infinite "a".
async function main(test)
{
while(1) await console.log(test);
}
main("a");
main("b");
While this code above logs infinite "a" and "b".
I wanted to understand async/await better but this behaviour above made me confused. Like how are those log functions handled by event loop?

An async function runs synchronously until the first await, return, or implicit return (the code falling off the end of the function). That's so it can start the asynchronous process that it will later report the completion of (by settling the promise it returns). So your first example logs a forever because that's done by the synchronous part of the first call to main; execution never reaches the second call to main.
Your second example introduces asynchronousness by adding await (it doesn't matter that what await is awaiting isn't a promise [console.log returns undefined]; it gets wrapped in one). That means that the first call to main runs synchronously until after the first console.log, then stops and returns a promise while it waits for the promise wrapped around undefined to settle.¹ That allows the second call to main to go forward, doing the same thing. At that point, the code making the calls to main is complete and any pending promise callbacks can be executed. The first pending promise callback is for the await in the first call to main, so that call does another loop iteration; the second pending promise is for the await in the second call to main, so it does another loop iteration; and so on, and so on, and so on.
You mentioned "concurrency." The two calls to main in your second example do run concurrently but are not multi-threaded; instead, they each alternately get access to the one main thread briefly until their next await. JavaScript works on a model of one thread per "realm" (loosely, per global environment). So you don't have to worry about threading issues like reordered operations or stale caches like you would in a multi-threaded environment (unless you're using shared memory, but you aren't in that code); there's only one thread.
¹ Technically, the promise is already settled in this case before await starts awaiting it (because it's just a promise wrapped around undefined), but when you attach to a promise, it never triggers your code synchronously, even if the promise is already settled.

Using Asynchronous functions within Javascript usually resolves data that is not readily available. Examples include calling from a database and scraping web pages for data.
The code provided endlessly loops because there is no end to the said loop (while). In the context of the code provided, while(1) will always remain true, as there is no condition to state otherwise. In practical use, loops are designed to cease when a condition is met.
Here's a basic example of using a while loop sourced from MDN Docs covering break
let n = 0;
while (n < 3) {
n++;
}
console.log(n);
// expected output: 3
To better understand asynchronous programming within Javascript, I highly recommend the MDN Docs on async functions.
To answer your question regarding console logging's interactions with await: JS will 'freeze' the script when it reads an await and work on completing its defined actions, then returns to read the following line. In the context of the code, Javascript is doing its job. It circles back to a lack of a break or condition for the loop.

Why wrapping your code in resolved prop makes your sync code act like async code?

Loop inside of the promise works like sync code, for example:
console.log('im working')
function proLoop(){
return Promise((resolve ,rejects)=>{
for (let i = 0; i < 1000000000000; i++) {}
console.log('loop is done')
})
}
proLoop();
console.log('im working')
So even if we write is like promise it will get more time and freezes our code In other words it will works synchronically.
i find a solution but why it works?
so the solution is just warp your code as like resolved promise
like this
return new Promise.resolve().then( ()=>{
for (let i = 0; i < 1000000000000; i++) {}
console.log('loop is done')
})
but why and how???

Couple of things you need to understand here:
Promises don't make something asynchronous - they are a notification mechanism for something that's already asynchronous. Promises notify you of the success or failure of an operation that's already asynchronous.
Callback function of the promise constructor is called synchronously; it is called synchronously to start an operation that's already asynchronous.
In your case, promise constructor contains synchronous code; this is why it blocks other code.
Moving the loop inside the callback function of then executes the loop asynchronously because the callback function of then is invoked asynchronously.
Note: Even though the callback function of then method is invoked asynchronously, once it starts executing, no other code will execute until the synchronous code inside the callback is executed completely.

In both cases, the for loop will block the main event loop. The difference is when the for loop starts.
In the first case, the function passed to new Promise is executed immediately. It therefore blocks the rest of the function that creates it.
In the second case, the function passed to then() is queued up to run next time the event loop is free. The function that created it therefore finishes first and then the for loop runs. (If that's the goal, then its a pretty unintuitive approach, and using setImmediate (in browsers) or Node's process.nextTick() would be neater.)
If you wanted to shunt the work done by the for loop off the main event loop so it didn't block anything you would use a Web Worker or Worker thread depending on what JavaScript runtime you were using.

Is non-parallel access to a method in node JS guaranteed?

Javascript is single threaded and - Node.js uses an asynchronous event-driven design pattern, which means that multiple actions are taken at the same time while executing a program.
With this in mind, I have a pseudo code:
myFunction() // main flow
var httpCallMade = false // a global variable
async myFunction() {
const someData = await callDB() // LINE 1 network call
renderMethod() // LINE 2 flow1
}
redisPubSubEventHandler() { // a method that is called from redis subscription asynchronously somewhere from a background task in the program
renderMethod() // LINE 3 flow2
}
renderMethod(){
if(!httpCallMade) {
httpCallMade = true //set a global flag
const res = makeHTTPCall() // an asynchronous network call. returns a promise.
} // I want to ensure that this block is "synchronized" and is not acessible by flow1 and flow2 simultaneously!
}
myFunction() is called in the main thread - while redisPubSubEventHandler() is called asynchronously from a background task in the program. Both flows would end in calling renderMethod(). The idea is to ensure makeHTTPCall() (inside renderMethod) is only allowed to be called once
Is it guaranteed that renderMethod() would never be executed in parallel by LINE2 and LINE3 at the same time? My understanding is that as soon as renderMethod() is executed - event loop will not allow anything else to happen in server - which guarantees that it is only executed once at a given time (even if it had a network call without await).
Is this understanding correct?
If not, how do I make synchronize/lock entry to renderMethod?

Javascript is single-threaded. Therefore, unless you are deliberately using threads (eg. worker_threads in node.js) no function in the current thread can be executed by two parallel threads at the same time.
This explains why javascript has no mutex or semaphore capability - because generally it is not needed (note: you can still have race conditions because asynchronous code may be executed in a sequence you did not expect).
There is a general confusion that asynchronous code means parallel code execution (multi-threaded). It can but most of the time when a system is labeled as asynchronous or non-blocking or event-oriented INSTEAD of multi-threaded it often means that the system is single-threaded.
In this case asynchronous means parallel WAIT. Not parallel code execution. Code is always executed sequentially - only, due to the ability of waiting in parallel you may not always know the sequence the code is executed in.
There are parts of javascript that execute in a separate thread. Modern browsers execute each tab and iframe in its own thread (but each tab or iframe are themselves single-threaded). But script cannot cross tabs, windows or iframes. So this is a non-issue. Script may access objects inside iframes but this is done via an API and the script itself cannot execute in the foreign iframe.
Node.js and some browsers also do DNS queries in a separate thread because there is no standardized cross-platform non-blocking API for DNS queries. But this is C code and not your javascript code. Your only interaction with this kind of multi-threading is when you pass a URL to fetch() or XMLHttpRequest().
Node.js also implement file I/O, zip compression and cryptographic functions in separate threads but again this is C code, not your javascript code. All results from these separate threads are returned to you asynchronously via the event loop so by the time your javascript code process the result we are back to executing sequentially in the main thread.
Finally both node.js and browsers have worker APIs (web workers for browsers and worker threads for node.js). However, both these API use message passing to transfer data (in node only a pointer is passed in the message thus the underlying memory is shared) and it still protects functions from having their variables overwritten by another thread.
In your code, both myFunction() and redisPubSubEventHandler() run in the main thread. It works like this:
myFunction() is called, it returns immediately when it encounters the await.
a bunch of functions are declared and compiled.
we reach the end of your script:
// I want to ensure that this method is "synchronized" and is not called by flow1 and flow2 simultaneously!
}
<----- we reach here
now that we have reached the end of script we enter the event loop...
either the callDB or the redis event completes, our process gets woken up
the event loop figures out which handler to call based on what event happened
either the await returns and call renderMethod() or redisPubSubEventHandler() gets executed and call renderMethod()
In either case both your renderMethod() calls will execute on the main thread. Thus it is impossible for renderMethod() to run in parallel.
It is possible for renderMethod() to be half executed and another call to renderMethod() happens IF it contains the await keyword. This is because the first call is suspended at the await allowing the interpreter to call renderMethod() again before the first call completes. But note that even in this case you are only in trouble if you have an await between if.. and httpCallMade = true.

You need to differentiate between synchronous and asynchronous, and single- and multi-threaded.
JavaScript is single-threaded so no two lines of the same execution context can run at the same time.
But JavaScript allows asynchronous code execution (await/async), so the code in the execution context does not need to be in the order it appears in the code but that different parts of the code can be executed interleaved (not overlapped) - which could be called "running in parallel", even so, I think this is misleading.
event-driven design pattern, which means that multiple actions are taken at the same time while executing a program.
There are certain actions that can happen at the same time, like IO, multiprocessing (WebWorkers), but that is (with respect to JavaScript Code execution) not multi-threaded.
Is it guaranteed that renderMethod() would never be executed in parallel by LINE2 and LINE3 at the same time?
Depends on what you define as parallel at the same time.
Parts of logic you describe in renderMethod() will (as you do the request asynchronously) run interleaved, so renderMethod(){ if(!httpCallMade) { could be executed multiple times before you get the response (not the Promise) back from makeHTTPCall but the code lines will never executed at the same time.
My understanding is that as soon as renderMethod() is executed - event loop will not allow anything else to happen in server - which guarantees that it is only executed once at a given time (even if it had a network call without await).
The problem here is, that you somehow need to get the data from your async response.
Therefore you either need to mark your function as async and use const res = await makeHTTPCall() this would allow code interleaving at the point of await. Or use .then(…) with a callback, which will be executed asynchronously at a later point (after you left the function)
But from the beginning of the function to the first await other the .then not interleaving could take place.
So your httpCallMade = true would prevent that another makeHTTPCall could take place, before the currently running is finished, under the assumption that you set httpCallMade to false only when the request is finished (in .then callback, or after the await)
// I want to ensure that this method is "synchronized" and is not called by flow1 and flow2 simultaneously!
As soon as a get a result in an asynchronous way, you can't go back to synchronous code execution. So you need to have a guard like httpCallMade to prevent that the logic described in renderMethod can run multiple times interleaved.

Your question really comes down to:
Given this code:
var flag = false;
function f() {
if (!flag) {
flag = true;
console.log("hello");
}
}
and considering that flag is not modified anywhere else, and many different, asynchronous events may call this function f...:
Can "hello" be printed twice?
The answer is no: if this runs on an ECMAScript compliant JS engine, then the call stack must be empty first before the next job is pulled from an event/job queue. Asynchronous tasks/reactions are pushed on an event queue. They don't execute before the currently executing JavaScript has run to completion, i.e. up until the call stack is empty. So they never interrupt running JavaScript code pre-emptively.
This is true even if these asynchronous tasks/events/jobs are scheduled by other threads, lower-level non-JS code,...etc. They all must wait their turn to be consumed by the JS engine. And this will happen one after the other.
For more information, see the ECMAScript specification on "Job". For instance 8.4 Jobs and Host Operations to Enqueue Jobs:
A Job is an abstract closure with no parameters that initiates an ECMAScript computation when no other ECMAScript computation is currently in progress.
[...]
Only one Job may be actively undergoing evaluation at any point in time.
Once evaluation of a Job starts, it must run to completion before evaluation of any other Job starts.
For example, promises generate such jobs -- See 25.6.1.3.2 Promise Resolve Functions:
When a promise resolve function is called with argument resolution, the following steps are taken:
[...]
Perform HostEnqueuePromiseJob(job.[[Job]], job.[[Realm]]).

It sounds like you want to do something like a 'debounce', where any event will cause makeHttpCall() execute, but it should only be executing once at a time, and should execute again after the last call if another event has occurred while it was executing. So like this:
DB Call is made, and makeHttpCall() should execute
While makeHttpCall() is executing, you get a redis pub/sub event that should execute makeHttpCall() again, but that is delayed because it is already executing
Still before the first call is done, another DB call is made and requires makeHttpCall() to execute again. But even though you have received two events, you only need to have it called one time to update something with the most recent information you have.
The first call to makeHttpCall() finishes, but since there have been two events, you need to make a call again.
const makeHttpCall = () => new Promise(resolve => {
// resolve after 2 seconds
setTimeout(resolve, 2000);
});
// returns a function to call that will call your function
const createDebouncer = (fn) => {
let eventCounter = 0;
let inProgress = false;
const execute = () => {
if (inProgress) {
eventCounter++;
console.log('execute() called, but call is in progress.');
console.log(`There are now ${eventCounter} events since last call.`);
return;
}
console.log(`Executing... There have been ${eventCounter} events.`);
eventCounter = 0;
inProgress = true;
fn()
.then(() => {
console.log('async function call completed!');
inProgress = false;
if (eventCounter > 0) {
// make another call if there are pending events since the last call
execute();
}
});
}
return execute;
}
let debouncer = createDebouncer(makeHttpCall);
document.getElementById('buttonDoEvent').addEventListener('click', () => {
debouncer();
});
<button id="buttonDoEvent">Do Event</button>

Difference between async await in python vs JavaScript

Note: this is not about multi threading or multi processing. This question is regarding a single process and single thread.
Python async.io and JavaScript async both are single thread concepts.
In python, async.io, we can use async await keywords to create a function so that when this function is invoked multiple times (via gather) they get executed concurrently. The way it works is that when an await keyword is encountered, other tasks can execute. As explained here we apply the async await keywords to function that we would like to execute concurrently. However while these tasks are running concurrently, the main thread is blocked.
In JavaScript async has evolved from callbacks, promise, async/await. In the main program, when async is encountered, then the function is sent to the event loop (where the function execution begins) and the main thread can continue working. Any subsequent async function also gets added to the event loop. Inside the event loop when the function execution encountered an await then other function is given a chance to execute untill await in encountered.
To get this behaviour in python, that is - allow main thread to continue while executing child tasks the only option is multithreading/multiprocessing. Because once we start the child thread/process, and untill we call .join the main thread is not blocked.
Is there anyway by which the python's async.io can make the main thread non blocking? If not, then is this the fundamental difference between async concept in JavaScript and python?

when async is encountered, then the function is sent to the event loop and the main thread can continue working.
This is close, but not quite right. In Javascript, execution won't stop until the callstack has been emptied - the await keyword will suspend the execution of a particular function until an event triggers, and in the mean time, control returns to its caller. This means the first part of any async function will execute as soon as it is called (it's not immediately put into the event loop), and will only pause as soon as an await is hit.
To get this behaviour in python, that is - allow main thread to continue while executing child tasks the only option is multithreading/multiprocessing.
The difference here is that by default, Javascript always has an event loop and python does not. In other words, python has an on/off switch for asynchronous programming while Javascript does not. When you run something such as loop.run_forever(), you're basically flipping the event loop on, and execution won't continue where you left off until the event loop gets turned back off. (calling it a "thread" isn't quite the right word here, as it's all single-threaded, as you already acknowledged. Instead, we generally call each task that we queue up, well, a "task")
You're asking if there's a way to let your code continue execution after starting up the event loop. I'm pretty sure the answer is no, nor should it be needed. Whatever you want to execute after the event loop has started can just be executed within the event loop.
If you want your python program to act more like Javascript, then the first thing you do can be to start up an event loop, and then any further logic can be placed within the first task that the event loop executes. In Javascript, this boiler plate essentially happens for you, and your source code is effectively that first task that's queued up in the event loop.
Update:
Because there seems to be some confusion with how the Javascript event loop works, I'll try to explain it a little further.
Remember that an event loop is simply a system where, when certain events happen, a block of synchronous code can be queued up to run as soon as the thread is not busy.
So let's see what the event loop does for a simple program like this:
// This async function will resolve
// after the number of ms provided has passed
const wait = ms => { ... }
async function main() {
console.log(2)
await wait(100)
console.log(4)
}
console.log(1)
main()
console.log(3)
When Javascript begins executing the above program, it'll begin with a single task queued up in it's "run these things when you're not busy" queue. This item is the whole program.
So, it'll start at the top, defining whatever needs to be defined, executes console.log(1), call the main function, enters into it and runs console.log(2), calls wait() which will conceptually cause a background timer to start, wait() will return a promise which we then await, at which point we immediately go back to the caller of main, main wasn't awaited so execution continues to console.log(3), until we finally finish at the end of the file. That whole path (from defining functions to console.log(3)) is a single, non-interruptible task. Even if another task got queued up, Javascript wouldn't stop to handle that task until it finished this chunk of synchronous logic.
Later on, our countdown timer will finish, and another task will go into our queue, which will cause our main() function to continue execution. The same logic as before applies here - our execution path could enter and exit other async functions, and will only stop when it reaches the end of, in this case, the main function (even hitting an await keywords doesn't actually make this line of syncrounous logic stop, it just makes it jump back to the caller). The execution of a single task doesn't stop until the callstack has been emptied, and when execution is continuing from an async function, the first entry of the callstack starts at that particular async function.
Python's async/await follows these same rules, except for the fact that in Python, the event loop isn't running by default.

javascript
const wait = async (s) => {
setTimeout(() => {
console.log("wating " + s + "s")
}, s * 1000)
}
async function read_file() {
console.log("initial read_file sleep(2.1)")
await wait(2)
console.log("read_file 1/2 wait(2)")
await wait(0.1)
console.log("read_file 2/2 wait(0.1)")
}
async function read_api() {
console.log("initial read_api wait(2)")
await wait(2)
console.log("read_api whole wait(2)")
}
read_file()
console.log("does not block")
read_api()
console.log("the second time, won't block")
// initial read_file sleep(2.1)
// does not block
// initial read_api wait(2)
// the second time, won't block
// read_file 1/2 wait(2)
// read_api whole wait(2)
// read_file 2/2 wait(0.1)
// !!! Wait a moment
// wating 0.1s
// wating 2s
// wating 2s
python
import asyncio
async def read_file():
print("initial read_file asyncio.sleep(2 + 0.1)")
await asyncio.sleep(2)
print("read_file 1/2 asyncio.sleep(2)")
await asyncio.sleep(0.1)
print("read_file 2/2 asyncio.sleep(0.1)")
async def read_api():
print("initial read_api asyncio.sleep(2)")
await asyncio.sleep(2)
print("read_api whole asyncio.sleep(2)")
async def gather():
await asyncio.gather(
asyncio.create_task(read_file()),
asyncio.create_task(read_api()))
asyncio.run(gather())
"""
initial read_file asyncio.sleep(2.1)
initial read_api asyncio.sleep(2)
!!! Wait a moment
read_file 1/2 asyncio.sleep(2)
read_api whole asyncio.sleep(2)
read_file 2/2 asyncio.sleep(0.1)
"""
await scope:
javascript: After the method is executed, wait for the Promise to resolve
await wait(2) Just wait(2) inside is guaranteed to be synchronous (or wait)
python: Suspend method for other methods to execute
await asyncio.sleep(2) Method read_file will release resources and suspend
btw, javascript's await/async is just Promise syntactic sugar

does async/await nesting have performance consequences?

lets consider we have N async functions like:
async function1(){
return await fetch(...);
}
.
.
.
async functionN(){
return await fetch(...);
}
then we have a function like:
async wrapper(){
let var = await fetch(...);
function1();
.
.
.
functionN();
}
would this create one big microtask queue that would effectively block ui thread going to next task before all called functions resolved their awaits?

There is nothing in the microtask queue until promises resolve. Until then other tasks and (UI) events can be processed.
This is because the await operator will make the corresponding async function return immediately, allowing other JS code to execute. In your case the promise is returned by fetch, which in practice will not resolve immediately. So there is nothing blocking here.
Then when the HTTP response makes the fetch promise resolve, a microtask will indeed be created, which, when when executed will restore the corresponding async function's execution context. Your example function has nothing else to do, so that is quickly done.
Note that it does not matter whether this function was originally called from within some other function: at this stage, only that particular function's execution context (in which an awaited promise resolved) is restored without any pre-existing callstack. So it does not return again to the wrapping function. That already happened in the first phase and will not happen again.
Then again there is free event processing until the next fetch promise resolves. And so it continues.