Javascript is single threaded and - Node.js uses an asynchronous event-driven design pattern, which means that multiple actions are taken at the same time while executing a program.
With this in mind, I have a pseudo code:
myFunction() // main flow
var httpCallMade = false // a global variable
async myFunction() {
const someData = await callDB() // LINE 1 network call
renderMethod() // LINE 2 flow1
}
redisPubSubEventHandler() { // a method that is called from redis subscription asynchronously somewhere from a background task in the program
renderMethod() // LINE 3 flow2
}
renderMethod(){
if(!httpCallMade) {
httpCallMade = true //set a global flag
const res = makeHTTPCall() // an asynchronous network call. returns a promise.
} // I want to ensure that this block is "synchronized" and is not acessible by flow1 and flow2 simultaneously!
}
myFunction() is called in the main thread - while redisPubSubEventHandler() is called asynchronously from a background task in the program. Both flows would end in calling renderMethod(). The idea is to ensure makeHTTPCall() (inside renderMethod) is only allowed to be called once
Is it guaranteed that renderMethod() would never be executed in parallel by LINE2 and LINE3 at the same time? My understanding is that as soon as renderMethod() is executed - event loop will not allow anything else to happen in server - which guarantees that it is only executed once at a given time (even if it had a network call without await).
Is this understanding correct?
If not, how do I make synchronize/lock entry to renderMethod?
Javascript is single-threaded. Therefore, unless you are deliberately using threads (eg. worker_threads in node.js) no function in the current thread can be executed by two parallel threads at the same time.
This explains why javascript has no mutex or semaphore capability - because generally it is not needed (note: you can still have race conditions because asynchronous code may be executed in a sequence you did not expect).
There is a general confusion that asynchronous code means parallel code execution (multi-threaded). It can but most of the time when a system is labeled as asynchronous or non-blocking or event-oriented INSTEAD of multi-threaded it often means that the system is single-threaded.
In this case asynchronous means parallel WAIT. Not parallel code execution. Code is always executed sequentially - only, due to the ability of waiting in parallel you may not always know the sequence the code is executed in.
There are parts of javascript that execute in a separate thread. Modern browsers execute each tab and iframe in its own thread (but each tab or iframe are themselves single-threaded). But script cannot cross tabs, windows or iframes. So this is a non-issue. Script may access objects inside iframes but this is done via an API and the script itself cannot execute in the foreign iframe.
Node.js and some browsers also do DNS queries in a separate thread because there is no standardized cross-platform non-blocking API for DNS queries. But this is C code and not your javascript code. Your only interaction with this kind of multi-threading is when you pass a URL to fetch() or XMLHttpRequest().
Node.js also implement file I/O, zip compression and cryptographic functions in separate threads but again this is C code, not your javascript code. All results from these separate threads are returned to you asynchronously via the event loop so by the time your javascript code process the result we are back to executing sequentially in the main thread.
Finally both node.js and browsers have worker APIs (web workers for browsers and worker threads for node.js). However, both these API use message passing to transfer data (in node only a pointer is passed in the message thus the underlying memory is shared) and it still protects functions from having their variables overwritten by another thread.
In your code, both myFunction() and redisPubSubEventHandler() run in the main thread. It works like this:
myFunction() is called, it returns immediately when it encounters the await.
a bunch of functions are declared and compiled.
we reach the end of your script:
// I want to ensure that this method is "synchronized" and is not called by flow1 and flow2 simultaneously!
}
<----- we reach here
now that we have reached the end of script we enter the event loop...
either the callDB or the redis event completes, our process gets woken up
the event loop figures out which handler to call based on what event happened
either the await returns and call renderMethod() or redisPubSubEventHandler() gets executed and call renderMethod()
In either case both your renderMethod() calls will execute on the main thread. Thus it is impossible for renderMethod() to run in parallel.
It is possible for renderMethod() to be half executed and another call to renderMethod() happens IF it contains the await keyword. This is because the first call is suspended at the await allowing the interpreter to call renderMethod() again before the first call completes. But note that even in this case you are only in trouble if you have an await between if.. and httpCallMade = true.
You need to differentiate between synchronous and asynchronous, and single- and multi-threaded.
JavaScript is single-threaded so no two lines of the same execution context can run at the same time.
But JavaScript allows asynchronous code execution (await/async), so the code in the execution context does not need to be in the order it appears in the code but that different parts of the code can be executed interleaved (not overlapped) - which could be called "running in parallel", even so, I think this is misleading.
event-driven design pattern, which means that multiple actions are taken at the same time while executing a program.
There are certain actions that can happen at the same time, like IO, multiprocessing (WebWorkers), but that is (with respect to JavaScript Code execution) not multi-threaded.
Is it guaranteed that renderMethod() would never be executed in parallel by LINE2 and LINE3 at the same time?
Depends on what you define as parallel at the same time.
Parts of logic you describe in renderMethod() will (as you do the request asynchronously) run interleaved, so renderMethod(){ if(!httpCallMade) { could be executed multiple times before you get the response (not the Promise) back from makeHTTPCall but the code lines will never executed at the same time.
My understanding is that as soon as renderMethod() is executed - event loop will not allow anything else to happen in server - which guarantees that it is only executed once at a given time (even if it had a network call without await).
The problem here is, that you somehow need to get the data from your async response.
Therefore you either need to mark your function as async and use const res = await makeHTTPCall() this would allow code interleaving at the point of await. Or use .then(…) with a callback, which will be executed asynchronously at a later point (after you left the function)
But from the beginning of the function to the first await other the .then not interleaving could take place.
So your httpCallMade = true would prevent that another makeHTTPCall could take place, before the currently running is finished, under the assumption that you set httpCallMade to false only when the request is finished (in .then callback, or after the await)
// I want to ensure that this method is "synchronized" and is not called by flow1 and flow2 simultaneously!
As soon as a get a result in an asynchronous way, you can't go back to synchronous code execution. So you need to have a guard like httpCallMade to prevent that the logic described in renderMethod can run multiple times interleaved.
Your question really comes down to:
Given this code:
var flag = false;
function f() {
if (!flag) {
flag = true;
console.log("hello");
}
}
and considering that flag is not modified anywhere else, and many different, asynchronous events may call this function f...:
Can "hello" be printed twice?
The answer is no: if this runs on an ECMAScript compliant JS engine, then the call stack must be empty first before the next job is pulled from an event/job queue. Asynchronous tasks/reactions are pushed on an event queue. They don't execute before the currently executing JavaScript has run to completion, i.e. up until the call stack is empty. So they never interrupt running JavaScript code pre-emptively.
This is true even if these asynchronous tasks/events/jobs are scheduled by other threads, lower-level non-JS code,...etc. They all must wait their turn to be consumed by the JS engine. And this will happen one after the other.
For more information, see the ECMAScript specification on "Job". For instance 8.4 Jobs and Host Operations to Enqueue Jobs:
A Job is an abstract closure with no parameters that initiates an ECMAScript computation when no other ECMAScript computation is currently in progress.
[...]
Only one Job may be actively undergoing evaluation at any point in time.
Once evaluation of a Job starts, it must run to completion before evaluation of any other Job starts.
For example, promises generate such jobs -- See 25.6.1.3.2 Promise Resolve Functions:
When a promise resolve function is called with argument resolution, the following steps are taken:
[...]
Perform HostEnqueuePromiseJob(job.[[Job]], job.[[Realm]]).
It sounds like you want to do something like a 'debounce', where any event will cause makeHttpCall() execute, but it should only be executing once at a time, and should execute again after the last call if another event has occurred while it was executing. So like this:
DB Call is made, and makeHttpCall() should execute
While makeHttpCall() is executing, you get a redis pub/sub event that should execute makeHttpCall() again, but that is delayed because it is already executing
Still before the first call is done, another DB call is made and requires makeHttpCall() to execute again. But even though you have received two events, you only need to have it called one time to update something with the most recent information you have.
The first call to makeHttpCall() finishes, but since there have been two events, you need to make a call again.
const makeHttpCall = () => new Promise(resolve => {
// resolve after 2 seconds
setTimeout(resolve, 2000);
});
// returns a function to call that will call your function
const createDebouncer = (fn) => {
let eventCounter = 0;
let inProgress = false;
const execute = () => {
if (inProgress) {
eventCounter++;
console.log('execute() called, but call is in progress.');
console.log(`There are now ${eventCounter} events since last call.`);
return;
}
console.log(`Executing... There have been ${eventCounter} events.`);
eventCounter = 0;
inProgress = true;
fn()
.then(() => {
console.log('async function call completed!');
inProgress = false;
if (eventCounter > 0) {
// make another call if there are pending events since the last call
execute();
}
});
}
return execute;
}
let debouncer = createDebouncer(makeHttpCall);
document.getElementById('buttonDoEvent').addEventListener('click', () => {
debouncer();
});
<button id="buttonDoEvent">Do Event</button>
Related
This is a predominantly theoretical question as I guess I would not use the fetched data outside of the .then() method.
I was just checking the behaviour of fetch.then() out of curiosity as I like to test and understand the principles of the languages I learn.
As expected
setTimeout(()=>{
console.log("myGlobVar in setTimeOut");
console.log(myGlobVar)}, 1000);
returns the fetched object, provided I use a delay long enough in setTimeOut (about 40ms in this case)
But I'm curious why the
console.log("myGlobVar after for loop");
console.log(myGlobVar); //returns undefined
does not work.
My rationale here was that running the loop long enough, would give enough time to fetch() to fetch the data and assign it to myGlobVar, which doesn't happen no matter how long it runs.
Actually -counterintuitively- for extremely high numbers in the loop (1000000000) even the
setTimeout(()=>{
console.log("myGlobVar in setTimeOut");
console.log(myGlobVar)}, 1000);
returns undefined.
<script>
"use strict"
let myGlobVar;
function loadPeaks() {
console.log("loadPeaks");
fetch('soundofChengdu.json')
.then(response => {
if (!response.ok) {
throw new Error("HTTP error: " + response.status);
}
return response.json();
})
.then(peaks => {
// wavesurfer.load('soundofChengdu.mp3', peaks.data);
myGlobVar= peaks;
})
.catch((e) => {
console.log({errorIs: e})
// console.error('error', e);
});
}
loadPeaks();
console.log("myGlobVar without waiting");
console.log(myGlobVar); //returns undefined
setTimeout(()=>{console.log("myGlobVar in setTimeOut"); console.log(myGlobVar)}, 1000); //returns undefined under +-40 ms, works above.
let b;
console.log("loop running");
for (let a=0; a<100000000; a++) {
b=b+a;
}
console.log("myGlobVar after for loop");
console.log(myGlobVar); //returns undefined
</script>
The first part is quite usual, you blocked the event-loop synchronously, no other task or even microtask can be executed in between so your variable is not set.
The second part about setTimeout() firing before the request's Promise gets resolved is a bit more interesting though, what you've discovered here is the task prioritization system.
All tasks don't have the same priority, and browsers are free to execute one kind of task before another.
For instance here, your browser gives more importance to timer tasks than to network tasks. And for the ones wondering "but fetch().then() is a microtask and should have higher priority", the returned Promise is resolved in a "normal" task (see this answer of mine), and that task is subject to the task prioritization system.
The code after loadPeaks() is synchronously called, meaning that the code will not wait for loadPeaks() to finish running and will run completely before loadPeaks() finishes.
This is the same for the for loop. Regardless of the iteration count, the for loop will finish before the request finishes. You can test this by adding a console.log() statement in your second then() call and see when it is printed in relation to the rest of your log statements.
Promise.then() on the other hand, waits for the request to finish. In your case, fetch() will run first and the then() calls will run sequentially, waiting for the previous call to finish before running.
My rationale here was that running the loop long enough, would give enough time to fetch() to fetch the data and assign it to myGlobVar, which doesn't happen no matter how long it runs.
JavaScript is actually strictly single-threaded, unlike most other languages you're probably familiar with. Instead of having threads running in parallel, it uses an Event Loop pattern where each operation runs, pushes follow-up operations onto the queue, and then yields back to the loop. I'll quote from the article I just linked to explain how this affects your code (emphasis mine):
"Run-to-Completion"
Each message is processed completely before any other message is processed.
This offers some nice properties when reasoning about your program, including the fact that whenever a function runs, it cannot be preempted and will run entirely before any other code runs (and can modify data the function manipulates). This differs from C, for instance, where if a function runs in a thread, it may be stopped at any point by the runtime system to run some other code in another thread.
This explains why the code after the for loop never has access to that global variable no matter how high the iteration count is: the event loop is still executing your original function, so none of those promise handlers have had any chance to execute.
#Kaiido's answer for your second question is much better than what I was going to write, so I won't bother. I'll leave the above up just in case a more basic explanation helps.
I couldn't understand the execution order of the two codes below.
async function main(test)
{
while(1) console.log(test);
}
main("a");
main("b");
This code above logs infinite "a".
async function main(test)
{
while(1) await console.log(test);
}
main("a");
main("b");
While this code above logs infinite "a" and "b".
I wanted to understand async/await better but this behaviour above made me confused. Like how are those log functions handled by event loop?
An async function runs synchronously until the first await, return, or implicit return (the code falling off the end of the function). That's so it can start the asynchronous process that it will later report the completion of (by settling the promise it returns). So your first example logs a forever because that's done by the synchronous part of the first call to main; execution never reaches the second call to main.
Your second example introduces asynchronousness by adding await (it doesn't matter that what await is awaiting isn't a promise [console.log returns undefined]; it gets wrapped in one). That means that the first call to main runs synchronously until after the first console.log, then stops and returns a promise while it waits for the promise wrapped around undefined to settle.¹ That allows the second call to main to go forward, doing the same thing. At that point, the code making the calls to main is complete and any pending promise callbacks can be executed. The first pending promise callback is for the await in the first call to main, so that call does another loop iteration; the second pending promise is for the await in the second call to main, so it does another loop iteration; and so on, and so on, and so on.
You mentioned "concurrency." The two calls to main in your second example do run concurrently but are not multi-threaded; instead, they each alternately get access to the one main thread briefly until their next await. JavaScript works on a model of one thread per "realm" (loosely, per global environment). So you don't have to worry about threading issues like reordered operations or stale caches like you would in a multi-threaded environment (unless you're using shared memory, but you aren't in that code); there's only one thread.
¹ Technically, the promise is already settled in this case before await starts awaiting it (because it's just a promise wrapped around undefined), but when you attach to a promise, it never triggers your code synchronously, even if the promise is already settled.
Using Asynchronous functions within Javascript usually resolves data that is not readily available. Examples include calling from a database and scraping web pages for data.
The code provided endlessly loops because there is no end to the said loop (while). In the context of the code provided, while(1) will always remain true, as there is no condition to state otherwise. In practical use, loops are designed to cease when a condition is met.
Here's a basic example of using a while loop sourced from MDN Docs covering break
let n = 0;
while (n < 3) {
n++;
}
console.log(n);
// expected output: 3
To better understand asynchronous programming within Javascript, I highly recommend the MDN Docs on async functions.
To answer your question regarding console logging's interactions with await: JS will 'freeze' the script when it reads an await and work on completing its defined actions, then returns to read the following line. In the context of the code, Javascript is doing its job. It circles back to a lack of a break or condition for the loop.
Note: this is not about multi threading or multi processing. This question is regarding a single process and single thread.
Python async.io and JavaScript async both are single thread concepts.
In python, async.io, we can use async await keywords to create a function so that when this function is invoked multiple times (via gather) they get executed concurrently. The way it works is that when an await keyword is encountered, other tasks can execute. As explained here we apply the async await keywords to function that we would like to execute concurrently. However while these tasks are running concurrently, the main thread is blocked.
In JavaScript async has evolved from callbacks, promise, async/await. In the main program, when async is encountered, then the function is sent to the event loop (where the function execution begins) and the main thread can continue working. Any subsequent async function also gets added to the event loop. Inside the event loop when the function execution encountered an await then other function is given a chance to execute untill await in encountered.
To get this behaviour in python, that is - allow main thread to continue while executing child tasks the only option is multithreading/multiprocessing. Because once we start the child thread/process, and untill we call .join the main thread is not blocked.
Is there anyway by which the python's async.io can make the main thread non blocking? If not, then is this the fundamental difference between async concept in JavaScript and python?
when async is encountered, then the function is sent to the event loop and the main thread can continue working.
This is close, but not quite right. In Javascript, execution won't stop until the callstack has been emptied - the await keyword will suspend the execution of a particular function until an event triggers, and in the mean time, control returns to its caller. This means the first part of any async function will execute as soon as it is called (it's not immediately put into the event loop), and will only pause as soon as an await is hit.
To get this behaviour in python, that is - allow main thread to continue while executing child tasks the only option is multithreading/multiprocessing.
The difference here is that by default, Javascript always has an event loop and python does not. In other words, python has an on/off switch for asynchronous programming while Javascript does not. When you run something such as loop.run_forever(), you're basically flipping the event loop on, and execution won't continue where you left off until the event loop gets turned back off. (calling it a "thread" isn't quite the right word here, as it's all single-threaded, as you already acknowledged. Instead, we generally call each task that we queue up, well, a "task")
You're asking if there's a way to let your code continue execution after starting up the event loop. I'm pretty sure the answer is no, nor should it be needed. Whatever you want to execute after the event loop has started can just be executed within the event loop.
If you want your python program to act more like Javascript, then the first thing you do can be to start up an event loop, and then any further logic can be placed within the first task that the event loop executes. In Javascript, this boiler plate essentially happens for you, and your source code is effectively that first task that's queued up in the event loop.
Update:
Because there seems to be some confusion with how the Javascript event loop works, I'll try to explain it a little further.
Remember that an event loop is simply a system where, when certain events happen, a block of synchronous code can be queued up to run as soon as the thread is not busy.
So let's see what the event loop does for a simple program like this:
// This async function will resolve
// after the number of ms provided has passed
const wait = ms => { ... }
async function main() {
console.log(2)
await wait(100)
console.log(4)
}
console.log(1)
main()
console.log(3)
When Javascript begins executing the above program, it'll begin with a single task queued up in it's "run these things when you're not busy" queue. This item is the whole program.
So, it'll start at the top, defining whatever needs to be defined, executes console.log(1), call the main function, enters into it and runs console.log(2), calls wait() which will conceptually cause a background timer to start, wait() will return a promise which we then await, at which point we immediately go back to the caller of main, main wasn't awaited so execution continues to console.log(3), until we finally finish at the end of the file. That whole path (from defining functions to console.log(3)) is a single, non-interruptible task. Even if another task got queued up, Javascript wouldn't stop to handle that task until it finished this chunk of synchronous logic.
Later on, our countdown timer will finish, and another task will go into our queue, which will cause our main() function to continue execution. The same logic as before applies here - our execution path could enter and exit other async functions, and will only stop when it reaches the end of, in this case, the main function (even hitting an await keywords doesn't actually make this line of syncrounous logic stop, it just makes it jump back to the caller). The execution of a single task doesn't stop until the callstack has been emptied, and when execution is continuing from an async function, the first entry of the callstack starts at that particular async function.
Python's async/await follows these same rules, except for the fact that in Python, the event loop isn't running by default.
javascript
const wait = async (s) => {
setTimeout(() => {
console.log("wating " + s + "s")
}, s * 1000)
}
async function read_file() {
console.log("initial read_file sleep(2.1)")
await wait(2)
console.log("read_file 1/2 wait(2)")
await wait(0.1)
console.log("read_file 2/2 wait(0.1)")
}
async function read_api() {
console.log("initial read_api wait(2)")
await wait(2)
console.log("read_api whole wait(2)")
}
read_file()
console.log("does not block")
read_api()
console.log("the second time, won't block")
// initial read_file sleep(2.1)
// does not block
// initial read_api wait(2)
// the second time, won't block
// read_file 1/2 wait(2)
// read_api whole wait(2)
// read_file 2/2 wait(0.1)
// !!! Wait a moment
// wating 0.1s
// wating 2s
// wating 2s
python
import asyncio
async def read_file():
print("initial read_file asyncio.sleep(2 + 0.1)")
await asyncio.sleep(2)
print("read_file 1/2 asyncio.sleep(2)")
await asyncio.sleep(0.1)
print("read_file 2/2 asyncio.sleep(0.1)")
async def read_api():
print("initial read_api asyncio.sleep(2)")
await asyncio.sleep(2)
print("read_api whole asyncio.sleep(2)")
async def gather():
await asyncio.gather(
asyncio.create_task(read_file()),
asyncio.create_task(read_api()))
asyncio.run(gather())
"""
initial read_file asyncio.sleep(2.1)
initial read_api asyncio.sleep(2)
!!! Wait a moment
read_file 1/2 asyncio.sleep(2)
read_api whole asyncio.sleep(2)
read_file 2/2 asyncio.sleep(0.1)
"""
await scope:
javascript: After the method is executed, wait for the Promise to resolve
await wait(2) Just wait(2) inside is guaranteed to be synchronous (or wait)
python: Suspend method for other methods to execute
await asyncio.sleep(2) Method read_file will release resources and suspend
btw, javascript's await/async is just Promise syntactic sugar
I have alot of syncrhounous functions that i want to execute before that are basic ajax requests, these request will render html to the DOM.
In order to do this i had to execute all of this synchrounous
requests one by one. But i somehow want to these synchrounous functions asynchrounous all at the same time and wait for them to finnish in order to speed things up. This has to happen inside a synchrounous function. But my understanding is that this doesnt work in javascript, but i would like to hear what you guys have to say.
So my attempt was to add all of these synchrounous requests into asynchrounous promises and then do a Promise.all call. I cant wait for the promise.all().then because the main thread will keep on execute the rest of the code after this main synchrounous thread/function. So i wonder if there is a way to block the main thread in order to wait for these asynchrounous calls
heres a short illustration of what im talking about
var syncfunc = () => {
var getPromise = () => {
return new Promise((resolve) => {
var asyncAjaxRequest = async function() {
doSomeStuff();
resolve();
}
})
}
var promises = [getPromse(), getPromse(), getPromse()];
Promise.all(promises);
console.log('i want this console.log to execute after all promises executed doSomeStuff');
/**
*
* Promise.all(promises).then(() => {
// I cant use this, because its script in other files that will execute if i wait like this
})
*/
}
I know .then will execute when all resolves are done, but i basiacally want to block this synchrounous thread waiting for all other asynchrounous to finish.
If i could i would ofcourse change the structure into my needs, but the problem and the reason why im trying to do this is because im using sitevision framework, and want to add some content to the dom before a print module opens the print window. To call every function synchrounous is just not the way to go, its to slow. Ive also tried to set window.print = null to make the print function disabled, and then add the print function back when promises resolves, but it simply doesnt work
You cannot make an asynchronous operation turn into a synchronous one in plain Javascript (without external code). The event driven JS engine just doesn't work that way.
By definition, an asynchronous operation starts the operation (handing execution off to native code) and then returns back to the interpreter which then continues to execute the code that follows. The native code will add an event to JS event queue when it finishes to allow the interpreter event loop to service the completion of the asynchronous operation. If you were the create some sort of "block" such as a semi-infinite while loop, that would "block" the interpreter from executing more code, you end up in a stalemate. The loop that is blocking the interpreter prevents the JS interpreter from ever getting to the point where it can ever process the event that signals the end of the asynchronous operation. So, you have a loop waiting for something to finish, but the thing it's waiting for can't finish until the loop finishes - stalemate.
So, because of the single threaded event loop nature of the JS interpreter, you can't (purely in Javascript) block waiting for the end of an asynchronous operation.
Pretty much always, the correct design is to refactor the surrounding code/infrastructure to work with an asynchronous operation and asynchronous result (callback or promise).
If this is node.js, there are a couple of horrific hacks that can get you this result, but they block the entire interpreter so are almost never a desired design.
The first option involves writing a custom nodejs plugin (async operations done in native code) that provides a blocking interface that just doesn't return until the operation is done.
The second option involves using the synchronous child_process operations (such as child_process.execFileSync() to create a blocking child process, run your code in that child process and then continue when that process finishes.
Both I could consider pretty bad hacks and pretty much never the desired way to solve such a problem. But, I did want to show you what has to be done in order to block for an asynchronous operation (it has to be moved out of Javascript or out of the process).
If you can't figure out how to solve your real problem with non-blocking, asynchronous operations, I'd suggest you post a new question where you describe in detail exactly what the real problem is and we can help you find an asynchronous design that would work for your situation. If you post a link to the new question in a comment here, some of the people engaged here may check in on the new question and attempt to help.
You could use async/await to solve this. This is how you do it:
async function promiseSolver() {
var getPromise = () => {
return new Promise((resolve) => {
var asyncAjaxRequest = async function() {
doSomeStuff();
resolve();
}
})
}
var promises = [getPromse(), getPromse(), getPromse()];
await Promise.all(promises);
console.log('i want this console.log to execute after all promises executed doSomeStuff');
/**
*
* Promise.all(promises).then(() => {
// I cant use this, because its script in other files that will execute if i wait like this
})
*/
}
Basically, your code will wait until the .all is completed and then will continue with processing. Take into consideration that while the code execution is synchronous, the code will be non blocking.
I've been reading nodebeginner
And I came across the following two pieces of code.
The first one:
var result = database.query("SELECT * FROM hugetable");
console.log("Hello World");
The second one:
database.query("SELECT * FROM hugetable", function(rows) {
var result = rows;
});
console.log("Hello World");
I get what they're supposed to do, they query the database to retrieve the answer to the query. And then console.log('Hello world').
The first one is supposedly synchronous code.
And the second one is asynchronous code.
The difference between the two pieces is very vague to me. What would the output be?
Googling on asynchronous programming didn't help me either.
The difference is that in the first example, the program will block in the first line. The next line (console.log) will have to wait.
In the second example, the console.log will be executed WHILE the query is being processed. That is, the query will be processed in the background, while your program is doing other things, and once the query data is ready, you will do whatever you want with it.
So, in a nutshell: The first example will block, while the second won't.
The output of the following two examples:
// Example 1 - Synchronous (blocks)
var result = database.query("SELECT * FROM hugetable");
console.log("Query finished");
console.log("Next line");
// Example 2 - Asynchronous (doesn't block)
database.query("SELECT * FROM hugetable", function(result) {
console.log("Query finished");
});
console.log("Next line");
Would be:
Query finished
Next line
Next line
Query finished
Note
While Node itself is single threaded, there are some task that can run in parallel. For example, File System operations occur in a different process.
That's why Node can do async operations: one thread is doing file system operations, while the main Node thread keeps executing your javascript code. In an event-driven server like Node, the file system thread notifies the main Node thread of certain events such as completion, failure, or progress, along with any data associated with that event (such as the result of a database query or an error message) and the main Node thread decides what to do with that data.
You can read more about this here: How the single threaded non blocking IO model works in Node.js
The difference between these two approaches is as follows:
Synchronous way:
It waits for each operation to complete, after that only it executes the next operation.
For your query:
The console.log() command will not be executed until & unless the query has finished executing to get all the result from Database.
Asynchronous way:
It never waits for each operation to complete, rather it executes all operations in the first GO only. The result of each operation will be handled once the result is available.
For your query:
The console.log() command will be executed soon after the Database.Query() method. While the Database query runs in the background and loads the result once it is finished retrieving the data.
Use cases
If your operations are not doing very heavy lifting like querying huge data from DB then go ahead with Synchronous way otherwise Asynchronous way.
In Asynchronous way you can show some Progress indicator to the user while in background you can continue with your heavy weight works. This is an ideal scenario for GUI apps.
This would become a bit more clear if you add a line to both examples:
var result = database.query("SELECT * FROM hugetable");
console.log(result.length);
console.log("Hello World");
The second one:
database.query("SELECT * FROM hugetable", function(rows) {
var result = rows;
console.log(result.length);
});
console.log("Hello World");
Try running these, and you’ll notice that the first (synchronous) example, the result.length will be printed out BEFORE the 'Hello World' line.
In the second (the asynchronous) example, the result.length will (most likely) be printed AFTER the "Hello World" line.
That's because in the second example, the database.query is run asynchronously in the background, and the script continues straightaway with the "Hello World". The console.log(result.length) is only executed when the database query has completed.
First, I realize I am late in answering this question.
Before discussing synchronous and asynchronous, let us briefly look at how programs run.
In the synchronous case, each statement completes before the next statement is run. In this case the program is evaluated exactly in order of the statements.
This is how asynchronous works in JavaScript. There are two parts in the JavaScript engine, one part that looks at the code and enqueues operations and another that processes the queue. The queue processing happens in one thread, that is why only one operation can happen at a time.
When an asynchronous operation (like the second database query) is seen, the code is parsed and the operation is put in the queue, but in this case a callback is registered to be run when this operation completes. The queue may have many operations in it already. The operation at the front of the queue is processed and removed from the queue. Once the operation for the database query is processed, the request is sent to the database and when complete the callback will be executed on completion. At this time, the queue processor having "handled" the operation moves on the next operation - in this case
console.log("Hello World");
The database query is still being processed, but the console.log operation is at the front of the queue and gets processed. This being a synchronous operation gets executed right away resulting immediately in the output "Hello World". Some time later, the database operation completes, only then the callback registered with the query is called and processed, setting the value of the variable result to rows.
It is possible that one asynchronous operation will result in another asynchronous operation, this second operation will be put in the queue and when it comes to the front of the queue it will be processed. Calling the callback registered with an asynchronous operation is how JavaScript run time returns the outcome of the operation when it is done.
A simple method of knowing which JavaScript operation is asynchronous is to note if it requires a callback - the callback is the code that will get executed when the first operation is complete. In the two examples in the question, we can see only the second case has a callback, so it is the asynchronous operation of the two. It is not always the case because of the different styles of handling the outcome of an asynchronous operation.
To learn more, read about promises. Promises are another way in which the outcome of an asynchronous operation can be handled. The nice thing about promises is that the coding style feels more like synchronous code.
Many libraries like node 'fs', provide both synchronous and asynchronous styles for some operations. In cases where the operation does not take long and is not used a lot - as in the case of reading a config file - the synchronous style operation will result in code that is easier to read.
In the synchronous case, the console.log command is not executed until the SQL query has finished executing.
In the asynchronous case, the console.log command will be directly executed. The result of the query will then be stored by the "callback" function sometime afterwards.
The main difference is with asynchronous programming, you don't stop execution otherwise. You can continue executing other code while the 'request' is being made.
Asynchronous programming in JS:
Synchronous
Stops execution of further code until this is done.
Because it this stoppage of further execution, synchronous code is called 'blocking'. Blocking in the sense that no other code will be executed.
Asynchronous
Execution of this is deferred to the event loop, this is a construct in a JS virtual machine which executes asynchronous functions (after the stack of synchronous functions is empty).
Asynchronous code is called non blocking because it doesn't block further code from running.
Example:
// This function is synchronous
function log(arg) {
console.log(arg)
}
log(1);
// This function is asynchronous
setTimeout(() => {
console.log(2)
}, 0);
log(3)
The example logs 1, 3, 2.
2 is logged last because it is inside a asynchronous function which is executed after the stack is empty.
The function makes the second one asynchronous.
The first one forces the program to wait for each line to finish it's run before the next one can continue. The second one allows each line to run together (and independently) at once.
Languages and frameworks (js, node.js) that allow asynchronous or concurrency is great for things that require real time transmission (eg. chat, stock applications).
Synchronous functions are blocking while asynchronous functions are not. In synchronous functions, statements complete before the next statement is run. In this case, the program is evaluated exactly in order of the statements and execution of the program is paused if one of the statements take a very long time.
Asynchronous functions usually accept a callback as a parameter and execution continue on the next line immediately after the asynchronous function is invoked. The callback is only invoked when the asynchronous operation is complete and the call stack is empty. Heavy duty operations such as loading data from a web server or querying a database should be done asynchronously so that the main thread can continue executing other operations instead of blocking until that long operation to complete (in the case of browsers, the UI will freeze).
Orginal Posted on Github: Link
Sync Programming
Programming languages like C, C#, Java are sync programming, what so ever you write will be execute in order of your writing.
-GET DATA FROM SQL.
//Suppose fetching data take 500 msec
-PERFORM SOME OTHER FUNCTION.
//Performing some function other will take 100 msec, but execution of other
//task start only when fetching of sql data done (i.e some other function
//can execute only after first in process job finishes).
-TOTAL TIME OF EXECUTION IS ALWAYS GREATER THAN (500 + 100 + processing time)
msec
Async
NodeJs comes up with async feature, it's non-blocking in nature, suppose in any I/O task which is taking time (fetching, writing, reading), nodejs won't kept idle and wait for the task to be finish, it'll start executing next tasks in the queue, and whenever that time taking task completed it will notify using callback.
Following example will help:
//Nodejs uses callback pattern to describe functions.
//Please read callback pattern to understand this example
//Suppose following function (I/O involved) took 500 msec
function timeConsumingFunction(params, callback){
//GET DATA FROM SQL
getDataFromSql(params, function(error, results){
if(error){
callback(error);
}
else{
callback(null, results);
}
})
}
//Suppose following function is non-blocking and took 100 msec
function someOtherTask(){
//some other task
console.log('Some Task 1');
console.log('Some Task 2');
}
console.log('Execution Start');
//Start With this function
timeConsumingFunction(params, function(error, results){
if(error){
console.log('Error')
}
else{
console.log('Successfull');
}
})
//As (suppose) timeConsumingFunction took 500 msec,
//As NodeJs is non-blocking, rather than remain idle for 500 msec, it will start
//execute following function immediately
someOtherTask();
In Short, Output is as:
Execution Start
//Roughly after 105 msec (5 msec it'll take in processing)
Some Task 1
Some Task 2
//Roughly After 510 msec
Error/Successful //depends on success and failure of DB function execution
Difference is clear where sync will definitely take more than 600 (500 + 100 + processing time) msec, async saves time.