We Keep Coding

Control Flow. Logic behind callbacks, promises, async/await. Kinda what's the point?

I think I know the answer to this but not sure. Wanting to confirm more than anything. The question applies (I think equally) to all three async approaches (callbacks, promises, async/await), but I'll ask it in the context of promises.
As I understand it, asynchronous programming is intended (at least) for event-driven applications to perform tasks in response to events, without the processes of doing those tasks blocking the ability to do any other tasks (presumably in response to other events) the application. eg. one event might trigger this series of tasks:
Query a database for some data
Wait for the response
Manipulate the data from the response (or handle errors as needed)
Write changes back to the database
done.
In more traditional programming (eg. C/C++) there'd be a main function that calls all that, and might get something back from it. But generally, the process of that main function, sitting around waiting for that to come back, blocks other operations (unless you start manually manipulating threads, or other stuff, that this JS async programming is presumably supposed to spare us from, right?).
But in the above example, supposedly, the (conceptual) "main function" doesn't need anything back from it. Step 5 isn't "return some result I need to act on", it's "done", end of story. If I understand correctly, the above is more likely called by a listener. That listener was set up by JS/node's equivalent of a "main function" -- the code that runs from the entry point of the app -- which has long since ended and the listeners are now running the show. Anything else this entire app needs to do will be triggered by some other independent event, caught by a listener. (Not that this is the only way but as I understand it this is pretty common). So perhaps the user sees the results, and hits another button or whatever to initiate some other, separate, independent, task with it. The above task is long deceased.
Ok... if all that's correct then there's this: For each of those 5 steps, we need to call them in succession. Each step relies on something provided by the one before it. So that process basically needs to be synchronous.
In promise code I believe it looks something like this:
askDbForData() // step 1
.then(responseFromDB => { // step 2
makeTheDesiredChangesToTheData(responseFromDB) // step 3
})
.then(changedData => writeBackToDB(changedData)) // step 4
Looks to me like ultimately this is chaining functions one after the other to perform what is otherwise essentially a synchronous task.
But but but... Synchronous... = blocking?
I've just realized how this doesn't appear to be very clear to me in most of the documentation/articles I've read on this. This is what I need to clear up...
I think the point is: the part where .next(...) picks up the result and sends it off to the next piece in the chain -- that part is "blocking" (though it happens in the blink of an eye so it's kinda moot), but each of those (presumably time consuming) functions (like askDbForData()) -- which are supposed to be asynchronous, and return promises -- do their thing separately and independently of any other control flow, etc, thus not blocking anything.
In other words, the promise chain itself is synchronous, but each piece along the way is asynchronous. The kicker, I think: Not asynchronous to other tasks in the same chain, but asynchronous to everything else the app is doing in other chains initiated by other events.
It might be that anyone reading this will read it and go "yeah well duh, that's the whole point". Perhaps that's what I'm hoping. But if it is, it hasn't been clear in any of my research so far, so it's be great to get it clear from someone(s) who "get" it. I think if all this IS correct, then it pretty much clears up nearly every other confusion I've had with this topic.
So... Yes this is long, but it is only one question. The question is:
Is that it -- is that the point -- or if not, what am I missing?
Thanks!

Looks to me like ultimately this is chaining functions one after the other to perform what is otherwise essentially a synchronous task
No, the code you show there only performs the synchronous part of askDbForData(), then returns. If properly coded, this first part is usually nothing.
I think the point is: the part where .next(...) picks up the result and sends it off to the next piece in the chain -- that part is "blocking" (though it happens in the blink of an eye so it's kinda moot)
Again no, the continuations are only called once the previous step is done. Once a continuation happens, the same split happens, first the synchronous (and hopefully minimal or non-existent) part runs then the rest of the function is registered as a continuation.
In other words, the promise chain itself is synchronous, but each piece along the way is asynchronous.
Absolutely not, the chain of promises is asynchronous. .next() stores a function reference to be called at a later date, it neither calls nor waits for the call of that function code.
The kicker, I think: Not asynchronous to other tasks in the same chain, but asynchronous to everything else the app is doing in other chains initiated by other events.
I've tried to avoid the word "chain" because it seems it's confusing you even more. It's not a chain, it's a tree, you can have promises executed in parallel or sequentially. So again, no.

Consider this.
You have a function A that returns a promise..a promise for something to come.
You also have a function B that does something else.
FunctionA().then(result=>{console.log(result)}).catch();
FunctionB();
If the promise in function A takes a long time to resolve..why not go ahead and do FunctionB while we wait?
Get it?

Performance impact during the multiple callback functions after sending response

Can anyone help me understand the function of NodeJS and performance impact for the below scenario.
a. Making the request to Rest API end point "/api/XXX". In this request, i am returning the response triggering the asynchronous function like below.
function update(req, res) {
executeUpdate(req.body); //Asynchronous function
res.send(200);
}
b. In this, I send the response back without waiting for the function to complete and this function executing four mongodb updates of different collection.
Questions:
As I read, the NodeJS works on the single thread, how this
asynchronous function is executing?
If there are multiple requests for same end point, how will be the
performance impact of NodeJS?
How exactly the NodeJS handles the asynchronous function of each
request, because as the NodeJS is runs on the single thread, is there
any possibility of the memory issue?

In short, it depends on what you are doing in your function.
The synchronous functions in node are executed on main thread, thus,
they will not preempt and execute until end of the function or until
return statement is encountered.
The async functions, on the other hand, are removed from main thread,
and will only be executed when async tasks are completed on a
separate worker thread.

There are, I think, two different parts in the answer to your question.
Actual Performance - which includes CPU & memory performance. It also obviously includes speed.
Understanding as the previous poster said, Sync and Async.
In dealing with #1 - actual performance the real only way to test it is to create or use a testing environment on your code. In a rudimentary way based upon the system you are using you can view some of the information in top (linux) or Glances will give you a basic idea of performance, but in order to know exactly what is going on you will need to apply some of the various testing environments or writing your own tests.
Approaching #2 - It is not only sync and async processes you have to understand, but also the ramifications of both. This includes the use of callbacks and promises.
It really all depends on the current process you are attempting to code. For instance, many Node programmers seem to prefer using promises when they make calls to MongoDB, especially when one requires more than one call based upon the return of the cursor.
There is really no written-in-stone formula for when you use sync or async processes. Avoiding callback hell is something all Node programmers try to do. Catching errors etc. is something you always need to be careful about. As I said some programmers will always opt for Promises or Async when dealing with returns of data. The famous Async library coupled with Bluebird are the choice of many for certain scenarios.
All that being said, and remember your question is general and therefore so is my answer, in order to properly know the implications on your performance, in memory, cpu and speed as well as in return of information or passing to the browser, it is a good idea to understand as best as you can sync, async, callbacks, promises and error catching. You will discover certain situations are great for sync (and much faster), while others do require async and/or promises.
Hope this helps somewhat.

Forever loop while waiting for asynchronous task?

I'm wondering if there's a way to cause JavaScript to wait for some variable-length code execution to finish before continuing using events and loops. Before answering with using timeouts, callbacks or referencing this as a duplicate, hear me out.
I want to expose a large API to a web worker. I want this API to feel 'native' in the sense that you can access each member using a getter which gets the information from the other thread. My initial idea was to compile the API and rebuild the entire object on the worker. While this works (and was a really fun project), it's slow at startup and cannot show changes made to the API without it being sent to the worker again after modification. Observers would solve part of this, and web workers transferrable objects would solve all, but they aren't adopted widely yet.
Since worker round-trip calls happen in a matter of milliseconds, I think stalling the thread for a few milliseconds may be an alright solution. Of course I would think about terminating in cases where calls take too long, but I'm trying to create a proof of concept first.
Let's say I want to expose the api object to the worker. I would define a getter for self.api which would fetch the first layer of properties. Each property would then be another getter and the process would continue until the final object is found.
worker.js
self.addEventListener('message', function(event) {
self.dataRecieved = true;
self.data = event.data; // would actually build new getters here
});
Object.defineProperty(self, 'api', {
get: function() {
self.dataRecieved = false;
self.postMessage('request api first-layer properties');
while(!self.dataRecieved);
return self.data; // whatever properties were received from host
}
});
For experimentation, we'll do a simple round-trip with no data processing:
index.html (only JS part)
var worker = new Worker("worker.js");
worker.onmessage = function() {
worker.postMessage();
};
If onmessage would interrupt the loop, the script should theoretically work. Then the worker could access objects like window.document.body.style on the fly.
My question really boils down to: is there a way to guarantee that an event will interrupt an executing code block?
From my understanding of events in JavaScript, I thought they did interrupt the current thread. Does it not because it's executing a blank statement over and over? What if I generated code to be executed and kept doing that until the data returned?

is there a way to guarantee that an event will interrupt an executing code block
As #slebetman suggests in comments, no, not in Javascript running in a browser's web-worker (with one possible exception that I can think of, see suggestion 3. below).
My suggestions, in decreasing order of preference:
Give up the desire to feel "native" (or maybe "local" might be a better term). Something like the infinite while loop that you suggest also seems to be very much fighting agains the cooperative multitasking environment offered by Javascript, including when thinking about a single web worker.
Communication between workers in Javascript is asynchronous. Perhaps it can fail, take longer than just a few milliseconds. I'm not sure what your use case is, but my feeling is that when the project grows, you might want to use those milliseconds for something else.
You could change your defined property to return a promise, and then the caller would do a .then on the response to retrieve the value, just like any other asynchronous API.
Angular Protractor/Webdriver has an API that uses a control flow to simulate a synchronous environment using promises, by always passing promises about. Taking the code from https://stackoverflow.com/a/22697369/1319998
browser.get(url);
var title = browser.getTitle();
expect(title).toEqual('My Title');
By my understanding, each line above adds a promise to the control flow to execute asynchronously. title isn't actually the title, but a promise that resolves to the title for example. While it looks like synchronous code, the getting and testing all happens asynchronously later.
You could implement something similar in the web worker. However, I do wonder whether it will be worth the effort. There would be a lot of code to do this, and I can't help feeling that the main consequence would be that it would end up harder to write code using this, and not easier, as there would be a lot of hidden behaviour.
The only thing that I know of that can be made synchronous in Javascript, is XMLHttpRequest when setting the async parameter to false https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest#Parameters. I wonder if you could come up with some sort of way to request to the server that maintains a connection with the main thread and pass data along that way. I have to say, my instinct is that this is quite an awful idea, and would be much slower than just requesting data from the main thread.

For what I know, there is not something native in JS to do this but it is relatively easy to do something similar. I made one some time ago for myself: https://github.com/xpy/whener/blob/master/whener.js .
You use it like when( condition, callback ) where condition is a function that should return true when your condition is met, and callback is the function that you want to execute at that time.

Is it possible to implement a sleep() in javascript?

This is a duplicate question. It has been asked many times before, with dozens of answers, some of them rated very highly. Unfortunately, as far as I have been able to tell, every single one of those answers is a variant of "You don't, it's bad programming practice. Use setTimeout instead".
This is Not. An. Answer!
There are some use cases - rare but they exist - where you might want the entire page's execution to halt for a second or two, and I find it very frustrating that nobody seems interested in answering the actual question. (have a look at the comments here for some examples).
I am sure it's possible to halt javascript executing; for instance, if I use firebug to insert a breakpoint, then the execution stops when it hits that point. So, firebug can do it. Is there some way that the program can halt execution of the current thread until some timeout occurs?
Just some thoughts: How does firebug do it? Is there some browser-specific method? Is it possible to trigger a stop, without specifying a timeout to continue? Could I programmatically insert a breakpoint, or remove one? Could I get a closure representing the current thread to pass to setTimeout?
I don't have a specific use case in mind; I am just looking for advise from someone who knows the browser/javascript design better than me, as to how this can most effectively be done.
So far, I have come up with only one solution:
endtime=Date.now()+1000;
while(Date.now() < endtime)
$.ajax(window.location.origin,{'async':false});
This appears to work. The problem with it is, it makes hundreds of excess requests. I would replace the location.origin with something like mysite/sleep?delay=X and write a server side script to provide the delay, which would but it down to one, but the whole thing still seems really hacky. There must be a better way to do this! How does the jquery.ajax function manage it? Or is there a busy-wait buried in it somewhere?
The following do not answer the question and will be downvoted, just because I am sick of seeing pages of answers that completely ignore the question in their rush to rant on the evils of sleep:
Sleep is evil, and you should do anything it takes to avoid needing it.
Refactor your code so that you can use setTimeout to delay execution.
Busy-wait (because it doesn't stop execution for the duration of the sleep).
Refactor your code to use deferred/promise semantics.

You should never do this, it's a bad idea...
... because the browser has been, traditionally, single-threaded. Sleeping freezes the UI as well as the script.
However, now that we have web workers and the like, that's not the case. You probably don't need a sleep, but having a worker busy-wait won't freeze the UI. Depending on just how much you want to freeze a particular thread, I've seen people use:
endtime = Date.now()+1000;
while (Date.now() < endtime)
or, curiously (this was in an older but corporate-sponsored analytics library):
endtime = new Date().getTime() + 1000;
while (new Date().getTime() < endtime)
which is probably slower. If you're running a busy wait, that doesn't necessarily matter, and allocating objects probably just burns memory and GC time.
Code using promises or timeouts tends to be more modular, but harder to read (especially when you first learn async techniques). That's not an excuse for not using it, as there are definite advantages, but maybe you need everything to stay synchronous for some reason.
If you have a debugger running and want some chunk of code to pause itself (very useful when you have a bunch of nested callbacks), you can use:
function foo() {
do.someStuff();
debugger;
do.otherStuff();
}
The browser should pause execution at the debugger statement. The debugger can almost always pause execution, because it is in control of the VM running the code; it can just tell the VM to stop running, and that ought to happen. You can't get quite to that level from a script, but if you take source as text (perhaps from a require.js plugin), you can modify it on the fly to include debugger statements, thus "programmatically inserting breakpoints." Bear in mind that they will only take effect when the debugger is already open, though.
To capture the state of a "thread" and persist it for later use, you may want to look into some of the more complicated functional programming concepts, particularly monads. These allow you to wrap a start value in a chain of functions, which modify it as they go, but always in the same way. You could either keep simple state (in some object), or record and reproduce everything the "thread" may have done by wrapping functions in functions. There will be performance implications, but you can pick up the last function later and call it, and you should be able to reproduce everything the thread may have done.
Those are all fairly complicated and specific-use solutions to avoid just deferring things idiomatically, but if you hypothetically need them, they could be useful.

No, it is not possible to implement a sleep in javascript in the traditional sense, as it is a single-threaded event based model. The act of sleeping this thread will lock up the browser it is running in and the user is presented with a message either telling them the browser has stopped responding (IE) or allowing them to abort the currently running code (Firefox).

Are Mutexes needed in javascript?

I have seen this link: Implementing Mutual Exclusion in JavaScript.
On the other hand, I have read that there are no threads in javascript, but what exactly does that mean?
When events occur, where in the code can they interrupt?
And if there are no threads in JS, do I need to use mutexes in JS or not?
Specifically, I am wondering about the effects of using functions called by setTimeout() and XmlHttpRequest's onreadystatechange on globally accessible variables.

Javascript is defined as a reentrant language which means there is no threading exposed to the user, there may be threads in the implementation. Functions like setTimeout() and asynchronous callbacks need to wait for the script engine to sleep before they're able to run.
That means that everything that happens in an event must be finished before the next event will be processed.
That being said, you may need a mutex if your code does something where it expects a value not to change between when the asynchronous event was fired and when the callback was called.
For example if you have a data structure where you click one button and it sends an XmlHttpRequest which calls a callback the changes the data structure in a destructive way, and you have another button that changes the same data structure directly, between when the event was fired and when the call back was executed the user could have clicked and updated the data structure before the callback which could then lose the value.
While you could create a race condition like that it's very easy to prevent that in your code since each function will be atomic. It would be a lot of work and take some odd coding patterns to create the race condition in fact.

The answers to this question are a bit outdated though correct at the time they were given. And still correct if looking at a client-side javascript application that does NOT use webworkers.
Articles on web-workers:
multithreading in javascript using webworkers
Mozilla on webworkers
This clearly shows that javascript via web-workers has multithreading capabilities. As concerning to the question are mutexes needed in javascript? I am unsure of this. But this stackoverflow post seems relevant:
Mutual Exclusion for N Asynchronous Threads

Yes, mutexes can be required in Javascript when accessing resources that are shared between tabs/windows, like localStorage.
For example, if a user has two tabs open, simple code like the following is unsafe:
function appendToList(item) {
var list = localStorage["myKey"];
if (list) {
list += "," + item;
}
else {
list = item;
}
localStorage["myKey"] = list;
}
Between the time that the localStorage item is 'got' and 'set', another tab could have modified the value. It's generally unlikely, but possible - you'd need to judge for yourself the likelihood and risk associated with any contention in your particular circumstances.
See the following articles for a more detail:
Wait, Don't Touch That: Mutual Exclusion Locks & JavaScript - Medium Engineering
JavaScript concurrency and locking the HTML5 localStorage - Benjamin Dumke-von der Eh, Stackoverflow

As #william points out,
you may need a mutex if your code does something where it expects a
value not to change between when the asynchronous event was fired and
when the callback was called.
This can be generalised further - if your code does something where it expects exclusive control of a resource until an asynchronous request resolves, you may need a mutex.
A simple example is where you have a button that fires an ajax call to create a record in the back end. You might need a bit of code to protect you from trigger happy users clicking away and thereby creating multiple records. there are a number of approaches to this problem (e.g. disable the button, enable on ajax success). You could also use a simple lock:
var save_lock = false;
$('#save_button').click(function(){
if(!save_lock){
//lock
save_lock=true;
$.ajax({
success:function()
//unlock
save_lock = false;
}
});
}
}
I'm not sure if that's the best approach and I would be interested to see how others handle mutual exclusion in javascript, but as far as i'm aware that's a simple mutex and it is handy.

JavaScript is single threaded... though Chrome may be a new beast (I think it is also single threaded, but each tab has it's own JavaScript thread... I haven't looked into it in detail, so don't quote me there).
However, one thing you DO need to worry about is how your JavaScript will handle multiple ajax requests coming back in not the same order you send them. So, all you really need to worry about is make sure your ajax calls are handled in a way that they won't step on eachother's feet if the results come back in a different order than you sent them.
This goes for timeouts too...
When JavaScript grows multithreading, then maybe worry about mutexes and the like....

JavaScript, the language, can be as multithreaded as you want, but browser embeddings of the javascript engine only runs one callback (onload, onfocus, <script>, etc...) at a time (per tab, presumably). William's suggestion of using a Mutex for changes between registering and receiving a callback should not be taken too literally because of this, as you wouldn't want to block in the intervening callback since the callback that will unlock it will be blocked behind the current callback! (Wow, English sucks for talking about threading.) In this case, you probably want to do something along the lines of redispatching the current event if a flag is set, either literally or with the likes of setTimeout().
If you are using a different embedding of JS, and that executes multiple threads at once, it can get a bit more dicey, but due to the way JS can use callbacks so easily and locks objects on property access explicit locking is not nearly as necessary. However, I would be surprised if an embedding designed for general code (eg, game scripting) that used multi threading didn't also give some explicit locking primitives as well.
Sorry for the wall of text!

Events are signaled, but JavaScript execution is still single-threaded.
My understanding is that when event is signaled the engine stops what it is executing at the moment to run event handler. After the handler is finished, script execution is resumed. If event handler changed some shared variables then resumed code will see these changes appearing "out of the blue".
If you want to "protect" shared data, simple boolean flag should be sufficient.