Related
I think I know the answer to this but not sure. Wanting to confirm more than anything. The question applies (I think equally) to all three async approaches (callbacks, promises, async/await), but I'll ask it in the context of promises.
As I understand it, asynchronous programming is intended (at least) for event-driven applications to perform tasks in response to events, without the processes of doing those tasks blocking the ability to do any other tasks (presumably in response to other events) the application. eg. one event might trigger this series of tasks:
Query a database for some data
Wait for the response
Manipulate the data from the response (or handle errors as needed)
Write changes back to the database
done.
In more traditional programming (eg. C/C++) there'd be a main function that calls all that, and might get something back from it. But generally, the process of that main function, sitting around waiting for that to come back, blocks other operations (unless you start manually manipulating threads, or other stuff, that this JS async programming is presumably supposed to spare us from, right?).
But in the above example, supposedly, the (conceptual) "main function" doesn't need anything back from it. Step 5 isn't "return some result I need to act on", it's "done", end of story. If I understand correctly, the above is more likely called by a listener. That listener was set up by JS/node's equivalent of a "main function" -- the code that runs from the entry point of the app -- which has long since ended and the listeners are now running the show. Anything else this entire app needs to do will be triggered by some other independent event, caught by a listener. (Not that this is the only way but as I understand it this is pretty common). So perhaps the user sees the results, and hits another button or whatever to initiate some other, separate, independent, task with it. The above task is long deceased.
Ok... if all that's correct then there's this: For each of those 5 steps, we need to call them in succession. Each step relies on something provided by the one before it. So that process basically needs to be synchronous.
In promise code I believe it looks something like this:
askDbForData() // step 1
.then(responseFromDB => { // step 2
makeTheDesiredChangesToTheData(responseFromDB) // step 3
})
.then(changedData => writeBackToDB(changedData)) // step 4
Looks to me like ultimately this is chaining functions one after the other to perform what is otherwise essentially a synchronous task.
But but but... Synchronous... = blocking?
I've just realized how this doesn't appear to be very clear to me in most of the documentation/articles I've read on this. This is what I need to clear up...
I think the point is: the part where .next(...) picks up the result and sends it off to the next piece in the chain -- that part is "blocking" (though it happens in the blink of an eye so it's kinda moot), but each of those (presumably time consuming) functions (like askDbForData()) -- which are supposed to be asynchronous, and return promises -- do their thing separately and independently of any other control flow, etc, thus not blocking anything.
In other words, the promise chain itself is synchronous, but each piece along the way is asynchronous. The kicker, I think: Not asynchronous to other tasks in the same chain, but asynchronous to everything else the app is doing in other chains initiated by other events.
It might be that anyone reading this will read it and go "yeah well duh, that's the whole point". Perhaps that's what I'm hoping. But if it is, it hasn't been clear in any of my research so far, so it's be great to get it clear from someone(s) who "get" it. I think if all this IS correct, then it pretty much clears up nearly every other confusion I've had with this topic.
So... Yes this is long, but it is only one question. The question is:
Is that it -- is that the point -- or if not, what am I missing?
Thanks!
Looks to me like ultimately this is chaining functions one after the other to perform what is otherwise essentially a synchronous task
No, the code you show there only performs the synchronous part of askDbForData(), then returns. If properly coded, this first part is usually nothing.
I think the point is: the part where .next(...) picks up the result and sends it off to the next piece in the chain -- that part is "blocking" (though it happens in the blink of an eye so it's kinda moot)
Again no, the continuations are only called once the previous step is done. Once a continuation happens, the same split happens, first the synchronous (and hopefully minimal or non-existent) part runs then the rest of the function is registered as a continuation.
In other words, the promise chain itself is synchronous, but each piece along the way is asynchronous.
Absolutely not, the chain of promises is asynchronous. .next() stores a function reference to be called at a later date, it neither calls nor waits for the call of that function code.
The kicker, I think: Not asynchronous to other tasks in the same chain, but asynchronous to everything else the app is doing in other chains initiated by other events.
I've tried to avoid the word "chain" because it seems it's confusing you even more. It's not a chain, it's a tree, you can have promises executed in parallel or sequentially. So again, no.
Consider this.
You have a function A that returns a promise..a promise for something to come.
You also have a function B that does something else.
FunctionA().then(result=>{console.log(result)}).catch();
FunctionB();
If the promise in function A takes a long time to resolve..why not go ahead and do FunctionB while we wait?
Get it?
There's a bit of someone else's code I am trying to add functionality to. It's using websockets to communicate with a server which I will most likely not be able to change (the server runs on a 3$ micro-controller...)
The pattern used, for instance when uploading data to the server, consists in using global variables, then sending a series of messages on the socket, as well as having an 'onmessage' which will handle the response. This seems clumsy, given that it assumes that there is only ever one socket call made at a time (I think the server guarantees that in fact). The messages sent by the server can be multiple, and even figuring out when the messages are finished is fiddly.
I am thinking of making things so that I have a better handle on things, mostly w.r.t. being able to know when the response has arrived (and finished), going to patterns like
function save_file(name, data, callback) {
}
And perhaps at some point I can even turn them into async functions.
So couple of ideas:
- is there some kind of identifier that I could find in the websocket object that might allow me to better string together request and response?
- short of that, what is the right pattern? I started using custom events, that allows me to much better tie the whole process, where I can supply a callback by attaching it to the event, but even doing removeEventListener is tricky because I need to keep reference to every single listener to make sure I can remove them later.
Any advice anyone?
It feels like this must be a stupid question but I don't understand something fundamental about making a request in indexedDB.
Why are the requests made before the event handlers are defined? For example, the request = objectStore.add(data) is made before the request.onsuccess and request.onerror functions are declared. Is this correct? Is it possible that the request could complete before the event handlers are registered?
I'm comparing it to the creation of an image element, followed by declaring the event handlers for onload and onerror, all before setting the source attribute to the location of a file and attempting to load it. But a request "element" can't be created before a request is made; so, there's nothing to attach the events to until the request has already been made.
Please let me know what I'm missing here. I've been writing and retrieving data from indexedDB without issue, and think I've been coding it correctly; but I want to make sure this is right and will always work.
Thank you.
Duplicate Response
I read that question and answer sometime ago, when I first started reading about indexedDB, and completely forgot about it. Had I found it again before I wrote this question, I likely would not have submitted it and would've just accepted that the code ought to work out whether I understand it or not. Handling error events and transaction aborts is what got me to thinking about the statement order again.
However, after reading the answer again, I don't understand enough to be any further along than just accepting it and hoping it will always work. I'm not trying to be snide. In one sense, it is just confusing for my limited capacity to think of event loops and epochs and that everything sort of happens all at once.
At the end of the epoch (or the start of the next, whatever you think is easier to understand), the underlying JS engine goes back and looks at what is registered to execute, and then executes everything nearly all at once.
There has to be an order of execution or nothing makes sense, asynchronous or not. I understand that the interpreter does not wait for an anysnchronous process to complete before starting to execute the next line of code. But aren't the synchronous statements processed completely in turn in the order they appear in the code and the asynchronous ones started off in the order they appear in the code, such that if an asynchronous process errored quickly, the event could be missed if the event handlers weren't declared in advance? The event handlers are not hoisted like function declarations, are they? This is the part that I still find confusing.
In this article by Jake Archibald on promises, in the introduction he presents an example concerning the loading of images and writes:
Unfortunately, in the example above, it's possible that the events happened before we started listening for them, so we need to work around that using the "complete" property of images.
and
This doesn't catch images that error'd before we got a chance to listen for them; unfortunately the DOM doesn't give us a way to do that. Also, this is loading one image, things get even more complex if we want to know when a set of images have loaded.
That gives the impression that order is important, such that, in the case of images, when possible, the source should be assigned after declaring all the event handlers, in order to not miss hearing events. The important part for me was the fact that an event could take place before the event handler was declared/registered.
I tried to follow the same pattern of making the request after declaring the event handlers in indexedDB, but it doesn't appear possible because there's nothing to attach the events to until the request is made.
Even when all the statements are asynchronous, such as in this example in the MDN Web Docs on Using IndexedDB, some things are still rather confusing. The objectStore.transaction.oncomplete is an interesting statement. We're waiting for the objectStore to be created before attempting to write data to it. (I think that's considered bad practice, writing data in an onupgradeneeded event; so, we don't use that statement.) But what is confusing is why we don't worry about the objectStore being created before creating an index in it. Why isn't the createIndex statement started at the same time the createObjectStore statement started, if everything is processed all at once? If the createObjectStore statement doesn't complete before the createIndex statement begins, shouldn't an event handler be required or it would fail because objectStore wouldn't yet exist?
I know it works because I've been using the same code pattern, but I really don't understand it.
These two items--the potential to miss events and why an event handler isn't needed in this indexedDB example--are what I'd like to better understand. I don't know if this makes my question different or not, but the answer to the duplicate question doesn't answer these for me. Perhaps, I'd have to understand the JS engine better to understand the answer to these questions.
const dbName = "the_name";
var request = indexedDB.open(dbName, 2);
request.onerror = function(event) {
// Handle errors.
};
request.onupgradeneeded = function(event) {
var db = event.target.result;
// Create an objectStore to hold information about our customers. We're
// going to use "ssn" as our key path because it's guaranteed to be
// unique - or at least that's what I was told during the kickoff meeting.
var objectStore = db.createObjectStore("customers", { keyPath: "ssn" });
// Create an index to search customers by name. We may have duplicates
// so we can't use a unique index.
objectStore.createIndex("name", "name", { unique: false });
// Create an index to search customers by email. We want to ensure that
// no two customers have the same email, so use a unique index.
objectStore.createIndex("email", "email", { unique: true });
// Use transaction oncomplete to make sure the objectStore creation is
// finished before adding data into it.
objectStore.transaction.oncomplete = function(event) {
// Store values in the newly created objectStore.
var customerObjectStore = db.transaction("customers", "readwrite").objectStore("customers");
customerData.forEach(function(customer) {
customerObjectStore.add(customer);
});
};
};
Clarification/Response to Answer/Comments
Thank you for taking the time to respond to my question and for the added explanation.
First, by "before" I simply mean the order in which the statements appear in the script.
I think I follow your analogy and it is a good one. I'm just still not clear on why the employees never submit their work to the secretary until the next day, when it is guaranteed that a secretary will be there to receive it.
It sounds similar to the fact that the javascript interpreter, when it performs its equivalent of compiling the script, hoists the function declarations, such that a function can be invoked in the code before the function declaration has been made.
It would appear that your saying, restated in my simple terms, that the JS engine, at some point before final execution, assigns the event handlers (the secretaries) to be registered in an earlier epoch than the one in which the requests (the employees) that utlimately trigger the events will complete. Therefore, it doesn't matter where in the code the request statements appear relative to the event handlers, that is, as long as they are defined within the same epoch.
The JS engine doesn't know when the request will complete, but only when the event handlers have been registered to start listening and when the request has commenced. As long as the JS engine has a process to order these steps properly independent of the order the statements appear in the code, such that an event cannot be missed, then it's no different to me than the hoisting of function declarations and I don't really have to think much about it any longer to accomplish my tasks.
However, I would still like to better understand what an epoch is, at least in terms of knowing that the statements are made within the same epoch. I don't see any mention of an epoch in the article on "Concurrecny Model and Event Loop" in MDN Web Docs. Would you mind pointing me to any good resources you know of?
Thank you.
Final Notes
I came across these two items through a link, here, on stack overflow. This same question was asked eight years ago and was answered about the same way but with different terminology; that is, instead of epochs, it was that javascript code will "run to completion" or has run-to-completion semantics. This question refers you to this document which can be searched for "run to completion" to read two exchanges about why there is not a race condition in this set up of making a request before registering the event handlers. The older JavaScript book I have by David Flanagan, in discussing the execution of JS "programs," states that, because JS has single-threaded execution, one never has to worry about race conditions; but I don't know if he was referring to this situation exactly.
Thus, this question has been asked and answered multiple times in the past and I guess I'm just another newbie asking an old question as if I was the first to have thought of it, and without enough knowledge of how JS processes.
The article "Concurrency Model and Event Loop", linked above, has a brief "Run-to-completion" section; but I didn't understand its implications until after reading the last document linked above.
I now take it to mean that all the code in a function will run to completion before any other code can begin, which seems to have two interpretations.
One is that the asynchronous request on the database is queued when the statement is reached in the function code but won't actually commence until all the other statements in the function are run, including the event handlers that are declared afterward.
Or, according to that last linked document above, the asynchronous request may run and even complete before the event handlers are registered, but the notification of its completion will remain in the queue and won't execute until after the rest of the statements in the function are run and the event handlers have been registered.
Interpretation 2 appears to be the accurate one but, whichever is the actual case, it all makes adequate sense to me now and explains why the secretary will always be there before the employee submits the work, and why, even if the employee finishes the work in a nanosecond, the employee won't submit the work until the next day when a secretary is guaranteed to be present to receive it. The employee may place the work-complete notification in the queue, but the queue won't sound the notification for a secretary to hear until the next day.
Thanks, Josh, for the additional explanation about what is meant by epochs and how that terminology works out in operation. I accepted your answer and appreciate you taking the time to write it all out.
Now that I seem to understand why the event-handler declarations can be made later in the code than the making of the request, I still don't understand why we can create an object store and then immediately create an index on that object store without having to wait until we know the object store has been successfully created, unless it is synchronous or something else special takes place in a versionchange transaction / onupgradeneeded event. There are no events mentioned in the MDN Web Docs description of createObjectStore and no examples that have any listeners on it; so; I'll just assume it's never necessary.
Thanks again.
Why are the requests made before the event handlers are defined?
It does not matter.
For example, the request = objectStore.add(data) is made before the request.onsuccess and request.onerror functions are declared. Is this correct?
Yes it is correct because again it does not matter.
I would be careful about your use of the word before. Maybe it means something different to me than it does to you. I can't tell. But maybe this is what is tripping you up.
Is it possible that the request could complete before the event handlers are registered?
If you register the event handlers in the same epoch as when you make the request, then no. The request only completes in a later epoch.
Ok, here is my attempt at explaining by example (sorry if this is bad!). Personification is generally a good educational technique, and is less intimidating then using raw technical terms, so let's go with that.
Let's say you are a boss, and have employees. Let's say you ask an employee to do some work for you. Then, you ask that employee to report back to your secretary when they completed the work. Immediately after asking the employee to go do that other work, you carry on doing your own work, without waiting for that employee to finish their work and report back. You are both basically doing work at the same time.
Now, in this situation, what happens if you don't have a secretary at the time you hand the employee a request to do something? Well, no problem. You go and hire another secretary before that employee finishes their work and before that employee even knows who to report back to, which is fine because all the employee knows is they report to your secretary. The employee does not know whether your secretary exists or not at the time of being assigned work, and does not need to know that. The missing secretary did not prevent that employee from getting started, or understanding the work to be done. And by the time that employee completes their work, you have a secretary ready and waiting. Or, you don't, because you don't happen to care to even acknowledge whether the work was actually completed, you just made a command and trusted the employee to do their job, whatever. You really only care about having them report back to your secretary if you need to do some other work that has to wait until after the first project is done, and that is a different concern.
Let's pretend you already had a secretary at the time you assigned the employee the work. What is the difference between this situation of already having a secretary, and the situation where you go and hire one shortly after assigning the work, but before it is done? There is NO difference.
Now, let's try and really address your concern. What you're suggesting is that it seems impossible to reliably go out and hire that secretary before you know whether the employee finished their assignment. This I think is the critical misunderstanding. It is entirely possible to do this. Why is that? I suppose it is not the easiest thing to grasp.
I am going to stretch this metaphor a bit and impose a strange rule. No matter how simple the project you hand off to the employee, even if it is just to run and get you coffee in the morning, they will never ever get back to you the same day. They will always finish their work some later day, at the earliest tomorrow. They might even finish their work within one fleeting nanosecond of you telling them, but they will NEVER get back to you or your secretary right away, they will always be delayed until tomorrow at the earliest.
This means you have all day to go and hire that secretary that did not exist at the time you gave the employee the order. So long as you do it before tomorrow, you're good. That secretary will exist and be working for you by the time the employee responds tomorrow, and will be able to receive the message from the employee.
Edit response to your added comments:
Yep, hoisting is similar in many respects. Things can happen in a different order then written in code. Hoisting is of course synchronous so it is not a perfect similarity, but the out-of-order aspect is still similar.
Epoch is just my own word i use for a single iteration of the event loop. Like in the case of a for loop using i for i from 0 to 2, there are 3 epochs, iteration 0, iteration 1, and iteration 2. i just call them epochs because it is like categories of time.
In a promise case, it might even be a microtask. In a js worker case, it might be thread-like (and workers are the new hotness over the old child-iframe technique). Basically these are all just ways to 'realize' doing more than one thing at a time. Node calls it a tick, and has things like nextTick() that defers code execution until the next tick of its loop. Within a single epoch, things happen in the order they are written (and notably hoisting is all in epoch 0). but some code may be async, and therefore happen across epochs, and therefore may run in a different order than it was written. Code written earlier may happen in a later epoch.
When you make a request, it says, start doing this thing, and get back to me at the earliest in the next epoch. You have up until the end of the current epoch to register handlers for the request.
Some code, like in the case of an image preloader as mentioned in your example, has to take into account that it attaches the listeners too late (images are being preloaded in an alternate timeline and some may already be loaded and in some browsers this means load will not fire), so it wants to check imageElement.complete to catch that case. In other cases of event listener implementations, some dispatcher implementations will fire events to newly added listeners for events that already happened where the new listener was not listening at the time of the event. But that is not a universal characteristic of event listener implementations, just a characteristic of certain implementations.
And in the case of the transaction.oncomplete thing from within onupgradeneeded, that is just not a great example. It is doing stuff it does not need to do.
This is the technical answer to your question:
https://html.spec.whatwg.org/multipage/webappapis.html#event-loops
The JS concurrency model is cooperative with "run-to-completion" semantics (no parallel processing of events in the same queue). This means that any async response will be posted as a message to the window event loop, and all the sequential code you see after the request is guaranteed to execute before the async response processing starts.
That said, from usability point of view, the IndexDB API is not affording the intent in the most expressive manner, and coming from other languages with preemptive threading you are excused for being confused :-)
Tx is committed when :
request success callback returns
- that means that multiple requests can be executed within transaction boundaries only when next request is executed from success callback of the previous one
when your task returns to event loop
It means that if no requests are submitted to it, it is not committed until it returns to event loop. These facts pose 2 problematic states :
placing a new IDB request by enqueuing a new task to event loop queue from within the success callback of previous request instead of submitting new request synchronously
in that case the first success callback immediately returns but another IDB request has been scheduled
are all the asynchronous requests executed within the single initial transaction? This is quite essential in case you want to implement result pulling with back-pressure where consumer gives you a feedback in form of a Future that it is ready to consume another response
creating a ReadWrite tx, not placing any requests against it and creating another one before returning to event loop
does creating a new one implicitly commits the previous tx ? If not, serious write lock starvations might occur, because :
If multiple "readwrite" transactions are attempting to access the same
object store (i.e. if they have overlapping scope), the transaction
that was created first must be the transaction which gets access to
the object store first. Due to the requirements in the previous
paragraph, this also means that it is the only transaction which has
access to the object store until the transaction is finished.
The example of enqueuing a new task to event loop queue from within the success callback by recursive request submission with back-pressure :
function recursiveFn(key) {
val req = store.get(key)
req.onsuccess = function() {
observer.onNext(req.result).onsuccess { recursiveFn(nextKey) }
}
}
Observer#onNext // returns Future[Ack] Ack is either Continue or Cancel
Now can onsuccess or onNext do a setTimeout(0) or not to make the whole thing be part of one transaction?
Bonus question :
I think that ReadOnly transactions are exposed to the consumer/user just because it would be hard to detect the end of a batch read if you recursively submit new requests from the success callback of the previous one right? Otherwise I don't see any other reason for them to be exposed to a user, right ?
I'm not sure I understand your question completely but I'll offer an answer on whether you can safely use IDB transaction events to move a state machine.
Yes and no. Yes in theory, no in practice.
I think you understand the transaction lifetime. But to rehash:
The lifetime of a transactions lasts as long as it's referenced: it's "active" so long as it's being referenced, after which it is said to be "finished" and the transaction is committed.
In theory, oncomplete should fire whenever a transaction successfully commits. There's a useful tip in the spec on this that suggests how you could loop:
To determine if a transaction has completed successfully, listen to the transaction’s complete event rather than the IDBObjectStore.add request’s success event, because the transaction may still fail after the success event fires.
To safely use this mechanism be sure to watch for non-success events including onblocked and onabort as well.
Practically speaking, I've found transactions to be flakey when long-lived or done consecutively in batches (as you've noted in another IDB comment). I'm generally not engineering my apps to require tricky behavior because, no matter how the spec says it should behavior, I'm seeing wonky transactions in both Firefox and Chromium (but mostly Blink, interestingly) especially when multiple tabs are open.
I spent many weeks rewriting dash to reuse transactions for supposed performance gains. In the end it could not pass even my basic write tests and I was forced to abandon simultaneous/queued/consecutive transactions and rewrite once again. This time I picked a one-transaction-at-a-time model which is slower but, for me, more reliable (and suggest to avoid my lib and use something like ydn for bulk inserts).
I'm not sure on your application requirements, but in my humble opinion tying in your I/O into your event loop seems like a disastrous idea. If I needed an event loop as what I understand to be the term I would definitely use requestAnimationFrame() and throttle that callback if I needed fewer ticks than one per ~33 milliseconds.
// synchronous Javascript
var result = db.get('select * from table1');
console.log('I am syncronous');
// asynchronous Javascript
db.get('select * from table1', function(result){
// do something with the result
});
console.log('I am asynchronous')
I know in synchronous code, console.log() executes after result is fetched from db, whereas in asynchronous code console.log() executes before the db.get() fetches the result.
Now my question is, how does the execution happen behind the scenes for asynchronous code and why is it non-blocking?
I have searched the Ecmascript 5 standard to understand how asynchronous code works but could not find the word asynchronous in the entire standard.
And from nodebeginner.org I also found out that we should not use a return statement as it blocks the event loop. But nodejs api and third party modules contain return statements everywhere. So when should a return statement be used and when shouldn't it?
Can somebody throw some light on this?
First of all, passing a function as a parameter is telling the function that you're calling that you would like it to call this function some time in the future. When exactly in the future it will get called depends upon the nature of what the function is doing.
If the function is doing some networking and the function is configured to be non-blocking or asychronous, then the function will execute, the networking operation will be started and the function you called will return right away and the rest of your inline javascript code after that function will execute. If you return a value from that function, it will return right away, long before the function you passed as a parameter has been called (the networking operation has not yet completed).
Meanwhile, the networking operation is going in the background. It's sending the request, listening for the response, then gathering the response. When the networking request has completed and the response has been collected, THEN and only then does the original function you called call the function you passed as a parameter. This may be only a few milliseconds later or it may be as long as minutes later - depending upon how long the networking operation took to complete.
What's important to understand is that in your example, the db.get() function call has long since completed and the code sequentially after it has also executed. What has not completed is the internal anonymous function that you passed as a parameter to that function. That's being held in a javascript function closure until later when the networking function finishes.
It's my opinion that one thing that confuses a lot of people is that the anonymous function is declared inside of your call to db.get and appears to be part of that and appears that when db.get() is done, this would be done too, but that is not the case. Perhaps that would look less like that if it was represented this way:
function getCompletionfunction(result) {
// do something with the result of db.get
}
// asynchronous Javascript
db.get('select * from table1', getCompletionFunction);
Then, maybe it would be more obvious that the db.get will return immediately and the getCompletionFunction will get called some time in the future. I'm not suggesting you write it this way, but just showing this form as a means of illustrating what is really happening.
Here's a sequence worth understanding:
console.log("a");
db.get('select * from table1', function(result){
console.log("b");
});
console.log("c");
What you would see in the debugger console is this:
a
c
b
"a" happens first. Then, db.get() starts its operation and then immediately returns. Thus, "c" happens next. Then, when the db.get() operation actually completes some time in the future, "b" happens.
For some reading on how async handling works in a browser, see How does JavaScript handle AJAX responses in the background?
jfriend00's answer explains asynchrony as it applies to most users quite well, but in your comment you seemed to want more details on the implementation:
[…] Can any body write some pseudo code, explaining the implementation part of the Ecmascript specification to achieve this kind of functionality? for better understanding the JS internals.
As you probably know, a function can stow away its argument into a global variable. Let's say we have a list of numbers and a function to add a number:
var numbers = [];
function addNumber(number) {
numbers.push(number);
}
If I add a few numbers, as long as I'm referring to the same numbers variable as before, I can access the numbers I added previously.
JavaScript implementations likely do something similar, except rather than stowing numbers away, they stow functions (specifically, callback functions) away.
The Event Loop
At the core of many applications is what's known as an event loop. It essentially looks like this:
loop forever:
get events, blocking if none exist
process events
Let's say you want to execute a database query like in your question:
db.get("select * from table", /* ... */);
In order to perform that database query, it will likely need to perform a network operation. Since network operations can take a significant amount of time, during which the processor is waiting, it makes sense to think that maybe we should, rather than waiting rather than doing some other work, just have it tell us when it's done so we can do other things in the mean time.
For simplicity's sake, I'll pretend that sending will never block/stall synchronously.
The functionality of get might look like this:
generate unique identifier for request
send off request (again, for simplicity, assuming this doesn't block)
stow away (identifier, callback) pair in a global dictionary/hash table variable
That's all get would do; it doesn't do any of the receiving bit, and it itself isn't responsible for calling your callback. That happens in the process events bit. The process events bit might look (partially) like this:
is the event a database response? if so:
parse the database response
look up the identifier in the response in the hash table to retrieve the callback
call the callback with the received response
Real Life
In real life, it's a little more complex, but the overall concept is not too different. If you want to send data, for example, you might have to wait until there's enough space in the operating system's outgoing network buffers before you can add your bit of data. When reading data, you might get it in multiple chunks. The process events bit probably isn't one big function, but itself just calling a bunch of callbacks (which then dispatch to more callbacks, and so on…)
While the implementation details between real life and our example are slightly different, the concept is the same: you kick off ‘doing something’, and a callback will be called through some mechanism or another when the work is done.