Suppose I have written a library class:
class ComplexThingDoer implements ThingDoer {
...
doComplexThing(arg1, arg2) {
// lots of complex code here
return finalResult;
}
...
}
And I have made this class available to others, who are using it as intended, completely ignorant of how doComplexThing actually does the complex thing and making use of the finalResult for whatever they're using it for.
Now, suppose I decide that I actually want to reimplement doComplexThing so that instead of synchronously running a lot of complex code here, it makes a request to a specialized server that returns the final result. Or makes a call to a database in which results of the complex thing have been precalculated. Or does one of those things the first time it's called with a particular set of arguments, and caches the result so that the next time it's called with the same arguments it can just return the previously-calculated value.
As far as I can tell, whether I use callbacks, Promises, or async/await, I am screwed - I can't do any of those things without breaking my interface. I have to return a Promise, or accept a callback, or something - there doesn't seem to be any way to just not return until I have the result, the way I've always done before. My callers don't care how I'm getting the result; they just want the result.
Am I missing something? Is there in fact a way to "de-promisify" a function so that its caller doesn't have to know that it's performing an asynchronous operation? Should I be pre-emptively writing everything as Promises just in case someday I might want to reimplement it in an asynchronous way (sounds like a terrible idea)?
Thanks!
Nope, you're not missing anything. If you make your function asynchronous, then every caller of it will have to treat it as such and also be asynchronous itself. The result of an asynchronous call will be available sometime later, on subsequent iterations of the underlying event loop. There is absolutely no way to block synchronous code to wait for that, because if you block in code the event loop won't advance and the result will never become available.
Switching an interface from synchronous to asynchronous is a breaking change.
Related
I think I know the answer to this but not sure. Wanting to confirm more than anything. The question applies (I think equally) to all three async approaches (callbacks, promises, async/await), but I'll ask it in the context of promises.
As I understand it, asynchronous programming is intended (at least) for event-driven applications to perform tasks in response to events, without the processes of doing those tasks blocking the ability to do any other tasks (presumably in response to other events) the application. eg. one event might trigger this series of tasks:
Query a database for some data
Wait for the response
Manipulate the data from the response (or handle errors as needed)
Write changes back to the database
done.
In more traditional programming (eg. C/C++) there'd be a main function that calls all that, and might get something back from it. But generally, the process of that main function, sitting around waiting for that to come back, blocks other operations (unless you start manually manipulating threads, or other stuff, that this JS async programming is presumably supposed to spare us from, right?).
But in the above example, supposedly, the (conceptual) "main function" doesn't need anything back from it. Step 5 isn't "return some result I need to act on", it's "done", end of story. If I understand correctly, the above is more likely called by a listener. That listener was set up by JS/node's equivalent of a "main function" -- the code that runs from the entry point of the app -- which has long since ended and the listeners are now running the show. Anything else this entire app needs to do will be triggered by some other independent event, caught by a listener. (Not that this is the only way but as I understand it this is pretty common). So perhaps the user sees the results, and hits another button or whatever to initiate some other, separate, independent, task with it. The above task is long deceased.
Ok... if all that's correct then there's this: For each of those 5 steps, we need to call them in succession. Each step relies on something provided by the one before it. So that process basically needs to be synchronous.
In promise code I believe it looks something like this:
askDbForData() // step 1
.then(responseFromDB => { // step 2
makeTheDesiredChangesToTheData(responseFromDB) // step 3
})
.then(changedData => writeBackToDB(changedData)) // step 4
Looks to me like ultimately this is chaining functions one after the other to perform what is otherwise essentially a synchronous task.
But but but... Synchronous... = blocking?
I've just realized how this doesn't appear to be very clear to me in most of the documentation/articles I've read on this. This is what I need to clear up...
I think the point is: the part where .next(...) picks up the result and sends it off to the next piece in the chain -- that part is "blocking" (though it happens in the blink of an eye so it's kinda moot), but each of those (presumably time consuming) functions (like askDbForData()) -- which are supposed to be asynchronous, and return promises -- do their thing separately and independently of any other control flow, etc, thus not blocking anything.
In other words, the promise chain itself is synchronous, but each piece along the way is asynchronous. The kicker, I think: Not asynchronous to other tasks in the same chain, but asynchronous to everything else the app is doing in other chains initiated by other events.
It might be that anyone reading this will read it and go "yeah well duh, that's the whole point". Perhaps that's what I'm hoping. But if it is, it hasn't been clear in any of my research so far, so it's be great to get it clear from someone(s) who "get" it. I think if all this IS correct, then it pretty much clears up nearly every other confusion I've had with this topic.
So... Yes this is long, but it is only one question. The question is:
Is that it -- is that the point -- or if not, what am I missing?
Thanks!
Looks to me like ultimately this is chaining functions one after the other to perform what is otherwise essentially a synchronous task
No, the code you show there only performs the synchronous part of askDbForData(), then returns. If properly coded, this first part is usually nothing.
I think the point is: the part where .next(...) picks up the result and sends it off to the next piece in the chain -- that part is "blocking" (though it happens in the blink of an eye so it's kinda moot)
Again no, the continuations are only called once the previous step is done. Once a continuation happens, the same split happens, first the synchronous (and hopefully minimal or non-existent) part runs then the rest of the function is registered as a continuation.
In other words, the promise chain itself is synchronous, but each piece along the way is asynchronous.
Absolutely not, the chain of promises is asynchronous. .next() stores a function reference to be called at a later date, it neither calls nor waits for the call of that function code.
The kicker, I think: Not asynchronous to other tasks in the same chain, but asynchronous to everything else the app is doing in other chains initiated by other events.
I've tried to avoid the word "chain" because it seems it's confusing you even more. It's not a chain, it's a tree, you can have promises executed in parallel or sequentially. So again, no.
Consider this.
You have a function A that returns a promise..a promise for something to come.
You also have a function B that does something else.
FunctionA().then(result=>{console.log(result)}).catch();
FunctionB();
If the promise in function A takes a long time to resolve..why not go ahead and do FunctionB while we wait?
Get it?
If you know that the Promise has already been resolved why can't you just call get() on it and receive the value? As opposed to using then(..) with a callback function.
So instead of doing:
promise.then(function(value) {
// do something with value
});
I want to be able to do the much simpler:
var value = promise.get();
Java offers this for it's CompletableFuture and I see no reason why JavaScript couldn't offer the same.
Java's get method "Waits if necessary for this future to complete", i.e. it blocks the current thread. We absolutely never want to do that in JavaScript, which has only one "thread".
It would have been possible to integrate methods in the API to determine synchronously whether and with what results the promise completed, but it's a good thing they didn't. Having only one single method, then, to get results when they are available, makes things a lot easier, safer and more consistent. There's no benefit in writing your own if-pending-then-this-else-that logic, it only opens up possibilities for mistakes. Asynchrony is hard.
Of course it not, because the task will run asynchronously so you can't get result immediately.
But you can use a sync/await to write sequential asynchronous code.
I've got this problem that I couldn't find a solution for by googling.
I've got a library, that I'm using (and do not want to edit, unless it's really necessary) that allows the user to select an item, then calls my custom callback function to modify the item and then continues working with it.
I need to perform some asynchronous tasks on it, which may take some time. This creates a race condition as my async tasks have not yet finished when the callback function is finished and the library continues its work on the item.
library.onItemSelectionCallback = function (item) {
myService.modifyItem(item).then(
function (modifiedItemProperty) {
item.newProperty = modifiedItemProperty;
});
myService.anotherModifyItem(item).then(
function (modifiedItemProperty) {
item.existingProperty = modifiedItemProperty;
});
}
How do I wait for both of my async tasks to finish, before allowing this callback to finish?
Only thing I could think of is looping with while and sleep every hundred or so milliseconds until both of the promises have been resolved, but that doesn't seem to be a very good solution.
I understand that this makes async requests quite synchronous and might possibly be detrimental for UX, but do not really see another way out.
EDIT: I know that i'm risking with removing the generic nature of the question and thus making it too localized, I will say that I'm trying to use angular-file-upload module, specifically, trying to mount a custom imageService, that would resize the picture before it's upload. I'm mounting it on the onBeforeUploadItem callback. The idea is that creating the resized image may take a while and that is why I need to return a promise from my imageService, that needs to be resolved before upload.
If modifyItem and anotherModifyItem work independently (that is, one does not rely on the other), you can just pipe them both into $q.all, eg
library.onItemSelectionCallback = function(item) {
var promises = {
newProperty: myService.modifyItem(item),
existingProperty: myService.anotherModifyItem(item)
};
return $q.all(promises).then(function(values) {
return angular.extend(item, values);
});
}
This will return a promise that resolves with item.
For the first part of my question -- Yes, I guess the only way to really wait for those two promises to be resolved would be something with a while and sleep, making them synchronous, which would probably work and not even be that bad (except for the site pausing until the requests are fulfilled), but would make me feel very, very bad about myself as a person and how my actions affect this world.
It is not possible to correctly mix callbacks and promises without hacks afaik.
For the second part of my question -- as per comments of #georgeawg, figured that an AngularJS module that implements HTML5 API and callbacks instead of $http service and promises is not how a good AngularJS module should be implemented, and so I moved towards a different module ng-file-upload, which, even though one could argue is less stylish, does the job very well and in an Angular way (ng-file-upload provides a simple $upload service, that returns a promise. If you want to modify files before upload, suggested way is to simply $watch and catch the moment user drag-drops or selects a file.).
I'm implementing an API which has a function:
get(url)
Which returns a Response Object, i.e. no callback. The Http Modules I have found implements node-style async functions with callbacks. I have tried to wait for the async function to return in several ways, using Fibers etc. Fibers solves the issue within the Fiber, but can't be used in my case as I need to return the value outside any Fiber.
It might be possible to wrap the Entire Execution (including the code using the API) but I really don't want this. Is there any module that does what I want?
If I understand the question, that you're trying to mix asynchronous code with synchronous code what you're trying to accomplish is not really possible without promises. As soon as you mix asynchronous code with synchronous code, you make the entire code asynchronous, or you risk the synchronous code returning a value before the result from the synchronous code is returned.
You could always try a timeout on the function that holds it returning until a given time period has expired, which hopes that the asynchronous code executes and has a return value before the synchronous code returns. However, this is extremely inefficient, and does not eliminate the problem as you could still end up returning before the asynchronous portion has finished.
I'd also agree with the comment from robertklep that node.js really does not fit your use case, and you might be better looking at another tool for the job.
// synchronous Javascript
var result = db.get('select * from table1');
console.log('I am syncronous');
// asynchronous Javascript
db.get('select * from table1', function(result){
// do something with the result
});
console.log('I am asynchronous')
I know in synchronous code, console.log() executes after result is fetched from db, whereas in asynchronous code console.log() executes before the db.get() fetches the result.
Now my question is, how does the execution happen behind the scenes for asynchronous code and why is it non-blocking?
I have searched the Ecmascript 5 standard to understand how asynchronous code works but could not find the word asynchronous in the entire standard.
And from nodebeginner.org I also found out that we should not use a return statement as it blocks the event loop. But nodejs api and third party modules contain return statements everywhere. So when should a return statement be used and when shouldn't it?
Can somebody throw some light on this?
First of all, passing a function as a parameter is telling the function that you're calling that you would like it to call this function some time in the future. When exactly in the future it will get called depends upon the nature of what the function is doing.
If the function is doing some networking and the function is configured to be non-blocking or asychronous, then the function will execute, the networking operation will be started and the function you called will return right away and the rest of your inline javascript code after that function will execute. If you return a value from that function, it will return right away, long before the function you passed as a parameter has been called (the networking operation has not yet completed).
Meanwhile, the networking operation is going in the background. It's sending the request, listening for the response, then gathering the response. When the networking request has completed and the response has been collected, THEN and only then does the original function you called call the function you passed as a parameter. This may be only a few milliseconds later or it may be as long as minutes later - depending upon how long the networking operation took to complete.
What's important to understand is that in your example, the db.get() function call has long since completed and the code sequentially after it has also executed. What has not completed is the internal anonymous function that you passed as a parameter to that function. That's being held in a javascript function closure until later when the networking function finishes.
It's my opinion that one thing that confuses a lot of people is that the anonymous function is declared inside of your call to db.get and appears to be part of that and appears that when db.get() is done, this would be done too, but that is not the case. Perhaps that would look less like that if it was represented this way:
function getCompletionfunction(result) {
// do something with the result of db.get
}
// asynchronous Javascript
db.get('select * from table1', getCompletionFunction);
Then, maybe it would be more obvious that the db.get will return immediately and the getCompletionFunction will get called some time in the future. I'm not suggesting you write it this way, but just showing this form as a means of illustrating what is really happening.
Here's a sequence worth understanding:
console.log("a");
db.get('select * from table1', function(result){
console.log("b");
});
console.log("c");
What you would see in the debugger console is this:
a
c
b
"a" happens first. Then, db.get() starts its operation and then immediately returns. Thus, "c" happens next. Then, when the db.get() operation actually completes some time in the future, "b" happens.
For some reading on how async handling works in a browser, see How does JavaScript handle AJAX responses in the background?
jfriend00's answer explains asynchrony as it applies to most users quite well, but in your comment you seemed to want more details on the implementation:
[…] Can any body write some pseudo code, explaining the implementation part of the Ecmascript specification to achieve this kind of functionality? for better understanding the JS internals.
As you probably know, a function can stow away its argument into a global variable. Let's say we have a list of numbers and a function to add a number:
var numbers = [];
function addNumber(number) {
numbers.push(number);
}
If I add a few numbers, as long as I'm referring to the same numbers variable as before, I can access the numbers I added previously.
JavaScript implementations likely do something similar, except rather than stowing numbers away, they stow functions (specifically, callback functions) away.
The Event Loop
At the core of many applications is what's known as an event loop. It essentially looks like this:
loop forever:
get events, blocking if none exist
process events
Let's say you want to execute a database query like in your question:
db.get("select * from table", /* ... */);
In order to perform that database query, it will likely need to perform a network operation. Since network operations can take a significant amount of time, during which the processor is waiting, it makes sense to think that maybe we should, rather than waiting rather than doing some other work, just have it tell us when it's done so we can do other things in the mean time.
For simplicity's sake, I'll pretend that sending will never block/stall synchronously.
The functionality of get might look like this:
generate unique identifier for request
send off request (again, for simplicity, assuming this doesn't block)
stow away (identifier, callback) pair in a global dictionary/hash table variable
That's all get would do; it doesn't do any of the receiving bit, and it itself isn't responsible for calling your callback. That happens in the process events bit. The process events bit might look (partially) like this:
is the event a database response? if so:
parse the database response
look up the identifier in the response in the hash table to retrieve the callback
call the callback with the received response
Real Life
In real life, it's a little more complex, but the overall concept is not too different. If you want to send data, for example, you might have to wait until there's enough space in the operating system's outgoing network buffers before you can add your bit of data. When reading data, you might get it in multiple chunks. The process events bit probably isn't one big function, but itself just calling a bunch of callbacks (which then dispatch to more callbacks, and so on…)
While the implementation details between real life and our example are slightly different, the concept is the same: you kick off ‘doing something’, and a callback will be called through some mechanism or another when the work is done.