I am working on a NestJS API which receives some article codes and saves them in a DB with status as queue and then i want to build a background process which searches for articles with queue status and fetches a website with that code and fills in the information about that article ID.
I want it to fetch codes in 5 parallel routines, each one should get a new code from the list as soon as it finished fetching the current code.
The way it's built right now, it gets 5 codes and uses Promise.all but what i don't like on this approach is that if one of the await is significantly slower than the other, they all way for the slowest one.
const promiseResult = await Promise.all(promiseList);
Which approach is the best one for me to learn in order to achieve what i am planning to do?
Basically you do your job when all the promises are fulfilled. You are correct when you say that you wait for the slowest one, as this is exactly what await Promise.all(promiseList) does.
Now you need to ask yourself the question whether you are able to process the quicker promises while you await for the slowest one. If so, then you can iterate your promises and define a .then() for them:
for (let promise of promiseList) {
promise.then((value) => {
//Do something
});
}
This way all your promises have a then handler for the case when the promise is fulfilled and your Javascript flows normally, without waiting for any of the promises. Once each promise is fulfilled, the event loop will detect that an event happened and call your callback function (in the case of the example above, the arrow function inside the then call).
Note that the then callback should never assume that the other promises were also completed, but that shouldn't be a problem in your case, because you want to work when your promise list was only partially completed as well.
Related
Does Promise.all() run in sequential or parallel in Javascript?
For Example:
const promises = [promise1(), promise2(), promise3()]
Promise.all(promises)
.then(data => {
// whatever
});
Does promise1() execute and resolve before moving onto promise2() or does promise1(), promise2(), and promise 3() all run in parallel at the same time? I would assume like Node, Javascript in the browser to be single threaded thus they don't run in parallel?
Javascript is a single threaded application. However, asynchronous calls allow you to not be blocked by that call. This is particularly useful when making REST API calls. For example your promise1() can make a REST API call and before it waits for the results, another REST API call can be made from promise2(). This pseudo-parallelism is thus achieved by not waiting on API servers to do the tasks and fire multiple such calls to either same or different API endpoints in parallel. This allows your code to continue executing that parts that are not dependent on resolution of the promises.
So yes, promise1(), promise2() and promise3() can be said to be running in parallel in that respect. And there is a chance that promise2() gets resolved before promise1() and so on. The function Promise.all() waits for all the promises provided to it to fulfill or at least one of them to fail.
Learn more about Javascript event loops in this video by Jake Archibald.
Promise.all does not make your promises run in parallel.
Promise.all does not make your promises run at all.
What Promise.all does, is just waiting for all the promises to complete.
The line of code that actually executes things is this one:
const promises = [promise1(), promise2(), promise3()]
Assuming that your promises make HTTP calls:
promise1() is executed -> 1 HTTP call going on
promise2() is executed -> 2 HTTP calls going on
promise3() is executed -> 3 HTTP calls going on
then after a while, in undetermined order, these promises complete.
These 3 HTTP calls could complete in the same time, but in your code, you will have 3 sequential completition. For this reason, you can use Promise.all to assure that your callback is executed only when all of your promises complete.
Remember that there's a limit on the amount of parallel HTTP connections you can have inyour environment, look at this: https://stackoverflow.com/a/985704/7327715
I'm aware this is a common problem asked by developers but I can't seem to find anyone who has the same issue as me.
I've got a game i'm creating, this game has lots of factories that when called return a game object, such as a player. Before the game even starts, i need to make sure all the images required are loaded, i did this by simply converting the .onload event into a promise, and placing that into an async function with an await call. However, due to changes in how i'm organizing the game, i will need to handle image loading in those factories that create game objects. This is a problem because now the game doesn't know when all the images are loaded. I thought I might be able to make those factories asynchronous and await on them, they will have an async function call that is also awaited on, but I don't believe this is possible. So abstracted, this is what I want:
I have a promise gotBanana, which resolves once I have gotten my banana.
this is in an async function called, getBananas(), which gets lots of bananas in the loop, it waits for the gotBanana promise to resolve every loop iteration.
I want to put this in another (probably async) function, but I want that function to await for the getBananas() function to finish.
Is it possible to do this or do i need to handle this situation in a different manner?
Maybe you're looking for Promise.all that resolves all promises in parallel and return promise when it will be done. So you just need to map all your assets (images or other) and await it.
I am relatively new to NodeJS. I was wondering if the await keyword will slow down the entire javascript/nodeJS program?
For example,
If I have many express routers written on a single server file, and one router function calls the 'await' for a promise to resolve, will all others routers and asynchronous functions stay on halt/paused until the promise is resolved? Or just that thread will be paused?
In such a case the await call will cause performance issues to the Javascript program?
No. While await sounds like it is blocking, it is fully asynchronous(non blocking) as it is also implied in the required function signature by async keyword. It is a (lot) nicer way to use promises. So go ahead and use it in your code.
You also mentioned threads, I suggest you ignore thread concept while developing node.js apps and trust the node.js event loop. Just never use blocking IO calls(which are explicitly named so by having 'Sync' in the name).
await waits for a promise to get resolved, but as we know that node is asynchronous in nature, therefore, other requests will be made to the application will have no effect on them they will not wait for the previous request promise to be resolved.
example
route 1 -> it will await and iterate through million rows and return sum in response
route 2 -> it will only return '1' in response
now when we'll call route 1 first then route 2 then you'll see that still you'll get the response from route 2 immediately and when route 1 will get completed you'll get that response.
I'm using Promises in Angular (4) project and I have a question about them that I couldn't find a response to it in the docs.
When I create a Promise, I basically wait for an async answer from a service/party. But how long should I expect this Promise to stay in pending state?
Is there any mechanism that will terminate it after a while?
How reliable is this concept of waiting/pending?
Let's suppose that I need to get some data from a busy service that can answer even after few minutes of waiting, maybe more, no matter if the computing of the response is a resource intensive process or that service is linked with another one that is responding very slow.
Is there anything on the client side that will somehow terminate my Promise and determine/force to create another one to ask again for my data?
Someone suggested to upgrade to Observables, and I will do that, but for now I want to keep using Promises, at least for some areas of the code.
Tks a lot
A Promise can be in pending state as long as the page is loaded.
You can wrap the call in another Promise where you introduce a timeout like shown in
let wrappingPromise = new Promise((resolve, reject) => {
var error = false;
setTimeout(function(){
reject("some error");
}, 3000);
this.http.get(...).toPromise().then(res => {
if(!error) {
resolve(res.json);
}
});
});
This will cause an error when the timeout is reached.
It will still wait to receive the full response.
An Observable might be able to forward a cancellation and close the connection, so that the result isn't even received anymore when the timeout is reached. This might depend on whether the concrete implementation and the browser used browser API supports that.
new Promise(() => {}) will never settle, like a callback never called.
A promise is a return object you attach callbacks to, instead of passing callbacks into the function. That's all. It is not a control surface of the asynchronous operation that was just started.
Instead, look to the asynchronous API you called for such controls, if it has them.
Creating promises
Most people are consumers of promises returned from asynchronous APIs. There's no reason to create a Promise other than to wrap a legacy callback API. In an ideal world there'd be no need.
I've got this problem that I couldn't find a solution for by googling.
I've got a library, that I'm using (and do not want to edit, unless it's really necessary) that allows the user to select an item, then calls my custom callback function to modify the item and then continues working with it.
I need to perform some asynchronous tasks on it, which may take some time. This creates a race condition as my async tasks have not yet finished when the callback function is finished and the library continues its work on the item.
library.onItemSelectionCallback = function (item) {
myService.modifyItem(item).then(
function (modifiedItemProperty) {
item.newProperty = modifiedItemProperty;
});
myService.anotherModifyItem(item).then(
function (modifiedItemProperty) {
item.existingProperty = modifiedItemProperty;
});
}
How do I wait for both of my async tasks to finish, before allowing this callback to finish?
Only thing I could think of is looping with while and sleep every hundred or so milliseconds until both of the promises have been resolved, but that doesn't seem to be a very good solution.
I understand that this makes async requests quite synchronous and might possibly be detrimental for UX, but do not really see another way out.
EDIT: I know that i'm risking with removing the generic nature of the question and thus making it too localized, I will say that I'm trying to use angular-file-upload module, specifically, trying to mount a custom imageService, that would resize the picture before it's upload. I'm mounting it on the onBeforeUploadItem callback. The idea is that creating the resized image may take a while and that is why I need to return a promise from my imageService, that needs to be resolved before upload.
If modifyItem and anotherModifyItem work independently (that is, one does not rely on the other), you can just pipe them both into $q.all, eg
library.onItemSelectionCallback = function(item) {
var promises = {
newProperty: myService.modifyItem(item),
existingProperty: myService.anotherModifyItem(item)
};
return $q.all(promises).then(function(values) {
return angular.extend(item, values);
});
}
This will return a promise that resolves with item.
For the first part of my question -- Yes, I guess the only way to really wait for those two promises to be resolved would be something with a while and sleep, making them synchronous, which would probably work and not even be that bad (except for the site pausing until the requests are fulfilled), but would make me feel very, very bad about myself as a person and how my actions affect this world.
It is not possible to correctly mix callbacks and promises without hacks afaik.
For the second part of my question -- as per comments of #georgeawg, figured that an AngularJS module that implements HTML5 API and callbacks instead of $http service and promises is not how a good AngularJS module should be implemented, and so I moved towards a different module ng-file-upload, which, even though one could argue is less stylish, does the job very well and in an Angular way (ng-file-upload provides a simple $upload service, that returns a promise. If you want to modify files before upload, suggested way is to simply $watch and catch the moment user drag-drops or selects a file.).