I'm experiencing some weird behavior when using promises inside of a map.
This isn't really a problem per se, but I'd like to understand what's going on.
let books = [
{Name: "Moby Dick",
AuthorId: 1
},
{Name: "The Great Gatsby",
AuthorId: 2}
]
let authors = [
{AuthorId: 1,
Name: "Herman Melville"
},
{AuthorId: 2,
Name: "F. Scott Fitzgerald"
}
]
const getAuthorName = (AuthorId) => {
return new Promise((resolve, reject) => {
setTimeout(() => {
resolve(authors.find((author) => {
return author.AuthorId === AuthorId
}).Name)
}, 1000
})
}
let responseArray = []
let promises = books.map((book) => (
getAuthorName(book.AuthorId).then((res) => {
responseArray.push({
...book,
AuthorName: res
})
})
))
setTimeout(() => console.log(responseArray), 500)
//I would expect to have to do this:
//Promise.all(promises).then((res) => console.log(res))
I would expect the
setTimeout(() => console.log(responseArray), 5000)
to log an empty string, because the array of promises hasn't been run through Promise.all yet, but it seems like even though the map should just be returning an array of promises, it's actually running the promises. Why is this?
Edit
I've edited the getAuthor promise to wait a second before resolving to further elaborate because that wasn't the point I was trying to get at.
I would expect mapping over an array, to return a new array with whatever is returned in the map. For example, if I do
let arrayOfPromises = books.map((book) => {
return getAuthor(book.AuthorId)
}
I would expect arrayOfPromises to be an array of getAuthor promises.
However, when I throw a .then() on the end of the promise that is being returned, it appears that the code in the .then() is evaluating.
So if I do
let promises = books.map((book) => {
return getAuthor(book.AuthorId).then((res) => {
console.log(res)
})
}
In the console I'll see "Herman Merville" and "F. Scott Fitzgerald" and in the promises var I'll have an array of promises.
While I thought that each getAuthor's .then would only evaluate once I call Promise.all(promises) because the getAutor promise being returned inside the map. Am I understanding something wrong here?
That's because the code inside your promise isn't asynchronous, so it's resolving instantly as there's nothing to wait for.
To expand a bit on this, what you are doing is the same as
const getAuthorName = (AuthorId) => {
return Promise.resolve(authors.find((author) => {
return author.AuthorId === AuthorId
}).Name)
}
It will allow you to chain a then syntax on the function, but that also mean that the then will be executed directly after the function call.
edit
Following your edit, here's a bit more.
You've modified the method to be asynchronous using the setTimeout. Which means that the map is now return an array of promises as you expected.
However there's another issue with your code, you have also encapsulated the console.log in a setTimeout, one with an even greater timer!
To better understand what is going on here, I highly recommend that you watch this video.
However in the spirit of StackOverflow, I'll add some written explanations here.
You are first creating an array of promises, executing callbacks after approximately 1s after their creations (they won't wait for the map to be over).
As soon as they are done, they go in what's called the event loop of your browser. Basically it's a pile of events that your browser will handle as soon as the main thread is freed.
So 1s after their creations the promises will resolve, and place their callbacks on this event loop.
Now what happens when you run your console.log.
Well if you didn't put it in its own setTimeout, it would print an empty array.
What if it was a setTimeout of 500ms? Here we can safely assume that the map would take less than 500ms to finish and start the new setTimeout so the console.log would reach the event loop first and print an empty array once again.
What about 5s? Both setTimeouts will be over long before the console.log is reached, so you will print an filled array.
I made a jsfiddle if you want to try it out for yourself.
I hope this helps clear things up.
Related
In JavaScript, I'm calling a promise which returns an id; we'll call this promise1. promise1 has a following .then() which, given the id returned from promise1, calls a for-each loop. On each iteration of this loop, promise2 is carried out
promsie2 also has a .then(). so the data returned from promise2 can be utilized.
mycode.js
exports.my_function = (req, res) => {
var data = req.body;
var name_array = ["Jeff", "Sophie", "Kristen"]
var personal_ID_arr = []
promise1.getID(data) //promise1 gets an int 'id'
.then(id => {
array.forEach(name => { //for each name
promise2.getPersonalID(name, id) //promise2 creates a personal id for each person using their name and the id generated from promise1
.then(personalID => {
personal_ID_arr.push(personalID) //add the generated personal id to an arr
})
})
})
//will need to operate on 'personal_ID_arr' here after the above finishes or possibly still within the promise1's .then; as long as the arr is filled fully
}
However, I'm running into the issue of synchronicity not allowing for this type of logic to happen, as things begin happening out of order once it gets to the loop aspect of the program. I need to carry out another function with the filled-out 'personal_ID_arr' after these promises are carried out as well
I've looked at other questions regarding promise chaining but this is a different case due to the fact the for loop needs to work with the data from the first promise and use .then() inside a .then(). Surely, I don't need to keep this structure though if a better one exists.
Any help would be greatly appreciated
Keeping an array of IDs that an asynchronous operation will fill, is not something that will help you carrying out more asynchronous operations on the array later. Instead, consider using an array of Promises for each name, and wait them all to resolve using Promise.all. Also, i removed the personal_ID_arr from the function my_function variables and moved it to the appropriate asynchronous operation. I did a simple console log of the array, you should fill your next step there.
Although maybe you need to write this logic with Promises, but I would suggest using async / await for this task, it will lead to more readable code.
exports.my_function = (req, res) => {
var data = req.body;
var name_array = ["Jeff", "Sophie", "Kristen"]
promise1.getID(data) //promise1 gets an int 'id'
.then(id =>
//map the array of Strings to array of Promises instead, wait for them to resolve
Promise.all(name_array.map(name =>
promise2.getPersonalID(name, id)
))
)
.then(personal_ID_arr => console.log(personal_ID_arr))
//will need to operate on 'personal_ID_arr' here after the above finishes or possibly still within the promise1's .then; as long as the arr is filled fully
}
I have been reading up on methods to implement a polling function and found a great article on https://davidwalsh.name/javascript-polling. Now using a setTimeout rather than setInterval to poll makes a log of sense, especially with an API that I have no control over and has shown to have varying response times.
So I tried to implement such a solution in my own code in order to challenge my understanding of callbacks, promises and the event loop. I have followed guidance outlined in the post to avoid any anti-patterns Is this a "Deferred Antipattern"? and to ensure promise resolution before a .then() promise resolve before inner promise resolved and this is where I am getting stuck. I have put some code together to simulate the scenario so I can highlight the issues.
My hypothetical scenario is this:
I have an API call to a server which responds with a userID. I then use that userID to make a request to another database server which returns a set of data that carries out some machine learning processing that can take several minutes.
Due to the latency, the task is put onto a task queue and once it is complete it updates a NoSql database entry with from isComplete: false to isComplete: true. This means that we then need to poll the database every n seconds until we get a response indicating isComplete: true and then we cease the polling. I understand there are a number of solutions to polling an api but I have yet to see one involving promises, conditional polling, and not following some of the anti-patterns mentioned in the previously linked post. If I have missed anything and this is a repeat I do apologize in advance.
So far the process is outlined by the code below:
let state = false;
const userIdApi = () => {
return new Promise((res, rej) => {
console.log("userIdApi");
const userId = "uid123";
setTimeout(()=> res(userId), 2000)
})
}
const startProcessingTaskApi = userIdApi().then(result => {
return new Promise((res, rej) => {
console.log("startProcessingTaskApi");
const msg = "Task submitted";
setTimeout(()=> res(msg), 2000)
})
})
const pollDatabase = (userId) => {
return new Promise((res, rej) => {
console.log("Polling databse with " + userId)
setTimeout(()=> res(true), 2000)
})
}
Promise.all([userIdApi(), startProcessingTaskApi])
.then(([resultsuserIdApi, resultsStartProcessingTaskApi]) => {
const id = setTimeout(function poll(resultsuserIdApi){
console.log(resultsuserIdApi)
return pollDatabase(resultsuserIdApi)
.then(res=> {
state = res
if (state === true){
clearTimeout(id);
return;
}
setTimeout(poll, 2000, resultsuserIdApi);
})
},2000)
})
I have a question that relates to this code as it is failing to carry out the polling as I need:
I saw in the accepted answer of the post How do I access previous promise results in a .then() chain? that one should "Break the chain" to avoid huge chains of .then() statements. I followed the guidance and it seemed to do the trick (before adding the polling), however, when I console logged out every line it seems that userIdApi is executed twice; once where it is used in the startProcessingTaskApi definition and then in the Promise.all line.
Is this a known occurrence? It makes sense why it happens I am just wondering why this is fine to send two requests to execute the same promise, or if there is a way to perhaps prevent the first request from happening and restrict the function execution to the Promise.all statement?
I am fairly new to Javascript having come from Python so any pointers on where I may be missing some knowledge to be able to get this seemingly simple task working would be greatly appreciated.
I think you're almost there, it seems you're just struggling with the asynchronous nature of javascript. Using promises is definitely the way to go here and understanding how to chain them together is key to implementing your use case.
I would start by implementing a single method that wraps setTimeout to simplify things down.
function delay(millis) {
return new Promise((resolve) => setTimeout(resolve, millis));
}
Then you can re-implement the "API" methods using the delay function.
const userIdApi = () => {
return delay(2000).then(() => "uid123");
};
// Make userId an argument to this method (like pollDatabase) so we don't need to get it twice.
const startProcessingTaskApi = (userId) => {
return delay(2000).then(() => "Task submitted");
};
const pollDatabase = (userId) => {
return delay(2000).then(() => true);
};
You can continue polling the database by simply chaining another promise in the chain when your condition is not met.
function pollUntilComplete(userId) {
return pollDatabase(userId).then((result) => {
if (!result) {
// Result is not ready yet, chain another database polling promise.
return pollUntilComplete(userId);
}
});
}
Then you can put everything together to implement your use case.
userIdApi().then((userId) => {
// Add task processing to the promise chain.
return startProcessingTaskApi(userId).then(() => {
// Add database polling to the promise chain.
return pollUntilComplete(userId);
});
}).then(() => {
// Everything is done at this point.
console.log('done');
}).catch((err) => {
// An error occurred at some point in the promise chain.
console.error(err);
});
This becomes a lot easier if you're able to actually use the async and await keywords.
Using the same delay function as in Jake's answer:
async function doItAll(userID) {
await startTaskProcessingApi(userID);
while (true) {
if (await pollDatabase(userID)) break;
}
}
I would like to do a synchronous loop in a part of my code.
The function saveInDatabase checks if the item title (string) already exists in the database. That's why it can't be resolved in parallel, otherwise the condition will never apply (and would create duplicates).
Promise.all(arr.map(item => {
saveInDatabase(item).then((myResult) => ... );
}));
I tried to encapsulate this function into separate promises, also tried with npm packages (synchronous.js, sync), but it seems that it does not fit with my code.
Maybe this solution is completely silly.
Do you think it's a better idea to replace promise.all by a synchronous loop (foreach for example) ? The problem is that I need the results of each iteration...
I'm using Node 6.11.2. Could you give me some tips to handle that ?
Thank you in advance.
Without using await (which is not in node.js v6.11.2, but would make this simpler), a classic pattern for serializing a bunch of async operations that return a promise is to use a reduce() loop like this:
arr.reduce(function(p, item) {
return p.then(function() {
return saveInDatabase(item).then((myResult) => ... );
});
}, Promise.resolve()).then(function() {
// all done here
}).catch(function(err) {
// error here
});
If you want to save all the results, you can use your .then(myResult => ...) handler to .push() the result into an array which you can access when done.
This will serialize all the calls to saveInDatabase(item) to it waits for the first one to be done before calling the second one, waits for the second one to be done before calling the third one, etc...
The default implementation here will stop if saveInDatabase(item) rejects. If you want to keep going (you don't say in your question), even when it gives an error, then you can add a .catch() to it to turn the rejected promise into a fulfilled promise.
In node.js v7+, you can use await in a regular for loop:
async function myFunc() {
let results = [];
for (let item of arr) {
let r = await saveInDatabase(item).then((myResult) => ... );
results.push(r);
}
return results;
}
myFunc().then(function(results) {
// all done here
}).catch(function(err) {
// error here
});
If you could run all the requests in parallel, then you could do that like this:
Promise.all(arr.map(item => {
return saveInDatabase(item).then((myResult) => ... );
})).then(function(results) {
// all done here
}).catch(function(err) {
// error here
});
In any of these, if you don't want it to stop upon a rejection, then add a .catch() to your saveInDatabase() promise chain to turn the rejection into a resolved promise with some known value or error value you can detect.
I have a sequence of events I'm trying to align, but I don't understand how to resolve a promise after a single line of asynchronous code. In this case, I'm making an API call which resolves to an array of objects. I'm then using Object.assign() to convert the array of objects into an object of objects. Once completed, I then want to call a function which does something with that new object of objects. The conversion process takes some time, and I'm not sure how to postpone my function call until after the object has been converted. My non-functioning code is given below.
this.queryCodes().then(() => {
new Promise((resolve) => {
resolve(() => {
// This code isn't executing.
this.floorCodesLookup = Object.assign({}, ...this.floorCodes);
});
}).then((data) => {
this.resolveCodesToDescriptions();
});
});
resolve should be given the result of your asynchronous operation, not the code to execute. The code you want to execute shouldn't go in the resolve argument, but instead in the callback you give to the promise constructor. So for your snippet:
this.queryCodes().then(() => {
return new Promise((resolve) => {
// Put the code you want to execute here...
// Call resolve when your work is done, and pass it the result if there is one.
resolve(/* maybe result here */);
}).then((data) => {
this.resolveCodesToDescriptions();
});
});
It's also worth noting that you don't need to use a promise constructor since you're already chaining off a promise to begin with. You could just:
this.queryCodes().then(() => {
// Do additional work here...
this.floorCodesLookup = Object.assign({}, ...this.floorCodes);
return this.resolveCodesToDescriptions();
});
resolve() doesn't take a callback as an argument. It takes the resolved value as an argument. Not sure where you got the callback notion from. That's why the callback is never called because that's not a supported feature of resolve().
Manually creating a promise inside a .then() handler is likely not needed and is probably an anti-pattern. You don't show enough of the real code here to understand what is going on, but this is likely way more complicated than it needs to be. From the code you show, I think you can just do this:
this.queryCodes().then(() => {
this.floorCodesLookup = Object.assign({}, ...this.floorCodes);
this.resolveCodesToDescriptions();
});
Lets say there is an array of numbers, periodically with setInterval you need to get and delete first 5 elements and based on those 5 numbers you need to do some async stuff as follows
for each of five numbers get something from DB
for each record in result set from DB do another async operation
error handling should be done in that way if any error occur along the way that number should be returned to array ensuring whole thing (1. and 2.) to be repeated for that number
Getting something from DB based on some number is totally independent of doing the same with another number. Same apply for fetched records. So in both 1. and 2. Promise.all sounds fantastic, but fact that Promise.all rejects if any passed promise rejects doesn't play nicely with described scenario. Basically if all calls to DB resolves with records except last one which rejects, i'm not able to continue with successful ones and their records.
So not being able to solve this with native Promise support, i looked into bluebird with whom i have very little/no experience which is the reason i'm posting this question. Going through API reference as it seems to me this could easily be solved with .reflect()? What i have tried:
setInterval(() => {
const firstFive = numbers.splice(0, 5)
Promise.all(firstFive.map((number) => firstAsync(number).reflect()))
.each((result, index) => {
if (result.isRejected()) {
return numbers.unshift(firstFive[index])
}
return Promise.all(result.value().map((record) => secondAsync(record).reflect()))
.each((result, index) => {
if(result.isRejected()) {
return numbers.unshift(firstFive[index])
}
})
})
}, 500)
Am i using .reflect() in intended way? Besides that im blindly guessing here that .each((result, index) => {...}) is syntactic sugar for something like .then((results) => results.forEach((result, index) => {...}), isn't?
I wonder why you are using Promise.all at all. You say that the five numbers are totally independent of each other, and it doesn't look like you needed to wait for all five of them to do anything.
So just handle them separately:
function doSomething(number) {
return firstAsync(number)
.then(secondAsync)
.catch(e => {
// something went wrong:
numbers.unshift(number); // do it again in next batch
});
}
setInterval(() => {
const firstFive = numbers.splice(0, 5);
firstFive.forEach(doSomething);
}, 500);