This question already has answers here:
How do I return the response from an asynchronous call?
(41 answers)
Closed 5 years ago.
I'm trying to chain promises together to make a series of requests. This is one of the few times I've used promises, so I'm not too sure what I'm doing. Hoping that someone can catch whats going on wrong. Basically, my first promise is working correctly, then the second function runs after the first promise is resolved, however, the third does not wait so the information I'm requesting is incomplete.
I can see that my functions are returning the correct information. So mainly my question is how do I format my promises so they each wait for each other to complete before running?
function getPaths(folderpath) {
return new Promise(function(resolve, reject) {
db.getPaths(folderpath, function(response) {
//get paths for files and folders
for (let i = 0; i < response.entries.length; i++) {
let entries = response.entries[i];
if (entries[".tag"] == "folder") {
folderPaths.push(response.entries[i].path_lower);
}
if (entries[".tag"] == "file") {
filePaths.push(response.entries[i].path_lower);
}
}
resolve();
});
});
}
function openFolders(folders) {
return new Promise(function(resolve, reject) {
for (let i = 0; i < folders.length; i++) {
db.getPaths(folders[i], function(response) {
for (let j = 0; j < response.entries.length; j++) {
let entries = response.entries[j];
if (entries[".tag"] == "file") {
filePaths.push(response.entries[j].path_lower);
}
}
//console.log(filePaths); //returns correct information
});
}
resolve();
});
}
getPaths("/path").then(function() {
openFolders(folderPaths);
}).then(function() {
console.log(filePaths); //returns incomplete information
});
You have a big problem here in bold
function openFolders(folders) {
return new Promise(function(resolve, reject) {
for (let i = 0; i < folders.length; i++) {
db.getPaths(folders[i], function(response) {
// ...
});
}
resolve();
});
}
You're running a synchrounous for loop around an asynchronous function call. The loop finishes iterating synchronously and then immediately calls resolve() - the code in the async handler (eg, filePaths.push(...)) does not run before the Promise is resolved
Your problems don't stop there though. You're using Promises unconventionally by modifying global state and then resolving and empty Promise - resolve() vs resolve(someValue). #FrankModica's answer has more to say about that.
As for remedies, I'd recommend you take a look at Promise.all - and in general, using Promises in a more conventional way
In the snippet below, I've mocked your db function to read from a fake in-memory database, fakedb. Then, I made a getPaths function which wraps db.getPaths but returns a Promise instead. Then I rewrite openFolders to create an array of Promises, pass them to Promise.all, then finally resolve the values I want
Worth noting: Promise.all will run your array of Promises in parallel - if this undesirable, you could run them in serial using .reduce and .then.
const db = {
getPaths: (path, k) => {
setTimeout(k, 50, fakedb[path])
}
}
function getPaths (path) {
return new Promise((resolve, reject) =>
db.getPaths(path, response => resolve(response)))
}
function openFolders (folders) {
return Promise.all(folders.map(getPaths))
.then(responses =>
responses.reduce((acc, {entries}) =>
acc.concat(entries.filter(x => x['.tag'] === 'file').map(x => x.path_lower)), []))
}
const fakedb = {
'cat': {
'entries': [
{ '.tag': 'dir', 'path_lower': './cat/.' },
{ '.tag': 'dir', 'path_lower': './cat/..' },
{ '.tag': 'file', 'path_lower': './cat/a' },
{ '.tag': 'file', 'path_lower': './cat/b' },
{ '.tag': 'file', 'path_lower': './cat/c' }
]
},
'dog': {
'entries': [
{ '.tag': 'dir', 'path_lower': './dog/.' },
{ '.tag': 'dir', 'path_lower': './dog/..' },
{ '.tag': 'file', 'path_lower': './dog/a' },
{ '.tag': 'file', 'path_lower': './dog/b' },
{ '.tag': 'file', 'path_lower': './dog/c' }
]
}
}
openFolders(['cat','dog']).then(console.log, console.error)
The operation I'm performing in openFolders might feel a little complex because it handles all transformations of the responses in a single .then handler - you could (optionally) separate the work into multiple .then calls which might result in a lighter cognitive load
function openFolders (folders) {
return Promise.all(folders.map(getPaths))
// get `.entries` of each response
.then(responses =>
responses.map(x => x.entries))
// flatten array of entries arrays into a single array
.then(arrOfEntries =>
arrOfEntries.reduce((acc, entries) =>
acc.concat(entries), []))
// only keep entries where `.tag` of each entry is `'file'`
.then(entries =>
entries.filter(x => x['.tag'] === 'file'))
// return `path_lower` of the resulting entries
.then(entries =>
entries.map(x => x.path_lower))
}
If I understand correctly, you are using some variables from the outer scope (folderPaths and filePaths). I'm not sure that's the best way to go about this, but I believe it will start working once you return the promise from openFolders. This way you wait until openFolders completes:
getPaths("/path").then(function() {
return openFolders(folderPaths);
}).then(function() {
console.log(filePaths);
});
But I'd recommend resolving the information required for the next call:
resolve(folderPaths);
And:
resolve(filePaths);
So your code can look more like:
getPaths("/path").then(function(folderPaths) {
return openFolders(folderPaths);
}).then(function(filePaths) {
console.log(filePaths);
});
Edit: Looks like you may have another issue mentioned by #naomik. If db.getPaths is async, in your openFolders function you may be resolving before all of the async calls in the loop complete. Unfortunately I don't have time right now to show the solution. Hopefully #naomik can!
Related
I've this usecase wherein I wish to do the following :
Set metadata in indexDb
Iterate over an array of images
See if img is already set in indexDb
if yes, do nothing, if not, download the img
set the downloaded img (as blob) in indexDb
Raise all images processed event at the end
ads data :
[{
ETag:"",
S3URL:"",
duration:30,
filename:"",
linear-gradient:"",
status:"",
timerRequired:"yes"
}]
My code at the moment :
this.Tvlocalforage.setItem('meta', newMeta).then(() => { //Step 1
for (let idx in ads) { //Step 2
this.localforage.getItem(ads[idx]['filename']).then(blob => {
if(!blob){ //Step 3
LSPromise = imgSrcToBlob(ads[idx]['S3URL'], undefined, 'Anonymous', 1).then((blob) => { //Step 4
return this.localforage.setItem(ads[idx]['filename'], blob); //Step 5
});
LSPromises.push(LSPromise);
}
});
}
}).then(() => {
if(LSPromises.length) {
Promise.all(LSPromises).then((data) => {
this.TvLSkeyCount = LSPromises.length;
this.fireLoadAssetsEvent(); //Step 6
});
}
});
Problems I am facing :
After the promise for setting metadata is resolved, it straightaway goes to then() block and by that time LSPromises is null. Of course I understand that internal nested promises haven't been resolved yet.
Resolution I tried :
Return LSGetter promises and download images later. this did not work either.
Code I tried :
this.Tvlocalforage.setItem('meta', newMeta).then(() => {
for (let idx in ads) {
let p = this.Tvlocalforage.getItem(ads[idx]['filename']);
LSPromises.push({'promise' : p, 'filename' : ads[idx]['filename'], 'url' : ads[idx]['S3URL']});
}
}).then(() => {
if(LSPromises.length){
Promise.all(LSPromises.map(obj => {
obj['promise'].then(blob => {
if(!blob){
imgSrcToBlob(obj['url'], undefined, 'Anonymous', 1).resolve(blob => {
return this.Tvlocalforage.setItem(obj['filename'], blob);
});
}
});
})).then((data) => {this.fireLoadAssetsEvent();});
}
I tried 2 more ways to wrapper up and try to return promise.all of download step from inside & on trying to resolve that, return promise.all of set downloaded images to LS. But it did not work.
Get rid of the for() loop since idx will not be what you want it to be inside the promise callback since loop will complete before promises do
Can use map() instead to create the array using a closure
Something like:
this.Tvlocalforage.setItem('meta', newMeta).then(() => { //Step 1
let LSPromises = ads.map(ad => {
return this.localforage.getItem(ads[idx]['filename']).then(blob => {
if (!blob) { //Step 3
return imgSrcToBlob(ad['S3URL'], undefined, 'Anonymous', 1).then((blob) => { //Step 4
return this.localforage.setItem(ad['filename'], blob); //Step 5
});
}
return null
});
});
return Promise.all(LSPromises).then((data) => {
this.TvLSkeyCount = data.filter(o => o).length;
this.fireLoadAssetsEvent(); //Step 6
// not sure what needs to be returned here
});
});
There might be other errors, but there is a missing return:
this.Tvlocalforage.setItem('meta', newMeta).then(() => { //Step 1
for (let idx in ads) { //Step 2
LSPromises.push(this.localforage.getItem(ads[idx]['filename']).then(blob => {
if(!blob){ //Step 3
return /* added return */ imgSrcToBlob(ads[idx]['S3URL'], undefined, 'Anonymous', 1).then((blob) => { //Step 4
return this.localforage.setItem(ads[idx]['filename'], blob); //Step 5
});
// LSPromises.push(LSPromise);
}
}));
}
// }).then(() => {
if(LSPromises.length) {
return /* <<<=== */ Promise.all(LSPromises).then((data) => {
this.TvLSkeyCount = LSPromises.length;
this.fireLoadAssetsEvent(); //Step 6
});
}
});
If the promise returned from Promise.all() is not returned, the caller can not wait for it to complete.
I could not test this, but you should try to flatten the nesting, chaining the thens at the outermost level. You can use Promise.all even more in order to pass the ad value through the chain together with the resolved values:
this.Tvlocalforage.setItem('meta', newMeta).then(() => // Step 1
Promise.all(ads.map( ad => // Step 2
Promise.all(ad, this.localforage.getItem(ad.filename))
))
).then(blobs =>
blobs.filter( ([ad, blob]) => !blob ) // Step 3
).then(blobs =>
Promise.all(blobs.map( ([ad]) =>
[ad, imgSrcToBlob(ad.S3URL, undefined, 'Anonymous', 1)] // Step 4
))
).then(blobs =>
Promise.all(blobs.map( ([ad, blob]) =>
this.localforage.setItem(ad.filename, blob) // Step 5
))
).then(data => {
this.TvLSkeyCount = data.length;
this.fireLoadAssetsEvent(); // Step 6
});
Say I have an array of objects which have asynchronous methods:
[
{
partOne: function(input) {
// Do something async
},
partTwo: function(result) {
// Do something w/ result of partOne
}
},
{
partOne: function(resultOfPrevious) {
// Do something async
},
partTwo: function(result) {
// Do something w/ result of partOne
}
},
{
partOne: function(resultOfPrevious) {
// Do something async
},
partTwo: function(result) {
// Do something w/ result of partOne
}
}
]
I want to execute partOne of the first object with my input, pass the result (async) to the partTwo callback, then pass the result of partTwo as input to partOne of the next object and so on. The array may be of one or more objects. I'm wondering what the best pattern to execute this kind of code is?
It is somewhat similar to the waterfall method of async.js: https://caolan.github.io/async/docs.html#waterfall, but I wonder how I can do this without a library and possibly with cleaner code?
Not sure if async/await might help here?
Another option without collecting every callback to an array, using async/await:
async function processWaterfallObject (data, input) {
let result = input
for (let entry of data) {
result = await entry.partOne(result)
result = await entry.partTwo(result)
}
return result
}
This assumes that functions in your data array are either async or return a Promise.
async/await is currently supported by every major browser and is available in node since 7.6.0.
Here is a simple function to invoke each asynchronous function in a stack
function drain(stack, initialArg) {
stack.reduce(function (sectArg, section) {
return Object.keys(section).reduce(async function (arg, key) {
return await section[key].call(null,arg)
}, sectArg)
}, initialArg)
}
To use it ensure that each function in you stack returns a value
var stack = [
{
partOne: function(input) {
// Do something async
console.log('section one partOne', input)
return 'section one partOne output'
},
partTwo: function(result) {
// Do something w/ result of partOne
console.log('section one partTwo', result)
return 'section one partTwo output'
}
},
{
partOne: function(resultOfPrevious) {
// Do something async
console.log('section two partOne', resultOfPrevious)
return 'section two partOne output'
},
partTwo: function(result) {
// Do something w/ result of partOne
console.log('section two partTwo', result)
return 'section two partTwo output'
}
},
{
partOne: function(resultOfPrevious) {
// Do something async
console.log('section three partOne', resultOfPrevious)
return 'section three partOne output'
},
partTwo: function(result) {
// Do something w/ result of partOne
console.log('section three partTwo', result)
return 'section three partTwo output'
}
}
]
So that you can invoke the stack like
drain(stack, 'initialArg')
See this jsfiddle: https://jsfiddle.net/kqj0rror/
Assuming your array of objects given in the original question is under a variable called waterfall
let collector = [];
for (waterfallObj of waterfall) {
let tempArr = Object.values(waterfallObj);//get the functions out of the object
for (waterfallFunc of tempArr) {
collector.push(waterfallFunc);
}
}
//now you have your functions in order in collector
function recursiveCallback(i) {
if (i>collector.length-1) {
return;//if there are no more to call then return
}
collector[i]().then(function(){
recursiveCallback(i+1);
});
}
If you want the next function to do something with the previous functions value then simply change the then to then(function(passedValue and then use that passedValue in the recursiveCallback call within it
I used to have the following code:
function makeCall(userInfo) {
api.postUser(userInfo).then(response => {
utils.redirect(response.url);
})
// other logic
return somethingElse;
}
And I was able to write a test that looked like this:
const successPromise = Promise.resolve({ url: 'successUrl' })
beforeEach(function() {
sinon.stub(api.postUser).returns(successPromise);
}
afterEach(function() {
api.postUser.restore();
}
it "calls API properly and redirects" do
makeCall({});
expect(api.postUser).calledWith(userInfo).toBe(true);
successPromise.then(() => {
expect(utils.redirect.calledWith('successUrl')).toBe(true);
done();
}
emd
And everything was green.
Now, I had to add another promise to make another external call, before doing the api postUser call, so my code looks like this:
function makeCall(names) {
fetchUserData(names).then(userData => {
return api.postUser(userData).then(response => {
utils.redirect(response.url);
})
})
// other logic
return somethingElse;
}
where fetchUseData is a chain of many promises, such like:
function fetchNames(names) {
// some name regions
return Promise.all(names);
}
function fetchUserData(names) {
fetchUsersByNames(names).then(users => {
// For now we just choose first user
{
id: users[0].id,
name: users[0].name,
}
});
}
And the tests I had fail. I am trying to understand how to change my tests to make sure that I am still testing that I do the final API call properly and the redirect is also done. I want to stub what fetchUserData(names), to prevent doing that HTTP call.
You're not using promises correctly. Your code doesn't have a single return statement, when it should have several (or it should be using arrow functions in such a way that you don't need them, which you're not doing).
Fix your code:
function makeCall(names) {
// v---- return
return fetchUserData(names).then(userData => {
// v---- return
return api.postUser(userData).then(response => {
utils.redirect(response.url);
})
})
}
function fetchUserData(names) {
// v---- return
return fetchUsersByNames(names).then(users => {
// For now we just choose first user
// v---- return
return {
id: users[0].id,
name: users[0].name,
}
});
}
Once you've done that, you can have your test wait for all of the operations to finish.
Test code:
makeCall(['name']).then(() =>
expect(api.postUser).calledWith(userInfo).toBe(true);
expect(utils.redirect.calledWith('successUrl')).toBe(true);
done();
});
You should add a return statement, otherwise you are not returning promises nowhere:
function fetchNames(names) {
// some name regions
return Promise.all(names);
}
function fetchUserData(names) {
return fetchUsersByNames(names).then(users => {
// For now we just choose first user
{
id: users[0].id,
name: users[0].name,
}
});
}
So when you are using Promise.all(), then you will have as result of the promise an array with all the value returned by all the promises.
So then this method should look like this when called:
fetchNames(names).then((arrayOfResolvedPromises) => {
// here you will have all your promised resolved and the array holds all the results
});
So inside your test you can move your done inside the block where all the promises will be resolved.
In addition, I strongly suggest you to use a library as chai-as-promised for testing promises.
It has a lot of nice methods for testing your promises.
https://github.com/domenic/chai-as-promised
I want to do a fetch then update the markers state. The problem is the state is updating before my fetch ends (due to asynchronous I think). I think there would be a solution with UnderscoreJS using the _.after function to set the state after my fetch ends. I don't know how to do that. Any idea?
Here my code:
onRegionChange(region) {
let latitude = region.latitude;
let longitude = region.longitude;
let markers = [];
_.filter(this.props.stations, (v) =>
{
if (this.getDistanceFromLatLonInKm(latitude,longitude,v.position.lat,v.position.lng) < 1) {
fetch('https://api.jcdecaux.com/vls/v1/stations/' + v.number + '?contract=Paris&apiKey=3a9169028401f05f02bcffd87f4a3963dcd52f63')
.then((response) => response.json())
.then((station) => {
console.log("station", station);
markers.push({
number: station.number,
coordinate: {
latitude: station.position.lat,
longitude: station.position.lng
},
title: station.name,
description: station.address,
banking: station.banking,
bonus: station.bonus,
status: station.status,
bike_stands: station.bike_stands,
available_bike_stands: station.available_bike_stands,
available_bikes: station.available_bikes,
last_update: station.last_update
});
})
.catch((error) => {
console.warn(error);
});
}
}
)
this.setState({
markers: markers
})
console.log("markers", this.state.markers);
}
Your mixing synchronous and asynchronous ideas and they don't mix well.
You're better off converting the synchronous values into promises (same thing the fetch is doing).
First I break out the drivel parts into functions. This cleans up the actual code and lowers the complexity.
function stationFetchUrl(station) {
return `https://api.jcdecaux.com/vls/v1/stations/${station.number}?contract=Paris&apiKey=3a9169028401f05f02bcffd87f4a3963dcd52f63`;
}
function convertStationData(station) {
return {
number: station.number,
coordinate: {
latitude: station.position.lat,
longitude: station.position.lng
},
title: station.name,
description: station.address,
banking: station.banking,
bonus: station.bonus,
status: station.status,
bike_stands: station.bike_stands,
available_bike_stands: station.available_bike_stands,
available_bikes: station.available_bikes,
last_update: station.last_update
};
}
function stationNearRegion(region, station) {
const distance = this.getDistanceFromLatLonInKm(
region.latitude,
region.longitude,
stationStub.position.lat,
stationStub.position.lng
);
return (distance < 1);
}
Next in the event handler I map over the stations and convert them to either null (in the case of the distance being >= 1) or fetch the new station data from the server (fetch()).
This results in an array of either promises or nulls. I use Underscore's .compact() to remove the null (same as filter did). Then I have an array of promises which I pass to Promise.all which waits till all the promises in the array resolve. When they do it passes the result of those promises (as an Array) to the .then() function which I dutifully use this.setState() on.
onRegionChange(region) {
const markerPromises = _.chain(this.props.stations)
.map(stationStub => {
if (stationNearRegion.call(this, region, stationStub) {
return fetch(stationFetchUrl(station))
.then(response => response.json())
.then(convertStationData);
}
})
.compact()
.value();
Promise.all(markerPromises)
.then(markers => this.setState({markers}))
.catch(error => console.log(error.toString()));
});
Since the onRegionChange is an event handler it would not return a promise. Instead we handle a possible error(s) by logging them in the .catch() block.
References
Chaining in Underscore.js (miniarray.com)
Promise - JavaScript | MDN (developer.mozilla.org)
You're Missing the Point of Promises (blog.domenic.me)
This question already has answers here:
How do I access previous promise results in a .then() chain?
(17 answers)
Closed 7 years ago.
I am trying to use Promises in JavaScript using ES6 to return data from 2 methods from an object, which would in production call out to an endpoint.
My MovieApi object I have 2 promises which I want to return firstly a list of movies, and then 1 movie by id. I want to use promises to avoid callback hell. I am following the approach listed as the answer to the question Arent promises just callbacks but I am clearly doing it wrong, as I have the error findMovie is not defined
let movieData = [
{
id: '1011',
name: 'Gleaming the cube',
year: "1989"
},
{
id: "1012",
name: "Airborne",
year: "1989"
}
]
let MovieApi = {
findMovie: function(id) {
return new Promise(function(resolve, reject) {
if(id === undefined) reject(Error('incorrect movie id'));
let movie = ''
for (let m of movieData) {
if (m.id.toLowerCase() === id.toLowerCase()) {
movie = m
break
}
}
resolve(movie)
});
},
findAllMovies: function() {
return new Promise(function(resolve, reject) {
if(movieData === undefined) reject(Error('Could not find any movies'))
resolve(movieData)
});
}
}
Call the movie promise like this... but I get an error trying to call my second .then() method
MovieApi.findAllMovies()
.then( function (movies){
return findMovie(req.params.id)
}).then(function(movie){
let MovieStore = { movie: movie, movies: movies }
}).catch(function(error) {
console.error("Failed!", error);
});
Is there a way to get out of callback hell here, or will I just have to make another call to the MovieApi object, essentially having the same readability issue as if I were using callbacks.
The problem is return findMovie(req.params.id), your findMovie is a property of the MovieApi object so
MovieApi.findAllMovies()
.then( function (movies){
return MovieApi.findMovie(req.params.id)
})
Demo: Fiddle