We Keep Coding

Iterate over array of queries and append results to object in JavaScript

I want to return results from two database queries in one object.
function route(start, end) {
return new Promise((resolve, reject) => {
const queries = routeQuery(start, end);
var empty_obj = new Array();
for (i=0; i<queries.length; i++) {
query(queries[i], (err, res) => {
if (err) {
reject('query error', err);
console.log(err);
return;
} else {
empty_obj.push(res.rows);
}});
}
console.log(empty_obj);
resolve({coords: empty_obj});
});
}
This is my code right now, the queries are working fine but for some reason, pushing each result into an empty array does not work. When I console log that empty object, it stays empty. The goal is to resolve the promise with the generated object containing the two query results. I'm using node-postgres for the queries.
Output of res is an object:
{
command: 'SELECT',
rowCount: 18,
oid: null,
rows: [
{ ...

I suggest you turn your query function into a Promise so that you can use Promise.all:
// turn the callback-style asynchronous function into a `Promise`
function queryAsPromise(arg) {
return new Promise((resolve, reject) => {
query(arg, (err, res) => {
if (err) {
console.error(err);
reject(err);
return;
}
resolve(res);
});
});
}
Then, you could do the following in your route function:
function route(start, end) {
const queries = routeQuery(start, end);
// use `Promise.all` to resolve with
// an array of results from queries
return Promise.all(
queries.map(query => queryAsPromise(query))
)
// use `Array.reduce` w/ destructing assignment
// to combine results from queries into a single array
.then(results => results.reduce(
(acc, item) => [...acc, ...item.rows],
[]
))
// return an object with the `coords` property
// that contains the final array
.then(coords => {
return { coords };
});
}
route(1, 10)
.then(result => {
// { coords: [...] }
})
.catch(error => {
// handle errors appropriately
console.error(error);
});
References:
Promise.all - MDN
Array.reduce - MDN
Destructing assignment - MDN
Hope this helps.

The issue you currently face is due to the fact that:
resolve({coords: empty_obj});
Is not inside the callback. So the promise resolves before the query callbacks are called and the rows are pushed to empty_obj. You could move this into the query callback in the following manner:
empty_obj.push(res.rows); // already present
if (empty_obj.length == queries.length) resolve({coords: empty_obj});
This would resolve the promises when all rows are pushed, but leaves you with another issue. Callbacks might not be called in order. Meaning that the resulting order might not match the queries order.
The easiest way to solve this issue is to convert each individual callback to a promise. Then use Promise.all to wait until all promises are resolved. The resulting array will have the data in the same order.
function route(start, end)
const toPromise = queryText => new Promise((resolve, reject) => {
query(queryText, (error, response) => error ? reject(error) : resolve(response));
});
return Promise.all(routeQuery(start, end).map(toPromise))
.then(responses => ({coords: responses.map(response => response.rows)}))
.catch(error => {
console.error(error);
throw error;
});
}

wait for Promise.all To continue executing

I would like to insert products into array and then continue using that array. I don't have an expirience using Promisses. Bellow is my code.
This is the function that I would like to return an array.
const findProduct = async function(request) {
const products = [];
await Promise.all(
request.products.forEach(async product => {
await db
.get()
.collection('products')
.findOne({ _id: product.productId.toString() }, (err, result) => {
if (err) throw new Error('exception!');
products.push(result);
});
})
)
.then(() => {
return products;
})
.catch(e => e);
};
Products is always undefined.
Shoud I maybe return Promise.all i save it into an array, then use that array?
This is how I plan to use array
const purchase = new Purchase(req.body);
const products = await findProduct(req.body);
purchase.products = [...products];

In the context of your findProduct function, using async/await syntax doesn't make a lot of sense if you intend to await the result of findProduct using const products = await findProduct(req.body);
Instead you want to define your function using only Promises.
Firstly, db.collection.findOne() makes use of a callback for asynchronous events, to use this in a Promise chain, you must wrap this in a Promise. Which can be done like so:
new Promise((resolve, reject) => {
db
.get()
.collection('products')
.findOne({ _id: product.productId.toString() }, (err, result) => { err ? reject(err) : resolve(result) })
});
Here (err, result) => { err ? reject(err) : resolve(result) } is just a concise form of
(err, result) => {
if (err) {
reject(err);
} else {
resolve(result);
}
}
Next, you must pass an array to Promise.all(). Because you already have the array request.products, you can convert (or map) each value to it's corresponding Promise to get it's data. This is done using Array.prototype.map. When used this way, Promise.all() will resolve to an array of product data objects in the same order of request.products.
Mixing this in to your code, yields:
const findProduct = function(request) {
return Promise.all(
request.products.map(product => {
return new Promise((resolve, reject) => {
db
.get()
.collection('products')
.findOne({ _id: product.productId.toString() }, (err, result) => err ? reject(err) : resolve(result))
});
})
);
};
Note here that I removed .then(()=>products) (because Promise.all will return products itself thanks to the promise above) and .catch(e=>e) (because you shouldn't be ignoring errors like this).
With these changes, you can now use findProduct as you have used it in your question.
Update:
Seems that the MongoDB API is Promises compliant, which means we can remove the Promise callback wrapper and simplify it down to:
const findProduct = function(request) {
return Promise.all(
request.products.map(product => {
return db
.get()
.collection('products')
.findOne({ _id: product.productId.toString() });
})
);
};

Promise.all() expects an array of Promises and map will return an array. Since db will also return a Promise we just feed those promises into Promise.all()
Edit: forgot to return Promise.all() which returns a promise so it can be consumed with the await findproducts() or findproducts().then()
const findProduct = function (request) {
return Promise.all(
request.products.map(product => {
return db
.get()
.collection('products')
.findOne({ _id: product.productId.toString() }
});
})
)};
Since quite a lot of people ask: "How do i get something from a Promise?" --> use the resolve value
Here a long explanation why this works: wait() will return a Promise that is mapped into an array with map() which returns this array. So when map is done we have an array with 3 Promises returned by the 3 wait() calls. This array is passed into Promise.all() which will wait for all the Promises to resolve and resolves itself to an array with all the resolve values of the Promises from the array. Since findProduct() will return this Promise from Promise.all() when can access it with findProduct().then() or await findProduct()
const start = { products: ['milk', 'butter', 'eggs'] }
const findProduct = function (request) {
return Promise.all(
request.products.map(product => {
return wait(product)
})
)
};
function wait(product) {
return new Promise((resolve) => {
setTimeout(() => { resolve(product) }, 5000)
})
}
findProduct(start).then(res => console.log(res))
// logs [ 'milk', 'butter', 'eggs' ]
Edit: To answer the comment about .then() returning a Promise
findProduct(start)
.then(res => console.log(res))
.then(res => console.log(res))
// logs [ 'milk', 'butter', 'eggs' ] and undefined
findProduct(start)
.then(res => {console.log(res); return res})
.then(res => console.log(res))
// logs 2x [ 'milk', 'butter', 'eggs' ]

React Native Wait All Image Prefetch Done

I'm trying to prefetch multiple image before navigating to another screen, but returnedStudents all undefined.
prepareStudentImages = async (students) => {
let returnedStudents = students.map(student => {
Image.prefetch(student.image)
.then((data) => {
...
})
.catch((data) => {
...
})
.finally(() => {
return student;
});
});
await console.log(returnedStudents); // ----> all items undefined
}

There are a couple of things to fix with this:
1) Your map() function does not return anything. This is why your console log is undefined.
2) Once your map functions work, you are logging an array of promises. To deal with multiple promises (an array), you can use Promise.all().
So I think to fix this, you can do:
prepareStudentImages = async (students) => {
const returnedStudents = students.map(student =>
Image.prefetch(student.image)
.then((data) => {
...
})
.catch((data) => {
...
})
.finally(() => {
return student
})
)
console.log(returnedStudents) // log the promise array
const result = await Promise.all(returnedStudents) // wait until all asyncs are complete
console.log(result) // log the results of each promise in an array
return result
}

Looping Results with an External API Call and findOneAndUpdate

I am trying to write a program that gets the documents from a mongo database with mongoose and process them using an API and then edits each document in the database with the results of the processing. My problem is that I have problems because I don't understand completely nodejs and the asynchronous. This is my code:
Model.find(function (err, tweets) {
if (err) return err;
for (var i = 0; i < tweets.length; i++) {
console.log(tweets[i].tweet);
api.petition(tweets[i].tweet)
.then(function(res) {
TweetModel.findOneAndUpdate({_id: tweets[i]._id}, {result: res}, function (err, tweetFound) {
if (err) throw err;
console.log(tweetFound);
});
})
.catch(function(err) {
console.log(err);
})
}
})
The problem is that in the findOneAndUpdate, tweets is undefined so it can't find that id. Any solution? Thanks

The core thing you are really missing is that the Mongoose API methods also use "Promises", but you seem to just be copying from documentation or old examples using callbacks. The solution to this is to convert to using Promises only.
Working with Promises
Model.find({},{ _id: 1, tweet: 1}).then(tweets =>
Promise.all(
tweets.map(({ _id, tweet }) =>
api.petition(tweet).then(result =>
TweetModel.findOneAndUpdate({ _id }, { result }, { new: true })
.then( updated => { console.log(updated); return updated })
)
)
)
)
.then( updatedDocs => {
// do something with array of updated documents
})
.catch(e => console.error(e))
Aside from the general conversion from callbacks, the main change is using Promise.all() to resolve the ouput from the Array.map() being processed on the results from .find() instead of the for loop. That is actually one of the biggest problems in your attempt, since the for cannot actually control when the async functions resolve. The other issue is "mixing callbacks", but that is what we are generally addressing here by only using Promises.
Within the Array.map() we return the Promise from the API call, chained to the findOneAndUpdate() which is actually updating the document. We also use new: true to actually return the modified document.
Promise.all() allows an "array of Promise" to resolve and return an array of results. These you see as updatedDocs. Another advantage here is that the inner methods will fire in "parallel" and not in series. This usually means a faster resolution, though it takes a few more resources.
Note also that we use the "projection" of { _id: 1, tweet: 1 } to only return those two fields from the Model.find() result because those are the only ones used in the remaining calls. This saves on returning the whole document for each result there when you don't use the other values.
You could simply just return the Promise from the findOneAndUpdate(), but I'm just adding in the console.log() so you can see the output is firing at that point.
Normal production use should do without it:
Model.find({},{ _id: 1, tweet: 1}).then(tweets =>
Promise.all(
tweets.map(({ _id, tweet }) =>
api.petition(tweet).then(result =>
TweetModel.findOneAndUpdate({ _id }, { result }, { new: true })
)
)
)
)
.then( updatedDocs => {
// do something with array of updated documents
})
.catch(e => console.error(e))
Another "tweak" could be to use the "bluebird" implementation of Promise.map(), which both combines the common Array.map() to Promise(s) implementation with the ability to control "concurrency" of running parallel calls:
const Promise = require("bluebird");
Model.find({},{ _id: 1, tweet: 1}).then(tweets =>
Promise.map(tweets, ({ _id, tweet }) =>
api.petition(tweet).then(result =>
TweetModel.findOneAndUpdate({ _id }, { result }, { new: true })
),
{ concurrency: 5 }
)
)
.then( updatedDocs => {
// do something with array of updated documents
})
.catch(e => console.error(e))
An alternate to "parallel" would be executing in sequence. This might be considered if too many results causes too many API calls and calls to write back to the database:
Model.find({},{ _id: 1, tweet: 1}).then(tweets => {
let updatedDocs = [];
return tweets.reduce((o,{ _id, tweet }) =>
o.then(() => api.petition(tweet))
.then(result => TweetModel.findByIdAndUpdate(_id, { result }, { new: true })
.then(updated => updatedDocs.push(updated))
,Promise.resolve()
).then(() => updatedDocs);
})
.then( updatedDocs => {
// do something with array of updated documents
})
.catch(e => console.error(e))
There we can use Array.reduce() to "chain" the promises together allowing them to resolve sequentially. Note the array of results is kept in scope and swapped out with the final .then() appended to the end of the joined chain since you need such a technique to "collect" results from Promises resolving at different points in that "chain".
Async/Await
In modern environments as from NodeJS V8.x which is actually the current LTS release and has been for a while now, you actually have support for async/await. This allows you to more naturally write your flow
try {
let tweets = await Model.find({},{ _id: 1, tweet: 1});
let updatedDocs = await Promise.all(
tweets.map(({ _id, tweet }) =>
api.petition(tweet).then(result =>
TweetModel.findByIdAndUpdate(_id, { result }, { new: true })
)
)
);
// Do something with results
} catch(e) {
console.error(e);
}
Or even possibly process sequentially, if resources are an issue:
try {
let cursor = Model.collection.find().project({ _id: 1, tweet: 1 });
while ( await cursor.hasNext() ) {
let { _id, tweet } = await cursor.next();
let result = await api.petition(tweet);
let updated = await TweetModel.findByIdAndUpdate(_id, { result },{ new: true });
// do something with updated document
}
} catch(e) {
console.error(e)
}
Noting also that findByIdAndUpdate() can also be used as matching the _id is already implied so you don't need a whole query document as a first argument.
BulkWrite
As a final note if you don't actually need the updated documents in response at all, then bulkWrite() is the better option and allows the writes to generally process on the server in a single request:
Model.find({},{ _id: 1, tweet: 1}).then(tweets =>
Promise.all(
tweets.map(({ _id, tweet }) => api.petition(tweet).then(result => ({ _id, result }))
)
).then( results =>
Tweetmodel.bulkWrite(
results.map(({ _id, result }) =>
({ updateOne: { filter: { _id }, update: { $set: { result } } } })
)
)
)
.catch(e => console.error(e))
Or via async/await syntax:
try {
let tweets = await Model.find({},{ _id: 1, tweet: 1});
let writeResult = await Tweetmodel.bulkWrite(
(await Promise.all(
tweets.map(({ _id, tweet }) => api.petition(tweet).then(result => ({ _id, result }))
)).map(({ _id, result }) =>
({ updateOne: { filter: { _id }, update: { $set: { result } } } })
)
);
} catch(e) {
console.error(e);
}
Pretty much all of the combinations shown above can be varied into this as the bulkWrite() method takes an "array" of instructions, so you can construct that array from the processed API calls out of every method above.

How do I flatten a nested promise dependency?

I'm using mondora/asteroid through a node app to use Meteor DDP via a promise pattern.
I have the following code I am rewriting from callback style, into a promise style, but am stuck on how to flatten it.
asteroid.call('Pony.search', { params })
.then(res => {
if (res.length === 1) {
// something
asteroid.call('Pony.finish', { params })
// this part feels wrong
.then(res => {
// something else
});
} else {
// nope
}
})
.catch(err => {
console.error(err);
});
There is a second asteroid.call & then inside the first promise response which is a promise. This part feels wrong, like it should be flat and not nested but I'm not sure how to get there.
edit:
Ended up using something like this (still not decided on whether to have the first then check for if length === 1 and potentially immediately reject it. Anyone know what is best practice on that?
asteroid.call('Pony.search', { params })
.then(res => res.length === 1 ? res : Promise.reject())
.then(res => asteroid.call('Pony.finish', { res[0].something }))
.then(res => {
// do something
})
.catch(err => {
// handle the no found
console.error(err);
});

Instead of nesting callbacks, chain promises together with .then()
A note:
I'm not sure what Promise library you are using, but the idea is to return a rejected promise from the first .then() if there is an error, otherwise you return a successful promise. The promise library will then handle the error handling for you, going to the catch block if there is a rejected promise.
asteroid.call('Pony.search', { params })
.then(res => {
res.length === 1 ? return asteroid.call('Pony.finish', { params }) : Promise.reject();
})
.then(res => {
//do stuff here
})
.catch(err => {
console.error(err);
});
edit:
The only issue is when you need to access both of the return values from the promises at the same time. When you flatten out a promise chain you lose access to the results from the previous promises.
You have a few options:
If you don't need the previous result then flatten out the promise chain like I did here
If you do need the previous value
2a. And you don't care about the ordering of execution then use Promise.all([promise1, promise2])
2b. And you do care about the ordering of the execution then you must use nested promises like you originally did.

If promises are nested, there is no difference between promises and callback.
Try changing your code to use promise chain:
asteroid.call('Pony.search', { params })
.then(res => {
if (res.length === 1) {
// something
let p1 = asteroid.call('Pony.finish', { params });
return p1;
}
else {
// nope
}
})
.then (function SomeThingElse(res2){
// something else
// res2 is return value from nested asteroid.call
})
.catch(err => {
console.error(err);
});
Since the first resolve handler returns a Promise p1, the invocation of the next functions in the chain (function SomeThingElse) is deferred until p1 is resolved.
Now expanding this example:
asteroid.call('Pony.search', { params })
.then(res => {
if (res.length === 1) {
// something
let p1 = asteroid.call('Pony.finish', { params });
return p1;
}
else {
// nope
}
})
.then (function SomeThingElse(res2){
// something else
// res2 is return value from nested asteroid.call
})
.then (function AnotherFunc(res3){
})
.catch(err => {
console.error(err);
});
If SomeThingElse returns a Promise, invocation of AnotherFunc is delayed until that promise is resolved.
If SomeThingElse does not return a Promise, AnotherFunc would be invoked immediately with the same parameters as 'SomeThingElse' received. In other words, both SomeThingElse and AnotherFunc are invoked when p1 is resolved.