I'm new in JavaScript and Node.js. I have the following code:
const populateSetup = async () => {
token = await getToken();
const promises = await constant.accounts.map(async (x) => {
const accountId = await createAccountRequest(x.account);
const peoid = await createPeopleRequests(x.people);
const pid = await createProjectsRequests(x.project);
return [accountId, pid, peoid];
});
const [accountId, pid, peoid] = await Promise.all(promises);
};
In the above, token is first fetched and is required to create account and then accountId returned is required to create people and projects. Let's say I have the following input:
exports.accounts = [
{ account: this.testAccountFirst, project: this.projectOne, people: this.testUserOne },
{ account: this.testAccountSecond, project: this.projectTwo, people: this.testUserTwo },
];
After running the populateSetup() in node environment my result is (not the console output but the output of the populateSetup():
testAccountFirst has 1 people -> testUserOne
testAccountSecond has 2 projects and 1 user -> projectOne, projectTwo, testUserTwo
expected result is:
testAccountFirst should have 1 project and 1 people -> projectOne, testUserOne
testAccountSecond should have 1 project and 1 people -> projectTwo, testUserTwo
The problem here is that the accountId of first account is not sent to the projectsRequest. I don't know how to resolve this. I have gone through this Stackoverflow question but still couldn't figure out.
I'm having a hard time understanding exactly what question you're asking, but .map() is not async savvy. That means that even though you declare the callback as async, .map() doesn't do anything with the promise that returns and thus it doesn't wait to start the 2nd iteration before the 1st iteration is done. Therefore, you end up running all the async operations from all the iterations of the loop in parallel and they can finish in any random order.
If you really want to run them sequentially, one after another, then switch your .map() to a for loop because a for loop will wait for an await in the first iteration of the loop before starting the 2nd iteration of the loop and so on...
return [accountId, pid, peoid]; is returning resolved promises, also, you wait one promise resolving after another. For example, if one resolving is 5 sec, then you need to wait 5+5+5=15 secs
But more of that, it is bad practice to use .map() with promises inside, because it is sync operator.
In your case, I would use something like that:
const populateSetup = async () => {
const token = await getToken();
const [accountId, peoid, pid] =[
createAccountRequest(x.account),
createPeopleRequests(x.people),
createProjectsRequests(x.project)
]
return Promise.all([accountId, peoid,pid])
};
Here, You return promise, that can be used like this:
const [accountId, peoid,pid] = await populateSetup()
Promise.all() do promise execution simultaniosly and wait for all to be resolved, so it is 5 secs instead of 15
Related
I have a sample code and questions along with that
'use strict';
const AWS = require('aws-sdk');
const promiseFunction = (item) => {
const database = new AWS.DynamoDB();
const params = {
TableName: 'some-table',
Item: item
}
try {
//database.putItem(params).promise(); //3
//Updated as per the comments..
return database.putItem(params).promise(); //3
} catch (err) {
console.log(`Error`);
}
}
const test = async () => {
const arr = [];
try {
for (let data=1;data<=1000;data++) {
const obj = { data: {N: data.toString()}}
arr.push(promiseFunction(obj)); //1
}
const result = await Promise.all(arr); //2
} catch (e) {
console.log(`Error`);
}
}
test();
Some follow-up questions:
At line-2, the result will contain the resolve/reject result once all the promise based function get executed on line //1. Right?
How, the promiseFunction at line-1 is executing and inserting item in dynamodb as I am just pushing it into an array rather than calling the dynamo putItem API. Technically, all 'putItem' should start execute in parallel at line-2. Please help here?
Is the above piece of code is valid in terms of Promise.all and executing insert operations in parallel. Please advice.
At line-2, the result will contain the resolve/reject result once all the promise based function get executed on line //1. Right?
The result will be assigned once all the promises in the arr have been fulfilled. Promise.all doesn't know (or care) how those promises were created.
Technically, all 'putItem' should start execute in parallel at line-2.
No. All the putItem() calls are executed in your loop. Promise.all doesn't execute anything, it only waits for promises created earlier. The request are done concurrently because the code starts them all at once, without waiting in between each loop iteration, not because it uses Promise.all.
Is the above piece of code is valid in terms of Promise.all and executing insert operations in parallel.
No. Your promiseFunction doesn't actually return a promise, since you forgot a return statement on the line marked as //3. Once you add that, the code is fine though.
I have N workspaces. The number N is dynamic.
Every workspace has to execute a few pre-defined queries.
What I am currently doing is looping through an array of workspaces(Synchronously) and executing all the queries using Promise.all() (which is asynchronous).
Goal: What I need is to run all the queries for all the workspaces asynchronously. So I want to get rid of the loop to go thru each workspace. The ideal result would be an array of array. For example, if there are 3 workspaces and 2 queries the result would be [[q1, q2], [q1,q2], [q1,q2]] each q1 and q2 are the results for every workspace.
Below is the sample code:
async function fetchingWorkspaceLogs (workspaceId) {
// Defining q1QueryString, q2QueryString so on...
// for azure "loganalytics" reader.
const [q1Result, q2Result, q3Result] = await Promise.all([
logAnalyticsReader.query(
q1QueryString,
workspaceId
),
logAnalyticsReader.query(
q2QueryString,
workspaceId
),
])
// return some promises
}
// Parse all workspaces for query
for (let j = 0; j < workspaceIdList.length; j++) {
workspaceId = workspaceIdList[j]
const queryResults = await fetchingWorkspaceLogs(workspaceId)
q1QueryResults = queryResults[0]
q2QueryResults = queryResults[1]
}
How can I create another promise object to make it async?
Feel free to ask if you need anything else to get more clarity.
If I understand you correctly, you can map() that workspaces array into Promises array and wrap it with Promise.all.
The below code is "pseudo" to make the point.
If it not reflects your situation, I'll probably need more information.
async function fetchingWorkspaceLogs (workspaceId) {
const [q1Result, q2Result] = await Promise.all([
Promise.resolve(`param1: ${workspaceId}`),
Promise.resolve(`param2: ${workspaceId}`),
]);
// In this example, the function returns the result to make the point of returning Promise with the information
return Promise.resolve([q1Result, q2Result]);
}
const workspaceIdList = ['workspace1', 'workspace2', 'workspace3'];
(async () => {
const result = await Promise.all(
workspaceIdList.map(workspace => fetchingWorkspaceLogs(workspace))
);
console.log(result);
})();
you need to use .map() function.
It should have given me not null output but I am getting null output. It is working fine if I console inside the block of code but outside it gives me null value
Here is the code:
app.post("/downloadDb",async (req,res)=>{
var docData = [];
var idList = [];
console.log("downloadBd");
const mahafuzCol = firestore.collection("Mahafuz")
await mahafuzCol.listDocuments()
.then( listDoc=>{
//List of id fetch
listDoc.forEach(data=>{
idList.push(data.id)
});
}).catch(e=>console.log(e));
//document is fetched
await idList.forEach(id=>{
mahafuzCol.doc(id).get().then(
doc=>{
docData.push(doc.data());
//Here I get desire output w=if I log with console
}
);
});
//Here I get null output
await console.log(docData);
});
Ok, looking at your piece of code, I would like to point out a few things.
You are using the latest and greatest ES7 async and await feature which is great. Why are you stuck with the old way of defining variables? Try to use let and const instead of var. Don't mix the ECMAScript versions like this. This is considered a bad practice.
Loops in Node.js are synchronous as of now (though asynchronous loops are in the pipeline of Node and we would see them soon). You cannot put asynchronous code inside a loop and expect them to work as expected. There is a whole concept of event loop in Node.js and how Node handles asynchronous tasks. So, if you have time, you should definitely go through the event loop concepts.
Here is the better way to write the code:
app.post('/downloadDb', async (req, res) => {
// always wrap asynchronous code in async/await in try/catch blocks
console.log('downloadBd');
const mahafuzCol = firestore.collection('Mahafuz');
try {
// assuming that listDocuments returns a promise
// await on it until it gets resolved
// all the listed documents will be assigned to docs
// once the promise is resolved
const docs = await mahafuzCol.listDocuments();
const idList = docs.map(data => data.id);
// again assuming that get() returns a promise
// pushing all the promises to an array so that
// we can use Promise.all to resolve all the promises
// at once
const promisesList = idList.map(id => mahafuzCol.doc(id).get());
// fetching document is here
// once all the promises are resolved, data contains
// the result of all the promises as an array
const data = await Promise.all(promisesList);
const docData = data.map(doc => doc.data());
console.log(docData);
// return some response
return res.status(200).send();
} catch (error) {
console.log('error: ', error);
// return some response
return res.status(500).send();
}
});
PS:
If you still somehow want to use asynchronous loops, have a look at this library
https://caolan.github.io/async/docs.html#each
Since forEach is async in nature, as soon as you write a promise in a forEach() loop. The next command gets executed, which in your case is:
// Here I get null output
await console.log(docData);
You'd need the typical for loop for this task:
Try the snippet below:
let idListLen = idList.length;
for (let i =0; i< idListLen; i++) {
mahafuzCol.doc(id).get().then(
doc=>{
docData.push(doc.data());
//Here I get desire output w=if I log with console
}
);
}
console.log(docData) //prints the list of data
I have this code that is checking if my userContacts ids exist in another collection, and I'm returning all the matches.
async function fetchCommonNumbers() {
var commonNumbers = [];
let contactsReference = admin.firestore().collection("user_contacts").doc("iNaYVsDCg3PWsDu67h75xZ9v2vh1").collection("contacts");
const dbContactReference = admin.firestore().collection('db_contacts_meta');
userContacts = await contactsReference.get();
userContacts = userContacts.docs;
await Promise.all(
userContacts.map(userContact => {
const DocumentID = userContact.ref.id;
//Check if Document exists
return dbContactReference.doc(DocumentID).get().then(dbContact => {
if (dbContact.exists) {
console.log(DocumentID);
commonNumbers.push(dbContact.data());
}
});
}));
return Promise.resolve(commonNumbers);
}
I need to only return X matches and not all since later I'll be having million of records and I want to reduce processing time.
How can I make the Promise.all to stop when commonNumbers has X items in it?
Currently there is not implementation of cancelable promises (more info can be found here enter link description here),
If you want, you can define your own "cancelable promise" wrapping a normal promise.
Reduce the processing time without "stopping" promises
You can't really make the promises stop. But since you're looking to reduce the number of database calls, what you can do is to selectively resolve your promises.
For example you can include a conditional statement in your map function. Like this
if commonNumbers.length < maxLength then return me a Promise containing the database call
Else, just resolve a random value (like false in my example)
Your promises will still be there, but you will have limited the number of DB calls to the necessary. It will look something like this
const arr = [1, 2, 3, 4, 5, 6];
const buffer = [];
const maxLenBuffer = 3;
const p = Promise.all(
arr.map(n => {
if (buffer.length < maxLenBuffer) {
buffer.push(n);
return Promise.resolve(n);
} else {
// There's still a promise to be resolved, but it's not a HTTP call
// This gives you the gain of performance you're looking for
return Promise.resolve(false);
}
})
);
p.then(() => console.log(buffer));
Note
While this can reduce your database calls, the actual number of calls can be a little higher than your maximum specified. This is due to the asynchronous nature of the calls
Instead of breaking the promise in between, I would suggest you use the limit method do firestore.
You can query only for X number of records and this X can be either hardcoded or can come from user. Something like:
documentRef.orderBy("name").limit(3).get()
sup guys, i'm working on this firebase project and i need to iterate trought a subcollection of all sales of all stores in the root collection and sum their values... the problem that i'm getting is that i'm getting the sum printed before the iteration. I'm new to TS and Firebase... this is what i got so far:
export const newBilling = functions.firestore.document('billings/{billId}').onCreate(event =>
{
const valueArray = []
const feeArray = []
const storesCollection = afs.collection('stores').where('active', '==', true).get().then(stores => {
stores.forEach(store => {
const salesCollection = afs.collection('stores').doc(store.id).collection('sales').get().then(sales => {
sales.forEach(sale => {
return valueArray.push(sale.data().value) + feeArray.push(sale.data().fee)
// other aproach
// valueArray.push(sale.data().value)
// feeArray.push(sale.data().fee)
})
})
})
}).catch(error => {console.log(error)})
let cashbackSum, feeSum : number
cashbackArray.forEach(value => {
cashbackSum += value
})
feeArray.forEach(value => {
feeSum += value
})
console.log(cashbackSum, feeSum)
return 0
})
TKS =)
You're not using promises correctly. You've got a lot of get() method call, each of which are asynchronous and return a promise, but you're never using them to case the entire function to wait for all the work to complete. Calling then() doesn't actually make your code wait - it just runs the next bit of code and returns another promise. Your final console.log is executing first because none of the work you kicked off ahead of it is complete yet.
Your code actually needs to be substantially different in order to work correctly, and you need to return a promise from the entire function that resolves only after all the work is complete.
You can learn better how to use promises in Cloud Functions by going through the video tutorials.