We Keep Coding

Resolving async function after a loop end

I expect when I call an async function to resolve promise at the end, not before.
const urls = await uploadImages({ imageChanges, questions });
// ...next step
// I will use urls
But after calling await uploadImages() it continues to run until const data = await fetch(image.src);
And then ...next step starts. How can I make it wait for imageChanges.forEach loop finish ? Should I create another nested function inside ?
const uploadImages = async ({ imageChanges, questions }) => {
if (!imageChanges.length) return null;
const storage = firebase.storage();
let urls;
try {
//** convert each new image's src from blob to downloadUrl. */
imageChanges.forEach(async image => {
const questionId = questions.findIndex(q => q.id === image.questionId);
const imagePath = `${questionId}.jpg`;
const storageRef = storage.ref(imagePath);
// **
const data = await fetch(image.src);
const blob = await data.blob();
const uploadTaskSnapshot = await storageRef.put(blob);
const downloadURL = await uploadTaskSnapshot.ref.getDownloadURL();
urls.push(downloadURL)
});
return urls;
} catch (error) {
console.log(error.message);
}
};

forEach with async doesn't work as expected. Read this answer for more info.
Try like this
const uploadImages = async ({ imageChanges, questions }) => {
if (!imageChanges.length) return null;
const storage = firebase.storage();
try {
const imageChangesUrlPromise = imageChanges.map(async () => {
const questionId = questions.findIndex(q => q.id === image.questionId);
const imagePath = `${questionId}.jpg`;
const storageRef = storage.ref(imagePath);
const data = await fetch(image.src);
const blob = await data.blob();
const uploadTaskSnapshot = await storageRef.put(blob);
const downloadURL = await uploadTaskSnapshot.ref.getDownloadURL();
return downloadURL;
})
return await Promise.all(imageChangesUrlPromise);
} catch (error) {
console.log(error.message);
}
};
and then
const urls = await uploadImages({ imageChanges, questions });
...

JavaScript does this because forEach is not promise-aware. It cannot support async and await. You cannot use await in forEach.
If you use await in a map, map will always return an array of promises. This is because asynchronous functions always return promises.
By littile modification to your code, this should work,
const uploadImages = async ({ imageChanges, questions }) => {
if (!imageChanges.length) return null;
const storage = firebase.storage();
let urls;
try {
//** convert each new image's src from blob to downloadUrl. */
await Promise.all(imageChanges.map(async image => {
const questionId = questions.findIndex(q => q.id === image.questionId);
const imagePath = `${questionId}.jpg`;
const storageRef = storage.ref(imagePath);
// **
const data = await fetch(image.src);
const blob = await data.blob();
const uploadTaskSnapshot = await storageRef.put(blob);
const downloadURL = await uploadTaskSnapshot.ref.getDownloadURL();
urls.push(downloadURL)
}));
return urls;
} catch (error) {
console.log(error.message);
}
};
const urls = await uploadImages({ imageChanges, questions });

How can I access functions output across Node.js?

I have 3 files:
Ingredients.js
const fs = require("fs");
const readline = require('readline');
const stream = require('stream');
const ingredients = () => {
const instream = fs.createReadStream('ingredients.txt');
const outstream = new stream;
const rl = readline.createInterface(instream, outstream);
const listIngredients = {};
rl.on('line', function (line) {
let lower = line.toLowerCase();
listIngredients[lower] = 0;
});
rl.on('close', function () {
console.log('listIngredients', listIngredients);
});
}
module.exports = ingredients;
cookbook.js:
let fs = require("fs");
const book = () => {
const regex = /\b(\w+)\b/g;
fs.readFile('cook-book.txt', 'utf8', function (err, data) {
let book = data;
let lower = book.toLowerCase();
let split = lower.match(regex);
console.log(split);
});
}
module.exports = book;
compare.js
const ingredients = require('./ingredients');
const book = require('./book');
I'm trying to increase the key values of ingredients every time they are mentioned in the cookbook. I think this should go into a different js file to make it cleaner.
Whilst i can console.log out the information from the above files, I cannot figure out how to actually access the data and make changes to the ingredients object in compare.js?

as others noticed your ingredients and book variables are functions having required information inside their scope and not returning it outside. to fix it, you have to return values.
as you're working with asynchronous stuff, your functions should be wrapped into Promise's to handle the flow correctly.
this code should help you:
const fs = require('fs');
const readline = require('readline');
const { Writable } = require('stream');
const fsp = fs.promises;
// ingredients.js
const getIngredients = async () => new Promise((resolve, reject) => {
const instream = fs.createReadStream('ingredients.txt');
const outstream = new Writable();
const rl = readline.createInterface(instream, outstream);
const listIngredients = {};
rl.on('line', line => {
const lower = line.toLowerCase();
listIngredients[lower] = 0;
});
rl.on('error', reject);
rl.on('close', () => resolve(listIngredients));
});
// cookbook.js
const getBookContent = async () => new Promise(async (resolve, reject) => {
try {
const wordRegEx = /\b(\w+)\b/g;
const book = await fsp.readFile('cook-book.txt', 'utf8')
const lower = book.toLowerCase();
return resolve(lower.match(wordRegEx));
} catch (error) {
return reject(error);
}
});
// compare.js
(async () => {
const ingredients = await getIngredients();
const words = await getBookContent();
console.log(ingredients);
console.log(words);
})();
the names of functions have been change for better representations of their instances.
i've also used an async iife to use async/await syntax, however you can still work with Promises themselves

Converting Promises Code to async await and mocking test cases?

How to convert the below function which has nested promise and await to just using await or only using promises ?
const test = (url, id) => {
return new Promise((_resolve, _reject) => {
if (isValidUrl(url)) {
let storage = new Storage(Indexdb, id);
const cae = new valueExtract(url);
cae.fetch()
.then(data => {
new zip(data)
.then(obj => obj.getZip())
.then(obj => obj.getList())
.then(list => {
return new Promise(async (resolve, reject) => {
try {
let sI = storage.connection;
await Promise.all(Object.keys(list).map(async (fileName, index) => {
let blob = await new FileExtractor(list[fileName]);
await sI.setItemForce(
fileName,
new StoreObject(
fileName,
'testData',
blob
).dataObject
)
}))
_resolve(sI);
} catch (err) {
_reject(err)
}
})
})
.catch(err => _reject(err))
})
.catch(err => _reject(err))
} else {
_reject('Invalid URL')
}
})
};
I was unable to do the same this what i tried but it never resolves
const test = async (url, id) => {
if (isValidUrl(url)) {
try {
let storage = new Storage(Indexdb, id);
const cae = new valueExtract(url);
const data = await cae.fetch();
return new ZIPExtractor(data)
.then(obj => obj.getZip())
.then(obj => obj.getList())
.then(list => {
return async (resolve, reject) => {
try {
let sI = storage.connection;
await Promise.all(Object.keys(list).map(async (fileName, index) => {
let blob = await new FileExtractor(list[fileName]);
await sI.setItemForce(
fileName,
new StoreObject(
fileName,
'testData',
blob
).dataObject
)
}))
} catch (err) {
throw new Error(err)
}
}
})
.catch(err => _reject(err))
} catch (e) {
throw new Error('Invalid URL')
}
}
};
Also how do we write test case for these kind of a function so that we need not pass in actual network url and mock in jest.

It should fulfill, but with the async (resolve, reject) { … } that you return. You never should've used this in the first place, you can just omit it:
const test = async (url, id) => {
if (!isValidUrl(url)) {
throw new Error('Invalid URL')
}
const storage = new Storage(Indexdb, id);
const cae = new valueExtract(url);
const data = await cae.fetch();
const obj = await new ZIPExtractor(data); // shudder. A constructor should never return a promise
const zip = await obj.getZip();
const list = await zip.getList();
const sI = storage.connection;
await Promise.all(Object.keys(list).map(async (fileName, index) => {
const blob = await new FileExtractor(list[fileName]);
const store = new StoreObject(fileName, 'testData', blob);
await sI.setItemForce(fileName, store.dataObject);
}));
return sI; // or something?
}

Choose which data to use using promise/async

I tried the below to acces both data and json values, however I can now only acces the data values, what can I do to acces the json values as well?
const getUser = user => new Promise(async (resolve, reject) => {
try {
const read = await snekfetch.get('https://www.website.nl/api/public/users?name=' + user);
const data = JSON.parse(read.text);
const result = await snekfetch.get('https://www.website.com/api/public/users/' + data.uniqueId + '/profile');
const json = JSON.parse(result.text);
resolve(data, json);
} catch (error) {
reject(error);
}
});
const promise = Promise.resolve(getUser(args[0]));
promise.then(function(data, json) {
const name = data.name;
const motto = data.motto;
const memberSince = data.memberSince;
const groups = json.groups.length;
const badges = json.badges.length;
const friends = json.friends.length;
const rooms = json.rooms.length;
message.channel.send(`${name}\n${motto}\n${memberSince}\n${groups || 'N/A'}\n${badges || 'N/A'}\n${friends || 'N/A'}\n${rooms || 'N/A'}\n`);
}).catch(function(err) {
console.log(err);
return message.reply(`${args[0]} does not exists.`);
});

when you resolve a promise, you pass in a single value with the data you want to resolve it to. If there are multiple pieces of data you want to resolve with, then stick them in an object and resolve with that object
const getUser = user => new Promise(async (resolve, reject) => {
try {
const read = await snekfetch.get('https://www.website.nl/api/public/users?name=' + user);
const data = JSON.parse(read.text);
const result = await snekfetch.get('https://www.website.com/api/public/users/' + data.uniqueId + '/profile');
const json = JSON.parse(result.text);
resolve({ data, json }); // <--- created an object with two properties
} catch (error) {
reject(error);
}
});
getUser('someUser')
.then((result) => {
console.log(result.data)
console.log(result.json)
})
Additionally, i want to point out that you're creating extra promises where they are not needed. async functions automatically create promises, so your getUser function can just be:
const getUser = async (user) => {
const read = await snekfetch.get('https://www.website.nl/api/public/users?name=' + user);
const data = JSON.parse(read.text);
const result = await snekfetch.get('https://www.website.com/api/public/users/' + data.uniqueId + '/profile');
const json = JSON.parse(result.text);
return { data, json };
}

javascript with firebase timing issue

As I run this code, I wished to add the array final to the firestore -> sendGrid collection but it's always empty, although when I print it, it actually has the values.
I believe this is because of the timing issue, I always get [] -> value is just evaluated now (warning) and when I expand it it has the value.
function test() {
let today = new Date();
let addedDate = new Date(today.addDays(7));
let final = [];
let counter = 0;
let adder = new Promise(function (resolve, reject) {
db.collection("email").get()
.then((querySnapshot) => {
console.log(querySnapshot);
if (querySnapshot.empty !== true) {
querySnapshot.forEach((data) => {
console.log(data.data());
console.log(data.id);
let db2 = db.collection("email").doc(data.id);
let foodArr = [];
if (data.data() !== null) {
console.log(addedDate);
if (addedDate >= userList[0].exxpiaryDate) {
console.log("True");
}
db2.collection("list").where("expiaryDate", "<", addedDate.getTime()).get()
.then((list) => {
if (list.empty !== true) {
list.forEach((food) => {
if (food !== null) {
let temp = {
name: food.data().name,
time: food.data().expiaryDate,
};
foodArr.push(temp);
console.log(foodArr);
}
})
}
if (foodArr.length !== 0) {
let emailArr = {
email: data.data().email,
food: foodArr
};
console.log(emailArr);
final[counter] = (emailArr);
counter++;
console.log(final[0]);
}
}).catch((err) => {
console.log(err);
});
}
});
}
console.log(final);
resolve(final);
}).catch((err) => {
console.log(err);
});
});
return adder;
}
async function add() {
let add = await test();
console.log(add);
db.collection("sendGrid").add({
response: add
}).then((item) => {
console.log(item);
}).catch((err) => {
console.log(err);
});
}

Some points: (google if unsure why)
- prefer const
- return early
- clean code (eg. from console.log:s)
- cache fn calls
- functional programming is neat, look up Array.(map, filter, reduce, ...)
- destructuring is neat
- use arr[arr.length] = x; or arr.push(x), no need to manage your own counter
- short-circuit is sometimes neat (condition && expression; instead of if (condition) expression;)
- is queryResult.empty a thing? If it's a normal array, use !arr.length
- define variables in the inner most possible scope it's used in
- if having a promise in an async, make sure to return it
- prefer arrow functions
I changed the code to follow those points:
const test = ()=> {
const today = new Date();
const addedDate = new Date(today.addDays(7));
return new Promise((resolve)=> {
const final = [];
const emailsQuery = db.collection("email")
// an async/promise/then that's inside another promise, but not returned/awaited
emailsQuery.get().then((querySnapshot) => {
querySnapshot
.map(data=> ({id: data.id, data: data.data()}))
.filter(o=> o.data)
.forEach(({id, data: {email}}) => {
const db2 = db.collection("email").doc(id);
const itemsQuery = db2.collection("list").where("expiaryDate", "<", addedDate.getTime())
// another one!
itemsQuery.get().then((items) => {
const food = items.filter(o=> o).map(o=> o.data()).filter(o=> o)
.forEach(({name, expiaryDate: time})=> ({name, time}))
food.length && final.push({email, food})
}).catch(console.error);
});
// resolving before the two async ones have finished!!
resolve(final);
}).catch(console.error);
});
}
const add = async ()=> {
let response = await test();
return db.collection("sendGrid").add({response})
.then((item) => console.log('item:', item))
.catch(console.error)
}
Now, we can see that there is an issue with the async flow ("timing issue" in your words). I'll add one more best practice:
- use async/await when possible
Changing using that one makes it more clear, and solves the issue:
const test = async ()=> {
const today = new Date();
const addedDate = new Date(today.addDays(7));
const emailsQuery = db.collection("email")
const querySnapshot = await emailsQuery.get()
const emailEntries = querySnapshot
.map(data=> ({id: data.id, data: data.data()}))
.filter(o=> o.data)
// invoking an async fn -> promise; map returns the result of all invoked fns -> array of promises
const promisedItems = emailEntries.map(async ({id, data: {email}}) => {
const db2 = db.collection("email").doc(id);
const itemsQuery = db2.collection("list").where("expiaryDate", "<", addedDate.getTime())
const items = await itemsQuery.get()
const food = items.filter(o=> o).map(o=> o.data()).filter(o=> o)
.forEach(({name, expiaryDate: time})=> ({name, time}))
return {email, food}
});
const items = await Promise.all(promisedItems)
return items.filter(item=> item.food.length)
}
const add = async ()=> {
let response = await test();
return db.collection("sendGrid").add({response})
.then((item) => console.log('item:', item))
.catch(console.error)
}
Now, the flow is clear!
.
Even more concise (though lack of var names -> less clear) - just for kicks:
// (spelled-fixed expiryDate); down to 23% loc, 42% char
const getUnexpiredFoodPerEmails = async ({expiryDateMax} = {
expiryDateMax: new Date(new Date().addDays(7)),
})=> (await Promise.all((await db.collection('email').get())
.map(data=> ({id: data.id, data: data.data()})).filter(o=> o.data)
.map(async ({id, data: {email}})=> ({
email,
food: (await db.collection('email').doc(id).collection('list')
.where('expiryDate', '<', expiryDateMax.getTime()).get())
.filter(o=> o).map(o=> o.data()).filter(o=> o)
.forEach(({name, expiryDate: time})=> ({name, time})),
}))
)).filter(item=> item.food.length)
const add = async ()=> db.collection('sendGrid').add({
response: await getUnexpiredFoodPerEmails(),
}).then(console.log).catch(console.error)
// ...or with names
const getUnexpiredFoodListForEmailId = async ({id, expiryDateMax} = {
expiryDateMax: new Date(new Date().addDays(7)),
})=> (await db.collection('email').doc(id).collection('list')
.where('expiryDate', '<', expiryDateMax.getTime()).get())
.filter(o=> o).map(o=> o.data()).filter(o=> o)
.forEach(({name, expiryDate})=> ({name, time: expiryDate}))
const getEmails = async ()=> (await db.collection('email').get())
.map(data=> ({id: data.id, data: data.data()})).filter(o=> o.data)
const getUnexpiredFoodPerEmails = async ({expiryDateMax} = {
})=> (await Promise.all((await getEmails()).map(async ({id, data})=> ({
email: data.email,
food: await getUnexpiredFoodListForEmailId({id, expiryDateMax}),
})))).filter(item=> item.food.length)
const add = async ()=> db.collection('sendGrid').add({
response: await getUnexpiredFoodPerEmails(),
}).then(console.log).catch(console.error)