This question already has answers here:
Using async/await with a forEach loop
(33 answers)
Why is my variable unaltered after I modify it inside of a function? - Asynchronous code reference
(7 answers)
Closed 7 months ago.
I have a situation where I need to read all the files in a directory and all its subdirectories. I wrote a basic function:
Although this is in React-Native, I'm not sure that matters (or maybe there is a react-native solution).
let fileList = [];
const readDir = async (dir) => {
// This line simply reads the contents of a directory, creating an array of strings
// of all the files and directories inside 'dir'.
const items = await FileSystem.readDirectoryAsync(dir);
items.forEach(async (item) => {
const f = await FileSystem.getInfoAsync(item); // Gets basic information about the item
if (f.isDirectory === true) {
readDir(f.uri); // f.uri is the location of the new directory to read
} else {
// runs if the item is a file
console.log("f.uri: ", f.uri);
fileList.push(f.uri); // A global variable
}
})
};
const parentDirectory = "parent_folder";
readDir(parentDirectory);
// Do more stuff here once all files have been read and added to 'fileList'
This seems to partially work as all the files in all the subdirectories are consoled out from inside the else {...} segment.
However, how can I know the loop is complete so I can continue the script and use 'fileList'?
Don't use forEach with async they don't work together and you can't await the loop. Use a standard for .. of loop, which works nicely with async/await
const readDir = async (dir) => {
const items = await FileSystem.readDirectoryAsync(dir);
for (let item of items) {
const f = await FileSystem.getInfoAsync(item);
if (f.isDirectory === true) {
await readDir(f.uri);
} else {
console.log("f.uri: ", f.uri);
fileList.push(f.uri);
}
}
}
And as readDir is async as well, you have of course either to await it also, or use then to continue once it is finished
async function foo() {
const parentDirectory = "parent_folder";
await readDir(parentDirectory);
//do some other stuff once readdir is finished.
}
or
const parentDirectory = "parent_folder";
readDir(parentDirectory).then(_ => {
//do some other stuff once readdir is finished.
}
Related
This question already has answers here:
Using async/await with a forEach loop
(33 answers)
Closed 4 months ago.
Need to query downloading of file array
I have an array with URL's of files to dowload:
const arrUrl = [
'http://server.domain/path/file1.jpg',
'http://server.domain/path/file2.jpg',
'http://server.domain/path/file3.jpg',
];
when i try to download files with Axios and code:
arrUrl.forEach(async (element: string, i:number) => {
const response = await new Axios({responseType: 'stream'}).get(element);
await writeFile('./files/'+i, response.data);
console.log(element);
});
All files start downloading at one time. But how can i download files in query mode? one-by-one
This can be achieved using async function with for-in (index) of for-of (element) loop. Since we need to deal with loop index for-in is suitable here.
const arrUrl = [
'http://server.domain/path/file1.jpg',
'http://server.domain/path/file2.jpg',
'http://server.domain/path/file3.jpg',
];
async function downloadFiles(arrUrl) {
for (let index in arrUrl) {
const element = arrUrl[index];
const response = await new Axios({ responseType: 'stream' }).get(element);
await writeFile('./files/' + index, response.data);
}
}
downloadFiles(arrUrl);
The following function runs after a drag and drop operation of multiple files.
function getFilesInfo(ev){
for (let i = 0; i < ev.dataTransfer.items.length; i++) {
if (ev.dataTransfer.items[i].kind === 'file') {
let file = ev.dataTransfer.items[i].getAsFile();
//getFileInfo adds string to DOM element
//Note the promise usage ...
file.arrayBuffer().then((data)=>getFileInfo(file.name,data));
}
}
}
I can't figure out how to call a function after all of the promises in this function finish.
Basically I want something like this, sequentially:
getFilesInfo(ev);
//getFileInfo(<file1>);
//getFileInfo(<file2>);
//getFileInfo(<file3>);
// etc.
//run only after all getFileInfo() calls have finished
processResults();
The tricky part is that reading the files generates a promise for each file that gets called when the file has been read into memory (part of the arrayBuffer() call). I can't figure out how to delay processResults because getFilesInfo finishes after all of the read calls have been triggered, not (from what I can tell), after the getFileInfo functions have finished.
It seems like perhaps I could somehow add all arrayBuffer calls to an array and then do some promise chaining (maybe?) but that seems awkward and I'm not even sure how I would do that.
You can use Promise.all to wait for an array of promise to finish:
async function getFilesInfo(ev) {
// create list of jobs
const jobs = [];
for (const item of ev.dataTransfer.items) {
if (item.kind === 'file') {
let file = item.getAsFile();
jobs.push(file.arrayBuffer().then(data => {
getFileInfo(file.name, data);
}));
}
}
// wait for all promise to fullfil
await Promise.all(jobs);
}
https://developer.mozilla.org/fr/docs/Web/JavaScript/Reference/Global_Objects/Promise/all
You could do it that way:
function getFilesInfo(ev){
return ev.dataTransfer.items.filter(item=>item.kind === 'file').map(item=>{
let file = item.getAsFile();
return file.arrayBuffer().then((data)=>getFileInfo(file.name,data));
});
}
Promise.all(...getFilesInfo(ev)).then(_=>{
processResults();
});
// or with async/await
(async ()=>{
await Promise.all(...getFilesInfo(ev));
processResults();
})()
async function getFilesInfo(ev) {
await Promise.all(ev.dataTransfer.items.map(async (i) => {
const file = i.getAsFile();
const data = await file.arrayBuffer();
return getFileInfo(file.name, data);
}));
}
await getFilesInfo(ev); // will be awaited until all the promises are resolved
processResults();
Let me know if that helps.
The conceptual hurdle I was running into was that I was thinking of the then function as returning the results, not promises. Also, many of the examples I've seen with Promise.all are usually just concatenating explicit calls, not building an array in a loop.
As suggested by Bergi, I simply added the calls to an array, and then passed that array into Promise.all
function getFilesInfo(ev) {
// create list of jobs
let jobs = [];
for (const item of ev.dataTransfer.items) {
if (item.kind === 'file') {
let file = item.getAsFile();
jobs.push(file.arrayBuffer().then(data => {
getFileInfo(file.name, data);
}));
}
}
return jobs;
}
//The call in the parent
let jobs = getFilesInfo(ev);
Promise.all(jobs).then(processResults);
I am trying to run parallel requests in batches to an API using a bunch of keywords in an array. Article by Denis Fatkhudinov.
The problem I am having is that for each keyword, I need to run the request again with a different page argument for as many times as the number in the pages variable.
I keep getting Cannot read property 'then' of undefined for the return of the chainNext function.
The parallel request in batches on its own, without the for loop, works great, I am struggling to incorporate the for loop on the process.
// Parallel requests in batches
async function runBatches() {
// The keywords to request with
const keywords = ['many keyword strings here...'];
// Set max concurrent requests
const concurrent = 5;
// Clone keywords array
const keywordsClone = keywords.slice()
// Array for future resolved promises for each batch
const promises = new Array(concurrent).fill(Promise.resolve());
// Async for loop
const asyncForEach = async (pages, callback) => {
for (let page = 1; page <= pages; page++) {
await callback(page);
}
};
// Number of pages to loop for
const pages = 2;
// Recursively run batches
const chainNext = (pro) => {
// Runs itself as long as there are entries left on the array
if (keywordsClone.length) {
// Store the first entry and conviently also remove it from the array
const keyword = keywordsClone.shift();
// Run 'the promise to be' request
return pro.then(async () => {
// ---> Here was my problem, I am declaring the constant before running the for loop
const promiseOperation = await asyncForEach(pages, async (page) => {
await request(keyword, page)
});
// ---> The recursive invocation should also be inside the for loop
return chainNext(promiseOperation);
});
}
return pro;
}
return await Promise.all(promises.map(chainNext));
}
// HTTP request
async function request(keyword, page) {
try {
// request API
const res = await apiservice(keyword, page);
// Send data to an outer async function to process the data
await append(res.data);
} catch (error) {
throw new Error(error)
}
}
runBatches()
The problem is simply that pro is undefined, because you haven't initialized it.
You basically execute this code:
Promise.all(new Array(concurrent).fill(Promise.resolve().map(pro => {
// pro is undefined here because the Promise.resolve had no parameter
return pro.then(async () => {})
}));
I'm not completely sure about your idea behind that, but this is your problem in a more condensed version.
I got it working by moving actual request promiseOperation inside the for loop and returning the recursive function there too
// Recursively run batches
const chainNext = async (pro) => {
if (keywordsClone.length) {
const keyword = keywordsClone.shift()
return pro.then(async () => {
await asyncForEach(pages, (page) => {
const promiseOperation = request(keyword, page)
return chainNext(promiseOperation)
})
})
}
return pro
}
Credit for the parallel requests in batches goes to https://itnext.io/node-js-handling-asynchronous-operations-in-parallel-69679dfae3fc
This question already has answers here:
Using async/await with a forEach loop
(33 answers)
Closed 2 years ago.
The below JavaScript is returning a strange version of an array. The below forEach logs nothing to console. I'm probably missing something quite fundamental here, but I know for a fact that the array is populated...unless it's not populated until later due to being async and Chrome just evaluates the console.log after the fact?
let damageRollStringArr = []
monster.damage_types.forEach(async (damageTypeName) => {
const damageType = await checkResponseAndReturnValue(await DamageTypesService.findDamageType(damageTypeName.trim()))
if (damageType !== null) {
damageRollStringArr.push(damageType.damage)
}
})
damageRollStringArr.forEach(el => console.log(el))
// Was returning an empty array[] with objects inside?
return damageRollStringArr
Thanks
demageRollStringArr.forEach (2nd foreach) wont wait for monster.demage_types.forEach (1st foreach) to execute although there is async await in the first forEach
To achieve what you want to do, try this,
let damageRollStringArr = []
const promises = monster.damage_types.map(async (damageTypeName) => {
const damageType = await checkResponseAndReturnValue(await DamageTypesService.findDamageType(damageTypeName.trim()))
if (damageType !== null) {
damageRollStringArr.push(damageType.damage)
}
})
await Promise.all(promises)
damageRollStringArr.forEach(el => console.log(el))
return damageRollStringArr
Or, you can use normal for() loop to replace 1st foreach
let damageRollStringArr = []
for(let i = 0; i < monster.demage_types.length; i++) {
const damageType = await checkResponseAndReturnValue(await DamageTypesService.findDamageType(damageTypeName.trim()))
if (damageType !== null) {
damageRollStringArr.push(damageType.damage)
}
}
damageRollStringArr.forEach(el => console.log(el))
return damageRollStringArr
1st solution will run the loop parallel,
2nd solution will run the loop sequential
This question already has answers here:
How do I return the response from an asynchronous call?
(41 answers)
Closed 3 years ago.
I am new to node js and I have never worked with promises before so I would greatly appreciate any advice.
I am using an async await function to read a .txt file line by line, it returns an array.
async function processLineByLine() {
const fileStream = fs.createReadStream('input.txt');
const array = [];
const rl = readline.createInterface({
input: fileStream,
crlfDelay: Infinity
});
for await (const line of rl) {
array.push(line)
}
return array
}
I want to create a new instance of a class using this array as an argument, like this:
let variableName = new ClassName(array)
So that I can call functions on this instance of an object and manipulate its state.
I have tried to do this:
async function input() {
var result = await processLineByLine();
return result
}
let variableName = ClassName(input())
variableName.someFunction()
But it fails as the state I am trying to access is undefined and console.log(input()) shows promise pending.
You need to put all the code after the await:
async function input() {
var result = await processLineByLine();
let variableName = ClassName(result)
variableName.someFunction();
}
input().catch(console.error);