We Keep Coding

JavaScript Asynchronous in Redux..?

Action in fuction (action for data imported from firstore)
Here you get index data.
export async function getUserIndex (data){
let db = loadFB().firestore();
console.log(data)
let date = moment(data).utc().format()
let query = db.collection('users').where("create_date", ">", date)
console.log(query)
return await query.get().then(docs=>{
let result = docs.size
console.log("result!!", result)
return result
})
}
components in function
async getuserIndex_component(date){
let number =""
number = await userAction.getUserIndex(date)
console.log("number",number)
return await number
}
const {user_list} = this.props; //user_list = [{id:aa, pw:bb},{id:cc, pw:dd}...} data
let data = user_list
let groups = {}
let number =0
for (let index_data of data) {
let date = moment(index_data.create_date).format("YYYY-MM-DD").slice(0, 10)
let index = this.getuserIndex_component(date) //<==here log Promise{<pendding>}
console.log(index)
if (groups[date]) {
let group = groups[date];
group.push({ index: index-number, ...index_data });
} else {
number =0;
groups[date] = [{ index: index-number, ...index_data }]
}
number++;
}
const dates = Object.keys(groups)
const user_list_result = []
for (let date of dates) {
user_list_result.push(...(groups[date]))
}
return(...)
I am using redux. in action
Made a "getUserIndex()."
The component will then access the "getUserIndex" and retrieve data. But there is only pending in the console.log, and the data I want does not appear.
In this situation, I would like to know how to deal with async in for of.
I want to know what part I'm making a mistake.
In summary, functions in components are executed later than the render. So the index in the render does not contain the data. What is the solution to this?

As getuserIndex_component is asynchronous, it will return a promise. You will need to wait for that promise to resolve before the index result is available.
Firstly, you should be able to simplify getuserIndex_component to
async getuserIndex_component(date) {
let number = await userAction.getUserIndex(date)
console.log("number", number)
return number
}
And when you remove the console.log it could simply be
async getuserIndex_component(date) {
return userAction.getUserIndex(date)
}
As I mentioned in the comments. You can send off all of the async calls, then just wait for all of the promises to resolve via Promise.all.
Something like this. (I couldn't test any of this, but it should demonstrate the general idea.)
const { user_list } = this.props; //user_list = [{id:aa, pw:bb},{id:cc, pw:dd}...} data
let data = user_list
let groups = {}
let number = 0
const dates = []
const promises = data.map((index_data) => {
const date = moment(index_data.create_date).format("YYYY-MM-DD").slice(0, 10)
dates.push(date) // To use after all promises resolve.
return this.getuserIndex_component(date) // The index promise.
})
const indexes = await Promise.all(promises)
for (let i = 0; i < dates.length; i++) {
const index = indexes[i]
const date = dates[i]
console.log(index)
if (groups[date]) {
let group = groups[date];
group.push({
index: index - number,
...index_data
});
} else {
number = 0;
groups[date] = [{
index: index - number,
...index_data
}]
}
number++;
}
const dates = Object.keys(groups)
const user_list_result = []
for (let date of dates) {
user_list_result.push(...(groups[date]))
}
return (...)

Nested for loop combined with fetching data from an API in Nodejs?

My scenario is as follows:
There is an API I would like to fetch from. The API returns a json that has an array named "assets". This array size will always be 20.
Now, I call the endpoint like so:
fetch(
`https://api.example.io/api/v1/assets/?offset=${offset}`
)
where if offset is 0 it will return the array of the 0 - 20 assets, if the offset is 20 it will return 20 to 40 and so on.
I want to check 1000 items which means I would like to call this fetch 1000/20 = 50 times.
Whenever I call the fetch I would like loop over these 20 items and insert them into my DB.
The problem is that I can't do something like this:
let offset=0;
for(let i = 0; i < 50; i++ {
fetch(
`https://api.example.io/api/v1/assets/?offset=${offset}`
)
for(let j = 0; j < 20; j++){
// Insert into DB
}
offset+=20;
}
Due to JS asynchronous nature. Whenever I try to do this it will call the fetch with the value 0 for offset a couple of times, it won't wait until the nested for loop finishes and than call it for 20 and later 40 and so on...
What is the correct way to achive this behavior?

I've nicked the following from one of my nodeJS repos as it employs async code for solving a very similar problem:
// Throttling is important because you don't want to
// overload the API
const createThrottle = require('async-throttle');
const throttle = createThrottle(2);
// First push all the links into an array using
// the offset
const links = [];
for (let offset = 0; offset < 100; offset += 20) {
links.push(`https://api.example.io/api/v1/assets/?offset=${offset}`);
}
// Next create an array of promises by `map`ing
// over the links using `fetch`.
// Notice I've used throttle here to slow down the hits on
// the API
const promises = links.map(link => throttle(async => () {
const res = await fetch(link);
return await res.json();
}));
// Once all the promises are complete, iterate over their datasets
Promise.all(promises).then(datasets => {
// iterate over datasets
});

Why not use Promise.all?
const promises = [];
for (let i = 0; i < 10; i++) {
promises.push(
fetch(`https://api.example.io/api/v1/assets/?offset=${i}`)
);
}
Promise.all(promises).then( _results => {
// All fetch calls have completed at this point. _results will be in array of your fetch results.
console.log(results);
// Do db insert here
});
One thing you could do is to make a function that is a promise. Within that function, have it execute the fetch and then execute the db insert all in one function (using .then). If you did it this way, the single Promise.all call would handle everything. If you don't, you'll have to loop through the promises again and insert those values into the db. It might look something like this:
const promises = [];
for (let i = 0; i < 10; i++) {
promises.push(fetchAndInsert(i));
}
Promise.all(promises).then( _results => {
console.log(results);
});
function fetchAndInsert(offset) {
return new Promise( (resolve, reject) => {
fetch(`https://api.example.io/api/v1/assets/?offset=${i}`).then (_result => {
// db.insert().then( _result => {
//resolve();
//})
})
})
}

You could use async and await. This should work:
async function fetchExample() {
for (let i = 0; i < 50; i++) {
let fetchedData = await fetch(
`https://api.example.io/api/v1/assets/?offset=${offset}`
);
for(data of fetchedData) {
// Insert into DB
}
offset+=20;
}
}

Instead of for..loop, you could use recursion or Promises.
Recursion::
let insertToDB = function (records, cb) {
if (!records.length) return cb();
let record = records.shift();//assuming records is an array
// Insert record into DB1
insertToDB(records, cb);
};
let loop = function (count, offset, cb) {
if (!count) return cb();
fetch(
`https://api.example.io/api/v1/assets/?offset=${offset}`
)
insertToDB(recordsFromFetch, function () {
offset += 20;
--count;
loop(count, offset, cb)
})
};
loop(50, 0, function () {
console.log("All Done");
})
Promise:: I have not run it, so might be some syntactical error, but you get the idea
let insertToDB = function (record) {
return new Promise(function (resolve, reject) {
// Insert record into DB then call resolve
resolve();
})
};
let fetchPhase = function (offset) {
fetch(
`https://api.example.io/api/v1/assets/?offset=${offset}`
)
let dbAll = [];
for (let j = 0; j < 20; j++) {
// assuming you get records from fetch , pass record to be added to db in the parameter
dbAll.push(insertToDB(records[j]))
}
return Promise.all(dbAll)
};
let all = [];
let offset = 0;
for (let i = 0; i < 50; i++) {
all.push(fetchPhase(i));
}
Promise.all(all)
.then(function () {
console.log("ALL DONe");
})

Promise.all(Array(50).fill(null).map((v,i) => {
const url = `https://api.example.io/api/v1/assets/?offset=${i*20}`;
return fetch(url).then(results => {
for (let result of results) {
// insert into DB
}
);
}).then(() => console.log("done"));

How to run "x" promises in parallel Javascript

I have a function "promiseFunction" which returns a promise, which is resolved at a later time.
I need to call this function many times, however I only want a set number of executions of this function to happen at once.
The function calls some external single threaded c code on my computer, if I call too many instances of this code at once I crash my system, but if I call it sequentially 1 at a time it's very slow as only one thread of my cpu is doing any work.
So I came up with the code below, however it doesn't work. It will call the first 10 promises in parallel, but slowly it starts to call less and less promises at once, until it's only calling 1 promise at a time.
var totalNumberOfPromises = // total number times to run promiseFunction;
var promiseCounter = 0; // keep track of which promise call this is
for(w=0;w<10;w++){ // run 10 promises at a time
promiseFunction().then(function(resolve) {
loadNewPromise();
})
promiseCounter++;
}
function loadNewPromise(){
if(promiseCounter<totalNumberOfPromises){
promiseFunction().then(function(resolve) { loadNewPromise(); });
}else{
alert("Finished");
}
promiseCounter++;
}
Is there anything wrong with the code above that causes this behavior? And is there a standard way of doing this?

Here's a function I prepared earlier (I've used this for a few years now for just such a thing
const multiQueue = length => {
length = (isNaN(length) || length < 1) ? 1 : length;
const q = Array.from({length}, () => Promise.resolve());
let index = 0;
const add = cb => {
index = (index + 1) % length;
return (q[index] = q[index].then(() => cb()));
};
return add;
};
// demo usage
const q = multiQueue(10);
let inFlight = 0;
let maxInFlight = 0;
const promiseFunction = (i) => {
inFlight++;
maxInFlight = Math.max(inFlight, maxInFlight);
const obj = {inFlight, maxInFlight, i};
return new Promise(resolve => {
setTimeout(() => {
inFlight--;
resolve(Object.assign(obj, {t:performance.now()}));
}, 10 );
})
};
for (let i = 0; i < 40; i++) {
q(() => promiseFunction(i)).then(v => console.log(JSON.stringify(v)));
}
You can see that at most there are 10 "inFlight" requests

How about something like this? If you construct your queue which is and Array of functions that return a Promise you can splice chunks out of it and process each with a Promise.all.
const fakePromise = (id) => {
return new Promise((resolve) => {
setTimeout(() => {
console.log(`Resolving promise ${id}`)
resolve(id)
}, 100)
})
}
const queue = Array(100).fill().map((_, i) => {
return () => fakePromise(i)
})
const batchProcessPromises = (promises, batchSize) => {
if (promises && promises.length) {
return Promise.all(
promises.splice(0, batchSize)
.map(promise => promise())
)
.then(() => {
console.log('Batch complete')
return batchProcessPromises(promises, batchSize)
})
}
console.log('Batch complete')
return Promise.resolve()
}
batchProcessPromises(queue, 10)
.then(() => {
console.log('Time to get one with my day')
})
How do you plan to construct all your promises? This function effects the original queue so you would need to ensure that the array being passed into batchProcessPromises isn't shared. To get around this you could potentially use the spread operator like so
batchProcessPromises([...queue], 10)
.then(() => {
console.log('Time to get one with my day', queue)
})
Fiddle here https://jsfiddle.net/stwilz/2bpdcxo6/24/

How to use fetch within a for-loop, wait for results and then console.log it

i have this problem: i want to make multiple fetch calls within a for-loop. The number of calls depend on the user input (in my example i have three). How can i make it loop through all the fetch requests and then console.log the number off calls?
function getPosts(){
let url = ["https://www.freecodecamp.org", "https://www.test.de/, http://www.test2.com"];
let array = new Array;
for (let i = 0; i < url.length; i++) {
console.log(url[i]);
fetch(url[i])
.then(res => {return res.text(); })
.then(res => {
let reg = /\<meta name="description" content\=\"(.+?)\"/;
res = res.match(reg);
array.push(res);
console.log(res);
}
)
.catch(status, err => {return console.log(status, err);})
}
console.log (array.length);
}
It console.logs 0 instead of 3, cause it doesn't wait for all the promises to be resolved. How can i make it to console.log 3?
If you know a solution, please help me out.

You can't call console.log(array.length) until after the promises are all done. So why not something like this?
let url = ["https://www.freecodecamp.org", "https://www.test.de/, http://www.test2.com"];
let array = new Array;
var fetches = [];
for (let i = 0; i < url.length; i++) {
console.log(url[i]);
fetches.push(
fetch(url[i])
.then(res => {return res.text(); })
.then(res => {
let reg = /\<meta name="description" content\=\"(.+?)\"/;
res = res.match(reg);
array.push(res);
console.log(res);
}
)
.catch(status, err => {return console.log(status, err);})
);
}
Promise.all(fetches).then(function() {
console.log (array.length);
});
}
Promise.all waits for all the fetches to finish, THEN it'll print the #.

You can use async/await with try/catch:
async function getPosts(){
let array = [];
let url = ["https://www.freecodecamp.org", "https://www.test.de/", "http://www.test2.com"];
let reg = /\<meta name="description" content\=\"(.+?)\"/;
for (let i = 0; i < url.length; i++) {
console.log('fetching',url[i]);
try {
let p1 = await fetch(url[i]);
let p2 = await p1.text();
let res = p2.match(reg);
array.push(res);
console.log('adding',res);
}
catch (e) {
console.error(e.message);
}
};
console.log ('length',array.length);
};
getPosts().then(()=>{console.log('done')});

You can try chaining your promises, Try the following:
function getPosts(){
let url = ["https://www.freecodecamp.org", "https://www.test.de/, http://www.test2.com"];
let array = new Array;
var promise = Promise.resolve();
for (let i = 0; i < url.length; i++) {
console.log(url[i]);
promise = promise.then(fetch(url[i]))
.then(res => {return res.text(); })
.then(res => {
let reg = /\<meta name="description" content\=\"(.+?)\"/;
res = res.match(reg);
array.push(res);
console.log(res);
}
)
.catch(status, err => {return console.log(status, err);})
}
promise.then(function(response){
console.log (array.length);
});
}

You can use async await ( async: function, await: operator ). Await operator simply wait for the promise to be resolved. First promise will be resolved then it will move to another one. Also, if it finds error in any fetch, it will catch the error right away.

You should use promise, A Promise is a proxy for a value not necessarily known when the promise is created. It allows you to associate handlers with an asynchronous action's eventual success value or failure reason. This lets asynchronous methods return values like synchronous methods: instead of immediately returning the final value, the asynchronous method returns a promise to supply the value at some point in the future.
const fetch = require('node-fetch')
let url = ["https://www.freecodecamp.org", "https://www.test.de/, http://www.test2.com"];
let array = new Array;
function get(url) {
return new Promise((resolve, reject) => {
fetch(url)
.then(res => { return res.text(); })
.then(res => {
let reg = /\<meta name="description" content\=\"(.+?)\"/;
res = res.match(reg);
resolve(res)
//console.log(res);
}
)
.catch(err => { reject(err) })
});
}
async function result() {
for (let i = 0; i < url.length; i++) {
const value = await get(url[i]);
array.push(value)
}
console.log(array.length)
}
result()
==> array.length = 2,

How to guarantee sequential order with angular http rest api in for loop?

I'm trying to create a form that allows you to create multiple resources in sequential order.
Example below
Floor 1
Floor 2
Floor 3
...
Floor 9
The problem with the code is that the order is not guarantee.
My code below
let startAt = this.addAreasForm.controls['startAt'].value
const name = this.addAreasForm.controls['name'].value
const newArea = {name: name}
for (let i = 1; i < (amount + 1); i++) {
newArea.name = name + ' ' + startAt
startAt++
this.areasService.createArea(newArea, parentId)
.subscribe(
area => this.added.emit(area)
)
}
Can come back like
Floor 2
Floor 3
Floor 1
Floor 5
Floor 4
How do you handle async api calls to guarantee sequential order?

You can use async / await for that purpose with the Promise resolve:
for (let i = 1; i < (amount + 1); i++) {
await new Promise(resolve => {
newArea.name = name + ' ' + startAt
startAt++
this.areasService.createArea(newArea, parentId)
.subscribe(
area => {
this.added.emit(area);
resolve();
});
});
}
Remember to put async before your function. See this demo on StackBlitz.

You can try something like this, I don't exactly all your code from your services, but the main idea is this: In order to execute async code in order, you can build an array of promises and then to use Promise.all to take each result in the same order from the creation:
Promise.all
let startAt = this.addAreasForm.controls['startAt'].value;
const name = this.addAreasForm.controls['name'].value;
const newArea = {name: name};
Keep your services into variables I don't know from where your context comes.
const areasService = this.areasService,
added = this.added;
Make a function that create a promise for your subscribe:
function createAreaPromise(newArea, parentId) {
return new Promise((resolve, reject) => {
areasService.createArea(newArea, parentId)
.subscribe(area => resolve(area));
});
}
Than another function to build multiple an array of promises:
function buildPromises() {
let promises = [];
for (let i = 1; i < (amount + 1); i++) {
newArea.name = name + ' ' + startAt
startAt++
promises.push(createAreaPromise(newArea, parentId));
}
return promises;
}
Then solve them with Promise.all, to obtain the same order from creation
let promises = buildPromises();
Promise.all(promises)
.then(results => {
results.forEach(result => added.emit(result));
});
Here a live example:
function random() {
return Math.floor(Math.random() * 5);
}
function makePromise(index) {
return new Promise((resolve) => {
setTimeout(() => {
resolve(index);
}, random() * 1000);
});
}
function buildPromises() {
let promises = [];
for(let i = 0; i < 5; i++) {
promises.push(makePromise(i));
}
return promises;
}
let promises = buildPromises();
Promise.all(promises)
.then(results => {
results.forEach(result => {
console.log(result);
});
});