We Keep Coding

Assigning new values doesn't work inside multiple api calls

I am calling multiple apis with axios and Promise.all and trying to assign some new values (changing image to blurhash code with plaiceholder) inside data returned from each api like this
let promises = [];
const urls = [
`https://api.themoviedb.org/3/trending/movie/week?api_key=${process.env.API_KEY}`,
`https://api.themoviedb.org/3/movie/popular?api_key=${process.env.API_KEY}&language=en-US&page=1`,
];
urls.forEach(function (url) {
promises.push(
axios.get(url).then(
(data) => {
data.data.results.map(async (one) => {
const { blurhash, img } = await getPlaiceholder(
`https://image.tmdb.org/t/p/w500${one.poster_path}`
);
// assigning new values and return the object
return {
...img,
...one,
blurhash,
};
});
return { success: true, data: data };
},
function () {
return { success: false };
}
)
);
});
const [trendingRes, popularRes] = await Promise.all(promises);
console.log(trendingRes.data.data) // new values don't get assigned
I am guessing it's because I used async await inside .then() so this return { success: true, data: data }; doesn't wait for the new data to get updated and return the old data. But I couldn't find a way to fix it.

issue with promise Node.js

I have been trying to figure this out for quite some time and it's too confusing so i thought i'd ask.
var listenKey = "";
const createListenKey = async () => {
await axios({
url: "/api/v3/userDataStream",
method: "POST",
baseURL: "https://api.binance.com",
headers: {
"X-MBX-APIKEY":
"H48w9CLuTtTi955qWjcjjEKhh0Ogb3jnnluYucXXXXXXXXXXXXXXXX",
},
}).then((response) => {
var key = response.data.listenKey;
console.log(key, "created");
return key;
});
};
listenKey = createListenKey();
listenKey.then((key) => {
console.log(key);
});
the console.log in the last but one line is printing undefined. Why is that?
Thanks in advance!

You did not return anything from the async function createListenKey
const asynF = async ()=>{
Promise.resolve(1).then(res=>{
//Simulating response from axios call
console.log(res)
})
// you are not returning anyting from this function equivalent of => return ;
}
asynF().then(res=>{
//this would log undefined
console.log(res)
})
As you know async function returns a promise you have two options make the outer wrapper a async function as well and just use await like below
const key = await createListenKey(config)
or else
you could simply do this
return createListenKey(config).then(res=>{
listenKey = res
})
can't say much more without knowing context.
Might I suggest not to mix then and async wait together

Because the createListenKey() function doesn't return anything. The return statement in the then clause inside that function is scoped in the then block. To return a value from an async function, you need to do something like this.
const createListenKey = async () => {
const response = await axios({
url: "/api/v3/userDataStream",
method: "POST",
baseURL: "https://api.binance.com",
headers: {
"X-MBX-APIKEY":
"H48w9CLuTtTi955qWjcjjEKhh0Ogb3jnnluYucXXXXXXXXXXXXXXXX",
},
})
var key = response.data.listenKey;
console.log(key, "created");
return key;
};

How do I make a long list of http calls in serial?

I'm trying to only make one http call at time but when I log the response from getUrl they are piling up and I start to get 409s (Too many requests)
function getUrl(url, i, cb) {
const fetchUrl = `https://api.scraperapi.com?api_key=xxx&url=${url.url}`;
fetch(fetchUrl).then(async res => {
console.log(fetchUrl, 'fetched!');
if (!res.ok) {
const err = await res.text();
throw err.message || res.statusText;
}
url.data = await res.text();
cb(url);
});
}
let requests = urls.map((url, i) => {
return new Promise(resolve => {
getUrl(url, i, resolve);
});
});
const all = await requests.reduce((promiseChain, currentTask) => {
return promiseChain.then(chainResults =>
currentTask.then(currentResult => [...chainResults, currentResult]),
);
}, Promise.resolve([]));
Basically I don't want the next http to start until the previous one has finished. Otherwise I hammer their server.
BONUS POINTS: Make this work with 5 at a time in parallel.

Since you're using await, it would be a lot easier to use that everywhere instead of using confusing .thens with reduce. It'd also be good to avoid the explicit Promise construction antipattern. This should do what you want:
const results = [];
for (const url of urls) {
const response = await fetch(url);
if (!response.ok) {
throw new Error(response); // or whatever logic you need with errors
}
results.push(await response.text());
}
Then your results variable will contain an array of response texts (or an error will have been thrown, and the code won't reach the bottom).
The syntax for an async function is an async keyword before the argument list, just like you're doing in your original code:
const fn = async () => {
const results = [];
for (const url of urls) {
const response = await fetch(url);
if (!response.ok) {
throw new Error(response); // or whatever logic you need with errors
}
results.push(await response.text());
}
// do something with results
};
To have a limited number of requests at a time, make a queue system - when a request completes, recursively call a function that makes another request, something like:
const results = [];
const queueNext = async () => {
if (!urls.length) return;
const url = urls.shift();
const response = await fetch(url);
if (!response.ok) {
throw new Error(response); // or whatever logic you need with errors
}
results.push(await response.text());
await queueNext();
}
await Promise.all(Array.from({ length: 5 }, queueNext));
// do something with results

You cannot use Array methods to sequentually run async operations because array methods are all synchronous.
The easiest way to achieve sequential async tasks is through a loop. Otherwise, you will need to write a custom function to imitate a loop and run .then after a async task ends, which is quite troublesome and unnecessary.
Also, fetch is already returning a Promise, so you don't have to create a Promise yourself to contain that promise returned by fetch.
The code below is a working example, with small changes to your original code (see comments).
// Fake urls for example purpose
const urls = [{ url: 'abc' }, { url: 'def', }, { url: 'ghi' }];
// To imitate actual fetching
const fetch = (url) => new Promise(resolve => {
setTimeout(() => {
resolve({
ok: true,
text: () => new Promise(res => setTimeout(() => res(url), 500))
});
}, 1000);
});
function getUrl(url, i, cb) {
const fetchUrl = `https://api.scraperapi.com?api_key=xxx&url=${url.url}`;
return fetch(fetchUrl).then(async res => { // <-- changes here
console.log(fetchUrl, 'fetched!');
if (!res.ok) {
const err = await res.text();
throw err.message || res.statusText;
}
url.data = await res.text();
return url; // <--- changes here
});
}
async function getAllUrls(urls){
const result = [];
for (const url of urls){
const response = await getUrl(url);
result.push(response);
}
return result;
}
getAllUrls(urls)
.then(console.log);

async/await is perfect for this.
Assuming you have an array of URLs as strings:
let urls = ["https://example.org/", "https://google.com/", "https://stackoverflow.com/"];
You simply need to do:
for (let u of urls) {
await fetch(u).then(res => {
// Handle response
}).catch(e => {
// Handle error
});
}
The loop will not iterate until the current fetch() has resolved, which will serialise things.
The reason array.map doesn't work is as follows:
async function doFetch(url) {
return await fetch(url).then(res => {
// Handle response
}).catch(e => {
// Handle error
});
}
let mapped = urls.map(doFetch);
is equivalent to:
let mapped;
for (u of urls) {
mapped.push(doFetch(u));
}
This will populate mapped with a bunch of Promises immediately, which is not what you want. The following is what you want:
let mapped;
for (u of urls) {
mapped.push(await doFetch(u));
}
But this is not what array.map() does. Therefore using an explicit for loop is necessary.

Many people provided answers using for loop. But in some situation await in for loop is not welcome, for example, if you are using Airbnb style guide.
Here is a solution using recursion.
// Fake urls for example purpose
const urls = [{ url: 'abc' }, { url: 'def', }, { url: 'ghi' }];
async function serialFetch(urls) {
return await doSerialRecursion(
async (url) => {
return result = await fetch(url)
.then((response) => {
// handle response
})
.catch((err) => {
// handle error
});
},
urls,
0
);
}
async function doSerialRecursion(fn, array, startIndex) {
if (!array[startIndex]) return [];
const currResult = await fn(array[startIndex]);
return [currResult, ...(await doSerialRecursion(array, fn, startIndex + 1))];
}
const yourResult = await serialFetch(urls);
The doSerialRecursion function will serially execute the function you passed in, which is fetch(url) in this example.

Function returns before ajax call in for loop end

I'm having some trouble with a function that returns before ajax call from for loop ends. I do several ajax call which I get the result from and use them on my then chain.
Here is my code
const delay = t => new Promise(resolve => setTimeout(resolve, t));
(async () => {
await delay(timeout);
let results: boolean[] = [];
for (let i = 0; i < times; i++) {
await this.ajaxService.postWithData("/api/v1/..",
arg,
(data) => {
results[i] = this.checkStep(data);
},
() => {
});
}
return results; // This returns before for loop containing ajaxServces calls ends
})().then((results) => {
// Do some things here using `results`
});
postWithData
public request = (options: Options, successCallback: Function, errorCallback?: Function): void => {
var that = this;
$.ajax({
url: options.url,
type: options.method,
data: options.data,
cache: false,
success: function (d) {
successCallback(d);
},
error: function (d) {
if (errorCallback) {
errorCallback(d);
return;
}
}
});
}
I've checked SO answers, e.g. https://stackoverflow.com/a/40329190/3264998

your request wrapper is not great and unfortunately Aaron's isn't much better. I haven't used jQuery much in recent years, but I think it has Promise-based APIs now too. Either way you'll want something generic that basically gives you a "promisified" version of $.ajax so you can work with await -
const defaultOpts =
{ method: 'GET', cache: false }
const request = (opts = {}) =>
new Promise
( (resolve, reject) =>
$.ajax({ ...defaultOpts, ...opts })
.done((req, status, res) => resolve(res))
.fail((req, status, err) => reject(err))
)
You haven't provided a Minimum, Complete, Verifiable Example (MVCE) so it makes it difficult help you. We don't know about times or checkStep so some guesses were made about your intentions. Something like this might help get your wheels turning again -
const main = async () =>
{ const steps =
[ request ("/api/v1/something..")
, request ("/api/v1/another..")
, request ("/api/v2/andthis..")
]
const res =
await Promise.all(steps)
console.log(res)
// [ <result1>, <result2>, <result3> ]
// I don't know what your checkStep does
// but something like this might work
res.forEach(r => this.checkStep(r))
// otherwise
// const checked =
// res.map(r => this.checkStep(r))
// ...
return res
}
main()
.then(res => console.log("all results", res))
// [ <result1>, <result2>, <result3> ]
.catch (e => ...)
If one request is dependent on another, putting the requests in an array and then calling Promise.all is not a great fit. In the example below we query all the posts for a particular author. The fictional API we're querying asks for the author's username but we only know the author's id. In this case, we look up the author's username first, then we can query the posts by username -
const authorById = id =>
request({ url: '/authors', data: { id } })
const postsByAuthor = username =>
request ({ url: '/posts', data: { author: username } })
const main = () =>
{ const author =
await authorById (10)
// { id: 10, username: alice, email: alice#email.lol }
// this request depends on the result of the first request
const posts =
await postsByAuthor (author.username)
// [ { postId: 9, title: "hello world" }, { postId: 11, ... }, ... ]
// ...
}

Something like this should work
public request = (options: Options, successCallback: Function, errorCallback?: Function): Promise => {
return new Promise( (resolve, reject) => {
var that = this;
$.ajax({
url: options.url,
type: options.method,
data: options.data,
cache: false,
success: function (d) {
resolve(successCallback(d));
},
error: function (d) {
if (errorCallback) {
reject(errorCallback(d));
}
}
})};
}
Also, you specified a return type for your function, would you be using TypeScript? Not sure if Promise return type is valid. But the idea is that you need to return a promise.

Promise, Async Await

setDeviceTimeout = id => timeout => {
const {onSetDevices, devices} = this.props;
var newDeviceList = devices.map(device => {
if (device.id === id) {
var newDevice = {
//...device,
timeout: timeout
};
deviceTable.oncePostDevice(newDevice).then( data => {
return newDevice = data
});
}
return device;
});
onSetDevices(newDeviceList);
}
So the issue I am having here is that the onSetDevices(newDeviceList) get's called before the devices.map() is finished. This is because the devices.map() has the call to a server oncePostDevice(newDevice), then returns the data and stores it in the newDevice variable and puts that into the newDeviceList array.
Because this happens onSetDevices doesn't include the the newDevice in the newDeviceList array of objects and when I set my redux state using onSetDevices, nothing has changed.
I am wondering how I turn this into an async, await or use a promise alone to finish the task of making onSetDevices wait for the devices.map() to finish.
Also here is the code for oncePostDevice:
export const oncePostDevice = (device) => new Promise(function(resolve, reject) {
fetch('https://url/devices/'+device.id, {
method: 'PUT',
headers: {
"Accept": "application/json",
"Content-Type": "application/json"
},
body: JSON.stringify(device)
})
.then(response => response.json())
.then(
data => {return resolve(data)},
error => {return reject(error)}
)
.catch(err => console.error(this.props.url, err.toString()));
});
As you can see I already have a promise in here working and returning the data afterwards.
I just need to know how to make my setDeviceTimeout inner mapping function finish before I hit onSetDevices.

Here's how you could do it (explanations inline in code):
// make the function async
setDeviceTimeout = id => async timeout => {
const {onSetDevices, devices} = this.props;
// make a list of promises, not of devices
// note: mapping a value via an async function will create promises
const newDeviceListPromises = devices.map(async device => {
if (device.id === id) {
const newDevice = {
...device,
timeout: timeout
};
return await deviceTable.oncePostDevice(newDevice);
}
return device;
});
// wait for all promises to finish and what they return will be the devices
const newDeviceList = await Promise.all(newDeviceListPromises);
onSetDevices(newDeviceList);
};