I am looping through ids that sends it to an async function and I want to return both success and failed data, right now I am only returning success data
const successContractSignature: LpContractSla[] = [];
for (const id of lpContractSlaIds) {
const data = await createLpSignature(context, id);
if (data) {
successContractSignature.push(data);
}
}
return successContractSignature;
If the createLpSignature throws error, do i need a try catch here? but wouldnt that exit the loop?
How can I push failed data to separate array without breaking the loop?
Unless there's a specific reason to avoid the call in parallel, it's always a good practice to avoid to start async calls in a for loop with await, since you are going to wait for each promise to resolve (or reject ) before executing the next one.
This is a better pattern which lets you also get all the results of the promises, either they resolved or rejected:
const successContractSignature: LpContractSla[] = await Promise.allSettled(lpContractSlaIds.map((id: string) => createLpSignature(context,id)))
return successContractSignature;
But if for some particular reason you need to make these calls in a sequence and not in parallel, you can wrap the single call in a try catch block ,that won't exit the loop:
for (const id of lpContractSlaIds) {
let data;
try {
data = await createLpSignature(context, id);
} catch(e) {
data = e
}
if (data) {
successContractSignature.push(data);
}
}
You can test it in this example:
const service = (id) =>
new Promise((res, rej) =>
setTimeout(
() => (id %2 === 0 ? res("ID: "+id) : rej('Error ID : '+id)),
1000
)
);
const ids = [1,2,3,4,5]
const testParallelService = async () => {
try {
const data = await Promise.allSettled(ids.map(id => service(id)))
return data.map(o => `${o.status}: ${o.reason ?? o.value}`)
} catch(e) {
console.log(e)
}
}
testParallelService().then(data => console.log("Parallel data: ", data))
const testSequentialService = async () => {
const res = [];
for (const id of ids) {
let data;
try {
data = await service(id);
} catch (e) {
data = e;
}
if (data) {
res.push(data);
}
}
return res;
};
testSequentialService().then((data) => console.log('Sequential Data: ', data));
Related
I have this test I made just to check an API, but then i tryied to add an URL from a second fetch using as parameter a value obtained in the first fetch and then return a value to add in the first fecth. The idea is to add the image URL to the link. thanks in advance.
function script() {
const url = 'https://pokeapi.co/api/v2/pokemon/?offset=20&limit=20'
const result = fetch(url)
.then( (res)=>{
if(res.ok) {
return res.json()
} else {
console.log("Error!!")
}
}).then( data => {
console.log(data)
const main = document.getElementById('main');
main.innerHTML=`<p><a href='${data.next}'>Next</a></p>`;
for(let i=0; i<data.results.length;i++){
main.innerHTML=main.innerHTML+`<p><a href=${getImageURL(data.results[i].url)}>${data.results[i].name}</a></p>`;
}
})
}
async function getImageURL(imgUrl) {
const resultImg = await fetch(imgUrl)
.then( (res)=> {
return res.json()
})
.then (data => {
console.log(data.sprites.other.dream_world.front_default);
})
return resultImg.sprites.other.dream_world.front_default;
}
In general, don't mix .then/.catch handlers with async/await. There's usually no need, and it can trip you up like this.
The problem is that your fulfillment handler (the .then callback) doesn't return anything, so the promise it creates is fulfilled with undefined.
You could return data, but really just don't use .then/.catch at all:
async function getImageURL(imgUrl) {
const res = await fetch(imgUrl);
if (!res.ok) {
throw new Error(`HTTP error ${res.status}`);
}
const resultImg = await res.json();
return resultImg.sprites.other.dream_world.front_default;
}
[Note I added a check of res.ok. This is (IMHO) a footgun in the fetch API, it doesn't reject its promise on HTTP errors (like 404 or 500), only on network errors. You have to check explicitly for HTTP errors. (I wrote it up on my anemic old blog here.)]
There's also a problem where you use getImageURL:
// Incorrent
for (let i = 0; i < data.results.length; i++) {
main.innerHTML=main.innerHTML+`<p><a href=${getImageURL(data.results[i].url)}>${data.results[i].name}</a></p>`;
}
The problen here is that getImageURL, like all async functions, returns a promise. You're trying to use it as those it returned the fulfillment value you're expecting, but it can't — it doesn't have that value yet.
Instead, you need to wait for the promise(s) youre creating in that loop to be fulfilled. Since that loop is in synchronous code (not an async function), we'd go back to .then/.catch, and since we want to wait for a group of things to finish that can be done in parallel, we'd do that with Promise.all:
// ...
const main = document.getElementById('main');
const html = `<p><a href='${data.next}'>Next</a></p>`;
Promise.all(data.results.map(async ({url, name}) => {
const realUrl = await getImageURL(url);
return `<p><a href=${realUrl}>${name}</a></p>`;
}))
.then(paragraphs => {
html += paragraphs.join("");
main.innerHTML = html;
})
.catch(error => {
// ...handle/report error...
});
For one, your
.then (data => {
console.log(//...
at the end of the promise chain returns undefined. Just remove it, and if you want to console.log it, do console.log(resultImg) in the next statement/next line, after await.
This the final version that accomplish my goal. Just want to leave this just in case someone finds it usefull. Thanks for those who answer!
function script() {
const url = 'https://pokeapi.co/api/v2/pokemon/?offset=20&limit=20'
const result = fetch(url)
.then( (res)=>{
if(res.ok) {
return res.json()
} else {
console.log("Error!!")
}
}).then( data => {
console.log(data)
const main = document.getElementById('main');
main.innerHTML=`<p><a href='${data.next}'>Proxima Página</a></p>`;
Promise.all(data.results.map(async ({url, name}) => {
const realUrl = await getImageURL(url);
return `<div><a href=${realUrl}>${name}</a></div>`;
}))
.then(paragraphs => {
main.innerHTML=main.innerHTML+paragraphs;
})
.catch(error => {
console.log(error);
});
})
}
async function getImageURL(imgUrl) {
const res = await fetch(imgUrl);
if(!res.ok) {
throw new Error(`HTTP Error ${res.status}`)
}
const resultImg = await res.json();
return resultImg.sprites.other.dream_world.front_default
}
This loop is going to run an arbitrary amount of times, and I want to get the result out of it after them all. Anything I try (promisifying, async/await, nesting functions, et al) seems to be a dead end. I don't get why I cannot just stick a .then on the API call, or on the function I made here. But I suspect the problem is more fundamental with my understanding, because I can't seem to even get just the "data" to return...same inability to wait on the API call. Wrapping it in a promise loses the "data" and pulling it with the "for loop" inside there doesn't work either. This is making me question my entire progress with JS/implementing other people's APIs.
const gotPeeps = () => {
challongeClient.tournaments.show({
id: tournamentURL,
callback: (err, data) => {
//return data //doesnt return "data" from gotPeeps?
for (const [key, value] of Object.entries(data.tournament.matches)) {
if (value.match.state === 'open') {
peepsList.push(value.match.player1Id, value.match.player2Id)
console.log(peepsList)
}}}})}
gotPeeps()
EDIT
To the comments:
I'm trying to get the results after the for loop is complete.
The "loop" I was referring to is the "for of" over the data Object.
"Putting the code after the loop but inside the callback" does not work. I have a previous question in my week of failing to solve this:
How to solve race condition in Javascript?
Here is the whole thing, with some past versions commented out.:
const tournamentModel = require('../models/tournamentSchema')
require('dotenv').config()
const challonge = require('challonge');
module.exports = {
name: 'getmatches',
aliases: ['gm'],
cooldown: 0,
description: 'Get Challonge data into console.',
execute(message, args, cmd, client, Discord, profileData) {
let peep = ''
let peepsList = ''
const tournamentURL = 'TESTING_Just_Sign_Up_If_You_See_This850587786533011496'
const challongeClient = challonge.createClient({
apiKey: process.env.CHALLONGE_API_KEY,
})
const getPlayer = (playerXId) => {
return new Promise((resolve, reject) => {
challongeClient.participants.show({
id: tournamentURL,
participantId: playerXId,
callback: (err, data) => {
if (err) {
reject(err);
return;
}
peep = data.participant.misc
peepsList.push(peep)
console.log(peepsList)
console.log('RUNNING GET PLAYER', playerXId, playerIndexCount)
resolve(peepsList);
}
});
});
}
const peepsList = []
const matchList = []
const gotPeeps = () => {
challongeClient.tournaments.show({
id: tournamentURL,
include_participants: 1,
include_matches: 1,
callback: (err, data) => {
for (const [key, value] of Object.entries(data.tournament.matches)) {
if (value.match.state === 'open') {
peepsList.push(value.match.player1Id, value.match.player2Id)
console.log(peepsList)
}
}
}
/*// GET PLAYERS
getPlayer(value.match.player1Id)
.then(() => {
getPlayer(value.match.player2Id)
})
}
*/
}
)
}
gotPeeps()
}}
You can make this function return a promise and await the function (as long as your function is an async function)
const gotPeeps = () => {
return new Promise((resolve, reject) => {
const peepsList = []; // declare the empty array to e filled
challongeClient.tournaments.show({
id: tournamentURL,
callback: (err, data) => {
for (const [key, value] of Object.entries(data.tournament.matches)) {
if (value.match.state === "open") {
peepsList.push(value.match.player1Id, value.match.player2Id);
}
}
resolve(peepsList); // resolve the promise with the filled array
// TODO: handle reject
},
});
})
};
(async () => {
try {
const result = await gotPeeps();
} catch (error) {
// TODO: handle error
}
})();
I'm trying to only make one http call at time but when I log the response from getUrl they are piling up and I start to get 409s (Too many requests)
function getUrl(url, i, cb) {
const fetchUrl = `https://api.scraperapi.com?api_key=xxx&url=${url.url}`;
fetch(fetchUrl).then(async res => {
console.log(fetchUrl, 'fetched!');
if (!res.ok) {
const err = await res.text();
throw err.message || res.statusText;
}
url.data = await res.text();
cb(url);
});
}
let requests = urls.map((url, i) => {
return new Promise(resolve => {
getUrl(url, i, resolve);
});
});
const all = await requests.reduce((promiseChain, currentTask) => {
return promiseChain.then(chainResults =>
currentTask.then(currentResult => [...chainResults, currentResult]),
);
}, Promise.resolve([]));
Basically I don't want the next http to start until the previous one has finished. Otherwise I hammer their server.
BONUS POINTS: Make this work with 5 at a time in parallel.
Since you're using await, it would be a lot easier to use that everywhere instead of using confusing .thens with reduce. It'd also be good to avoid the explicit Promise construction antipattern. This should do what you want:
const results = [];
for (const url of urls) {
const response = await fetch(url);
if (!response.ok) {
throw new Error(response); // or whatever logic you need with errors
}
results.push(await response.text());
}
Then your results variable will contain an array of response texts (or an error will have been thrown, and the code won't reach the bottom).
The syntax for an async function is an async keyword before the argument list, just like you're doing in your original code:
const fn = async () => {
const results = [];
for (const url of urls) {
const response = await fetch(url);
if (!response.ok) {
throw new Error(response); // or whatever logic you need with errors
}
results.push(await response.text());
}
// do something with results
};
To have a limited number of requests at a time, make a queue system - when a request completes, recursively call a function that makes another request, something like:
const results = [];
const queueNext = async () => {
if (!urls.length) return;
const url = urls.shift();
const response = await fetch(url);
if (!response.ok) {
throw new Error(response); // or whatever logic you need with errors
}
results.push(await response.text());
await queueNext();
}
await Promise.all(Array.from({ length: 5 }, queueNext));
// do something with results
You cannot use Array methods to sequentually run async operations because array methods are all synchronous.
The easiest way to achieve sequential async tasks is through a loop. Otherwise, you will need to write a custom function to imitate a loop and run .then after a async task ends, which is quite troublesome and unnecessary.
Also, fetch is already returning a Promise, so you don't have to create a Promise yourself to contain that promise returned by fetch.
The code below is a working example, with small changes to your original code (see comments).
// Fake urls for example purpose
const urls = [{ url: 'abc' }, { url: 'def', }, { url: 'ghi' }];
// To imitate actual fetching
const fetch = (url) => new Promise(resolve => {
setTimeout(() => {
resolve({
ok: true,
text: () => new Promise(res => setTimeout(() => res(url), 500))
});
}, 1000);
});
function getUrl(url, i, cb) {
const fetchUrl = `https://api.scraperapi.com?api_key=xxx&url=${url.url}`;
return fetch(fetchUrl).then(async res => { // <-- changes here
console.log(fetchUrl, 'fetched!');
if (!res.ok) {
const err = await res.text();
throw err.message || res.statusText;
}
url.data = await res.text();
return url; // <--- changes here
});
}
async function getAllUrls(urls){
const result = [];
for (const url of urls){
const response = await getUrl(url);
result.push(response);
}
return result;
}
getAllUrls(urls)
.then(console.log);
async/await is perfect for this.
Assuming you have an array of URLs as strings:
let urls = ["https://example.org/", "https://google.com/", "https://stackoverflow.com/"];
You simply need to do:
for (let u of urls) {
await fetch(u).then(res => {
// Handle response
}).catch(e => {
// Handle error
});
}
The loop will not iterate until the current fetch() has resolved, which will serialise things.
The reason array.map doesn't work is as follows:
async function doFetch(url) {
return await fetch(url).then(res => {
// Handle response
}).catch(e => {
// Handle error
});
}
let mapped = urls.map(doFetch);
is equivalent to:
let mapped;
for (u of urls) {
mapped.push(doFetch(u));
}
This will populate mapped with a bunch of Promises immediately, which is not what you want. The following is what you want:
let mapped;
for (u of urls) {
mapped.push(await doFetch(u));
}
But this is not what array.map() does. Therefore using an explicit for loop is necessary.
Many people provided answers using for loop. But in some situation await in for loop is not welcome, for example, if you are using Airbnb style guide.
Here is a solution using recursion.
// Fake urls for example purpose
const urls = [{ url: 'abc' }, { url: 'def', }, { url: 'ghi' }];
async function serialFetch(urls) {
return await doSerialRecursion(
async (url) => {
return result = await fetch(url)
.then((response) => {
// handle response
})
.catch((err) => {
// handle error
});
},
urls,
0
);
}
async function doSerialRecursion(fn, array, startIndex) {
if (!array[startIndex]) return [];
const currResult = await fn(array[startIndex]);
return [currResult, ...(await doSerialRecursion(array, fn, startIndex + 1))];
}
const yourResult = await serialFetch(urls);
The doSerialRecursion function will serially execute the function you passed in, which is fetch(url) in this example.
I'm using async await inside of for loop as below.
for (let i = 0; i < result.length; i += 1) {
try{
const client = await axios.get(
`${process.env.user}/client/${result[i].id}`
);
} catch(error){
console.log(error)
}
if (client.data.success === true) {
result[i].Name = rider.data.client.Name;
result[i].PhoneNumber = rider.data.client.Number;
}
}
But I want to make this using 'new Promise' and 'promiss.all' to make it asynclously.
But I don'k know how to make this correctly doing error handle well.
Could you recommend some advice for this? Thank you for reading it.
This can be a basic solution, i think
let playList = []
for (let i = 0; i < result.length; i += 1) {
playList.push(axios.get(
`${process.env.user}/client/${result[i].id}`
).then(res => {
if (res.data.success === true) {
result[i].Name = rider.data.client.Name;
result[i].PhoneNumber = rider.data.client.Number;
}
}).catch(ex => console.log(ex)));
}
await Promise.all(playList)
This can also be done by using a foreach loop.
The for/foreach loop can be simplified by using a map function.
Js map function is equivalent of c# select linq function.
The fat arrow in js map function is not bound to return a value unlike c# select inner function which must return a value.
await Promise.all(result.map(async r => {
let client;
try {
client = await axios.get(`${process.env.user}/client/${r.id}`);
} catch (error) {
console.log(error)
}
if (client.data.success === true) {
r.Name = rider.data.client.Name;
r.PhoneNumber = rider.data.client.Number;
}
}));
Try this
var promises = result.map(r => axios.get(`${process.env.user}/client/${r.id}`);
Promise.all(promises).then(function(values) {
console.log('All promises done');
});
The idea is that if you are awaiting something, that is promise, you can await it, or call it to get promise
Example:
function Foo()
{
return new Promise(...); // Promise of int for example
}
you can do
var p = Foo(); //you will get promise
Or
var v = await Foo(); // you will get int value when promise resolved
This is how you do it with async/await + Promise.all:
const myResult = await Promise.all(result.map(({ id }) => {
return axios.get(`${process.env.user}/client/${id}`);
}));
// deal with the result of those requests
const parsed = myResult.map(data => /* your code here */);
Here is an example using Array.map to call your function along with Promise.all. I wrapped the axios request in a function so if one of your request fails, it wont stop every other requests. If you don't mind stopping when you got an issue, look at others answers to your question.
function fakeRequest() {
return new Promise((resolve, reject) => {
setTimeout(() => {
resolve({
data: {
success: true,
client: {
Name: 'Todd',
Number: 5,
},
},
});
}, 300);
});
}
(async() => {
const result = [{}, {}, {}];
await Promise.all(result.map(async(x, xi) => {
try {
const client = await fakeRequest();
if (client.data.success === true) {
result[xi].Name = client.data.client.Name;
result[xi].PhoneNumber = client.data.client.Number;
}
} catch (err) {
console.log(err)
}
}));
console.log(result);
})();
I'm new to node.js and javascript in general but I am having issues understanding why the writeFile function is writing a blank file. I think the for loop should be a Promise but I am not sure how to write it.
const removeProviders = function () {
readline.question('Where is the file located? ', function(filePath) {
let providerArray = fs.readFileSync(filePath).toString().split('\r\n');
console.log(providerArray);
let importFile = '';
for (let providerId in providerArray) {
getProvider(providerArray[providerId]).then((response) => {
let providerInfo = response.data;
return providerInfo;
}).then((providerInfo) => {
let entry = createImportEntry(providerInfo);
importFile += "entry";
})
}
fs.writeFile('C:/path/to/file.txt', importFile);
You can collect all the promises from the for-loop into an Array and then pass them into Promise.all() which will resolve only after all of them resolved. If one of the promises are rejected, it will reject as well:
const promises = providerArray.map((item) => {
return getProvider(item)
.then((response) => {
let providerInfo = response.data;
return providerInfo;
})
.then((providerInfo) => {
let entry = createImportEntry(providerInfo);
importFile += "entry";
});
});
Promise.all(promises).then(() => {
fs.writeFile('C:/path/to/file.txt', importFile, err => {
if (err) {
console.error(err);
}
});
});
While doing this you could also get rid of the importFile variable and collect directly the results of your promises. Promise.all().then(results => {}) will then give you an array of all results. Thus no need for an updatable variable.
The below approach may be useful.
I don't know your getProvider return Promise. you can set promise in getProvider method
const getProviderValue = async function(providerArray) {
return new Promise((resolve, reject) {
let importFile = '';
for (let providerId in providerArray) {
await getProvider(providerArray[providerId]).then((response) => {
let providerInfo = response.data;
return providerInfo;
}).then((providerInfo) => {
let entry = createImportEntry(providerInfo);
importFile += "entry";
})
}
resolve(importFile)
})
}
const removeProviders = async function () {
readline.question('Where is the file located? ', function(filePath) {
let providerArray = fs.readFileSync(filePath).toString().split('\r\n');
console.log(providerArray);
let importFile = await getProviderValue(providerArray)
fs.writeFile('C:/path/to/file.txt', importFile);
})
}
Your for loop does not wait for the promises to resolve. A better way to approach this problem would be to use reduce.
providerArray.reduce(
(p, _, i) => {
p.then(_ => new Promise(resolve =>
getProvider(providerArray[providerId]).then((response) => {
let providerInfo = response.data;
return providerInfo;
}).then((providerInfo) => {
let entry = createImportEntry(providerInfo);
importFile += entry;
resolve();
}))
);
}
, Promise.resolve() );
You can also use Promise.all but the out data may not be in the order that you expect if you append to the importFile variable.