Using fetch API and async/await, is it possible to continue polling indefinitely, regardless of availability of a URL? I anticipate that a URL might become available eventually, so I want to keep trying until a condition is met. Tried to come up with a minimum viable code sample and I'm not sure I pulled it off:
// this is just a placeholder. It will eventually be a function
// that evaluates something real.
// Assume validContinue gets updated elsewhere.
function shouldContinue() {
return validContinue;
}
async function wonderPoll(someUrl) {
// just a delay mechanism
function wait(ms = 1000) {
return new Promise(resolve => {
setTimeout(resolve, ms);
});
}
// the actual individual poll
async function pollingFunction(url) {
const response = await fetch(url, {
cache: 'no-store'
});
if (response.ok) {
return response;
} else {
Promise.reject(response);
}
}
// allegedly keep polling until condition is met.
// But the rejected Promise is breaking out!
while (shouldContinue()) {
await wait();
result = await pollingFunction(someUrl);
}
// when the fetch hits a rejected state, we never get here!
console.log('done with the while loop, returning last successful result')
return result;
}
const sampleUrl = 'https://get.geojs.io/v1/ip/country.json?ip=8.8.8.8';
const sampleUrl2 = 'http://totallybroken_fo_sho';
// swap the URL to test
wonderPoll(sampleUrl)
.then((result) => {
console.log('got a result', result)
})
.catch((err) => {
console.log('got an error', err)
});
I see what's happening (I think). The parent call ultimately executes the polling function, which rejects on the Promise. The condition to continue is still theoretically met, but the rejection breaks out of the While loop and sends to rejection directly up. This propagates all the way up to the catch method of the original/initial Promise. It doesn't even hit any code that would have come after the While loop in the case of resolved Promises.
What I don't know is how to prevent that from happening. I think I don't understand the syntax for intercepting and resolving the promise. When I replace Promise.reject in the response parser with Promise.resolve(response), it still ends up rejecting up to the top.
If the URL I provide is valid, it will continue until the condition is no longer met.
Here's a fiddle: https://jsfiddle.net/gregpettit/qf495bjm/5/
To use the fiddle, the "stop" button simulates the condition being met, and I've provided two different URLs that have to be manually swapped (by passing someUrl or someUrl2) to test.
Expected results:
with good URL, continuous polling (will have to dig into network in dev tools) until condition is met (by pressing Stop!) and then the calling function's 'then' can show the result.
with bad URL, continuous polling until condition is met, and then calling function's 'catch' shows the error
Actual results:
positive test case is OK
negative test case goes directly to the catch
You can try…catch it to prevent breaking out of loop.
while (shouldContinue()) {
try {
await wait();
result = await pollingFunction(someUrl);
} catch (e) {}
}
Change the code in while loop to try/catch so you can catch the error
result can hold a value when there's no error, or a reason when there is an error
Once the loop is stopped, you either return the value, or throw with the reason
As below
async function wonderPoll(someUrl) {
// just a delay mechanism
function wait(ms = 1000) {
return new Promise(resolve => {
setTimeout(resolve, ms);
});
}
// the actual individual poll
async function pollingFunction(url) {
const response = await fetch(url, {
cache: 'no-store'
});
if (response.ok) {
return response;
} else {
Promise.reject(response);
}
}
// allegedly keep polling until condition is met. But the rejected Promise is breaking out!
while (shouldContinue()) {
try {
await wait();
const value = await pollingFunction(someUrl);
result = {value};
} catch (reason) {
result = {reason};
}
}
// when the fetch hits a rejected state, we never get here!
console.log('done with the while loop, returning last successful result')
if (result.reason) {
throw result.reason;
}
return result.value;
}
Running example https://jsfiddle.net/twkbo9pg/
the example includes status in the result, but that is unnecessary (I borrowed code from my Promise.allSettled polyfill and forgot to remove that property)
you might want to check out observable streams! If you're going to have a lot of data coming in over time, that's rxjs's whole thing.
There's actually a few ways to do this if this feels janky (it kinda does haha).
import { ajax } from "rxjs/ajax";
import { duration } from "moment-timezone"; // I copied this from some old code... whatever.
import { catchError, map, share, switchMap } from "rxjs/operators";
const baseUrl = "http://foo.bar"
const base = (method, headers = {}) => ({
method,
headers: {
Accept: "application/json",
...headers,
},
crossDomain: true,
withCredentials: true,
})
const ajaxGet = url => ajax({ ...base("GET"), url })
export const userEpic = timer(0, duration(5, "minutes").asMilliseconds()).pipe(
switchMap(() =>
ajaxGet(`${baseUrl}/users`).pipe(
map(({ response }) => getUsersSuccess(response)),
catchError(e => of(getUsersError(e))),
)
),
share()
)
Two things
} else {
Promise.reject(response);
}
should return that. It's working "by accident" right now.
} else {
return Promise.reject(response);
}
Secondly, result = await pollingFunction(someUrl); might want to add .catch to it:
result = await pollingFunction(someUrl).catch(_=>null); or whatever can be tested for in the enclosing while
But I think you can simplify the whole thing thus:
export async function wonderPoll(someUrl) {
while (shouldContinue()) {
await wait();
const response = await fetch(someUrl, { cache: 'no-store' });
if (response.ok)
return response;
}
return Promise.reject(); // only if !shouldContinue()
}
Related
i have an array of ids, and i call api like this, i'm not sure if it's correct, what if on some id it'll fail?
const myFunc = async () => {
try {
for (const todoId of todoIds) {
await fetch(
`https://abcd123poiuy.mockapi.io/users/${
user!.id
}/todos/${todoId}`,
{
method: 'PUT',
headers: {
Accept: 'application/json',
'Content-Type': 'application/json',
},
body: JSON.stringify({complete: completeAll}),
},
);
}
setLoading(false);
} catch (clearCompletedError) {
setError(clearCompletedError);
setLoading(false);
}
}
should while loop or maybe recursion is good? pls help
I'd push all the promises in an array and do Promise.all since you want it to fail if one fails.
const promises = []
for (const todoId of todoIds) {
promises.push(fetch(`someUrlwith${todoId}`));
}
Promise.all(promises).then(() => {
// success callback
}).catch(() => {
// at least one of them failed
})
Another solution if you want to call them one at a time
function loopFetch(i) {
if (i >= todoIds.length) return; // stop when reached end
return fetch(`someUrl${todoIds[index]}`).then(() => {
return loopFetch(++i);
});
}
loopFetch(0);
Promise chaining is a concept that executes sequence of asynchronous tasks sequentially. The developer has control over the each Promise block and take action on next sequence of Promise execution.
Parallel Promise - (or) - Parallel Microtasks:
With Promise.all, Promise.race and Promise.allSettled you gain control over the settlement of all the triggered promise.
Advantage: Parallely trigger all asynchronous tasks and save time.
Remember : Promise.all will return only the first rejection despite there may be more than one rejection in your AJAX calls & Promise.race will return result of either rejected or resolved promise which ever happens sooner.
Realtime UseCase: (No chaining) using Promise.allSettled
In real world its prudent that all fetch calls will be fired at once and resolved / rejected status is know in future. In this case it makes sense to know what all fetch calls are passed or failed and take corrective actions to only that require status attention. This way save network round trips!
Example below for Promise.allSettled & Promise.all to show the difference.(Read code comments)
const dummy = [{ //Dummy data
value: 1,
delay: 100,
reject: true, //flag to explicitly reject this data
},{
value: 2,
delay: 200
},
{
value: 3,
delay: 300,
},
{
value: 4,
delay: 400,
reject:true
}];
const fakeFetch = function(req){ // Fake fetch call
return new Promise(function(resolve, reject) {
console.log("API:", req.value);
setTimeout(req.reject ? reject : resolve, req.delay, req.value);
})
}
const promiserSettled = async ()=>{
let collate = [];
for(let req of dummy){
collate.push(fakeFetch(req));
}
try{ // Promise.allSettle code block
const everthing = await Promise.allSettled(collate).then((values)=>{
throw values.filter((val) => val.status === "rejected");
})
console.log("Are we done: ", everthing);
}catch(e){
console.log("Promise.allSettled - Rejected Value:", e); // Logs all rejected data
}
try{ // Promise.all code block
const everthing = await Promise.all(collate);
console.log("Are we done: ", everthing);
}catch(e){
console.log("Promise.all - Rejected Value:", e); // Logs only one rejected data
}
};
promiserSettled(); // Execute main
I have some difficulties with a nested promise (below).
For now I'm using an async function in the catch to trigger authentication Errors.
But is it really a good way and isn't there a better way ?
The function have to send a POST request. If an authentication Error is thrown then login(), else throw the error.
If login() is fulfilled : retry the POST (and then return the results), else throw the error;
function getSomeData() {
return post('mySpecialMethod');
}
function login() {
const params = { /* some params */ }
return post('myLoginMethod', params).then(result => {
/* some processing */
return { success: true };
});
}
const loginError = [1, 101];
function post(method, params, isRetry) {
const options = /* hidden for brevity */;
return request(options)
// login and retry if authentication Error || throw the error
.catch(async ex => {
const err = ex.error.error || ex.error
if (loginError.includes(err.code) && !isRetry) { // prevent infinite loop if it's impossible to login -> don't retry if already retried
await login();
return post(method, params, true)
} else {
throw err;
}
})
// return result if no error
.then(data => {
// data from 'return request(options)' or 'return post(method, params, true)'
return data.result
});
}
Use
getSomeData.then(data => { /* do something with data */});
I'd suggest that for complex logic at least you use the async/await syntax.
Of course .then() etc is perfectly valid, however you will find the nesting of callbacks awkward to deal with.
My rule (like a lot of things in programming) is use context to make your decision. .then() works nicely when you're dealing with a limited number of promises. This starts to get awkward when you're dealing with more complex logic.
Using async / await for more involved logic allows you to structure your code more like synchronous code, so it's more intuitive and readable.
An example of two approaches is shown below (with the same essential goal). The async / await version is the more readable I believe.
Async / await also makes looping over asynchronous tasks easy, you can use a for loop or a for ... of loop with await and the tasks will be performed in sequence.
function asyncOperation(result) {
return new Promise(resolve => setTimeout(resolve, 1000, result));
}
async function testAsyncOperationsAwait() {
const result1 = await asyncOperation("Some result 1");
console.log("testAsyncOperationsAwait: Result1:", result1);
const result2 = await asyncOperation("Some result 2");
console.log("testAsyncOperationsAwait: Result2:", result2);
const result3 = await asyncOperation("Some result 3");
console.log("testAsyncOperationsAwait: Result3:", result3);
}
function testAsyncOperationsThen() {
return asyncOperation("testAsyncOperationsThen: Some result 1").then(result1 => {
console.log("testAsyncOperationsThen: Result1:", result1);
return asyncOperation("testAsyncOperationsThen: Some result 2").then(result2 => {
console.log("testAsyncOperationsThen: Result2:", result2);
return asyncOperation("testAsyncOperationsThen: Some result 3").then(result3 => {
console.log("testAsyncOperationsThen: Result3:", result3);
})
})
})
}
async function test() {
await testAsyncOperationsThen();
await testAsyncOperationsAwait();
}
test();
... But is it really a good way and isn't there a better way ?
No it's not a good idea because it hurts code readability.
You're mixing 2 interchangeable concepts, Promise.then/catch and async/await. Mixing them together creates readability overhead in the form of mental context switching.
Anyone reading your code, including you, would need to continuously switch between thinking in terms of asynchronous flows(.then/.catch) vs synchronous flows (async/await).
Use one or the other, preferably the latter since it's more readable, mostly because of it's synchronous flow.
Although I don't agree with how you're handling logins, here's how I would rewrite your code to use async/await and try...catch for exception handling:
function getSomeData() {
return post('mySpecialMethod')
}
async function login() {
const params = { } // some params
await post('myLoginMethod', params)
return { success: true }
}
const loginError = [1, 101]
async function post(method, params, isRetry) {
const options = {} // some options
try {
const data = await request(options)
return data.result
} catch (ex) {
const err = ex.error.error || ex.error
if (err) throw err
if (loginError.includes(err.code) && !isRetry) {
await login()
return post(method, params, true)
}
throw err
}
}
I obviously cannot/didn't test the above.
Also worth exploring the libraries which provides retry functionalities.
something like https://www.npmjs.com/package/async-retry
Generally this is not a big problem. You can chain/encapsulate async calls like this.
When it comes to logic it depends on your needs. I think the login state of a user should be checked before calling any API methods that require authentication.
I can't understand why adding .take(1) at the end of my observable, triggers the result, and if I don't it keeps pending:
function generateToken(identifier){
return new Observable<string>((observer) => {
jwt.sign(identifier, 'devsecret', (err, token) => {
if (err) {
observer.error(err);
} else if (token) {
observer.next(token);
}
});
}).pipe( take(1));
}
Does anyone know why? Care to share the reason and whether this is a proper implementation? Mind that I'm not subscribing to this function anywhere else, but I keep piping the result.
here is where I call the method and return a response with a authorization header
public login(identifier): Observable<any> {
return generateToken(identifier).pipe(
catchError((err: Error) => of(err)),
map(token => {
return {'Authorization': token}
}));
}
and last but not least this function is converted in a promise and the response is returned as an http request
function async userLogin(identifier) {
return await login(identifier).toPromise();
}
Thanks for your time and patience
This explains your issue:
return await login(identifier).toPromise();
Promise resolves on Observable completion or rejects if it errors, so it works with take(1) because it takes the first Observable value and completes it.
You can also get the output if you complete it. And it looks a bit more appropriate:
} else if (token) {
observer.next(token);
observer.complete(); <---
}
take(1) makes sure the subscriber.complete() method is called right after the first item is emitted. BTW, this can be done directly by calling observer.complete() after the observer.next().
toPromise() will only resolve once the stream is completed, not on every emission.
Try
.pipe( () => take(1) );
Or
.pipe( take );
Same behavior happens in promises. The callback wants a function not a statement.
I'm quite a newbie in JavaScript and in Promises.
I'm trying to build an array of objects that I get from an API.
To do so, I've build two functions in a file MyFile.js.
The first one returns a promise when an axios promise is resolved. It's
function get_items (url) {
return new Promise((resolve, reject) => {
let options = {
baseURL: url,
method: 'get'
}
axios(options)
.then(response => {
resolve(response.data)
})
.catch(error => {
reject(error.stack)
})
})
}
The second one looks like this:
let output = []
let next_url = 'https://some_url.com/api/data'
async function get_data () {
try {
let promise = new Promise((resolve, reject) => {
if (next_url) {
get_items(next_url)
.then(response => {
output.push(...response.results)
if (response.next) {
next_url = response.next
console.log('NEXT_URL HERE', next_url)
get_data()
} else {
console.log('else')
next_url = false
get_data()
}
})
.catch(error => {
reject(error.stack)
})
} else {
console.log('before resolve')
resolve(output)
}
})
return await promise
} catch(e) {
console.log(e)
}
}
It's where I'm grinding my teeth.
What I think I understand of this function, is that:
it's returning the value of a promise (that's what I understand return await promise is doing)
it's a recursive function. So, if there is a next_url, the function continues on. But if there is not, it gets called one last time to go into the else part where it resolves the array output which contains the results (values not state) of all the promises. At least, when I execute it, and check for my sanity checks with the console.log I wrote, it works.
So, output is filled with data and that's great.
But, when I call this function from another file MyOtherFile.js, like this:
final_output = []
MyFile.get_data()
.then(result => {
console.log('getting data')
final_output.push(...result)
})
it never gets into the then part. And when I console.log MyFile.get_data(), it's a pending promise.
So, what I would like to do, is be able to make get_data() wait for all the promises result (without using Promise.all(), to have calls in serie, not in parallel, that would be great for performances, I guess?) and then be able to retrieve that response in the then part when calling this function from anywhere else.
Keep in mind that I'm really a newbie in promises and JavaScript in general (I'm more of a Python guy).
Let me know if my question isn't clear enough.
I've been scratching my head for two days now and it feels like I'm running in circle.
Thanks for being an awesome community!
This is a bit untested
const api_url = 'https://some_url.com/api/data';
get_data(api_url).then((results) => {
console.log(results);
}).catch((error) => {
// console.error(error);
});
function get_items (url) {
const options = {
baseURL: url,
method: 'get'
};
return axios(options).then((response) => response.data);
}
async function get_data(next_url) {
const output = [];
while (next_url) {
const { results, next } = await get_items(next_url);
output.push(...results);
next_url = next;
}
return output;
}
Basically it makes things a bit neater. I suggest to look at more examples with Promises and the advantage and when to ease await/async. One thing to keep in mind, if you return a Promise, it will follow the entire then chain, and it will always return a Promise with a value of the last then.. if that makes sense :)
There are a few problems. One is that you never resolve the initial Promise unless the else block is entered. Another is that you should return the recursive get_data call every time, so that it can be properly chained with the initial Promise. You may also consider avoiding the explicit promise construction antipattern - get_items already returns a Promise, so there's no need to construct another one (same for the inside of get_items, axios calls return Promises too).
You might consider a plain while loop, reassigning the next_url string until it's falsey:
function get_items (baseURL) {
const options = {
baseURL: url,
method: 'get'
}
// return the axios call, handle errors in the consumer instead:
return axios(options)
.then(res => res.data)
}
async function get_data() {
const output = []
let next_url = 'https://some_url.com/api/data'
try {
while (next_url) {
const response = await get_items(next_url);
output.push(...response.results)
next_url = response.next;
}
} catch (e) {
// handle errors *here*, perhaps
console.log(e)
}
return output;
}
Note that .catch will result in a Promise being converted from a rejected Promise to a resolved one - you don't want to .catch everywhere, because that will make it difficult for the caller to detect errors.
Another way of doing it is to not use async at all and just recursively return a promise:
const getItems = (url) =>
axios({
baseURL: url,
method: 'get',
}).then((response) => response.data);
const getData = (initialUrl) => {
const recur = (result, nextUrl) =>
!nextUrl
? Promise.resolve(result)
: getItems(nextUrl).then((data) =>
recur(result.concat([data.results]), data.next),
);
return recur([],initialUrl)
.catch(e=>Promise.reject(e.stack));//reject with error stack
};
As CertainPerformance noted; you don't need to catch at every level, if you want getData to reject with error.stack you only need to catch it once.
However; if you had 100 next urls and 99 of them were fine but only the last one failed would you like to reject in a way that keeps the results so far so you can try again?
If you do then the code could look something like this:
const getData = (initialUrl) => {
const recur = (result, nextUrl) =>
!nextUrl
? Promise.resolve(result)
: getItems(nextUrl)
.catch(e=>Promise.reject([e,result]))//reject with error and result so far
.then((data) =>
recur(result.concat([data.results]), data.next),
);
return recur([],initialUrl);//do not catch here, just let it reject with error and result
};
Take the following contrived example:
const housekeepingStuff = async function (data) {
const result = await notImportant(data);
result.more = 'yawn';
storeInDatabase(result);
};
const getStuff = async function () {
try {
const data = await getData();
data.extra = 'wow';
housekeepingStuff(data); // <---- don't want to await... but need to for error catching
return Promise.resolve(data);
} catch (err) {
return Promise.reject(err);
}
};
try {
const myData = await doSomeStuff();
res.send(myData);
} catch (err) {
console.log(err);
res.sendStatus(400);
}
I want to return the data from getStuff () ASAP without waiting for housekeepingStuff() but if I don't await that function then I have an uncaught error.
I could call housekeepingStuff() outside the getStuff() function, after getting and sending the data to whoever wants it:
try {
const myData = await doSomeStuff();
res.send(myData);
await housekeepingStuff(data); // <---- am awaiting but who cares because nothing follows
} catch (err) {
console.log(err);
res.sendStatus(400);
}
But that doesn't seem right because I don't want to have to remember to call housekeepingStuff() every time I call doSomeStuff()... it should ideally be handled "internally".
What is the correct approach here?
A promise (or async) function has 2 possible outcomes:
A successful outcome
An error outcome
To get either outcome, you must wait for it. You can't wait for 1 condition and not for the other, because the entire thing needs to execute so you can find out what the outcome was.
Otherwise you're really asking the javascript engine: Please predict for me if the function will fail, and if it does, await it.
The correct approach therefore is to just await it.
However, if you don't care about either successful or failed outcomes of this function, just call the function via another async function that eats all the errors:
async function doSomeStuffAndIgnoreError() {
try {
await doSomeStuff();
} catch (e) {
console.error(e);
}
}