I want to create a task that only builds the entire project if something changed. For that, I am comparing hashes (irrelevant to the question).
const buildIfChanged = async () => {
const hash = await getHash();
const newHash = await getNewHash();
if (hash !== newHash) {
console.log("START");
const task = series(build, cleanup)();
console.log("END", task);
}
};
In this example, task is undefined, so I cannot add a .on("end", ...) and resolve the promise after that. I also cannot await it.
The problem is, because I am not waiting for it to complete, the buildIfChanged task completes before build even has a chance to run.
Is there any way to do this with modern gulpfiles?
I have found a solution on how to do this in modern gulpfiles. The function call of parallel and series actually take a done function as a parameter.
So to solve this, you can do:
const buildIfChanged = async () => {
return new Promise(async (res, rej) => {
try {
const hash = await getHash();
const newHash = await getNewHash();
if (hash !== newHash) {
console.log("START");
const task = series(build, cleanup)(() => res()); // add the done function here
console.log("END", task);
}
} catch(e) {
rej(e);
}
})
};
This question already has answers here:
Any difference between await Promise.all() and multiple await?
(6 answers)
Closed 3 years ago.
I have 2 codes
1 code
async function foo() {
const result1 = asyncFunc1();
const result2 = asyncFunc2();
return [result1, result2];
}
2 code
async function foo() {
const [result1, result2] = await Promise.all([
asyncFunc1(),
asyncFunc2(),
]);
return [result1,result2];
}
question
Is there any difference between the two?
As the comments state, you did not await the function calls in the first snippet, so you will get promises as return values. However if you used await, one major difference is that in the first snippet, asyncFunc2 isn't executed until asyncFunc1 is resolved (assuming you use await). Whereas, in the second snippet, asyncFunc2 is executed right after asyncFunc1 regardless of whether it has resolved.
Promise.all will return the results in the order in which you pass the functions.
See docs for Promise.All and Await
As per the comments, there is essentially no difference in the two, other than the fact that you have not awaited the results from the two async functions, so you're gonna end up with just the promise objects. Async functions don't await scoped promises / async code automatically, they just allow you to use the await keyword, which is just stopping the execution until the underlying promise is resolved.
Here's an attempt to illustrate those differences.
What's important to note is that the returned values for the first function are not 1, 2 as would be expected.
const asyncFunc = (a) => {
return new Promise((resolve) => {
setTimeout(() => resolve(a), 1000);
})
}
const asyncFunc1 = () => asyncFunc(1);
const asyncFunc2 = () => asyncFunc(2);
async function foo() {
const result1 = asyncFunc1();
const result2 = asyncFunc2();
return JSON.stringify({ result1, result2 });
}
async function foo2() {
return [result1, result2] = await Promise.all([
asyncFunc1(),
asyncFunc2(),
]);
}
(async () =>{
const el = document.createElement('div');
el.appendChild(document.createTextNode(await foo()));
el.appendChild(document.createTextNode(await foo2()));
document.querySelector('body').appendChild(el);
})();
// {"result1":{},"result2":{}} 1,2
Here's a fiddle to play around with; jsfiddle.
Just find it out! Here is a snippet that shows all major differences:
const timer = ms => new Promise(res => setTimeout(res, ms));
async function one() {
console.log("1 one called");
await timer(2000);
console.log("1 one done");
}
async function two() {
console.log("2 two called");
await timer(1000);
console.log("2 two done");
// throw new Error; // Uncomment to see another interesting behaviour
}
(async function main() {
const startParallel = Date.now();
await Promise.all([one(), two()]);
console.log(`parallel done, took ${Date.now() - startParallel}`);
const startSerial = Date.now();
await one();
await two();
console.log(`serial done, took ${Date.now() - startSerial}`);
})();
I am trying to scrape some links with headless-chrome/puppeteer while scrolling down like this:
let interval
const linkScraper = async () => {
return new Promise(async (resolve,reject) => {
interval = setInterval(async () => {
const visiblePosts = await page.$$("div[class*='wrapper']")
const data = await handleVisiblePosts(visiblePosts)
allPosts = {...allPosts, ...data}
await scroll()
const stop = await areWeAtTheBottom()
if (stop) {
console.log('STOPPING')
clearInterval(interval)
resolve()
}
}, 100);
})
}
problem? clearInterval doesn't actually stop the interval. stopping gets printed multiple times.
I suspect it could also be because setinterval is async, which it needs to be in order to use await.
I can find the following possible reasons why your interval would not get stopped:
You are never getting to the stop condition.
You are overwriting the interval variable somehow so the actual interval you want to stop is no longer saved.
You are getting a rejected promise.
There does not appear to be any reason why the interval variable needs to be outside the linkScraper function and putting it inside the function will prevent it from getting overwritten in any way.
With this many await calls, it seems wise to add a try/catch to catch any rejected promises and stop the interval if there's an error.
If you see the STOPPING being logged, then you are apparently hitting the stop condition so it appears it would have to be an overwritten interval variable.
Here's a version that cannot overwrite the interval variable and makes a few other changes for code cleanliness:
const linkScraper = async () => {
return new Promise((resolve, reject) => {
const interval = setInterval(async () => {
try {
const visiblePosts = await page.$$("div[class*='wrapper']");
const data = await handleVisiblePosts(visiblePosts);
allPosts = { ...allPosts, ...data};
await scroll();
const stop = await areWeAtTheBottom();
if (stop) {
console.log('STOPPING');
clearInterval(interval);
resolve();
}
} catch(e) {
clearInterval(interval);
reject(e);
}
}, 100);
});
}
In cleaning up this code, I ran into a couple questions:
Do all four of your functions that you use await with actually return a promise?
And, where is allPosts declared?
EDIT: Just discovered another issue. setInterval() isn't aware of the await calls inside your function. Remember, the outer function doesn't actually block. It returns immediately as soon as you hit an await. That means that you can get another setInterval() callback while you were still processing the async operations of the first one. That will mess things up. Here's a way around that:
function delay(t) {
return new Promise(resolve => {
setTimeout(resolve, t);
});
}
const linkScraper = () => {
console.log("starting linkScraper");
async function run() {
const visiblePosts = await page.$$("div[class*='wrapper']");
const data = await handleVisiblePosts(visiblePosts);
allPosts = { ...allPosts, ...data};
await scroll();
const stop = await areWeAtTheBottom();
if (stop) {
console.log('STOPPING');
return "stop";
}
return "continue";
}
return run().then(result => {
if (result === "continue") {
return delay(100).then(run);
}
})
});
}
I accepted jfriend00's solution as it pointed me in the right direction, my slightly modified, final and working version looks like this:
const linkScraper = async () => {
return new Promise(async (resolve, reject) => {
const run = async () => {
console.log("running")
const visiblePosts = await page.$$("div[class*='wrapper']");
const data = await handleVisiblePosts(visiblePosts);
allPosts = {...allPosts, ...data};
await scroll();
const stop = await areWeAtTheBottom();
if (stop) {
console.log('STOPPING');
resolve()
} else {
await page.waitFor(100)
await run()
}
}
await run()
})
}
I have a hard time understanding how async and await works behind the scenes. I know we have promises which make our non blocking code by using the "then" function we can place all the work we need to do after the promise is resolved. and the work we want to do parallel to promise we just write it outside our then function. Hence the code becomes non blocking. However i don't understand how the async await makes non-blocking code.
async function myAsyncFunction() {
try {
let data = await myAPICall('https://jsonplaceholder.typicode.com/posts/1');
// It will not run this line until it resolves await.
let result = 2 + 2;
return data;
}catch (ex){
return ex;
}
}
See the above code. I cannot move forward until the API call is resolved. If it makes my code blocking code, how is it any better then promises? Or is there something I missed about async and await? Where do i put my code that is not dependent to the await call? so it can keep on working without await holding the execution?
I am adding a Promise code that i would like to replicate in an async await example.
function myPromiseAPI() {
myAPICall('https://jsonplaceholder.typicode.com/posts/1')
.then(function (data) {
// data
});
// runs parallel
let result = 2 + 2;
}
Just as its name implies, the await keyword will cause the function to "wait" until the corresponding promise resolves before executing the next line. The whole point of await is to provide a way to wait for an asynchronous operation to complete before continuing.
The difference between this and blocking code is that the world outside the function can continue executing while the function is waiting for the asynchronous operations to finish.
async and await are just syntactic sugar on top of promises. They allow you to write code that looks a lot like ordinary synchronous code even though it uses promises under the covers. If we translated your example there to something that explicitly worked with the promises, it would look something like:
function myAsyncFunction() {
return myAPICall('https://jsonplaceholder.typicode.com/posts/1')
.then(function (data) {
let result = 2 + 2;
return data;
})
.catch(function (ex) {
return ex;
});
}
As we can see here, the let result = 2 + 2; line is inside a .then() handler, which means it's not going to execute until myAPICall() has resolved. It's the same when you use await. await just abstracts away the .then() for you.
One thing to bear in mind (and I think the point you're looking for) is that you don't have to use await right away. If you wrote your function like this, then you could execute your let result = 2 + 2; line right away:
const timeout =
seconds => new Promise(res => setTimeout(res, seconds * 1000));
function myAPICall() {
// simulate 1 second wait time
return timeout(1).then(() => 'success');
}
async function myAsyncFunction() {
try {
console.log('starting');
// just starting the API call and storing the promise for now. not waiting yet
let dataP = myAPICall('https://jsonplaceholder.typicode.com/posts/1');
let result = 2 + 2;
// Executes right away
console.log('result', result);
// wait now
let data = await dataP;
// Executes after one second
console.log('data', data);
return data;
} catch (ex) {
return ex;
}
}
myAsyncFunction();
After some clarification, I can see that what you really wanted to know about is how to avoid having to wait for two async operations one by one and instead have them execute in parallel. Indeed, if you use one await after the other, the second won't start executing until the first has finished:
const timeout =
seconds => new Promise(res => setTimeout(res, seconds * 1000));
function myAPICall() {
// simulate 1 second wait time
return timeout(1).then(() => 'success');
}
async function myAsyncFunction() {
try {
console.log('starting');
let data1 = await myAPICall('https://jsonplaceholder.typicode.com/posts/1');
// logs after one second
console.log('data1', data1);
let data2 = await myAPICall('https://jsonplaceholder.typicode.com/posts/2');
// logs after one more second
console.log('data2', data2);
} catch (ex) {
return ex;
}
}
myAsyncFunction();
To avoid this, what you can do is start both async operations by executing them without awaiting them, assigning their promises to some variables. Then you can await both promises:
const timeout =
seconds => new Promise(res => setTimeout(res, seconds * 1000));
function myAPICall() {
// simulate 1 second wait time
return timeout(1).then(() => 'success');
}
async function myAsyncFunction() {
try {
console.log('starting');
// both lines execute right away
let dataP1 = myAPICall('https://jsonplaceholder.typicode.com/posts/1');
let dataP2 = myAPICall('https://jsonplaceholder.typicode.com/posts/2');
let data1 = await dataP1;
let data2 = await dataP2;
// logs after one second
console.log('data1', data1);
console.log('data2', data2);
} catch (ex) {
return ex;
}
}
myAsyncFunction();
One alternative way to do this is to use Promise.all() with some array decomposition:
const timeout =
seconds => new Promise(res => setTimeout(res, seconds * 1000));
function myAPICall() {
// simulate 1 second wait time
return timeout(1).then(() => 'success');
}
async function myAsyncFunction() {
try {
console.log('starting');
// both myAPICall invocations execute right away
const [data1, data2] = await Promise.all([
myAPICall('https://jsonplaceholder.typicode.com/posts/1'),
myAPICall('https://jsonplaceholder.typicode.com/posts/2'),
]);
// logs after one second
console.log('data1', data1);
console.log('data2', data2);
} catch (ex) {
return ex;
}
}
myAsyncFunction();
All four functions are called below in update return promises.
async function update() {
var urls = await getCdnUrls();
var metadata = await fetchMetaData(urls);
var content = await fetchContent(metadata);
await render(content);
return;
}
What if we want to abort the sequence from outside, at any given time?
For example, while fetchMetaData is being executed, we realize we no longer need to render the component and we want to cancel the remaining operations (fetchContent and render). Is there a way to abort/cancel these operations from outside the update function?
We could check against a condition after each await, but that seems like an inelegant solution, and even then we will have to wait for the current operation to finish.
The standard way to do this now is through AbortSignals
async function update({ signal } = {}) {
// pass these to methods to cancel them internally in turn
// this is implemented throughout Node.js and most of the web platform
try {
var urls = await getCdnUrls({ signal });
var metadata = await fetchMetaData(urls);
var content = await fetchContent(metadata);
await render(content);
} catch (e) {
if(e.name !== 'AbortError') throw e;
}
return;
}
// usage
const ac = new AbortController();
update({ signal: ac.signal });
ac.abort(); // cancel the update
OLD 2016 content below, beware dragons
I just gave a talk about this - this is a lovely topic but sadly you're not really going to like the solutions I'm going to propose as they're gateway-solutions.
What the spec does for you
Getting cancellation "just right" is actually very hard. People have been working on just that for a while and it was decided not to block async functions on it.
There are two proposals attempting to solve this in ECMAScript core:
Cancellation tokens - which adds cancellation tokens that aim to solve this issue.
Cancelable promise - which adds catch cancel (e) { syntax and throw.cancel syntax which aims to address this issue.
Both proposals changed substantially over the last week so I wouldn't count on either to arrive in the next year or so. The proposals are somewhat complimentary and are not at odds.
What you can do to solve this from your side
Cancellation tokens are easy to implement. Sadly the sort of cancellation you'd really want (aka "third state cancellation where cancellation is not an exception) is impossible with async functions at the moment since you don't control how they're run. You can do two things:
Use coroutines instead - bluebird ships with sound cancellation using generators and promises which you can use.
Implement tokens with abortive semantics - this is actually pretty easy so let's do it here
CancellationTokens
Well, a token signals cancellation:
class Token {
constructor(fn) {
this.isCancellationRequested = false;
this.onCancelled = []; // actions to execute when cancelled
this.onCancelled.push(() => this.isCancellationRequested = true);
// expose a promise to the outside
this.promise = new Promise(resolve => this.onCancelled.push(resolve));
// let the user add handlers
fn(f => this.onCancelled.push(f));
}
cancel() { this.onCancelled.forEach(x => x); }
}
This would let you do something like:
async function update(token) {
if(token.isCancellationRequested) return;
var urls = await getCdnUrls();
if(token.isCancellationRequested) return;
var metadata = await fetchMetaData(urls);
if(token.isCancellationRequested) return;
var content = await fetchContent(metadata);
if(token.isCancellationRequested) return;
await render(content);
return;
}
var token = new Token(); // don't ned any special handling here
update(token);
// ...
if(updateNotNeeded) token.cancel(); // will abort asynchronous actions
Which is a really ugly way that would work, optimally you'd want async functions to be aware of this but they're not (yet).
Optimally, all your interim functions would be aware and would throw on cancellation (again, only because we can't have third-state) which would look like:
async function update(token) {
var urls = await getCdnUrls(token);
var metadata = await fetchMetaData(urls, token);
var content = await fetchContent(metadata, token);
await render(content, token);
return;
}
Since each of our functions are cancellation aware, they can perform actual logical cancellation - getCdnUrls can abort the request and throw, fetchMetaData can abort the underlying request and throw and so on.
Here is how one might write getCdnUrl (note the singular) using the XMLHttpRequest API in browsers:
function getCdnUrl(url, token) {
var xhr = new XMLHttpRequest();
xhr.open("GET", url);
var p = new Promise((resolve, reject) => {
xhr.onload = () => resolve(xhr);
xhr.onerror = e => reject(new Error(e));
token.promise.then(x => {
try { xhr.abort(); } catch(e) {}; // ignore abort errors
reject(new Error("cancelled"));
});
});
xhr.send();
return p;
}
This is as close as we can get with async functions without coroutines. It's not very pretty but it's certainly usable.
Note that you'd want to avoid cancellations being treated as exceptions. This means that if your functions throw on cancellation you need to filter those errors on the global error handlers process.on("unhandledRejection", e => ... and such.
You can get what you want using Typescript + Bluebird + cancelable-awaiter.
Now that all evidence point to cancellation tokens not making it to ECMAScript, I think the best solution for cancellations is the bluebird implementation mentioned by #BenjaminGruenbaum, however, I find the usage of co-routines and generators a bit clumsy and uneasy on the eyes.
Since I'm using Typescript, which now support async/await syntax for es5 and es3 targets, I've created a simple module which replaces the default __awaiter helper with one that supports bluebird cancellations: https://www.npmjs.com/package/cancelable-awaiter
Unfortunately, there is no support of cancellable promises so far. There are some custom implementations e.g.
Extends/wraps a promise to be cancellable and resolvable
function promisify(promise) {
let _resolve, _reject
let wrap = new Promise(async (resolve, reject) => {
_resolve = resolve
_reject = reject
let result = await promise
resolve(result)
})
wrap.resolve = _resolve
wrap.reject = _reject
return wrap
}
Usage: Cancel promise and stop further execution immediately after it
async function test() {
// Create promise that should be resolved in 3 seconds
let promise = new Promise(resolve => setTimeout(() => resolve('our resolved value'), 3000))
// extend our promise to be cancellable
let cancellablePromise = promisify(promise)
// Cancel promise in 2 seconds.
// if you comment this line out, then promise will be resolved.
setTimeout(() => cancellablePromise.reject('error code'), 2000)
// wait promise to be resolved
let result = await cancellablePromise
// this line will never be executed!
console.log(result)
}
In this approach, a promise itself is executed till the end, but the caller code that awaits promise result can be 'cancelled'.
Unfortunately, no, you can't control execution flow of default async/await behaviour – it does not mean that the problem itself is impossible, it means that you need to do change your approach a bit.
First of all, your proposal about wrapping every async line in a check is a working solution, and if you have just couple places with such functionality, there is nothing wrong with it.
If you want to use this pattern pretty often, the best solution, probably, is to switch to generators: while not so widespread, they allow you to define each step's behaviour, and adding cancel is the easiest. Generators are pretty powerful, but, as I've mentioned, they require a runner function and not so straightforward as async/await.
Another approach is to create cancellable tokens pattern – you create an object, which will be filled a function which wants to implement this functionality:
async function updateUser(token) {
let cancelled = false;
// we don't reject, since we don't have access to
// the returned promise
// so we just don't call other functions, and reject
// in the end
token.cancel = () => {
cancelled = true;
};
const data = await wrapWithCancel(fetchData)();
const userData = await wrapWithCancel(updateUserData)(data);
const userAddress = await wrapWithCancel(updateUserAddress)(userData);
const marketingData = await wrapWithCancel(updateMarketingData)(userAddress);
// because we've wrapped all functions, in case of cancellations
// we'll just fall through to this point, without calling any of
// actual functions. We also can't reject by ourselves, since
// we don't have control over returned promise
if (cancelled) {
throw { reason: 'cancelled' };
}
return marketingData;
function wrapWithCancel(fn) {
return data => {
if (!cancelled) {
return fn(data);
}
}
}
}
const token = {};
const promise = updateUser(token);
// wait some time...
token.cancel(); // user will be updated any way
I've written articles, both on cancellation and generators:
promise cancellation
generators usage
To summarize – you have to do some additional work in order to support canncellation, and if you want to have it as a first class citizen in your application, you have to use generators.
Here is a simple exemple with a promise:
let resp = await new Promise(function(resolve, reject) {
// simulating time consuming process
setTimeout(() => resolve('Promise RESOLVED !'), 3000);
// hit a button to cancel the promise
$('#btn').click(() => resolve('Promise CANCELED !'));
});
Please see this codepen for a demo
Using CPromise (c-promise2 package) this can be easily done in the following way
(Demo):
import CPromise from "c-promise2";
async function getCdnUrls() {
console.log(`task1:start`);
await CPromise.delay(1000);
console.log(`task1:end`);
}
async function fetchMetaData() {
console.log(`task2:start`);
await CPromise.delay(1000);
console.log(`task2:end`);
}
function* fetchContent() {
// using generators is the recommended way to write asynchronous code with CPromise
console.log(`task3:start`);
yield CPromise.delay(1000);
console.log(`task3:end`);
}
function* render() {
console.log(`task4:start`);
yield CPromise.delay(1000);
console.log(`task4:end`);
}
const update = CPromise.promisify(function* () {
var urls = yield getCdnUrls();
var metadata = yield fetchMetaData(urls);
var content = yield* fetchContent(metadata);
yield* render(content);
return 123;
});
const promise = update().then(
(v) => console.log(`Done: ${v}`),
(e) => console.warn(`Fail: ${e}`)
);
setTimeout(() => promise.cancel(), 2500);
Console output:
task1:start
task1:end
task2:start
task2:end
task3:start
Fail: CanceledError: canceled
Just like in regular code you should throw an exception from the first function (or each of the next functions) and have a try block around the whole set of calls. No need to have extra if-elses. That's one of the nice bits about async/await, that you get to keep error handling the way we're used to from regular code.
Wrt cancelling the other operations there is no need to. They will actually not start until their expressions are encountered by the interpreter. So the second async call will only start after the first one finishes, without errors. Other tasks might get the chance to execute in the meantime, but for all intents and purposes, this section of code is serial and will execute in the desired order.
This answer I posted may help you to rewrite your function as:
async function update() {
var get_urls = comPromise.race([getCdnUrls()]);
var get_metadata = get_urls.then(urls=>fetchMetaData(urls));
var get_content = get_metadata.then(metadata=>fetchContent(metadata);
var render = get_content.then(content=>render(content));
await render;
return;
}
// this is the cancel command so that later steps will never proceed:
get_urls.abort();
But I am yet to implement the "class-preserving" then function so currently you have to wrap every part you want to be able to cancel with comPromise.race.
I created a library called #kaisukez/cancellation-token
The idea is to pass a CancellationToken to every async function, then wrap every promise in AsyncCheckpoint. So that when the token is cancelled, your async function will be cancelled in the next checkpoint.
This idea came from tc39/proposal-cancelable-promises
and conradreuter/cancellationtoken.
How to use my library
Refactor your code
// from this
async function yourFunction(param1, param2) {
const result1 = await someAsyncFunction1(param1)
const result2 = await someAsyncFunction2(param2)
return [result1, result2]
}
// to this
import { AsyncCheckpoint } from '#kaisukez/cancellation-token'
async function yourFunction(token, param1, param2) {
const result1 = await AsyncCheckpoint.after(token, () => someAsyncFunction1(param1))
const result2 = await AsyncCheckpoint.after(token, () => someAsyncFunction2(param2))
return [result1, result2]
}
Create a token then call your function with that token
import { CancellationToken, CancellationError } from '#kaisukez/cancellation-token'
const [token, cancel] = CancellationToken.source()
// spawn background task (run async function without using `await`)
CancellationError.ignoreAsync(() => yourAsyncFunction(token, param1, param2))
// ... do something ...
// then cancel the background task
await cancel()
So this is the solution of the OP's question.
import { CancellationToken, CancellationError, AsyncCheckpoint } from '#kaisukez/cancellation-token'
async function update(token) {
var urls = await AsyncCheckpoint.after(token, () => getCdnUrls());
var metadata = await AsyncCheckpoint.after(token, () => fetchMetaData(urls));
var content = await AsyncCheckpoint.after(token, () => fetchContent(metadata));
await AsyncCheckpoint.after(token, () => render(content));
return;
}
const [token, cancel] = CancellationToken.source();
// spawn background task (run async function without using `await`)
CancellationError.ignoreAsync(() => update(token))
// ... do something ...
// then cancel the background task
await cancel()
Example written in Node with Typescript of a call which can be aborted from outside:
function cancelable(asyncFunc: Promise<void>): [Promise<void>, () => boolean] {
class CancelEmitter extends EventEmitter { }
const cancelEmitter = new CancelEmitter();
const promise = new Promise<void>(async (resolve, reject) => {
cancelEmitter.on('cancel', () => {
resolve();
});
try {
await asyncFunc;
resolve();
} catch (err) {
reject(err);
}
});
return [promise, () => cancelEmitter.emit('cancel')];
}
Usage:
const asyncFunction = async () => {
// doSomething
}
const [promise, cancel] = cancelable(asyncFunction());
setTimeout(() => {
cancel();
}, 2000);
(async () => await promise)();