As far as I know, async/await is just syntactic sugar over promise.then. Consider this code snippet:
function sleep(n){
return new Promise(res => setTimeout(res, n));
}
function* range(n){
var i = 0;
while(i < n) yield i++;
}
async function doStuff(){
for(let n of range(10)){
console.log(n); // print the number
await sleep(1000); // wait for 1 second
}
}
async/await makes the code very linear, efficient and easy to understand. One thing to keep in mind is that range does not have to have an actual end for this to work.
The problem now is how this can be rewritten using pre-ES7 era's promise.then. Here's a possible implementation of the same loop:
function doStuff(){
return Array.from(range(10)).reduce((acc, ele) => {
return acc
.then(() => console.log(ele))
.then(() => sleep(1000))
}, Promise.resolve());
}
Ignoring the fact that the code isn't quite elegant, the use of Array.from(range(10))
creates an extra array that isn't needed, and
assumes range(10) will end some point in the future.
Doesn't look like a good conversion.
We can also completely reinvent the wheel by using yield as await, but that would make the syntax non ES5-compliant. The goal here is to:
Rewrite using ES5-compliant syntax
Use the promise-returning sleep function
Dynamically chain the sleep promise while allowing the iterator to not have an end
doStuff can be chained:
doStuff().finally(cleanUp); // clean up if something failed
(Optional) Code should not be overly complex
Any idea?
I think the following may do the trick, your example doesn't show what to do with resolve value and how it relates to the iterator values so I made a change to how sleep is called.
Some promise polyfils may run out of stack space with high promise chains so you should check your polyfil (if its implementation returns and continues with a setTimeout the stack should clear but some polyfils may not implement it this way).
function sleep(n){
return new Promise(res => setTimeout(_=>res(n/100), n));
}
function* range(n){
var i = 0;
while(i < n) yield i++;
}
function doStuff(){
const processValue =
resolve => {
console.log("resolved with:",resolve);
// if(resolve===3){throw "nope";}
return sleep(resolve*100);
},
rec = (p,iter) => {
const result = iter.next();
if (result.done){
return p;
}
p = p.then(_=>processValue(result.value))
return p.then(
resolve=>{
return rec(p,iter)
}
);
},
iter = range(10),
firstResult = iter.next();
if(firstResult.done){
return processValue(firstResult.value);
}
return rec(processValue(firstResult.value),iter);
}
doStuff()
.then(
x=>console.log("done:",x)
,reject=>console.warn("fail:",reject)
);
I've always said that if you need an asynchronous design pattern first look at the async library. In this case, since you're using promises, take a look at the promisified async-q library. The translation is straight forward:
var n = 0;
async.whilst(() => n < 10, () => {
n++;
console.log(n);
return sleep(1000);
})
Related
Say I have the following generator that produces an infinite range:
const generateInfiniteRange = function* () {
for (let i = 0; true; i++) {
yield i;
}
};
const infiniteRange = generateInfiniteRange();
const infiniteRange$ = from(infiniteRange);
I can do something like this with RxJS:
const predicate = i => i < 10;
infiniteRange$.pipe(
takeWhile(predicate),
);
Now let's say that the predicate is asynchronous:
const predicate = async i => i < 10;
infiniteRange$.pipe(takeWhile(predicate)).subscribe(console.log);
How can I make this code work?
infiniteRange$.pipe(takeWhile(predicate));
I've tried using map as follows:
infiniteRange$.pipe(
map(async i => ({
i,
predicateResult: await predicate(i),
})),
takeWhile(({predicateResult}) => predicateResult),
pluck('i'),
);
but that just ends up mapping everything to a promise that always coerces to a truthy value, so everything passes through the takeWhile
I figured based on a previous question I've asked (for which originally I asked this question as an addendum in a comment of an answer, before deciding a question was more appropriate) that I could use concatMap, but that just generates an infinite emission before any of the internal observables hit the pipe.
If I understand this right, you can not achieve what you want.
The simple reason is that when, with from you create an Observable out of a generator, then you are creating a synchronous Observable, i.e. an Observable which emits all its values synchronously.
You can see clearly this looking at the fromIterable source code
function fromIterable<T>(iterable: Iterable<T>) {
return new Observable((subscriber: Subscriber<T>) => {
for (const value of iterable) {
subscriber.next(value);
if (subscriber.closed) {
return;
}
}
subscriber.complete();
});
}
As you can see, the for loop is exited only when the subscriber is closed. But in our case the subscriber will be closed asynchronously, i.e. we need Node to stop the execution of the loop since no instructions are left and pick the next callback, the one that runs the predicate. This will never happen since the for loop will never end.
So the summary is that you can not have an async predicate working with a synchronous infinite stream of values, which is what you create using a Generator.
By the way, in order for the code to compile, you need to use concatMap to transform the value notified by the source stream to the Object used by the predicate. So a code that compiles is this one
infiniteRange$.pipe(
tap(i => {
console.log(i)
}),
concatMap(async i => {
console.log('concatMap hit');
return {
i,
predicateResult: await predicate(i),
}
}),
takeWhile(({predicateResult}) => {
console.log('takeWhile hit');
return predicateResult
}),
pluck('i'),
).subscribe(console.log);
Running this snippet, you will see that you enter once in the concatMap input function (i.e. "concatMap hit" will be printed once) while you never enter the function passed to takeWhile (i.e. "takeWhile hit" will never be printed).
A SOLUTION WITH ASYNC GENERATORS
Actually, if you change the generator to be async, then using concatMap in the pipe we can reach the result you are looking for.
This is how the code would look like
const generateInfiniteRangeAsync = async function* () {
for (let i = 0; true; i++) {
await new Promise(resolve => setTimeout(resolve, 1000));
yield i;
}
};
const predicate = async i => i < 10;
const infiniteRangeAsync = generateInfiniteRangeAsync();
const infiniteRangeAsync$ = from(infiniteRangeAsync);
infiniteRangeAsync$.pipe(
concatMap(async i => {
console.log('concatMap hit');
return {
i,
predicateResult: await predicate(i),
}
}),
takeWhile(({predicateResult}) => {
console.log('takeWhile hit');
return predicateResult
}),
pluck('i'),
)
.subscribe(console.log);
A simple solution, using iter-ops, which can handle asynchronous predicates out of the box:
import {pipeAsync, stop} from 'iter-ops';
// your infinite generator:
const generateInfiniteRange = function * () {
for (let i = 0; true; i++) {
yield i;
}
};
// create our iterable:
const i = pipeAsync(generateInfiniteRange(), stop(async a => a >= 10));
// test:
(async function () {
for await(const a of i) {
console.log(a); //=> 0, 1, 2, 3, 4, 5, 6, 7, 8, 9
}
})();
P.S. I am the author of the library.
I have a sudoku board implemented as an HTML table and a button that when clicked solves the sudoku board using a recursive backtracking algorithm in javascript. Now I want to make it so that you can see the adjustments being made by not altering the HTML immediately in my recursive function. I tried making the function async and then calling this function
function sleep(ms) {
return new Promise(resolve => setTimeout(resolve, ms));
}
like so
for (let choice = 1; choice <= 9; choice++) {
this.boardArray[row][col] = choice;
// put delay here
await sleep(250);
currEntry.textContent = choice;
if (this.choiceOkay(row, col)) {
const solved = this.solveBoard(nextRow, nextCol);
if (solved) {
return true;
}
}
This does give me the desired behavior initially but only for a portion of the board and then it seems to just stop. I removed the calls to the sleep function and tried it with the only alteration being "async" in front of the function declaration and it still only did a portion of the board but this time all at once without the visual delay. I'm wondering why making this function async causes this logic error?
Thanks in advance!
Also, this is my first question on Stack Overflow so let me know if I need to be more specific or anything along those lines.
The async keyword in front of a function basically says:
This function will return a Promise
Your sleep function already returns a Promise, so writing the keyword async is useless.
What matters is the await keyword which basically says:
On my right, there may be an async function (idem a Promise). Wait for my return before continuing
If you write async function sleep, but omit the await keyword when you call sleep, you throw the function call in the nature and never waits for its return to come back (hence your code running "without delay")
If you want delay, it is better to let your algorithm be as is, and let the caller of your algorithm tell your algorithm to continue or not.
You can for that matter use generators.
(An other possibility could be trampolining).
function* runAlgo () {
for (let choice = 1; choice <= 9; choice++) {
yield; // gives control back to delayer
console.log('choice : ', choice)
// do your algo and put some yield wherever you like
}
}
// no need for async, a Promise is already returned
function sleep (t) {
return new Promise((ok, ko) => setTimeout(ok, t))
}
async function delayer () { // need for async so we can use the await keyword below
const it = runAlgo()
let next = it.next()
while (!next.done) {
await sleep(1000)
next = it.next()
}
}
delayer()
However, it is very likely that solving your board is what freezes your ui.
So you want to wait inside of solvingBoard as well:
function* solveBoard () {
let i = 0
while (i < 5) { //freezes the ui if not yielding
yield i++
}
}
function* runAlgo () {
for (let choice = 1; choice <= 9; choice++) {
yield 'choice : '+choice; // gives control back to caller
yield* solveBoard()
}
}
// no need for async, a Promise is already returned
function sleep (t) {
return new Promise((ok, ko) => setTimeout(ok, t))
}
async function delayer () { // need for async so we can use the await keyword below
const it = runAlgo()
let next = it.next()
while (!next.done) {
await sleep(1000)
next = it.next()
console.log('data', next.value)
}
}
delayer()
I have this class that has method next returning a Promise.
class PromiseGenerator {
constructor() {
this.limit = 100;
this.counter = 0;
}
next() {
this.counter++;
return new Promise((resolve, reject) => {
setTimeout(() => {
resolve(this.counter <= this.limit ? this.counter : false);
}, 500);
});
}
}
Though example shows 100 it could return unlimited number of promises.
I need to execute all the promises in sequential order.
How can I achieve it?
Only solution I came up so far is recursive:
const source = new PromiseGenerator();
(function loop() {
source.next().then(counter => {
if (counter) {
console.log(counter);
loop();
} else {
console.log('Done');
}
});
})();
As I understand Node at the moment does not optimize tail calls and it may lead to stack growth.
Is there a better way of doing this?
if some Promise library has it it will work but it would be nice to understand how to achieve it without library as well.
Update 1: Sorry I didn't make it clear right away: I am not in control of PromiseGenerator class, it is something I can use but cannot change. So the question is how to handle this situation.
Update 2: I went with #eikooc solution: Without generators but just with async/await. See example below.
Thanks!
Generators are a perfect match for this. Construct a generator with the function* keyword:
function* promiseGenerator() {
while(!someCondition) {
yield new Promise((resolve, reject) => {})
}
}
And then call it with:
const source = promiseGenerator()
source.next()
This will continue to give you new values. The return looks like this {value: Promise, done: false} until it is finished.
When the generator finishes the done value will change to true
If you want to keep using the class and just want a loop. You can also combine your class with a async function:
async function loop() {
const source = new PromiseGenerator()
while (true) {
const result = await source.next()
if (result) {
console.log(result)
} else {
console.log('done')
break
}
}
}
loop()
I was just reading this fantastic article «Generators» and it clearly highlights this function, which is a helper function for handling generator functions:
function async(makeGenerator){
return function () {
var generator = makeGenerator.apply(this, arguments);
function handle(result){
// result => { done: [Boolean], value: [Object] }
if (result.done) return Promise.resolve(result.value);
return Promise.resolve(result.value).then(function (res){
return handle(generator.next(res));
}, function (err){
return handle(generator.throw(err));
});
}
try {
return handle(generator.next());
} catch (ex) {
return Promise.reject(ex);
}
}
}
which I hypothesize is more or less the way the async keyword is implemented with async/await. So the question is, if that is the case, then what the heck is the difference between the await keyword and the yield keyword? Does await always turn something into a promise, whereas yield makes no such guarantee? That is my best guess!
You can also see how async/await is similar to yield with generators in this article where he describes the 'spawn' function ES7 async functions.
Well, it turns out that there is a very close relationship between async/await and generators. And I believe async/await will always be built on generators. If you look at the way Babel transpiles async/await:
Babel takes this:
this.it('is a test', async function () {
const foo = await 3;
const bar = await new Promise(resolve => resolve('7'));
const baz = bar * foo;
console.log(baz);
});
and turns it into this
function _asyncToGenerator(fn) {
return function () {
var gen = fn.apply(this, arguments);
return new Promise(function (resolve, reject) {
function step(key, arg) {
try {
var info = gen[key](arg);
var value = info.value;
} catch (error) {
reject(error);
return;
}
if (info.done) {
resolve(value);
} else {
return Promise.resolve(value).then(function (value) {
return step("next", value);
}, function (err) {
return step("throw", err);
});
}
}
return step("next");
});
};
}
this.it('is a test', _asyncToGenerator(function* () { // << now it's a generator
const foo = yield 3; // <<< now it's yield, not await
const bar = yield new Promise(resolve => resolve(7));
const baz = bar * foo;
console.log(baz);
}));
you do the math.
This makes it look like the async keyword is just that wrapper function, but if that's the case then await just gets turned into yield, there will probably be a bit more to the picture later on when they become native.
You can see more of an explanation for this here:
https://www.promisejs.org/generators/
yield can be considered to be the building block of await. yield takes the value it's given and passes it to the caller. The caller can then do whatever it wishes with that value (1). Later the caller may give a value back to the generator (via generator.next()) which becomes the result of the yield expression (2), or an error that will appear to be thrown by the yield expression (3).
async-await can be considered to use yield. At (1) the caller (i.e. the async-await driver - similar to the function you posted) will wrap the value in a promise using a similar algorithm to new Promise(r => r(value) (note, not Promise.resolve, but that's not a big deal). It then waits for the promise to resolve. If it fulfills, it passes the fulfilled value back at (2). If it rejects, it throws the rejection reason as an error at (3).
So the utility of async-await is this machinery that uses yield to unwrap the yielded value as a promise and pass its resolved value back, repeating until the function returns its final value.
what the heck is the difference between the await keyword and the yield keyword?
The await keyword is only to be used in async functions, while the yield keyword is only to be used in generator function*s. And those are obviously different as well - the one returns promises, the other returns generators.
Does await always turn something into a promise, whereas yield makes no such guarantee?
Yes, await will call Promise.resolve on the awaited value.
yield just yields the value outside of the generator.
tl;dr
Use async/await 99% of the time over generators. Why?
async/await directly replaces the most common workflow of promise chains allowing code to be declared as if it was synchronous, dramatically simplifying it.
Generators abstract the use case where you would call a series of async-operations that depend on each other and eventually will be in a "done" state. The most simple example would be paging through results that eventually return the last set but you would only call a page as needed, not immediately in succession.
async/await is actually an abstraction built on top of generators to make working with promises easier.
See very in-depth Explanation of Async/Await vs. Generators
Try this test programs which I used to understand await/async with promises.
Program #1: without promises it doesn't run in sequence
function functionA() {
console.log('functionA called');
setTimeout(function() {
console.log('functionA timeout called');
return 10;
}, 15000);
}
function functionB(valueA) {
console.log('functionB called');
setTimeout(function() {
console.log('functionB timeout called = ' + valueA);
return 20 + valueA;
}, 10000);
}
function functionC(valueA, valueB) {
console.log('functionC called');
setTimeout(function() {
console.log('functionC timeout called = ' + valueA);
return valueA + valueB;
}, 10000);
}
async function executeAsyncTask() {
const valueA = await functionA();
const valueB = await functionB(valueA);
return functionC(valueA, valueB);
}
console.log('program started');
executeAsyncTask().then(function(response) {
console.log('response called = ' + response);
});
console.log('program ended');
Program #2: with promises
function functionA() {
return new Promise((resolve, reject) => {
console.log('functionA called');
setTimeout(function() {
console.log('functionA timeout called');
// return 10;
return resolve(10);
}, 15000);
});
}
function functionB(valueA) {
return new Promise((resolve, reject) => {
console.log('functionB called');
setTimeout(function() {
console.log('functionB timeout called = ' + valueA);
return resolve(20 + valueA);
}, 10000);
});
}
function functionC(valueA, valueB) {
return new Promise((resolve, reject) => {
console.log('functionC called');
setTimeout(function() {
console.log('functionC timeout called = ' + valueA);
return resolve(valueA + valueB);
}, 10000);
});
}
async function executeAsyncTask() {
const valueA = await functionA();
const valueB = await functionB(valueA);
return functionC(valueA, valueB);
}
console.log('program started');
executeAsyncTask().then(function(response) {
console.log('response called = ' + response);
});
console.log('program ended');
In many ways, generators are a superset of async/await. Right now async/await has cleaner stack traces than co, the most popular async/await-like generator based lib. You can implement your own flavor of async/await using generators and add new features, like built-in support for yield on non-promises or building it on RxJS observables.
So, in short, generators give you more flexibility and generator-based libs generally have more features. But async/await is a core part of the language, it's standardized and won't change under you, and you don't need a library to use it. I have a blog post with more details on the difference between async/await and generators.
The yield+gen.next()-as-a-language-feature can be used to describe (or implement) the underlying control-flow that await-async has abstracted away.
As other answers suggest, await-as-a-language-feature is (or can be thought of) an implementation on top of yield.
Here is a more intutive understanding for that:
Say we have 42 awaits in an async function, await A -> await B -> ...
Deep down it is equivalent to having yield A -> tries resolve this as a Promise [1]
-> if resolvable, we yield B, and repeat [1] for B
-> if not resolveable, we throw
And so we end up with 42 yields in a generator. And in our controller we simply keep doing gen.next() until it is completed or gets rejected. (ie this is the same as using await on an async function that contains 42 await.)
This is why lib like redux-saga utilizes generator to then pipe the promises to the saga middleware to be resolved all at one place; thus decoupling the Promises constructions from their evaluations, thus sharing close resemblance to the Free Monad.
The idea is to recursively chain then() calls to replicate the behavior of await which allows one to invoke async routines in a synchronous fashion. A generator function is used to yield back control (and each value) from the callee to the caller, which happens to be the _asyncToGenerator() wrapper function.
As mentioned above, this is the trick that Babel uses to create polyfills. I slightly edited the code to make it more readable and added comments.
(async function () {
const foo = await 3;
const bar = await new Promise((resolve) => resolve(7));
const baz = bar * foo;
console.log(baz);
})();
function _asyncToGenerator(fn) {
return function () {
let gen = fn(); // Start the execution of the generator function and store the generator object.
return new Promise(function (resolve, reject) {
function step(func, arg) {
try {
let item = gen[func](arg); // Retrieve the function object from the property name and invoke it. Similar to eval(`gen.${func}(arg)`) but safer. If the next() method is called on the generator object, the item value by the generator function is saved and the generator resumes execution. The value passed as an argument is assigned as a result of a yield expression.
if (item.done) {
resolve(item.value);
return; // The executor return value is ignored, but we need to stop the recursion here.
}
// The trick is that Promise.resolve() returns a promise object that is resolved with the value given as an argument. If that value is a promise object itself, then it's simply returned as is.
return Promise.resolve(item.value).then(
(v) => step("next", v),
(e) => step("throw", e)
);
} catch (e) {
reject(e);
return;
}
}
return step("next");
});
};
}
_asyncToGenerator(function* () { // <<< Now it's a generator function.
const foo = yield 3; // <<< Now it's yield, not await.
const bar = yield new Promise((resolve, reject) => resolve(7)); // <<< Each item is converted to a thenable object and recursively enclosed into chained then() calls.
const baz = bar * foo;
console.log(baz);
})();
I have a function connectImpl referenced in multiple places. I am trying to invoke this promise and return its value out to the calling function synchronously through intermediating via a generator. If I call .next() on the generator it is returned in a pending state
{ value: { state: 'pending' }, done: false }
I would like to wait on the value of this generator until it is no longer pending. I have tried multiple versions of waitOn to accomplish this, but I cannot seem to make it work properly.
I am open to implementation suggestions. This is driving me a bit batty. Surprisingly the final created log of the promise fires later in the execution chain - after the generator is done. I am obviously missing something:
let models = null ;
let connectImpl = function() {
console.log('connectImpl')
let orm = setupImpl()
let config = getConfigImpl()
let qInitialize = q.nbind(orm.initialize, orm)
if(models) {
console.log('connectImpl:cached')
return q(models)
} else {
console.log('connectImpl:create')
return qInitialize(config).then(function(m){
console.log('connectImpl:created')
models = m
return models
})
}
}
let waitOn = function(generator){
console.log('waitOn')
let done = false ;
let generatorValue = null
while(!done){
var generatorResult = generator.next()
console.log(generatorResult)
done = generatorResult.done
generatorValue = generatorResult.value
}
return generatorValue
}
let domainImpl = function() {
console.log('domainImpl')
let getConnection = function *() {
console.log('domainImpl:getConnection')
yield connectImpl()
}
var generator = getConnection()
return waitOn(generator)
}
console.log('START')
console.log(domainImpl())
console.log('END')
I am able to invoke and get the
START
domainImpl
waitOn
domainImpl:getConnection
connectImpl
connectImpl:create
{ value: { state: 'pending' }, done: false }
{ value: undefined, done: true }
undefined
END
connectImpl:created
I am able to add execute the connectImpl promise to work with the middleware via this function - but I can't seem to adapt this to my above use case:
let domainMiddlewareImpl = function () {
return function *(next) {
let models = yield connectImpl()
this.request.models = models.collections;
this.request.connections = models.connections;
yield next
};
};
This looks fun. Let's see how we can yield promises. Our end goal is to write something like:
waitOn(function*(){
console.log("hello");
yield Q.delay(2000); // a placeholder, your calls in your example
console.log("World"); // this should run two seconds late.
});
Your issue here is that you're yielding them without waiting for them in advance. First of all, you can skip to the end for a 'ready' solution (don't!) and here is a fiddle of what we're making. Let's go through implementing waitOn with generators:
Let's start:
function waitOn(gen){
}
So, our function takes a generator, the first thing we'll have to do is invoke it since we need to execute the generator to get its results:
function waitOn(gen){
let sequence = gen(); // call the generator
}
Next, we'll want to wrap everything in a Promise since our waitOn will yield promises and return a promise for being done itself:
function waitOn(gen){
let sequence = gen(); // call the generator
return Promise.resolve(); // this is Q.resolve with Q
}
Now, what cases do we have:
The generator is done and returned a value - that is a return
The generator yielded a regular value and we do not have to wait for it
The generator yielded a promise and we have to wait for it. We also have to deal with exceptions (what if we yield a promise that rejects?)
So our basic structure is something like:
function waitOn(gen){
let sequence = gen(); // call the generator
return Promise.resolve().then(function cont(value){
let {value, done} = en.next(value); // get the next item
// depending on the case do what's appropriate
});
}
Note the destructuring assignment - I assume that's ok since your code has ES6 statements in it too. Note since this is the first call, value is undefined but generally we'll want to pass the value from our last call on. Now to handle the cases:
function waitOn(gen){
let sequence = gen(); // call the generator
return Promise.resolve().then(function cont(value){
let {done, value} = en.next(value); // get the next item
if(done) return value; // return case
if(!value || !value.then) return cont(value); // value case, recurse
return value.catch(e => gen.throw(e)).then(cont); // promise case
});
}
Note the .catch clause - we're throwing our code from the promise back to the generator for it to handle so we can try/catch the promises.
That's it! In 9 lines of JavaScript we've implemented generators for promises. Now to your code, you can yield any promise:
let conn = q.nBind(orm.initialize, orm);
waitOn(function*(){
console.log("Starting")
let handle = yield conn(config);
console.log("Handle created!", handle); // connected here
});
Happy coding and enjoy the power of coroutines. After we've had our fun - it's worth mentioning that Q already ships with Q.async and other newer promise libraries like Bluebird ship with their own (Bluebird has Promise.coroutine). If you're using a promise library - you can utilise those. This implementation works with native promises too.