Async generator class stuck on infinite loop javascript - javascript

I'm trying to get the following async generator to work:
class MyIterator {
constructor(m) {
this.collection = m;
}
async *[Symbol.iterator]() {
for (let item of this.collection) {
const resultItem = await Promise.resolve(item)
console.log("item: ", resultItem)
yield resultItem
}
}
}
(async () => {
const iterator = new MyIterator([1,2,3])
let times = 0
for await (let thing of iterator) {
console.log("thing: ", thing)
// this is here to avoid an infinite loop
times++
if (times > 1000) break
}
})()
But it ends up in an infinite loop, and thing is always undefined.
item: 1
thing: undefined
item: 2
thing: undefined
item: 3
thing: undefined (x999)
I've tried a similar code, but this time without the Promise/async behaviour, and it seems to work just fine.
class MyIterator {
constructor(m) {
this.collection = m;
}
*[Symbol.iterator]() {
for (let item of this.collection) {
console.log("item: ", item)
yield item
}
}
}
const iterator = new MyIterator([1,2,3])
for (let thing of iterator) {
console.log("thing: ", thing)
}
item: 1
thing: 1
item: 2
thing: 2
item: 3
thing: 3

The for await..of construct will attempt to iterate over an async iterator.
An async iterator is defined using the ##asyncIterator well-known symbol:
class MyIterator {
constructor(m) {
this.collection = m;
}
async *[Symbol.asyncIterator]() { //<-- this is async
for (let item of this.collection) {
const resultItem = await Promise.resolve(item)
//console.log("item: ", resultItem)
yield resultItem
}
}
}
(async () => {
const iterator = new MyIterator([1,2,3])
let times = 0
for await (let thing of iterator) {
//no infinite loop
console.log("thing: ", thing)
}
})()
for await..of can also consume plain iterables that produce promises:
const promiseArray = [Promise.resolve("a"), Promise.resolve("b"), Promise.resolve("c")];
(async function() {
for await(const item of promiseArray) {
console.log(item);
}
})()
Attempting to make a regular iterator that is an async method/function does not work.
If you want to keep your ##iterator defined method your the best choice is to make it produce promises instead:
class MyIterator {
constructor(m) {
this.collection = m;
}
*[Symbol.iterator]() { // not async
for (let item of this.collection) {
yield Promise.resolve(item); //produce a promise
}
}
}
(async () => {
const iterator = new MyIterator([1,2,3])
let times = 0
for await (let thing of iterator) {
console.log("thing: ", thing)
}
})()
Although, that's might be a bad practice if any of the promises rejects:
const wait = (ms, val) =>
new Promise(res => setTimeout(res, ms, val));
const fail = (ms, val) =>
new Promise((_, rej) => setTimeout(rej, ms, val));
const arr = [
wait(100, 1),
wait(150, 2),
fail(0, "boom"),
wait(200, 3)
];
(async function(){
try {
for await (const item of arr) {
console.log(item);
}
} catch (e) {
console.error(e);
}
})()
/* result in the browser console:
Uncaught (in promise) boom
1
2
boom
*/
However, be aware that there is a difference in semantics between these:
A regular iterator produces an IteratorResult - an object with value and done properties.
const syncIterable = {
[Symbol.iterator]() {
return {
next() {
return {value: 1, done: true}
}
}
}
}
const syncIterator = syncIterable[Symbol.iterator]();
console.log("sync IteratorResult", syncIterator.next());
An async generator produces a promise for an IteratorResult
const asyncIterable = {
[Symbol.asyncIterator]() {
return {
next() {
return Promise.resolve({value: 2, done: true});
}
}
}
}
const asyncIterator = asyncIterable[Symbol.asyncIterator]();
asyncIterator.next().then(result => console.log("async IteratorResult", result));
Finally, an iterator that produces promises will have an IteratorResult where value is a promise:
const promiseSyncIterable = {
[Symbol.iterator]() {
return {
next() {
return {value: Promise.resolve(3), done: true}
}
}
}
}
const promiseSyncIterator = promiseSyncIterable[Symbol.iterator]();
const syncPromiseIteratorResult = promiseSyncIterator.next();
console.log("sync IteratorResult with promise", syncPromiseIteratorResult);
syncPromiseIteratorResult.value
.then(value => console.log("value of sync IteratorResult with promise", value));
Side-note on nomenclature: MyIterator is not an iterator. An iterator is an object with a next() method which produces an IteratorResult. An object that you can iterate over has an ##iterator (or ##asyncIterable) method and it is called iterable (or async iterable respectively).

As #VLAZ pointed out in a comment to my question, I was using Symbol.iterator instead of Symbol.asyncIterator. The following implementation works as expected:
class MyIterator {
constructor(m) {
this.collection = m;
}
async *[Symbol.asyncIterator]() {
for (let item of this.collection) {
const resultItem = await Promise.resolve(item)
console.log("item: ", resultItem)
yield resultItem
}
}
}
(async () => {
const iterator = new MyIterator([1,2,3])
for await (let thing of iterator) {
console.log("thing: ", thing)
}
})()

Related

Optimal Syntax for object creation requiring async calls in Javascript

I have an object that has properties that are generated via async functions. I feel like it is calling one, then calling the next, etc. I don't need it to do that but I do need it all be complete before moving on. Below is how I currently do it but I am looking for a more efficient method. I have thought about calling all the functions using Promise.All() in variables. Then creating the object and setting the properties to the variables but I have a feeling someone has a more elegant solution somewhere.
let obj = {
prop1 : await functionName(something),
prop2 : await anotherFunction(somethingElse)}
Interesting question.
I was thinking and come up with 2 solutions.
The 1st one is by using Promise.all as you mentioned:
function delay(ms = 0) {
return new Promise(resolve => setTimeout(resolve, ms))
}
async function functionName() {
await delay(1000)
return 'value1'
}
async function anotherFunction() {
await delay(500)
return 'value2'
}
async function createAsyncObject() {
const obj = {}
// Set properties
await Promise.all([
functionName().then(v => obj.prop1 = v),
anotherFunction().then(v => obj.prop2 = v)
])
return obj
}
const obj = await createAsyncObject()
The 2nd one I make use of Proxy object to make it handle the Promises
to set the object's properties:
function delay(ms = 0) {
return new Promise(resolve => setTimeout(resolve, ms))
}
async function functionName() {
await delay(1000)
return 'value1'
}
async function anotherFunction() {
await delay(500)
return 'value2'
}
function asyncObjectWrapper(obj) {
const promises = []
const then = function then(callback) {
return Promise.all(promises).then(() => callback())
}
return new Proxy(obj, {
get: (target, property, receiver) => {
if (property !== 'then') return
return then
},
set: (target, property, value) => {
if (!(value instanceof Promise)) return true
promises.push(value)
value.then(v => target[property] = v)
return true
}
})
}
async function createAsyncObject() {
const obj = {}
const wrapper = asyncObjectWrapper(obj)
// Set properties
await Object.assign(wrapper, {
prop1: functionName(),
prop2: anotherFunction()
})
return obj
}
const obj = await createAsyncObject()
You could create a list of name, value pairs by using Promise.all() and your property retrieval functions.
Once you have this list of entries you can use Object.fromEntries() to create your new object.
In this example, the property retrieval functions are getA(), getB() and getC(), they could, of course be called anything.
function getA() {
return new Promise(resolve => setTimeout(resolve, 500, 'value a'))
}
function getB() {
return new Promise(resolve => setTimeout(resolve, 500, 'value b'))
}
function getC() {
return new Promise(resolve => setTimeout(resolve, 500, 'value c'))
}
async function getProperty(name, fn) {
let value = await fn();
return [name, value];
}
async function getObj() {
console.log('Getting properties...');
const entries = await Promise.all([ getProperty('a', getA), getProperty('b', getB), getProperty('c', getC) ]);
console.log('Properties:', entries);
const newObj = Object.fromEntries(entries);
console.log('New obj:', newObj)
}
getObj()
.as-console-wrapper { max-height: 100% !important; }

Use async iterator triggered by a custom function in main scope

What I want to do is to create an iterator, which is only triggered when an external function is called, say an external event.
An iterator that simply waits for custom events.
function createIteratorWithFunction() {
var thingThatResolves;
var asyncIterable = {
thingThatResolves,
[Symbol.asyncIterator]() {
return {
next() {
return (new Promise((resolve, reject) => asyncIterable.thingThatResolves = (resolve))).then(_ => ({
value: _,
done: false
}));
},
return () {
return {
done: true
}
}
};
}
};
return asyncIterable;
}
iter = createIteratorWithFunction();
(async function() {
for await (let val of iter) {
console.log(val);
}
})()
<button onclick="iter.thingThatResolves('execute');iter.thingThatResolves(3)">execute next!</button>
As you can see, it only resolves 'execute', but not 3, of course because promises can't be resolved more than once, and it only is updated asynchronously, I understand this, but since the iterator is async, how would I create a queue, so that any values that could've synchronously been triggered are retrieved by next(), as well?
I have this feeling that there's a more elegant solution involving promise chains, but it's escaping me at the moment. :-) See inline comments:
function createIteratorWithFunction() {
// Our pending promise, if any
let promise = null;
// The `resolve` function for our `pending` promise
let resolve = null;
// The values in the queue
const values = [];
// The async iterable
const asyncIterable = {
add(value) {
// Add a value to the queue; if there's a pending promise, fulfill it
values.push(value);
const r = resolve;
resolve = pending = null;
r?.();
},
[Symbol.asyncIterator]() {
return {
async next() {
// If we don't have a value...
while (!values.length) {
// ...we need to wait for one; make sure we have something
// to wait for
if (!resolve) {
pending = new Promise(r => { resolve = r; });
}
await pending;
}
// Get the value we waited for and return it
const value = values.shift();
return {
value,
done: false,
};
},
return() {
return {
done: true,
};
}
};
}
};
return asyncIterable;
}
const iter = createIteratorWithFunction();
(async function() {
for await (let val of iter) {
console.log(val);
}
})();
document.getElementById("execute").addEventListener("click", () => {
iter.add("execute");
iter.add(3);
});
<button id="execute">execute next!</button>
One of the key things here is that an async iterable can have overlapping iterations, and it has to not get confused by that. This implementation avoids that by creating the promise it'll wait on synchronously if it needs one.
function createIteratorWithFunction() {
// Our pending promise, if any
let promise = null;
// The `resolve` function for our `pending` promise
let resolve = null;
// The values in the queue
const values = [];
// The async iterable
const asyncIterable = {
add(value) {
// Add a value to the queue; if there's a pending promise, fulfill it
values.push(value);
const r = resolve;
resolve = pending = null;
r?.();
},
[Symbol.asyncIterator]() {
return {
async next() {
// If we don't have a value...
while (!values.length) {
// ...we need to wait for one; make sure we have something
// to wait for
if (!resolve) {
pending = new Promise(r => { resolve = r; });
}
await pending;
}
// Get the value we waited for and return it
const value = values.shift();
return {
value,
done: false,
};
},
return() {
return {
done: true,
};
}
};
}
};
return asyncIterable;
}
const iter = createIteratorWithFunction();
(async function() {
for await (let val of iter) {
console.log("first:", val);
}
})();
(async function() {
for await (let val of iter) {
console.log("second:", val);
}
})();
document.getElementById("execute").addEventListener("click", () => {
iter.add("execute");
iter.add(3);
});
<button id="execute">execute next!</button>
I'm never happy when I have to make the promise's resolve function accessible outside the promise executor function (the function you pass new Promise), but as I say, the elegant solution with promise chains is escaping me. I sense strongly that it's there...somewhere... :-)
Another idea & way of doing this, you could use custom events, one advantage is that the code is much easier to reason with.
Below I've knocked up a simple example, it also allows you to cancel the iterator & handle errors. makeIter simple gives you 4 functions,
add = use this to add an item to the itterator.
iter = this is the iterator you can for await on.
done = if were done, you can call this and let the GC do it's thing.
error = allows you to put an error into the iterator, you can test this by un-commenting the last line.
To prevent any race conditions I've simply used a stack..
function makeIter() {
const obj = new EventTarget();
const evName = 'my-iter';
const stack = [];
obj.addEventListener(evName, e => {
stack.push(e.detail);
resolve();
});
async function *iter() {
while (true) {
await new Promise(r => resolve = r);
while (stack.length) {
const s = stack.shift();
if (s.resolve) yield(s.resolve);
if (s.reject) throw s.reject;
if (s.cancel) return;
}
}
}
function ev(p) {
obj.dispatchEvent(new CustomEvent(evName, {detail:p}));
}
return {
error: (e) => ev({reject: e}),
done: () => ev({cancel: true}),
add: item => ev({resolve: item}),
iter: iter()
}
}
///testing...
const test = makeIter();
(async function () {
try {
for await (const item of test.iter) {
console.log(item);
}
} finally {
console.log('iter done');
}
}());
test.add('hello');
setTimeout(() => test.add('there'), 100);
setTimeout(() => {test.add('1'); test.add('2'); test.add('3'); }, 200);
setTimeout(() => test.add('4'), 400);
setTimeout(() => test.done(), 1000);
//setTimeout(() => test.error(new Error('oops')), 1000);

How to "multicast" an async iterable?

Can an async generator be somehow broadcast or multicast, so that all its iterators ("consumers"? subscribers?) receive all values?
Consider this example:
const fetchMock = () => "Example. Imagine real fetch";
async function* gen() {
for (let i = 1; i <= 6; i++) {
const res = await fetchMock();
yield res.slice(0, 2) + i;
}
}
const ait = gen();
(async() => {
// first "consumer"
for await (const e of ait) console.log('e', e);
})();
(async() => {
// second...
for await (const é of ait) console.log('é', é);
})();
Iterations "consume" a value, so only one or the other gets it.
I would like for both of them (and any later ones) to get every yielded value, if such a generator is possible to create somehow. (Similar to an Observable.)
This is not easily possible. You will need to explicitly tee it. This is similar to the situation for synchronous iterators, just a bit more complicated:
const AsyncIteratorProto = Object.getPrototypeOf(Object.getPrototypeOf(async function*(){}.prototype));
function teeAsync(iterable) {
const iterator = iterable[Symbol.asyncIterator]();
const buffers = [[], []];
function makeIterator(buffer, i) {
return Object.assign(Object.create(AsyncIteratorProto), {
next() {
if (!buffer) return Promise.resolve({done: true, value: undefined});
if (buffer.length) return buffer.shift();
const res = iterator.next();
if (buffers[i^1]) buffers[i^1].push(res);
return res;
},
async return() {
if (buffer) {
buffer = buffers[i] = null;
if (!buffers[i^1]) await iterator.return();
}
return {done: true, value: undefined};
},
});
}
return buffers.map(makeIterator);
}
You should ensure that both iterators are consumed at about the same rate so that the buffer doesn't grow too large.
Here's a solution using Highland as an intermediary. From the docs:
A stream forked to multiple consumers will pull values, one at a time, from its source as only fast as the slowest consumer can handle them.
import _ from 'lodash'
import H from 'highland'
export function fork<T>(generator: AsyncGenerator<T>): [
AsyncGenerator<T>,
AsyncGenerator<T>
] {
const source = asyncGeneratorToHighlandStream(generator).map(x => _.cloneDeep(x));
return [
highlandStreamToAsyncGenerator<T>(source.fork()),
highlandStreamToAsyncGenerator<T>(source.fork()),
];
}
async function* highlandStreamToAsyncGenerator<T>(
stream: Highland.Stream<T>
): AsyncGenerator<T> {
for await (const row of stream.toNodeStream({ objectMode: true })) {
yield row as unknown as T;
}
}
function asyncGeneratorToHighlandStream<T>(
generator: AsyncGenerator<T>
): Highland.Stream<T> {
return H(async (push, next) => {
try {
const result = await generator.next();
if (result.done) return push(null, H.nil);
push(null, result.value);
next();
} catch (error) {
return push(error);
}
});
}
Usage:
const [copy1, copy2] = fork(source);
Works in Node, browser untested.
I built a library to do this here: https://github.com/tjenkinson/forkable-iterator
Means you can do something like:
import { buildForkableIterator, fork } from 'forkable-iterator';
function* Source() {
yield 1;
yield 2;
return 'return';
}
const forkableIterator = buildForkableIterator(Source());
console.log(forkableIterator.next()); // { value: 1, done: false }
const child1 = fork(forkableIterator);
// { value: 2, done: false }
console.log(child1.next());
// { value: 2, done: false }
console.log(forkableIterator.next());
// { value: 'return', done: true }
console.log(child1.next());
// { value: 'return', done: true }
console.log(forkableIterator.next());
If you no longer need to keep consuming from a fork providing you loose references to it there also shouldn’t be a memory leak.

Wrap a resultset callback function with a generator/iterator

I'm working on converting a legacy callback-based API into an async library. But I just can't wrap my head around getting a "resultset" to work as a generator (Node 10.x).
The original API works like this:
api.prepare((err, rs) => {
rs.fetchRows(
(err, row) => {
// this callback is called as many times as rows exist
console.log("here's a row:", row);
},
() => {
console.log("we're done, data exausted");
}
);
});
But here is how I want to use it:
const wrapped = new ApiWrapper(api);
const rs = await wrapped.prepare({});
for (let row of rs.rows()) {
console.log("here's a row:", row);
}
let row;
while(row = await rs.next()) {
console.log("here's a row:", row);
}
I thought I had it under control with generators, but it looks like you cannot use yield inside a callback. It actually seems logical if you think about.
class ApiWrapper {
constructor(api) {
this.api = api;
}
prepare() {
return new Promise((resolve, reject) => {
this.api.prepare((err, rs) => {
if (err) {
reject(err);
} else {
resolve(rs);
}
});
});
}
*rows() {
this.api.fetchRows((err, row) => {
if (err) {
throw err;
} else {
yield row; // nope, not allowed here
}
});
}
next() { ... }
}
So what alternatives do I have?
Important: I don't want to store anything in an array then iterate that, we're talking giga-loads of row data here.
Edit
I'm able to simulate the behavior I want using stream.Readable but it warns me that it's an experimental feature. Here's a simplified array-based version of the issue I'm trying to solve using stream:
const stream = require('stream');
function gen(){
const s = new stream.Readable({
objectMode: true,
read(){
[11, 22, 33].forEach(row => {
this.push({ value: row });
});
this.push(null)
}
});
return s;
}
for await (let row of gen()) {
console.log(row);
}
// { value: 11 }
// { value: 22 }
// { value: 33 }
(node:97157) ExperimentalWarning: Readable[Symbol.asyncIterator] is an experimental feature. This feature could change at any time
I finally realized I needed something similar to Go's channels that were async/await compatible. Basically the answer is to synchronize an async iterator and a callback, making them wait for each other as next() iterations are consumed.
The best (Node) native solution I found was to use a stream as an iterator, which is supported in Node 10.x but tagged experimental. I also tried to implement it with the p-defer NPM module, but that turned out to be more involved than I expected. Finally ran across the https://www.npmjs.com/package/#nodeguy/channel module, which was exactly what I needed:
const Channel = require('#nodeguy/channel');
class ApiWrapper {
// ...
rows() {
const channel = new Channel();
const iter = {
[Symbol.asyncIterator]() {
return this;
},
async next() {
const val = await channel.shift();
if (val === undefined) {
return { done: true };
} else {
return { done: false, value: val };
}
}
};
this.api.fetchRows(async (err, row) => {
await channel.push(row);
}).then(() => channel.close());
return iter;
}
}
// then later
for await (let row of rs.rows()) {
console.log(row)
}
Note how each iterating function core, next() and rows(), have a await that will throttle how much data can be pushed across the channel, otherwise the producing callback could end up pushing data uncontrollably into the channel queue. The idea is that the callback should wait for data to be consumed by the iterator next() before pushing more.
Here's a more self-contained example:
const Channel = require('#nodeguy/channel');
function iterating() {
const channel = Channel();
const iter = {
[Symbol.asyncIterator]() {
return this;
},
async next() {
console.log('next');
const val = await channel.shift();
if (val === undefined) {
return { done: true };
} else {
return { done: false, value: val };
}
}
};
[11, 22, 33].forEach(async it => {
await channel.push(it);
console.log('pushed', it);
});
console.log('returned');
return iter;
}
(async function main() {
for await (let it of iterating()) {
console.log('got', it);
}
})();
/*
returned
next
pushed 11
got 11
next
pushed 22
got 22
next
pushed 33
got 33
next
*/
Like I said, Streams and/or Promises can be used to implement this, but the Channel module solves some of the complexity that make it more intuitive.
The original question has two nested callback taking async functions
api.prepare((err,res) => ...)
rs.fetchRows((err,res) => ...)
The first one runs the callback only once so just promisifying it as follows is sufficient.
function promisfied(f){
return new Promise((v,x) => f(x,v));
}
However the second function will invoke it's callback multiple times and we wish to generate an async iterable from this function such that we can consume it in a for await of loop.
This is also possible by employing async generators as follows;
async function* rowEmitterGenerator(rs){
let _v, // previous resolve
_x, // previous reject
_row = new Promise((v,x) => (_v = v, _x = x));
rs.fetchRows((err, row) => ( err ? _x(err) : _v(row)
, _row = new Promise((v,x) => (_v = v, _x = x))
));
while(true){
try {
yield _row;
}
catch(e){
console.log(e);
}
}
}
Then putting all together in a top level await context;
const rows = await promisified(api.prepare),
rowEmitter = rowEmitterGenerator(rows);
for await (let row of rowEmitter){
console.log(`Received row: ${row}`);
// do something with the row
}

Filtering an array with a function that returns a promise

Given
let arr = [1,2,3];
function filter(num) {
return new Promise((res, rej) => {
setTimeout(() => {
if( num === 3 ) {
res(num);
} else {
rej();
}
}, 1);
});
}
function filterNums() {
return Promise.all(arr.filter(filter));
}
filterNums().then(results => {
let l = results.length;
// length should be 1, but is 3
});
The length is 3 because Promises are returned, not values. Is there a way to filter the array with a function that returns a Promise?
Note: For this example, fs.stat has been replaced with setTimeout, see https://github.com/silenceisgolden/learn-esnext/blob/array-filter-async-function/tutorials/array-filter-with-async-function.js for the specific code.
Here is a 2017 elegant solution using async/await :
Very straightforward usage:
const results = await filter(myArray, async num => {
await doAsyncStuff()
return num > 2
})
The helper function (copy this into your web page):
async function filter(arr, callback) {
const fail = Symbol()
return (await Promise.all(arr.map(async item => (await callback(item)) ? item : fail))).filter(i=>i!==fail)
}
Demo:
// Async IIFE
(async function() {
const myArray = [1, 2, 3, 4, 5]
// This is exactly what you'd expect to write
const results = await filter(myArray, async num => {
await doAsyncStuff()
return num > 2
})
console.log(results)
})()
// Arbitrary asynchronous function
function doAsyncStuff() {
return Promise.resolve()
}
// The helper function
async function filter(arr, callback) {
const fail = Symbol()
return (await Promise.all(arr.map(async item => (await callback(item)) ? item : fail))).filter(i=>i!==fail)
}
I'll even throw in a CodePen.
As mentioned in the comments, Array.prototype.filter is synchronous and therefore does not support Promises.
Since you can now (theoretically) subclass built-in types with ES6, you should be able to add your own asynchronous method which wraps the existing filter function:
Note: I've commented out the subclassing, because it's not supported by Babel just yet for Arrays
class AsyncArray /*extends Array*/ {
constructor(arr) {
this.data = arr; // In place of Array subclassing
}
filterAsync(predicate) {
// Take a copy of the array, it might mutate by the time we've finished
const data = Array.from(this.data);
// Transform all the elements into an array of promises using the predicate
// as the promise
return Promise.all(data.map((element, index) => predicate(element, index, data)))
// Use the result of the promises to call the underlying sync filter function
.then(result => {
return data.filter((element, index) => {
return result[index];
});
});
}
}
// Create an instance of your subclass instead
let arr = new AsyncArray([1,2,3,4,5]);
// Pass in your own predicate
arr.filterAsync(async (element) => {
return new Promise(res => {
setTimeout(() => {
res(element > 3);
}, 1);
});
}).then(result => {
console.log(result)
});
Babel REPL Demo
For typescript folk (or es6 just remove type syntax)
function mapAsync<T, U>(array: T[], callbackfn: (value: T, index: number, array: T[]) => Promise<U>): Promise<U[]> {
return Promise.all(array.map(callbackfn));
}
async function filterAsync<T>(array: T[], callbackfn: (value: T, index: number, array: T[]) => Promise<boolean>): Promise<T[]> {
const filterMap = await mapAsync(array, callbackfn);
return array.filter((value, index) => filterMap[index]);
}
es6
function mapAsync(array, callbackfn) {
return Promise.all(array.map(callbackfn));
}
async function filterAsync(array, callbackfn) {
const filterMap = await mapAsync(array, callbackfn);
return array.filter((value, index) => filterMap[index]);
}
es5
function mapAsync(array, callbackfn) {
return Promise.all(array.map(callbackfn));
}
function filterAsync(array, callbackfn) {
return mapAsync(array, callbackfn).then(filterMap => {
return array.filter((value, index) => filterMap[index]);
});
}
edit: demo
function mapAsync(array, callbackfn) {
return Promise.all(array.map(callbackfn));
}
function filterAsync(array, callbackfn) {
return mapAsync(array, callbackfn).then(filterMap => {
return array.filter((value, index) => filterMap[index]);
});
}
var arr = [1, 2, 3, 4];
function isThreeAsync(number) {
return new Promise((res, rej) => {
setTimeout(() => {
res(number === 3);
}, 1);
});
}
mapAsync(arr, isThreeAsync).then(result => {
console.log(result); // [ false, false, true, false ]
});
filterAsync(arr, isThreeAsync).then(result => {
console.log(result); // [ 3 ]
});
Here's a way:
var wait = ms => new Promise(resolve => setTimeout(resolve, ms));
var filter = num => wait(1).then(() => num == 3);
var filterAsync = (array, filter) =>
Promise.all(array.map(entry => filter(entry)))
.then(bits => array.filter(entry => bits.shift()));
filterAsync([1,2,3], filter)
.then(results => console.log(results.length))
.catch(e => console.error(e));
The filterAsync function takes an array and a function that must either return true or false or return a promise that resolves to true or false, what you asked for (almost, I didn't overload promise rejection because I think that's a bad idea). Let me know if you have any questions about it.
var wait = ms => new Promise(resolve => setTimeout(resolve, ms));
var filter = num => wait(1).then(() => num == 3);
var filterAsync = (array, filter) =>
Promise.all(array.map(entry => filter(entry)))
.then(bits => array.filter(entry => bits.shift()));
filterAsync([1,2,3], filter)
.then(results => console.log(results.length))
.catch(e => console.error(e));
var console = { log: msg => div.innerHTML += msg + "<br>",
error: e => console.log(e +", "+ (e.lineNumber-25)) };
<div id="div"></div>
Promise Reducer to the rescue!
[1, 2, 3, 4].reduce((op, n) => {
return op.then(filteredNs => {
return new Promise(resolve => {
setTimeout(() => {
if (n >= 3) {
console.log("Keeping", n);
resolve(filteredNs.concat(n))
} else {
console.log("Dropping", n);
resolve(filteredNs);
}
}, 1000);
});
});
}, Promise.resolve([]))
.then(filteredNs => console.log(filteredNs));
Reducers are awesome. "Reduce my problem to my goal" seems to be a pretty good strategy for anything more complex than what the simple tools will solve for you, i.e. filtering an array of things that aren't all available immediately.
asyncFilter method:
Array.prototype.asyncFilter = async function(f){
var array = this;
var booleans = await Promise.all(array.map(f));
return array.filter((x,i)=>booleans[i])
}
Late to the game but since no one else mentioned it, Bluebird supports Promise.map which is my go-to for filters requiring aysnc processing for the condition,
function filterAsync(arr) {
return Promise.map(arr, num => {
if (num === 3) return num;
})
.filter(num => num !== undefined)
}
Two lines, completely typesafe
export const asyncFilter = async <T>(list: T[], predicate: (t: T) => Promise<boolean>) => {
const resolvedPredicates = await Promise.all(list.map(predicate));
return list.filter((item, idx) => resolvedPredicates[idx]);
};
In case someone is interested in modern typescript solution (with fail symbol used for filtering):
const failSymbol = Symbol();
export async function filterAsync<T>(
itemsToFilter: T[],
filterFunction: (item: T) => Promise<boolean>,
): Promise<T[]> {
const itemsOrFailFlags = await Promise.all(
itemsToFilter.map(async (item) => {
const hasPassed = await filterFunction(item);
return hasPassed ? item : failSymbol;
}),
);
return itemsOrFailFlags.filter(
(itemOrFailFlag) => itemOrFailFlag !== failSymbol,
) as T[];
}
There is a one liner to to do that.
const filterPromise = (values, fn) =>
Promise.all(values.map(fn)).then(booleans => values.filter((_, i) => booleans[i]));
Pass the array into values and the function into fn.
More description on how this one liner works is available here.
For production purposes you probably want to use a lib like lodasync:
import { filterAsync } from 'lodasync'
const result = await filterAsync(async(element) => {
await doSomething()
return element > 3
}, array)
Under the hood, it maps your array by invoking the callback on each element and filters the array using the result. But you should not reinvent the wheel.
You can do something like this...
theArrayYouWantToFilter = await new Promise(async (resolve) => {
const tempArray = [];
theArrayYouWantToFilter.filter(async (element, index) => {
const someAsyncValue = await someAsyncFunction();
if (someAsyncValue) {
tempArray.push(someAsyncValue);
}
if (index === theArrayYouWantToFilter.length - 1) {
resolve(tempArray);
}
});
});
Wrapped within an async function...
async function filter(theArrayYouWantToFilter) {
theArrayYouWantToFilter = await new Promise(async (resolve) => {
const tempArray = [];
theArrayYouWantToFilter.filter(async (element, index) => {
const someAsyncValue = await someAsyncFunction();
if (someAsyncValue) {
tempArray.push(someAsyncValue);
}
if (index === theArrayYouWantToFilter.length - 1) {
resolve(tempArray);
}
});
});
return theArrayYouWantToFilter;
}
A valid way to do this (but it seems too messy):
let arr = [1,2,3];
function filter(num) {
return new Promise((res, rej) => {
setTimeout(() => {
if( num === 3 ) {
res(num);
} else {
rej();
}
}, 1);
});
}
async function check(num) {
try {
await filter(num);
return true;
} catch(err) {
return false;
}
}
(async function() {
for( let num of arr ) {
let res = await check(num);
if(!res) {
let index = arr.indexOf(num);
arr.splice(index, 1);
}
}
})();
Again, seems way too messy.
A variant of #DanRoss's:
async function filterNums(arr) {
return await arr.reduce(async (res, val) => {
res = await res
if (await filter(val)) {
res.push(val)
}
return res
}, Promise.resolve([]))
}
Note that if (as in current case) you don't have to worry about filter() having
side effects that need to be serialized, you can also do:
async function filterNums(arr) {
return await arr.reduce(async (res, val) => {
if (await filter(val)) {
(await res).push(val)
}
return res
}, Promise.resolve([]))
}
Late to the party, and I know that my answer is similar to other already posted answers, but the function I'm going to share is ready for be dropped into any code and be used.
As usual, when you have to do complex operations on arrays, reduce is king:
const filterAsync = (asyncPred) => arr =>
arr.reduce(async (acc,item) => {
const pass = await asyncPred(item);
if(pass) (await acc).push(item);
return acc;
},[]);
It uses modern syntax so make sure your target supports it. To be 100% correct you should use Promise.resolve([]) as the initial value, but JS just doesn't care and this way it is way shorter.
Then you can use it like this:
var wait = ms => new Promise(resolve => setTimeout(resolve, ms));
const isOdd = x => wait(1).then(()=>x%2);
(filterAsync(isOdd)([1,2,3,4,4])).then(console.log) // => [1,3]
Here's a shorter version of #pie6k's Typescript version:
async function filter<T>(arr: T[], callback: (val: T) => Promise<Boolean>) {
const fail = Symbol()
const result = (await Promise.all(arr.map(async item => (await callback(item)) ? item : fail))).filter(i => i !== fail)
return result as T[] // the "fail" entries are all filtered out so this is OK
}
An efficient way of approaching this is by processing arrays as iterables, so you can apply any number of required operations in a single iteration.
The example below uses library iter-ops for that:
import {pipe, filter, toAsync} from 'iter-ops';
const arr = [1, 2, 3]; // synchronous iterable
const i = pipe(
toAsync(arr), // make our iterable asynchronous
filter(async (value, index) => {
// returns Promise<boolean>
})
);
(async function() {
for await (const a of i) {
console.log(a); // print values
}
})();
All operators within the library support asynchronous predicates when inside an asynchronous pipeline (why we use toAsync), and you can add other operators, in the same way.
Use of Promise.all for this is quite inefficient, because you block the entire array from any further processing that can be done concurrently, which the above approach allows.

Categories