Related
context: Two javascript classes in separate files, each integrating a different external service and being called in a express.js router.
See "problematic code" below:
route
routes.post('/aws', upload.single('file'), async (req, res) => {
const transcribeParams = JSON.parse(req.body.options)
const bucket = 'bucket-name'
const data = await ( await ( await awsTranscribe.Upload(req.file, bucket)).CreateJob(transcribeParams)).GetJob()
res.send(data)
})
S3 class
class AmazonS3 {
constructor() {
this.Upload = this.Upload
}
async Upload(file, bucket) {
const uploadParams = {
Bucket: bucket,
Body: fs.createReadStream(file.path),
Key: file.filename,
}
this.data = await s3.upload(uploadParams).promise()
return this
}
}
Transcribe class
class Transcribe extends AwsS3 {
constructor() {
super()
this.CreateJob = this.CreateJob
this.GetJob = this.GetJob
}
async CreateJob(params) {
if(this.data?.Location) {
params.Media = { ...params.Media, MediaFileUri: this.data.Location }
}
this.data = await transcribeService.startTranscriptionJob(params).promise()
return this
}
async GetJob(jobName) {
if(this.data?.TranscriptionJob?.TranscriptionJobName) {
jobName = this.data.TranscriptionJob.TranscriptionJobName
}
this.data = await transcribeService.getTranscriptionJob({TranscriptionJobName: jobName}).promise()
return this
}
}
problem: the problem is with the chained awaits in the router file:
await ( await ( await awsTranscribe.Upload...
Yes, it does work, but it would be horrible for another person to maintain this code in the future.
How can i make so it would be just
awsTranscribe.Upload(req.file, bucket).CreateJob(transcribeParams).GetJob() without the .then?
The problem is with the chained awaits in the router file: await ( await ( await awsTranscribe.Upload...
No, that's fine. In particular it would be trivial to refactor it to separate lines:
routes.post('/aws', upload.single('file'), async (req, res) => {
const transcribeParams = JSON.parse(req.body.options)
const bucket = 'bucket-name'
const a = await awsTranscribe.Upload(req.file, bucket);
const b = await b.CreateJob(transcribeParams);
const c = await b.GetJob();
res.send(c);
});
Your actual problem is that a, b, and c all refer to the same object awsTranscribe. Your code would also "work" if it was written
routes.post('/aws', upload.single('file'), async (req, res) => {
const transcribeParams = JSON.parse(req.body.options)
const bucket = 'bucket-name'
await awsTranscribe.Upload(req.file, bucket);
await awsTranscribe.CreateJob(transcribeParams);
await awsTranscribe.GetJob();
res.send(awsTranscribe);
});
The horrible thing is that you are passing your data between these methods through the mutable awsTranscribe.data property - even storing different kinds of data in it at different times! One could change the order of method calls and it would completely break in non-obvious and hard-to-debug ways.
Also it seems that multiple requests share the same awsTranscribe instance. This will not work with concurrent requests. Anything is possible from just "not working" to responding with the job data from a different user (request)! You absolutely need to fix that, then look at ugly syntax later.
What you really should do is get rid of the classes. There's no reason to use stateful objects here, this is plain procedural code. Write simple functions, taking parameters and returning values:
export async function uploadFile(file, bucket) {
const uploadParams = {
Bucket: bucket,
Body: fs.createReadStream(file.path),
Key: file.filename,
};
const data = s3.upload(uploadParams).promise();
return data.Location;
}
export async function createTranscriptionJob(location, params) {
params = {
...params,
Media: {
...params.Media,
MediaFileUri: location,
},
};
const data = await transcribeService.startTranscriptionJob(params).promise();
return data.TranscriptionJob;
}
async function getTranscriptionJob(job) {
const jobName = job.TranscriptionJobName;
return transcribeService.getTranscriptionJob({TranscriptionJobName: jobName}).promise();
}
Then you can import and call them as
routes.post('/aws', upload.single('file'), async (req, res) => {
const transcribeParams = JSON.parse(req.body.options)
const bucket = 'bucket-name'
const location = await uploadFile(req.file, bucket);
const job = await createTranscriptionJob(location, transcribeParams);
const data = await getTranscriptionJob(job);
res.send(c);
});
I got interested in whether it was possible to take an object with several async methods and somehow make them automatically chainable. Well, you can:
function chain(obj, methodsArray) {
if (!methodsArray || !methodsArray.length) {
throw new Error("methodsArray argument must be array of chainable method names");
}
const methods = new Set(methodsArray);
let lastPromise = Promise.resolve();
const proxy = new Proxy(obj, {
get(target, prop, receiver) {
if (prop === "_promise") {
return function() {
return lastPromise;
}
}
const val = Reflect.get(target, prop, receiver);
if (typeof val !== "function" || !methods.has(prop)) {
// no chaining if it's not a function
// or it's not listed as a chainable method
return val;
} else {
// return a stub function
return function(...args) {
// chain a function call
lastPromise = lastPromise.then(() => {
return val.apply(obj, args);
//return Reflect.apply(val, obj, ...args);
});
return proxy;
}
}
}
});
return proxy;
}
function delay(t) {
return new Promise(resolve => {
setTimeout(resolve, t);
});
}
function log(...args) {
if (!log.start) {
log.start = Date.now();
}
const delta = Date.now() - log.start;
const deltaPad = (delta + "").padStart(6, "0");
console.log(`${deltaPad}: `, ...args)
}
class Transcribe {
constructor() {
this.greeting = "Hello";
}
async createJob(params) {
log(`createJob: ${this.greeting}`);
return delay(200);
}
async getJob(jobName) {
log(`getJob: ${this.greeting}`);
return delay(100);
}
}
const t = new Transcribe();
const obj = chain(t, ["getJob", "createJob"]);
log("begin");
obj.createJob().getJob()._promise().then(() => {
log("end");
});
There's a placeholder for your Transcribe class that has two asynchronous methods that return a promise.
Then, there's a chain() function that returns a proxy to an object that makes a set of passed in method names be chainable which allows you to then do something like this:
const t = new Transcribe();
// make chainable proxy
const obj = chain(t, ["getJob", "createJob"]);
obj.createJob().getJob()
or
await obj.createJob().getJob()._promise()
I wouldn't necessarily say this is production-ready code, but it is an interesting feasibility demonstration and (for me) a chance to learn more about a Javascript proxy object.
Here's a different approach that (instead of the proxy object) adds method stubs to a promise to make things chainable:
function chain(orig, methodsArray) {
let masterP = Promise.resolve();
function addMethods(dest) {
for (const m of methodsArray) {
dest[m] = function(...args) {
// chain onto master promise to force sequencing
masterP = masterP.then(result => {
return orig[m].apply(orig, ...args);
});
// add methods to the latest promise befor returning it
addMethods(masterP);
return masterP;
}
}
}
// add method to our returned promise
addMethods(masterP);
return masterP;
}
function delay(t) {
return new Promise(resolve => {
setTimeout(resolve, t);
});
}
function log(...args) {
if (!log.start) {
log.start = Date.now();
}
const delta = Date.now() - log.start;
const deltaPad = (delta + "").padStart(6, "0");
console.log(`${deltaPad}: `, ...args)
}
class Transcribe {
constructor() {
this.greeting = "Hello";
this.cntr = 0;
}
async createJob(params) {
log(`createJob: ${this.greeting}`);
++this.cntr;
return delay(200);
}
async getJob(jobName) {
log(`getJob: ${this.greeting}`);
++this.cntr;
return delay(100);
}
}
const t = new Transcribe();
log("begin");
chain(t, ["getJob", "createJob"]).createJob().getJob().then(() => {
log(`cntr = ${t.cntr}`);
log("end");
});
Since this returns an actual promise (with additional methods attached), you can directly use .then() or await with it without the separate ._promise() that the first implementation required.
So, you can now do something like this:
const t = new Transcribe();
chain(t, ["getJob", "createJob"]).createJob().getJob().then(() => {
log(`cntr = ${t.cntr}`);
});
or:
const t = new Transcribe();
await chain(t, ["getJob", "createJob"]).createJob().getJob();
log(`cntr = ${t.cntr}`);
And, here's a third version where it creates a thenable object (a pseudo-promise) with the added methods on it (if it bothers you to add methods to an existing promise):
function chain(orig, methodsArray) {
if (!methodsArray || !methodsArray.length) {
throw new Error("methodsArray argument must be array of chainable method names");
}
let masterP = Promise.resolve();
function makeThenable() {
let obj = {};
for (const m of methodsArray) {
obj[m] = function(...args) {
// chain onto master promise to force sequencing
masterP = masterP.then(result => {
return orig[m].apply(orig, ...args);
});
return makeThenable();
}
}
obj.then = function(onFulfill, onReject) {
return masterP.then(onFulfill, onReject);
}
obj.catch = function(onReject) {
return masterP.catch(onReject);
}
obj.finally = function(onFinally) {
return masterP.finally(onFinally);
}
return obj;
}
return makeThenable();
}
function delay(t) {
return new Promise(resolve => {
setTimeout(resolve, t);
});
}
function log(...args) {
if (!log.start) {
log.start = Date.now();
}
const delta = Date.now() - log.start;
const deltaPad = (delta + "").padStart(6, "0");
console.log(`${deltaPad}: `, ...args)
}
class Transcribe {
constructor() {
this.greeting = "Hello";
this.cntr = 0;
}
async createJob(params) {
log(`createJob: ${this.greeting}`);
++this.cntr;
return delay(200);
}
async getJob(jobName) {
log(`getJob: ${this.greeting}`);
++this.cntr;
return delay(100);
}
}
const t = new Transcribe();
log("begin");
chain(t, ["getJob", "createJob"]).createJob().getJob().then(() => {
log(`cntr = ${t.cntr}`);
log("end");
});
In javascript if i have a function defined like so
function Person() {
}
Person.prototype.someFunction = function() {
// Does soome logic
return this
}
Person.prototype.anotherFunction = function () {
// Does soome logic
return this;
};
And i want to implement chaining i will do something like this
const person = new Person();
person.someFunction().anotherFunction()
And this works as each method returns the instance of Person.
Now if i have a method which has some async action how do i return the this instsance in an async method
function someApiCall() {
return new Promise((res) => {
setTimeout(() => {
res('got data');
}, 2000);
});
}
Person.prototype.asyncFunction = function () {
someApiCall()
.then()
.catch()
// HOW DO I RETURN THIS INSTANCE HERE ???? as someAPICALL is async
};
So that i can use it as
person.someFunction().asyncFunction().anotherFunction()
Option 1 (Not executed in order):
Person.prototype.asyncFunction1 = function () {
someApiCall()
.then((e) => console.log(e))
.catch()
return this;
};
p.anotherFunction().asyncFunction1().anotherFunction()
All functions get called, but not in order. If you want to execute it in order, just do it like this:
Option 2 (Executed in order):
Person.prototype.asyncFunction2 = async function () {
const ans = await someApiCall();
console.log(ans);
return this;
};
// t represents this as you return it in asyncFunction2
p.anotherFunction().asyncFunction2().then((t) => t.anotherFunction())
You're trying to apply a synchronous programming paradigm to an asynchronous code approach, and that's just not going to work. When working with promises, which is what async does for you in an automatic way, rather than instance chaining, your code logic now needs to deal with promise chaining instead.
First, let's stop using legacy prototype syntax and look at modern (where "modern" is over five years old by now) class syntax:
class Person {
async someFunction() {
return ...
}
async anotherFunction() {
return ...
}
}
Because async is just a convenient promise wrapping, we have two options:
const person = new Person();
person
.someFunction()
.then(result => {
person
.anotherFunction()
.then(result => ...);
.catch(e => console.error(e));
})
.catch(e => console.error(e));
but this is both cumbersome and ugly. Let's use awaits instead:
const person = new Person();
try {
const someResult = await person.someFunction();
const anotherResult = await person..anotherFunction();
...
} catch (e) {
console.error(e);
}
Much better. We don't need instance chaining anymore when we're using async patterns, it's a pattern from a previous era of JS, and writing modern code does not benefit from trying to force it back in.
Some people are telling you that it can never be done or it's just not going to work. It's not their fault for misunderstanding but you don't need to suffer the same way as them.
Let' say you have an ordinary class, Account, with a few async methods -
class Account {
constructor(balance) {
this.balance = balance
}
async withdraw (x) {
await sleep(1000)
this.balance -= x
}
async deposit (x) {
await sleep(1000)
this.balance += x
}
}
sleep is a simple function which delays the program for ms milliseconds -
const sleep = ms =>
new Promise(r => setTimeout(r, ms))
Now we can write a chain function -
const chain = t =>
new Proxy(Promise.resolve(t), { get: get(t) })
const get = t => (target, prop) => (...args) =>
prop === "then"
? target[prop](...args)
: chain(target.then(async v => (await v[prop](...args), v)))
This seemingly allows us to mix synchronous and asynchronous behaviour -
const A = new Account(100)
const B = new Account(200)
chain(A).deposit(5).withdraw(20).then(a => console.log("A", a))
chain(B).withdraw(20).withdraw(30).deposit(10).then(b => console.log("B", b))
Run the snippet below to verify the result in your own browser -
const sleep = ms =>
new Promise(r => setTimeout(r, ms))
const get = t => (target, prop) => (...args) =>
prop === "then"
? target[prop](...args)
: chain(target.then(async v => (await v[prop](...args), v)))
const chain = t =>
new Proxy(Promise.resolve(t), { get: get(t) })
class Account {
constructor(balance) {
this.balance = balance
}
async withdraw (x) {
await sleep(1000)
this.balance -= x
}
async deposit (x) {
await sleep(1000)
this.balance += x
}
}
const A = new Account(100)
const B = new Account(200)
chain(A).deposit(5).withdraw(20).then(a => console.log("A", a))
chain(B).withdraw(20).withdraw(30).deposit(10).then(b => console.log("B", b))
console.log("running...")
A { balance: 85 }
B { balance: 160 }
Invent your own convenience. It's your program to do with as you please.
I am building a javascript object where some of the values are defined by asynchronous functions. My problem is the object gets defined faster than the asynchronous functions can return values:
const array = ['a', 'b', 'c', 'd']
const myArrayObj = [];
function returnKey1 () {
// make async call then store it in the return key1val:
return key1val
}
function returnKey2 () {
// make async call then store it in the return key2val:
return key2val
}
function returnKey3 () {
// make async call then store it in the return key3val:
return key3val
}
_.forEach( array, function ( arr ) {
myArrayObj.push({
key1: returnKey1(), // returns undefined
key2: returnKey2(), // returns undefined
key3: returnKey3(), // returns undefined
});
});
Does anyone know the correct way I should be doing this? Thanks in advance!
The nature of asynchronicity is that you must wait for an async process to finish if you want to access its end result.
In your case, you could achieve this using promises with not a lot of code.:
// promise that resolves after 2 seconds
const timeoutPromise = (str) => new Promise(resolve => setTimeout(() => resolve(str), 2000));
// functions that return promises that will eventually resolve to appropriate key values
function returnKey1() {
return timeoutPromise('key3');
}
function returnKey2() {
return timeoutPromise('key2');
}
function returnKey3() {
return timeoutPromise('key3');
}
// helper function that returns a promise which will resolve with all the keys when all key-returning promises resolve
function getKeys() {
return Promise.all([
returnKey1(),
returnKey2(),
returnKey3()
])
}
// usage
getKeys().then((keys) => {
console.log(
keys[0],
keys[1],
keys[2]
);
});
The old-school approach would be to use callbacks instead of promises, which have larger browser support but are much cruder and primitive.
Note: With modern transpilers and/or promise libraries you can obtain wide browser support for promises as well.
// function that calls its callback after 2 seconds
const timeoutCallback = (cb, key) => setTimeout(() => cb(key), 2000);
// functions that eventually call their callbacks with appropriate key values
function returnKey1(cb) {
return timeoutCallback(cb, 'key1');
}
function returnKey2(cb) {
return timeoutCallback(cb, 'key2');
}
function returnKey3(cb) {
return timeoutCallback(cb, 'key3');
}
// helper function that calls its callback when all the keys are obtained
function getKeys(cb) {
let keys = [undefined, undefined, undefined];
let hasAllKeys = () => keys.every(key => typeof key === 'string');
function makeReturnKeyCallback(idx) {
return (key) => {
keys[idx] = key;
if (hasAllKeys()) {
cb(keys);
}
};
}
returnKey1(makeReturnKeyCallback(0));
returnKey2(makeReturnKeyCallback(1));
returnKey3(makeReturnKeyCallback(2));
}
// usage
getKeys((keys) => {
console.log(
keys[0],
keys[1],
keys[2]
);
});
Whatever you do will end up using promises. You can use Promise.all directly, or if you are willing to spend a bit more time you could write your own version of Promise.all for objects; finally, you could do this using async functions, which are a way to write promise-based code in a way which looks sort of synchronous.
Using Promise.all:
// Just examples for testing purposes.
function returnKey1() { return promise(); }
function returnKey2() { return promise(); }
function returnKey3() { return promise(); }
// Make a promise returning a random value after random time.
function promise() {
return new Promise(resolve => {
const random = Math.random() * 1000;
setTimeout(() => resolve(random), random);
});
}
// Wait for all promises to finish, then construct result.
Promise.all([returnKey1(), returnKey2(), returnKey3()])
.then(([key1, key2, key3]) => ({key1, key2, key3}))
.then(console.log);
Writing your own version of Promise.all for objects
We could also create an analog to Promise.all for objects. Some libraries have such a routine. We will call it promiseAllKeys. We will pass it an object each of whose property values in a promise; it returns a promise for an object with all the values filled in. This saves us the trouble of converting an array coming back from Promise.all into the desired object.
function returnKey1() { return promise(); }
function returnKey2() { return promise(); }
function returnKey3() { return promise(); }
function promise() {
return new Promise(resolve => {
const random = Math.random() * 1000;
setTimeout(() => resolve(random), random);
});
}
// Convenience routine to construct an object from arrays of keys and values.
function objectFromKeysAndValues(keys, values) {
const result = {};
for (const i = 0; i < keys.length; i++) result[keys[i]] = values[i];
return result;
}
function promiseAllKeys(promises) {
return Promise.all(Object.values(promises))
.then(values => objectFromKeysAndValues(Object.keys(promises), values));
}
promiseAllKeys({key1: returnKey1(), key2: returnKey2(), key3: returnKey3()})
.then(console.log);
Using async/await
You could simplify your code using async functions. However, this code as written will wait until each promise completes before running the next one.
function returnKey1() { return promise(); }
function returnKey2() { return promise(); }
function returnKey3() { return promise(); }
function promise() {
return new Promise(resolve => {
const random = Math.random() * 1000;
setTimeout(() => resolve(random), random);
});
}
async function makeObject() {
return {key1: await returnKey1(), key2: await returnKey2(), key3: await returnKey3()};
}
makeObject().then(console.log);
Simple but messy:
const array = ['a', 'b', 'c', 'd']
const myArrayObj = [];
function returnKey1 (cb) {
// make async call then store it in the return key1val:
cb ("key1val");
}
function returnKey2 (cb) {
// make async call then store it in the return key2val:
cb("key2val");
}
function returnKey3 (cb) {
// make async call then store it in the return key3val:
cb("key3val");
}
_.forEach( array, function ( arr ) {
var o ={};
returnKey1(function(key){
o.key1=key;
returnKey2(function(key){
o.key2=key;
returnKey3(function(key){
o.key3=key
myArrayObj.push(obj);
})
})
})
});
});
Inside _.mapValues I want get some modified values with some latency (for example from DB), but I was faced with problem: while I modify values in sync mode everything is good, when i try use promises or callback its work incorrectly (in first case I get Promise object, in second: undefined value).
Here some simplified example, how i can rewrite code inside mapValues to solve this problem?
'use strict';
const _ = require('lodash');
const Promise = require('bluebird');
let obj = {
one: 1,
two: 2,
};
let increment = (value) => {
return value + 1;
};
let incrementProm = (value) => {
return Promise.resolve(value + 1);
};
let incrementCb = (value, cb) => {
let res = value + 1;
let err = null;
setTimeout(cb.bind(undefined, err, res), 100);
};
let t1 = _.mapValues(obj, (value) => {
return increment(value);
});
let t2 = _.mapValues(obj, (value) => {
return incrementProm(value);
});
let t3 = _.mapValues(obj, (value) => {
let temp;
incrementCb(value, (err, res) => {
temp = res;
});
return temp;
});
console.log('Sync res:');
console.log(t1);
console.log('Promise res:');
console.log(t2);
console.log('Callback res:');
console.log(t3);
You can use bluebird's props() function to resolve all properties with promises.
Promise.props(_.mapValues(obj, incrementProm))
.then(result => console.log(result));
var obj = {
one: 1,
two: 2
};
var incrementProm = value => Promise.resolve(value + 1);
Promise.props(_.mapValues(obj, incrementProm))
.then(result => console.log(result));
<script src="https://cdn.jsdelivr.net/lodash/4.13.1/lodash.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/bluebird/3.4.1/bluebird.js"></script>
You're mapping onto promises, so something I do might be:
var _ = require('lodash');
let incrementProm = (value) => {
return Promise.resolve(value + 1);
};
let obj = {
foo: 1,
bar: 2
}
let keys = _.keys(obj);
let promises = _.map(keys, k => {
return incrementProm(obj[k])
.then(newValue => { return { key: k, value: newValue } });
})
Promise.all(promises).then(values => {
values.forEach(v => obj[v.key] = v.value)
})
.then(() => {
// now your object is updated: foo = 2 and bar = 3
console.log(obj);
});
You can implement an async version of mapValues. Something like...
async function mapValuesAsync(object, asyncFn) {
return Object.fromEntries(
await Promise.all(
Object.entries(object).map(async ([key, value]) => [
key,
await asyncFn(value, key, object)
])
)
);
}
Then call it like you would call _.mapValues:
let t2 = await mapValuesAsync(obj, (value) => {
return incrementProm(value);
});
or, simply
let t2 = await mapValuesAsync(obj, incrementProm);
Here's a TypeScript solution, built on top of lodash functions:
import { fromPairs, toPairs, ObjectIterator } from 'lodash';
export async function mapValuesAsync<T extends object, TResult>(
obj: T | null | undefined,
callback: ObjectIterator<T, Promise<TResult>>,
): Promise<{ [P in keyof T]: TResult }> {
return fromPairs(
await Promise.all(
toPairs(obj).map(async ([key, value]) => [
key,
await callback(value, key, obj),
]),
),
) as { [P in keyof T]: TResult };
}
If you don't need types, here's the JavaScript version:
import { fromPairs, toPairs } from 'lodash';
export async function mapValuesAsync(obj, callback) {
return fromPairs(
await Promise.all(
toPairs(obj).map(async ([key, value]) => [
key,
await callback(value, key, obj),
]),
),
);
}
And here's a test, just in case you need it:
import { mapValuesAsync } from './map-values-async';
describe('mapValuesAsync', () => {
it('should map values with an async callback', async () => {
const result = await mapValuesAsync(
{
a: 1,
b: 2,
},
async (value) => {
return await Promise.resolve(value * 2);
},
);
const expected = {
a: 2,
b: 4,
};
expect(result).toEqual(expected);
});
});
This is inspired by Rui Castro's response, posted above, which elegantly makes use of Object.entries and Object.fromEntries. However, that requires es2019 or later to work. Lodash has equivalent functions, toPairs and fromPairs, which work with any flavour of JavaScript.
Given
let arr = [1,2,3];
function filter(num) {
return new Promise((res, rej) => {
setTimeout(() => {
if( num === 3 ) {
res(num);
} else {
rej();
}
}, 1);
});
}
function filterNums() {
return Promise.all(arr.filter(filter));
}
filterNums().then(results => {
let l = results.length;
// length should be 1, but is 3
});
The length is 3 because Promises are returned, not values. Is there a way to filter the array with a function that returns a Promise?
Note: For this example, fs.stat has been replaced with setTimeout, see https://github.com/silenceisgolden/learn-esnext/blob/array-filter-async-function/tutorials/array-filter-with-async-function.js for the specific code.
Here is a 2017 elegant solution using async/await :
Very straightforward usage:
const results = await filter(myArray, async num => {
await doAsyncStuff()
return num > 2
})
The helper function (copy this into your web page):
async function filter(arr, callback) {
const fail = Symbol()
return (await Promise.all(arr.map(async item => (await callback(item)) ? item : fail))).filter(i=>i!==fail)
}
Demo:
// Async IIFE
(async function() {
const myArray = [1, 2, 3, 4, 5]
// This is exactly what you'd expect to write
const results = await filter(myArray, async num => {
await doAsyncStuff()
return num > 2
})
console.log(results)
})()
// Arbitrary asynchronous function
function doAsyncStuff() {
return Promise.resolve()
}
// The helper function
async function filter(arr, callback) {
const fail = Symbol()
return (await Promise.all(arr.map(async item => (await callback(item)) ? item : fail))).filter(i=>i!==fail)
}
I'll even throw in a CodePen.
As mentioned in the comments, Array.prototype.filter is synchronous and therefore does not support Promises.
Since you can now (theoretically) subclass built-in types with ES6, you should be able to add your own asynchronous method which wraps the existing filter function:
Note: I've commented out the subclassing, because it's not supported by Babel just yet for Arrays
class AsyncArray /*extends Array*/ {
constructor(arr) {
this.data = arr; // In place of Array subclassing
}
filterAsync(predicate) {
// Take a copy of the array, it might mutate by the time we've finished
const data = Array.from(this.data);
// Transform all the elements into an array of promises using the predicate
// as the promise
return Promise.all(data.map((element, index) => predicate(element, index, data)))
// Use the result of the promises to call the underlying sync filter function
.then(result => {
return data.filter((element, index) => {
return result[index];
});
});
}
}
// Create an instance of your subclass instead
let arr = new AsyncArray([1,2,3,4,5]);
// Pass in your own predicate
arr.filterAsync(async (element) => {
return new Promise(res => {
setTimeout(() => {
res(element > 3);
}, 1);
});
}).then(result => {
console.log(result)
});
Babel REPL Demo
For typescript folk (or es6 just remove type syntax)
function mapAsync<T, U>(array: T[], callbackfn: (value: T, index: number, array: T[]) => Promise<U>): Promise<U[]> {
return Promise.all(array.map(callbackfn));
}
async function filterAsync<T>(array: T[], callbackfn: (value: T, index: number, array: T[]) => Promise<boolean>): Promise<T[]> {
const filterMap = await mapAsync(array, callbackfn);
return array.filter((value, index) => filterMap[index]);
}
es6
function mapAsync(array, callbackfn) {
return Promise.all(array.map(callbackfn));
}
async function filterAsync(array, callbackfn) {
const filterMap = await mapAsync(array, callbackfn);
return array.filter((value, index) => filterMap[index]);
}
es5
function mapAsync(array, callbackfn) {
return Promise.all(array.map(callbackfn));
}
function filterAsync(array, callbackfn) {
return mapAsync(array, callbackfn).then(filterMap => {
return array.filter((value, index) => filterMap[index]);
});
}
edit: demo
function mapAsync(array, callbackfn) {
return Promise.all(array.map(callbackfn));
}
function filterAsync(array, callbackfn) {
return mapAsync(array, callbackfn).then(filterMap => {
return array.filter((value, index) => filterMap[index]);
});
}
var arr = [1, 2, 3, 4];
function isThreeAsync(number) {
return new Promise((res, rej) => {
setTimeout(() => {
res(number === 3);
}, 1);
});
}
mapAsync(arr, isThreeAsync).then(result => {
console.log(result); // [ false, false, true, false ]
});
filterAsync(arr, isThreeAsync).then(result => {
console.log(result); // [ 3 ]
});
Here's a way:
var wait = ms => new Promise(resolve => setTimeout(resolve, ms));
var filter = num => wait(1).then(() => num == 3);
var filterAsync = (array, filter) =>
Promise.all(array.map(entry => filter(entry)))
.then(bits => array.filter(entry => bits.shift()));
filterAsync([1,2,3], filter)
.then(results => console.log(results.length))
.catch(e => console.error(e));
The filterAsync function takes an array and a function that must either return true or false or return a promise that resolves to true or false, what you asked for (almost, I didn't overload promise rejection because I think that's a bad idea). Let me know if you have any questions about it.
var wait = ms => new Promise(resolve => setTimeout(resolve, ms));
var filter = num => wait(1).then(() => num == 3);
var filterAsync = (array, filter) =>
Promise.all(array.map(entry => filter(entry)))
.then(bits => array.filter(entry => bits.shift()));
filterAsync([1,2,3], filter)
.then(results => console.log(results.length))
.catch(e => console.error(e));
var console = { log: msg => div.innerHTML += msg + "<br>",
error: e => console.log(e +", "+ (e.lineNumber-25)) };
<div id="div"></div>
Promise Reducer to the rescue!
[1, 2, 3, 4].reduce((op, n) => {
return op.then(filteredNs => {
return new Promise(resolve => {
setTimeout(() => {
if (n >= 3) {
console.log("Keeping", n);
resolve(filteredNs.concat(n))
} else {
console.log("Dropping", n);
resolve(filteredNs);
}
}, 1000);
});
});
}, Promise.resolve([]))
.then(filteredNs => console.log(filteredNs));
Reducers are awesome. "Reduce my problem to my goal" seems to be a pretty good strategy for anything more complex than what the simple tools will solve for you, i.e. filtering an array of things that aren't all available immediately.
asyncFilter method:
Array.prototype.asyncFilter = async function(f){
var array = this;
var booleans = await Promise.all(array.map(f));
return array.filter((x,i)=>booleans[i])
}
Late to the game but since no one else mentioned it, Bluebird supports Promise.map which is my go-to for filters requiring aysnc processing for the condition,
function filterAsync(arr) {
return Promise.map(arr, num => {
if (num === 3) return num;
})
.filter(num => num !== undefined)
}
Two lines, completely typesafe
export const asyncFilter = async <T>(list: T[], predicate: (t: T) => Promise<boolean>) => {
const resolvedPredicates = await Promise.all(list.map(predicate));
return list.filter((item, idx) => resolvedPredicates[idx]);
};
In case someone is interested in modern typescript solution (with fail symbol used for filtering):
const failSymbol = Symbol();
export async function filterAsync<T>(
itemsToFilter: T[],
filterFunction: (item: T) => Promise<boolean>,
): Promise<T[]> {
const itemsOrFailFlags = await Promise.all(
itemsToFilter.map(async (item) => {
const hasPassed = await filterFunction(item);
return hasPassed ? item : failSymbol;
}),
);
return itemsOrFailFlags.filter(
(itemOrFailFlag) => itemOrFailFlag !== failSymbol,
) as T[];
}
There is a one liner to to do that.
const filterPromise = (values, fn) =>
Promise.all(values.map(fn)).then(booleans => values.filter((_, i) => booleans[i]));
Pass the array into values and the function into fn.
More description on how this one liner works is available here.
For production purposes you probably want to use a lib like lodasync:
import { filterAsync } from 'lodasync'
const result = await filterAsync(async(element) => {
await doSomething()
return element > 3
}, array)
Under the hood, it maps your array by invoking the callback on each element and filters the array using the result. But you should not reinvent the wheel.
You can do something like this...
theArrayYouWantToFilter = await new Promise(async (resolve) => {
const tempArray = [];
theArrayYouWantToFilter.filter(async (element, index) => {
const someAsyncValue = await someAsyncFunction();
if (someAsyncValue) {
tempArray.push(someAsyncValue);
}
if (index === theArrayYouWantToFilter.length - 1) {
resolve(tempArray);
}
});
});
Wrapped within an async function...
async function filter(theArrayYouWantToFilter) {
theArrayYouWantToFilter = await new Promise(async (resolve) => {
const tempArray = [];
theArrayYouWantToFilter.filter(async (element, index) => {
const someAsyncValue = await someAsyncFunction();
if (someAsyncValue) {
tempArray.push(someAsyncValue);
}
if (index === theArrayYouWantToFilter.length - 1) {
resolve(tempArray);
}
});
});
return theArrayYouWantToFilter;
}
A valid way to do this (but it seems too messy):
let arr = [1,2,3];
function filter(num) {
return new Promise((res, rej) => {
setTimeout(() => {
if( num === 3 ) {
res(num);
} else {
rej();
}
}, 1);
});
}
async function check(num) {
try {
await filter(num);
return true;
} catch(err) {
return false;
}
}
(async function() {
for( let num of arr ) {
let res = await check(num);
if(!res) {
let index = arr.indexOf(num);
arr.splice(index, 1);
}
}
})();
Again, seems way too messy.
A variant of #DanRoss's:
async function filterNums(arr) {
return await arr.reduce(async (res, val) => {
res = await res
if (await filter(val)) {
res.push(val)
}
return res
}, Promise.resolve([]))
}
Note that if (as in current case) you don't have to worry about filter() having
side effects that need to be serialized, you can also do:
async function filterNums(arr) {
return await arr.reduce(async (res, val) => {
if (await filter(val)) {
(await res).push(val)
}
return res
}, Promise.resolve([]))
}
Late to the party, and I know that my answer is similar to other already posted answers, but the function I'm going to share is ready for be dropped into any code and be used.
As usual, when you have to do complex operations on arrays, reduce is king:
const filterAsync = (asyncPred) => arr =>
arr.reduce(async (acc,item) => {
const pass = await asyncPred(item);
if(pass) (await acc).push(item);
return acc;
},[]);
It uses modern syntax so make sure your target supports it. To be 100% correct you should use Promise.resolve([]) as the initial value, but JS just doesn't care and this way it is way shorter.
Then you can use it like this:
var wait = ms => new Promise(resolve => setTimeout(resolve, ms));
const isOdd = x => wait(1).then(()=>x%2);
(filterAsync(isOdd)([1,2,3,4,4])).then(console.log) // => [1,3]
Here's a shorter version of #pie6k's Typescript version:
async function filter<T>(arr: T[], callback: (val: T) => Promise<Boolean>) {
const fail = Symbol()
const result = (await Promise.all(arr.map(async item => (await callback(item)) ? item : fail))).filter(i => i !== fail)
return result as T[] // the "fail" entries are all filtered out so this is OK
}
An efficient way of approaching this is by processing arrays as iterables, so you can apply any number of required operations in a single iteration.
The example below uses library iter-ops for that:
import {pipe, filter, toAsync} from 'iter-ops';
const arr = [1, 2, 3]; // synchronous iterable
const i = pipe(
toAsync(arr), // make our iterable asynchronous
filter(async (value, index) => {
// returns Promise<boolean>
})
);
(async function() {
for await (const a of i) {
console.log(a); // print values
}
})();
All operators within the library support asynchronous predicates when inside an asynchronous pipeline (why we use toAsync), and you can add other operators, in the same way.
Use of Promise.all for this is quite inefficient, because you block the entire array from any further processing that can be done concurrently, which the above approach allows.