I have a function getC which depends on the output of getB which depends on the output of getA. I want to create an observable stream, so that when I subscribe to it I get the output of each function, as shown in the snippet.
Is there an operator in RxJS that will give me the same behaviour as in the snippet? Or is nesting switchMap, concat and of the only way?
const { concat, from, of } = rxjs
const { switchMap } = rxjs.operators
const getA = async() => new Promise(resolve => setTimeout(() => resolve({
a: 'A'
}), 500));
const getB = async value => new Promise(resolve => setTimeout(() => resolve({
a: value.a,
b: 'B',
}), 500));
const getC = async value => new Promise(resolve => setTimeout(() => resolve({
a: value.a,
b: value.b,
c: 'C',
}), 500));
const stream = from(getA()).pipe(
switchMap(a => concat(
of(a),
from(getB(a)).pipe(
switchMap(b => concat(
of(b),
getC(b)
)
)
)
))
);
stream.subscribe(value => {
console.log(value);
});
<script src="https://unpkg.com/rxjs#7.3.0/dist/bundles/rxjs.umd.min.js"></script>
If you know your dependencies in advance you can hardcode them in the different streams:
c$ requires the latest emission from b$
b$ requires the latest emission from a$
Run the streams in "order" whilst respecting their dependencies so you end up with three emissions, each (except for the first) depending on the previous one.
const a$ = of('A').pipe(delay(600));
const b$ = of('B').pipe(delay(400), combineLatestWith(a$), map(([b, a]) => a + b));
const c$ = of('C').pipe(delay(200), combineLatestWith(b$), map(([c, b]) => b + c));
concat(a$, b$, c$).subscribe(x => console.log(x));
<script src="https://cdnjs.cloudflare.com/ajax/libs/rxjs/7.3.0/rxjs.umd.min.js" integrity="sha512-y3JTS47nnpKORJX8Jn1Rlm+QgRIIZHtu3hWxal0e81avPrqUH48yk+aCi+gprT0RMAcpYa0WCkapxe+bpBHD6g==" crossorigin="anonymous" referrerpolicy="no-referrer"></script>
<script>
const {of, concat} = rxjs;
const {delay, combineLatestWith, map} = rxjs.operators;
</script>
You can do some refactoring to make your function recursive
const getValue = (propertyName, value = null) => new Promise(resolve =>
resolve(value ? {...value, [propertyName]: propertyName.toUpperCase()} : {[propertyName]: propertyName.toUpperCase()})
);
Then you can use a magic rxjs operator EXPAND
import { from, EMPTY } from 'rxjs';
import { expand } from 'rxjs/operators';
from(getValue('a')).pipe(
expand((previousData) => {
if(!previousData.hasOwnProperty('b')) {
return from(getValue('b', previousData))
} else if(!previousData.hasOwnProperty('c')) {
return from(getValue('c', previousData))
} else {
return EMPTY
}
}
)
).subscribe(console.log);
HERE is a working code https://stackblitz.com/edit/typescript-czpotj
how can I add async & await to this sequence of filters to make sure that the results of one filter is returned before the next filter is called?
let rooms = [
{
type: "bedroom",
name: "Caroline",
utility: {
size: "90-150",
shape: "square",
}
let hasType = async (type, object) => object.type === type;
let resultType = rooms.filter(x => hasType("bedroom", x));
console.log(resultType);
let hasSize = (size, object) => object.utility.size === size;
let resultTypePlusSize = resultType.filter(x => hasSize("90-150", x));
console.log(resultTypePlusSize);
let hasShape = (shape, object) => object.utility.shape === shape;
let resultTypePlusSizePlusShape = resultTypePlusSize.filter(
x => hasShape("square", x));
console.log(resultTypePlusSizePlusShape);
You could use reduce to build up promise chains and then Promise.all to execute multiple:
function chain(...mappers) {
return {
exec(start) {
return mappers.reduce((value, fn) => {
if(value instanceof Promise) {
return value.then(fn);
} else {
return fn(value);
}
}, start);
},
many(values) {
return Promise.all(values.map(val => this.exec(val)));
}
};
}
To be used as:
const transformUsers = chain(
user => (user.name = "test", user),
async user => user.name,
console.log
);
transformUsers.many([{ }, { }]);
I need to return an additional value other than Observable, key string from flatMap from below setup:
this.subscribeInit = this.tradingService.getInProgress()
.pipe(flatMap((s) => {
this.keys = s.map((tdi) => tdi.key);
return this.keys;
}),
flatMap((key) =>{
var res = this.tradingService.getTradingData(key);//returns `Observable<any>`
return res;
}))
.subscribe(res => {
console.log('Key is : ' + res.key);
},
undefined,
() => this.onComplete());
Something like:
flatMap((key) =>{
var res = this.tradingService.getTradingData(key);
return {res,key};
}))
How to achieve this?
Just use map ...
flatMap((key) =>{
return this.tradingService.getTradingData(key).pipe(map((res) => {
return {res,key};
}));
}))
ps: i am a bit confused: isnt flatMap deprecated and replaced my mergeMap, switchMap, ... ?
edit: apparently the problem was caused by using flatMap instead of mergeMap.
You can destructur object as function parameters
this.subscribeInit = this.tradingService.getInProgress()
.pipe(flatMap((s) => {
this.keys = s.map((tdi) => tdi.key);
return this.keys;
}),
flatMap((key) =>{
var res = this.tradingService.getTradingData(key);
return {res,key}; // pack res and key into object
}))
.subscribe(({res,key}) => { // destructuring previously returned object
console.log('Key is : ' + res[key]);
},
undefined,
() => this.onComplete());
const val1 = 'some val1', val2 = 'some val2';
Promise.resolve({val1, val2}).then(({val1, val2}) => {
console.log(val1);
console.log(val2);
});
As I run this code, I wished to add the array final to the firestore -> sendGrid collection but it's always empty, although when I print it, it actually has the values.
I believe this is because of the timing issue, I always get [] -> value is just evaluated now (warning) and when I expand it it has the value.
function test() {
let today = new Date();
let addedDate = new Date(today.addDays(7));
let final = [];
let counter = 0;
let adder = new Promise(function (resolve, reject) {
db.collection("email").get()
.then((querySnapshot) => {
console.log(querySnapshot);
if (querySnapshot.empty !== true) {
querySnapshot.forEach((data) => {
console.log(data.data());
console.log(data.id);
let db2 = db.collection("email").doc(data.id);
let foodArr = [];
if (data.data() !== null) {
console.log(addedDate);
if (addedDate >= userList[0].exxpiaryDate) {
console.log("True");
}
db2.collection("list").where("expiaryDate", "<", addedDate.getTime()).get()
.then((list) => {
if (list.empty !== true) {
list.forEach((food) => {
if (food !== null) {
let temp = {
name: food.data().name,
time: food.data().expiaryDate,
};
foodArr.push(temp);
console.log(foodArr);
}
})
}
if (foodArr.length !== 0) {
let emailArr = {
email: data.data().email,
food: foodArr
};
console.log(emailArr);
final[counter] = (emailArr);
counter++;
console.log(final[0]);
}
}).catch((err) => {
console.log(err);
});
}
});
}
console.log(final);
resolve(final);
}).catch((err) => {
console.log(err);
});
});
return adder;
}
async function add() {
let add = await test();
console.log(add);
db.collection("sendGrid").add({
response: add
}).then((item) => {
console.log(item);
}).catch((err) => {
console.log(err);
});
}
Some points: (google if unsure why)
- prefer const
- return early
- clean code (eg. from console.log:s)
- cache fn calls
- functional programming is neat, look up Array.(map, filter, reduce, ...)
- destructuring is neat
- use arr[arr.length] = x; or arr.push(x), no need to manage your own counter
- short-circuit is sometimes neat (condition && expression; instead of if (condition) expression;)
- is queryResult.empty a thing? If it's a normal array, use !arr.length
- define variables in the inner most possible scope it's used in
- if having a promise in an async, make sure to return it
- prefer arrow functions
I changed the code to follow those points:
const test = ()=> {
const today = new Date();
const addedDate = new Date(today.addDays(7));
return new Promise((resolve)=> {
const final = [];
const emailsQuery = db.collection("email")
// an async/promise/then that's inside another promise, but not returned/awaited
emailsQuery.get().then((querySnapshot) => {
querySnapshot
.map(data=> ({id: data.id, data: data.data()}))
.filter(o=> o.data)
.forEach(({id, data: {email}}) => {
const db2 = db.collection("email").doc(id);
const itemsQuery = db2.collection("list").where("expiaryDate", "<", addedDate.getTime())
// another one!
itemsQuery.get().then((items) => {
const food = items.filter(o=> o).map(o=> o.data()).filter(o=> o)
.forEach(({name, expiaryDate: time})=> ({name, time}))
food.length && final.push({email, food})
}).catch(console.error);
});
// resolving before the two async ones have finished!!
resolve(final);
}).catch(console.error);
});
}
const add = async ()=> {
let response = await test();
return db.collection("sendGrid").add({response})
.then((item) => console.log('item:', item))
.catch(console.error)
}
Now, we can see that there is an issue with the async flow ("timing issue" in your words). I'll add one more best practice:
- use async/await when possible
Changing using that one makes it more clear, and solves the issue:
const test = async ()=> {
const today = new Date();
const addedDate = new Date(today.addDays(7));
const emailsQuery = db.collection("email")
const querySnapshot = await emailsQuery.get()
const emailEntries = querySnapshot
.map(data=> ({id: data.id, data: data.data()}))
.filter(o=> o.data)
// invoking an async fn -> promise; map returns the result of all invoked fns -> array of promises
const promisedItems = emailEntries.map(async ({id, data: {email}}) => {
const db2 = db.collection("email").doc(id);
const itemsQuery = db2.collection("list").where("expiaryDate", "<", addedDate.getTime())
const items = await itemsQuery.get()
const food = items.filter(o=> o).map(o=> o.data()).filter(o=> o)
.forEach(({name, expiaryDate: time})=> ({name, time}))
return {email, food}
});
const items = await Promise.all(promisedItems)
return items.filter(item=> item.food.length)
}
const add = async ()=> {
let response = await test();
return db.collection("sendGrid").add({response})
.then((item) => console.log('item:', item))
.catch(console.error)
}
Now, the flow is clear!
.
Even more concise (though lack of var names -> less clear) - just for kicks:
// (spelled-fixed expiryDate); down to 23% loc, 42% char
const getUnexpiredFoodPerEmails = async ({expiryDateMax} = {
expiryDateMax: new Date(new Date().addDays(7)),
})=> (await Promise.all((await db.collection('email').get())
.map(data=> ({id: data.id, data: data.data()})).filter(o=> o.data)
.map(async ({id, data: {email}})=> ({
email,
food: (await db.collection('email').doc(id).collection('list')
.where('expiryDate', '<', expiryDateMax.getTime()).get())
.filter(o=> o).map(o=> o.data()).filter(o=> o)
.forEach(({name, expiryDate: time})=> ({name, time})),
}))
)).filter(item=> item.food.length)
const add = async ()=> db.collection('sendGrid').add({
response: await getUnexpiredFoodPerEmails(),
}).then(console.log).catch(console.error)
// ...or with names
const getUnexpiredFoodListForEmailId = async ({id, expiryDateMax} = {
expiryDateMax: new Date(new Date().addDays(7)),
})=> (await db.collection('email').doc(id).collection('list')
.where('expiryDate', '<', expiryDateMax.getTime()).get())
.filter(o=> o).map(o=> o.data()).filter(o=> o)
.forEach(({name, expiryDate})=> ({name, time: expiryDate}))
const getEmails = async ()=> (await db.collection('email').get())
.map(data=> ({id: data.id, data: data.data()})).filter(o=> o.data)
const getUnexpiredFoodPerEmails = async ({expiryDateMax} = {
})=> (await Promise.all((await getEmails()).map(async ({id, data})=> ({
email: data.email,
food: await getUnexpiredFoodListForEmailId({id, expiryDateMax}),
})))).filter(item=> item.food.length)
const add = async ()=> db.collection('sendGrid').add({
response: await getUnexpiredFoodPerEmails(),
}).then(console.log).catch(console.error)
Given
let arr = [1,2,3];
function filter(num) {
return new Promise((res, rej) => {
setTimeout(() => {
if( num === 3 ) {
res(num);
} else {
rej();
}
}, 1);
});
}
function filterNums() {
return Promise.all(arr.filter(filter));
}
filterNums().then(results => {
let l = results.length;
// length should be 1, but is 3
});
The length is 3 because Promises are returned, not values. Is there a way to filter the array with a function that returns a Promise?
Note: For this example, fs.stat has been replaced with setTimeout, see https://github.com/silenceisgolden/learn-esnext/blob/array-filter-async-function/tutorials/array-filter-with-async-function.js for the specific code.
Here is a 2017 elegant solution using async/await :
Very straightforward usage:
const results = await filter(myArray, async num => {
await doAsyncStuff()
return num > 2
})
The helper function (copy this into your web page):
async function filter(arr, callback) {
const fail = Symbol()
return (await Promise.all(arr.map(async item => (await callback(item)) ? item : fail))).filter(i=>i!==fail)
}
Demo:
// Async IIFE
(async function() {
const myArray = [1, 2, 3, 4, 5]
// This is exactly what you'd expect to write
const results = await filter(myArray, async num => {
await doAsyncStuff()
return num > 2
})
console.log(results)
})()
// Arbitrary asynchronous function
function doAsyncStuff() {
return Promise.resolve()
}
// The helper function
async function filter(arr, callback) {
const fail = Symbol()
return (await Promise.all(arr.map(async item => (await callback(item)) ? item : fail))).filter(i=>i!==fail)
}
I'll even throw in a CodePen.
As mentioned in the comments, Array.prototype.filter is synchronous and therefore does not support Promises.
Since you can now (theoretically) subclass built-in types with ES6, you should be able to add your own asynchronous method which wraps the existing filter function:
Note: I've commented out the subclassing, because it's not supported by Babel just yet for Arrays
class AsyncArray /*extends Array*/ {
constructor(arr) {
this.data = arr; // In place of Array subclassing
}
filterAsync(predicate) {
// Take a copy of the array, it might mutate by the time we've finished
const data = Array.from(this.data);
// Transform all the elements into an array of promises using the predicate
// as the promise
return Promise.all(data.map((element, index) => predicate(element, index, data)))
// Use the result of the promises to call the underlying sync filter function
.then(result => {
return data.filter((element, index) => {
return result[index];
});
});
}
}
// Create an instance of your subclass instead
let arr = new AsyncArray([1,2,3,4,5]);
// Pass in your own predicate
arr.filterAsync(async (element) => {
return new Promise(res => {
setTimeout(() => {
res(element > 3);
}, 1);
});
}).then(result => {
console.log(result)
});
Babel REPL Demo
For typescript folk (or es6 just remove type syntax)
function mapAsync<T, U>(array: T[], callbackfn: (value: T, index: number, array: T[]) => Promise<U>): Promise<U[]> {
return Promise.all(array.map(callbackfn));
}
async function filterAsync<T>(array: T[], callbackfn: (value: T, index: number, array: T[]) => Promise<boolean>): Promise<T[]> {
const filterMap = await mapAsync(array, callbackfn);
return array.filter((value, index) => filterMap[index]);
}
es6
function mapAsync(array, callbackfn) {
return Promise.all(array.map(callbackfn));
}
async function filterAsync(array, callbackfn) {
const filterMap = await mapAsync(array, callbackfn);
return array.filter((value, index) => filterMap[index]);
}
es5
function mapAsync(array, callbackfn) {
return Promise.all(array.map(callbackfn));
}
function filterAsync(array, callbackfn) {
return mapAsync(array, callbackfn).then(filterMap => {
return array.filter((value, index) => filterMap[index]);
});
}
edit: demo
function mapAsync(array, callbackfn) {
return Promise.all(array.map(callbackfn));
}
function filterAsync(array, callbackfn) {
return mapAsync(array, callbackfn).then(filterMap => {
return array.filter((value, index) => filterMap[index]);
});
}
var arr = [1, 2, 3, 4];
function isThreeAsync(number) {
return new Promise((res, rej) => {
setTimeout(() => {
res(number === 3);
}, 1);
});
}
mapAsync(arr, isThreeAsync).then(result => {
console.log(result); // [ false, false, true, false ]
});
filterAsync(arr, isThreeAsync).then(result => {
console.log(result); // [ 3 ]
});
Here's a way:
var wait = ms => new Promise(resolve => setTimeout(resolve, ms));
var filter = num => wait(1).then(() => num == 3);
var filterAsync = (array, filter) =>
Promise.all(array.map(entry => filter(entry)))
.then(bits => array.filter(entry => bits.shift()));
filterAsync([1,2,3], filter)
.then(results => console.log(results.length))
.catch(e => console.error(e));
The filterAsync function takes an array and a function that must either return true or false or return a promise that resolves to true or false, what you asked for (almost, I didn't overload promise rejection because I think that's a bad idea). Let me know if you have any questions about it.
var wait = ms => new Promise(resolve => setTimeout(resolve, ms));
var filter = num => wait(1).then(() => num == 3);
var filterAsync = (array, filter) =>
Promise.all(array.map(entry => filter(entry)))
.then(bits => array.filter(entry => bits.shift()));
filterAsync([1,2,3], filter)
.then(results => console.log(results.length))
.catch(e => console.error(e));
var console = { log: msg => div.innerHTML += msg + "<br>",
error: e => console.log(e +", "+ (e.lineNumber-25)) };
<div id="div"></div>
Promise Reducer to the rescue!
[1, 2, 3, 4].reduce((op, n) => {
return op.then(filteredNs => {
return new Promise(resolve => {
setTimeout(() => {
if (n >= 3) {
console.log("Keeping", n);
resolve(filteredNs.concat(n))
} else {
console.log("Dropping", n);
resolve(filteredNs);
}
}, 1000);
});
});
}, Promise.resolve([]))
.then(filteredNs => console.log(filteredNs));
Reducers are awesome. "Reduce my problem to my goal" seems to be a pretty good strategy for anything more complex than what the simple tools will solve for you, i.e. filtering an array of things that aren't all available immediately.
asyncFilter method:
Array.prototype.asyncFilter = async function(f){
var array = this;
var booleans = await Promise.all(array.map(f));
return array.filter((x,i)=>booleans[i])
}
Late to the game but since no one else mentioned it, Bluebird supports Promise.map which is my go-to for filters requiring aysnc processing for the condition,
function filterAsync(arr) {
return Promise.map(arr, num => {
if (num === 3) return num;
})
.filter(num => num !== undefined)
}
Two lines, completely typesafe
export const asyncFilter = async <T>(list: T[], predicate: (t: T) => Promise<boolean>) => {
const resolvedPredicates = await Promise.all(list.map(predicate));
return list.filter((item, idx) => resolvedPredicates[idx]);
};
In case someone is interested in modern typescript solution (with fail symbol used for filtering):
const failSymbol = Symbol();
export async function filterAsync<T>(
itemsToFilter: T[],
filterFunction: (item: T) => Promise<boolean>,
): Promise<T[]> {
const itemsOrFailFlags = await Promise.all(
itemsToFilter.map(async (item) => {
const hasPassed = await filterFunction(item);
return hasPassed ? item : failSymbol;
}),
);
return itemsOrFailFlags.filter(
(itemOrFailFlag) => itemOrFailFlag !== failSymbol,
) as T[];
}
There is a one liner to to do that.
const filterPromise = (values, fn) =>
Promise.all(values.map(fn)).then(booleans => values.filter((_, i) => booleans[i]));
Pass the array into values and the function into fn.
More description on how this one liner works is available here.
For production purposes you probably want to use a lib like lodasync:
import { filterAsync } from 'lodasync'
const result = await filterAsync(async(element) => {
await doSomething()
return element > 3
}, array)
Under the hood, it maps your array by invoking the callback on each element and filters the array using the result. But you should not reinvent the wheel.
You can do something like this...
theArrayYouWantToFilter = await new Promise(async (resolve) => {
const tempArray = [];
theArrayYouWantToFilter.filter(async (element, index) => {
const someAsyncValue = await someAsyncFunction();
if (someAsyncValue) {
tempArray.push(someAsyncValue);
}
if (index === theArrayYouWantToFilter.length - 1) {
resolve(tempArray);
}
});
});
Wrapped within an async function...
async function filter(theArrayYouWantToFilter) {
theArrayYouWantToFilter = await new Promise(async (resolve) => {
const tempArray = [];
theArrayYouWantToFilter.filter(async (element, index) => {
const someAsyncValue = await someAsyncFunction();
if (someAsyncValue) {
tempArray.push(someAsyncValue);
}
if (index === theArrayYouWantToFilter.length - 1) {
resolve(tempArray);
}
});
});
return theArrayYouWantToFilter;
}
A valid way to do this (but it seems too messy):
let arr = [1,2,3];
function filter(num) {
return new Promise((res, rej) => {
setTimeout(() => {
if( num === 3 ) {
res(num);
} else {
rej();
}
}, 1);
});
}
async function check(num) {
try {
await filter(num);
return true;
} catch(err) {
return false;
}
}
(async function() {
for( let num of arr ) {
let res = await check(num);
if(!res) {
let index = arr.indexOf(num);
arr.splice(index, 1);
}
}
})();
Again, seems way too messy.
A variant of #DanRoss's:
async function filterNums(arr) {
return await arr.reduce(async (res, val) => {
res = await res
if (await filter(val)) {
res.push(val)
}
return res
}, Promise.resolve([]))
}
Note that if (as in current case) you don't have to worry about filter() having
side effects that need to be serialized, you can also do:
async function filterNums(arr) {
return await arr.reduce(async (res, val) => {
if (await filter(val)) {
(await res).push(val)
}
return res
}, Promise.resolve([]))
}
Late to the party, and I know that my answer is similar to other already posted answers, but the function I'm going to share is ready for be dropped into any code and be used.
As usual, when you have to do complex operations on arrays, reduce is king:
const filterAsync = (asyncPred) => arr =>
arr.reduce(async (acc,item) => {
const pass = await asyncPred(item);
if(pass) (await acc).push(item);
return acc;
},[]);
It uses modern syntax so make sure your target supports it. To be 100% correct you should use Promise.resolve([]) as the initial value, but JS just doesn't care and this way it is way shorter.
Then you can use it like this:
var wait = ms => new Promise(resolve => setTimeout(resolve, ms));
const isOdd = x => wait(1).then(()=>x%2);
(filterAsync(isOdd)([1,2,3,4,4])).then(console.log) // => [1,3]
Here's a shorter version of #pie6k's Typescript version:
async function filter<T>(arr: T[], callback: (val: T) => Promise<Boolean>) {
const fail = Symbol()
const result = (await Promise.all(arr.map(async item => (await callback(item)) ? item : fail))).filter(i => i !== fail)
return result as T[] // the "fail" entries are all filtered out so this is OK
}
An efficient way of approaching this is by processing arrays as iterables, so you can apply any number of required operations in a single iteration.
The example below uses library iter-ops for that:
import {pipe, filter, toAsync} from 'iter-ops';
const arr = [1, 2, 3]; // synchronous iterable
const i = pipe(
toAsync(arr), // make our iterable asynchronous
filter(async (value, index) => {
// returns Promise<boolean>
})
);
(async function() {
for await (const a of i) {
console.log(a); // print values
}
})();
All operators within the library support asynchronous predicates when inside an asynchronous pipeline (why we use toAsync), and you can add other operators, in the same way.
Use of Promise.all for this is quite inefficient, because you block the entire array from any further processing that can be done concurrently, which the above approach allows.