How to chain a list of promises in RXJS? - javascript

How can i chain a list of promises in RXJS? Every promise needs to be executed when the previous is resolved (work todo is stateful).
The way i'm doing it now feels primitive:
const workTodo = []; // an array of work
const allWork = Observable.create(observer => {
const next= () => {
const currentTodo = workTodo.shift();
if (currentTodo ) {
doTodoAsync(currentTodo)
.then(result => observer.onNext(result))
.then(next);
} else {
observer.onCompleted();
}
};
next();
});
I was thinking something like this:
const workTodo = []; // an array of work
const allWork = Observable
.fromArray(workTodo)
.flatMap(doTodoAsync);
But that basically executes all promises at once.

It seems you were pretty close with your attempt.
You may either specify maximum concurrency of 1 for .flatMap like:
Observable.fromArray(workTodo)
.flatMap(doTodoAsync, 1)
or equivalently use .concatMap instead of .flatMap:
Observable.fromArray(workTodo)
.concatMap(doTodoAsync)
I would use concatMap as it feels more idiomatic.
UPDATE: DEMO

How about some recursion?
First create a recursive function and call it recursiveDoToDo:
const recursiveDoToDo = (currentTodo, index) =>
Observable
.fromPromise(doTodoAsync(currentTodo))
.map(resolved => ({resolved, index}));
The code above simply wraps your doTodoAsync into an Observable, and then we map the results to return the resolved promise and the index of the array, for recursion use later.
Next, we will recursively call the recursiveDoToDo with the .expand() operator.
recursiveDoToDo(worktodo[0], 0)
.expand(res => recursiveDoToDo(worktodo[res.index + 1], res.index + 1))
.take(worktodo.length)
All you need to do for your recursion is just to increment the index by 1. Because .expand() will recursively run forever, the .take() operator is there to tell the observable when to end the stream, which is the length of your worktodo.
Now you can simply subscribe to it:
recursion.subscribe(x => console.log(x));
Here is the working JS Bin

Related

Promises being evaluated before Promise.all in nested forEach, resulting in empty Promise.all

I'm getting some unusual behaviour.
Basically as part of my code I have a function which utilizes nested for loops to build a promise and add it to a list of promises.
After the nested loops are complete, I'd like to evaluate the promise list using promise.all().
I have successfully managed to do this with a single forEach loop in the past, the nesting seems to cause some issues, namely, testing reveals that the Promise.all is being called before the nested forEach loop terminates, resulting in it being called on an empty list and hence returning an empty list.
I have a feeling that the issue is that I'm missing a return statement somewhere in my nested forEach loop as mentioned in this answer but I have not been able to determine where.
culprit.js
const otherModule = require("blablabla")
const otherOtherModule = require("blablabla2")
function nestedFunction(list){
var promises = [];
list.forEach(element => {
otherModule.getSublist(element).then(sublist => {
sublist.forEach(subelement => {
promises.push(otherOtherModule.promiseResolvingFunction(subelement));
});
});
});
return Promise.all(promises);
}
module.exports = {
nestedFunction : nestedFunction
}
culprit.test.js
const culprit = require("culpritpath")
// for mocking
const otherModule = require("blablabla")
otherModule.getSublist = jest.fn(() => Promise.resolve([{}, {}, {}]))
const otherOtherModule = require("blablabla2")
otherOtherModule.promiseResolvingFunction = jest.fn(() => Promise.resolve())
describe("nestedFunction()", ()=>{
it("returns an array of resolved promises", () => {
return culprit.nestedFunction([{}, {}]).then(res => {
expect(res).toHaveLength(6);
})
})
})
instead I get that res is []. Further tests show that promiseResolvingFunction is being called the right number of times, so as I understand, Promise.all is being called before the nested forEach loop finishes.
PS: I am still getting started with promises and TDD, I am more than happy to hear feedback on any code smell.
Yeah so the problem I see is that your for each loop is calling asynchronous code and expecting it to execute synchronously.
I'd probably do something like...
var promises = list.map(element => {
return otherModule.getSublist(element).then(sublist => {
// Map all of the sublists into a promise
return Promise.all(sublist.map(subelement => {
return otherOtherModule.promiseResolvingFunction(subelement));
}));
});
});
return Promise.all(promises);
Of course then you'd end up with an array of arrays. If you wanted to keep the result a flat array of sublist items, another option would be to first fetch all of your lists, then fetch all of your sublists from those results...
return Promise.all(list.map( element => otherModule.getSublist(element)))
.then((sublists) => {
let subListPromises = [];
// Loop through each sublist, turn each item in it into a promise
sublists.forEach( sublist => {
sublistPromises = [
...sublistPromises,
sublist.map( subelement => otherOtherModule.promiseResolvingFunction(subelement))
]
})
// Return a promise dependent on *all* of the sublist elements
return Promise.all(sublistPromises)
})
You execute Promise.all before the array has been populated (which happens asynchronously).
It may look difficult to deal with nested promises, but just apply Promise.all to the inner arrays of promises, and then at the outer level, apply Promise.all to all those from the inner level.
Then you're not ready yet, as now you have a promise that resolves to an array of arrays (corresponding to the originally nested promises), so you need to flatten that with the quite new .flat method, or with [].concat:
function nestedFunction(list) {
// Get promise for the array of arrays of sub values
return Promise.all(list.map(element => {
return otherModule.getSublist(element).then(sublist => {
// Get promise for the array of sub values
return Promise.all(sublist.map(subelement => {
return otherOtherModule.promiseResolvingFunction(subelement);
}));
});
})).then(matrix => [].concat(...matrix)); // flatten the 2D array
}
You need to nest your promise resolution. Something like this:
const otherModule = require("blablabla")
const otherOtherModule = require("blablabla2")
function nestedFunction(list){
var promises =
list.map(element => {
return otherModule.getSublist(element).then(sublist => {
return Promise.all(
sublist.map(subelement => {
return otherOtherModule.promiseResolvingFunction(subelement);
})
);
});
});
return Promise.all(promises);
}
module.exports = {
nestedFunction : nestedFunction
}

Chaining http calls in angular 2 in a for loop

I have some code that looks like
//service.ts
addProduct(productId) {
this.http.post('someUrl', ReqData).map(json).subscribe(doStuff);
}
//component.ts
addAllproducts(productsIds) {
productIds.forEach(productId => service.addProduct(productId);
}
What I want is to be able to wait for each call to finish before calling for the next productId, without using window.setTimeout ..
How about some recursive calls using .expand()?
First, create a recursive function and map the data for recursive use:
const recursiveAddProduct = (currentProductId, index, arr)=>{
return service.addProduct(currentProductId)
.map((response)=>{
return {
data:response,
index: index+1,
arr:arr
}
})
};
Now, call it recursively in your component:
//productIds is an array of Ids
//start of using the first index of item, where index = 0
let reduced = recursiveAddProduct(productIds[0],0,productIds)
.expand((res)=>{
return res.index>res.arr.length-1 ? Observable.empty(): recursiveAddProduct(productIds[res.index],res.index,productIds)
});
reduced.subscribe(x=>console.log(x));
Here is a working JSBin
Benefit of using .expand operator:
You are still using Observables and can chain whatever operators you want to.
You are calling one http after another, which is your requirement.
You don't need to worry about error handling, they are all chained to a single stream. Just call a .catch to your observables.
You can do anything to your recursion method (data manipulation,etc)
You can set the condition when to terminate the recursion call.
One-liner (almost) code.
Edit
You can use .take() operator to terminate your recursion, if you don't like the inline ternary, like this:
let reduced = recursiveAddProduct(productIds[0],0,productIds)
.expand(res=>recursiveAddProduct(productIds[res.index],res.index,productIds))
.take(productIds.length)
Working JSBin
First return the observable from your service method:
addProduct(productId) {
return this.http.post('someUrl', ReqData).map(json).subscribe(doStuff);
}
And use a recursive function and call it in the subscribe callback for each of the items in your array:
let loop = (id: number) => {
service.addProduct(id)
.subscribe((result) => {
// This logic can be modified to any way you want if you don't want to mutate the `producIds` array
if (productIds.length) {
loop(productIds.shift())
}
})
}
loop(productIds.shift())
You can use the Observable.merge().
Try something like that
addProduct(productId):Observable<Response> {
return this.http.post('someUrl', productId);
}
addAllproducts(productsIds) {
let productedsObservable:Observable<Response>[]=[];
for(let productID in productsIds){
productedsObservable.push(this.addProduct(productID));
}
return merge(productedsObservable)
}
You need to subscribe to the requested function for it the execute the HTTP request.
You can read more about combination operators (like merge) here

How to execute promises in series?

var promiseReturningFuncs = [];
for(var i = 0; i < 5; i++){
promiseReturningFuncs.push(askQuestion);
}
var programmers = [];
Promise.reduce(promiseReturningFuncs, function(resp, x) {
console.log(typeof resp);
if(typeof resp != "function") {
programmers.push(resp);
}
return x();
})
.then(function(resp) {
programmers.push(resp);
console.log(programmers);
});
My goal: execute the askQuestion function in series and resolve an array of objects created by that function. (this function must execute in series so that it can respond to user input)
So imagine that the askQuestion function returns a promise that resolves a object I want to add to an array.
This is my messy way of doing it.
I am looking to find a cleaner way of doing it, ideally, i wouldn't even need to push to an array, I would just have a final .then, where the response is an array.
Since you appear to be using the Bluebird promise library, you have a number of built-in options for sequencing your promise returning functions. You can use Promise.reduce(), Promise.map() with a concurrency value of 1, Promise.mapSeries or Promise.each(). If the iterator function returns a promise, all of these will wait for the next iteration until that promise resolves. Which to use depends more upon the mechanics of how your data is structured and what result you want (neither of which you actually show or describe).
Let's suppose you have an array of promise returning functions and you want to call them one at a time, waiting for the one to resolve before calling the next one. If you want all the results, then I'd suggest Promise.mapSeries():
let arrayOfPromiseReturningFunctions = [...];
// call all the promise returning functions in the array, one at a time
// wait for one to resolve before calling the next
Promise.mapSeries(arrayOfPromiseReturningFunctions, function(fn) {
return fn();
}).then(function(results) {
// results is an array of resolved results from all the promises
}).catch(function(err) {
// process error here
});
Promise.reduce() could also be used, but it would accumulate a single result, passing it from one to the next and end with one final result (like Array.prototype.reduce() does).
Promise.map() is a more general version of Promise.mapSeries() that lets you control the concurrency number (the number of async operations in flight at the same time).
Promise.each() will also sequence your functions, but does not accumulate a result. It assumes you either don't have a result or you are accumulating the result out-of-band or via side effects. I tend to not like to use Promise.each() because I don't like side effect programming.
You could solve this in pure JS using ES6 (ES2015) features:
function processArray(arr, fn) {
return arr.reduce(
(p, v) => p.then((a) => fn(v).then(r => a.concat([r]))),
Promise.resolve([])
);
}
It applies the function given to the array in series and resolves to an array of the results.
Usage:
const numbers = [0, 4, 20, 100];
const multiplyBy3 = (x) => new Promise(res => res(x * 3));
// Prints [ 0, 12, 60, 300 ]
processArray(numbers, multiplyBy3).then(console.log);
You'll want to double check browser compatibility but this works on reasonably current Chrome (v59), NodeJS (v8.1.2) and probably most others.
You can use recursion so that you can move to the next iteration in a then block.
function promiseToExecuteAllInOrder(promiseReturningFunctions /* array of functions */) {
var resolvedValues = [];
return new Promise(function(resolve, reject) {
function executeNextFunction() {
var nextFunction = promiseReturningFunctions.pop();
if(nextFunction) {
nextFunction().then(function(result) {
resolvedValues.push(result);
executeNextFunction();
});
} else {
resolve(resolvedValues);
}
}
executeNextFunction();
}
}
Executing one after another using a recursive function( in a non promise way):
(function iterate(i,result,callback){
if( i>5 ) callback(result);askQuestion().then(res=>iterate(i+1,result.concat([res]),callback);
})(0,[],console.log);
For shure this can be wrapped in a promise:
function askFive(){
return new Promise(function(callback){
(function iterate(i,result){
if( i>5 ) callback(result);askQuestion().then(res=>iterate(i+1,result.concat([res]),callback);
})(0,[],console.log);
});
}
askFive().then(console.log);
Or:
function afteranother(i,promise){
return new Promise(function(resolve){
if(!i) return resolve([]);
afteranother(i-1,promise).then(val=>promise().then(val2=>resolve(val.concat([val2])));
});
}
afteranother(5,askQuestion).then(console.log);

How can I fill the array after forEach loop

const tileApiPromises = [];
response.data.forEach((tile) => {
return getPartnerToken(tile.id).then((response) => {
tileApiPromises.push(getTileContent(tile, response));
});
});
console.log(tileApiPromises) // should give me an array of promises
Promise.all(tileApiPromises) // The goal is to execute this.
I am getting empty array of course in the log. How can I get array with promises outside the forEach. Thanks for your help!
The problem with your code is that the push is done asynchronously - therefore, there's nothing in the array yet!
you say you tried map (in the comment) - did you try it like this?
const tileApiPromises = response.data.map((tile) => {
return getPartnerToken(tile.id).then((response) => {
return getTileContent(tile, response);
});
});
or, sexier
const tileApiPromises = response.data.map(tile =>
getPartnerToken(tile.id).then(response => getTileContent(tile, response))
);
You can use Array.map function instead of forEach.
forEach always return undefined.
while map function iterates over array provided and returns accumulated array of values returned in callback provided in map function.
Here is nice documentation on map function.
https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Global_Objects/Array/map

While loop using bluebird promises

I am trying to implement a while loop using promises.
The method outlined here seems to work.
http://blog.victorquinn.com/javascript-promise-while-loop
it uses a function like this
var Promise = require('bluebird');
var promiseWhile = function(condition, action) {
var resolver = Promise.defer();
var loop = function() {
if (!condition()) return resolver.resolve();
return Promise.cast(action())
.then(loop)
.catch(resolver.reject);
};
process.nextTick(loop);
return resolver.promise;
};
This seems to use anti-patterns and deprecated methods like cast and defer.
Does anyone know a better or more modern way to accomplish this?
Thanks
cast can be translated to resolve. defer should indeed not be used.
You'd create your loop only by chaining and nesting then invocations onto an initial Promise.resolve(undefined).
function promiseWhile(predicate, action, value) {
return Promise.resolve(value).then(predicate).then(function(condition) {
if (condition)
return promiseWhile(predicate, action, action());
});
}
Here, both predicate and action may return promises. For similar implementations also have a look at Correct way to write loops for promise. Closer to your original function would be
function promiseWhile(predicate, action) {
function loop() {
if (!predicate()) return;
return Promise.resolve(action()).then(loop);
}
return Promise.resolve().then(loop);
}
I prefer this implementation as its easier to simulate break and continue with it:
var Continue = {}; // empty object serves as unique value
var again = _ => Continue;
var repeat = fn => Promise.try(fn, again)
.then(val => val === Continue && repeat(fn) || val);
Example 1: stops when either the source or the destination indicate an error
repeat(again =>
source.read()
.then(data => destination.write(data))
.then(again)
Example 2: stop randomly if the coin flip given 90% probability results with a 0
var blah = repeat(again =>
Promise.delay(1000)
.then(_ => console.log("Hello"))
.then(_ => flipCoin(0.9) && again() || "blah"));
Example 3: Loop with condition that returns the sum:
repeat(again => {
if (sum < 100)
return fetchValue()
.then(val => sum += val)
.then(again));
else return sum;
})

Categories