We Keep Coding

RxJS `repeatWhen` with notifier repeats rapidly

I am trying to make a network retry-er using RxJS repeatWhen operator. The idea is that when a new request receives to the scheduler, then I try the request directly and if it results in a network failure result, I add it to a pool to be retried on sometime later. So the entering point of my scheduler is the queue function which does the job like this:
queue({ operation, body }) {
const behaviourSubject = new BehaviorSubject();
const task = {
operation,
body,
behaviourSubject,
};
this.doTask(task).subscribe({
error: _ => this.notifier.next({ tasks: [task], remove: false }),
complete: () => console.log('Task successfully done on first try: ', task),
});
return behaviourSubject;
}
and this.notifier is a Subject which is used as the notifier of the worker. So the worker itself is like this:
const rawWorker = new Observable(subscriber => {
const doneTasks = [];
const jobs = [];
for (const task of this.tasks) {
jobs.push(
this.doTask(task).pipe(
tap(_ => {
doneTasks.push(task);
}),
catchError(error => { task.behaviourSubject.next(error); return of(error); }),
)
);
}
if (jobs.length > 0) {
forkJoin(...jobs).subscribe(_ => {
this.notifier.next({ tasks: doneTasks, remove: true });
subscriber.next(`One cycle of worker done. #${doneTasks.length} task(s) done and #${this.tasks.length} remaining.`);
// subscriber.complete();
if (this.tasks.length > 0) {
this.notifier.next();
}
})
} else {
subscriber.complete();
}
});
const scheduledWorker = rawWorker.pipe( // TODO: delay should be added to retry and repeat routines
retry(),
repeatWhen(_ => this.notifier.pipe(
filter(_ => this.tasks.length > 0),
)),
);
and also the notifier keeps track of all undone requests in an array like this:
this.notifierSubscription = this.notifier
.pipe(
filter(data => data && data.tasks)
)
.subscribe({
next: ({ tasks = [], remove = false }) => {
if (remove) {
console.log('removing tasks: ', tasks);
this.tasks = this.tasks.filter(task => !tasks.some(tsk => task === tsk));
} else {
console.log('inserting: ', tasks);
this.tasks.push.apply(
this.tasks,
tasks,
);
}
console.log('new tasks array: ', this.tasks);
}
});
As I know if a cycle of the worker is not completed then repeatWhen has nothing to do. For example if I remove the part:
else {
subscriber.complete();
}
from the worker, on the first try of the worker (an empty cycle) the Observable does not complete and repeatWhen won't do anything. But on the other hand as you see I have commented // subscriber.complete(); when there exist jobs but the repeat is occurring. And the worst part of the problem is that many different instances of the worker run in parallel which makes many duplicate requests.
I have spent a lot of time on this problem but do not have any clue to trace.

Async/Await inside the Observable

How can I use async/await inside the Observable??
With this code I'm unable to trigger the unsubscribe function within observable thus interval is not cleared.
const { Observable } = require("rxjs");
const test = () => new Observable(async (subscriber) => {
await Promise.resolve();
const a = setInterval(() => {
subscriber.next(Math.random());
console.log("zz");
}, 500);
return () => {
console.log("asdsad");
clearInterval(a);
};
});
const xyz = test().subscribe(console.log);
setTimeout(() => {
xyz.unsubscribe();
}, 3000);

Async/Await inside an observable is not supported. However, it can be done with a behavior subject and an asynchronous nested function.
Create a behavior subject, convert it to an observable (.asObservable()), execute the asynchronous nested function, return the observable. Here's an example.
function getProgress() {
// Change this value with latest details
const value = new BehaviorSubject('10%');
const observable = value.asObservable();
// Create an async function
const observer = async() => {
// Perform all tasks in here
const wait1 = await new Promise(resolve => setTimeout(resolve, 3000));
value.next('66%');
const wait2 = await new Promise(resolve => setTimeout(resolve, 3000));
value.next('100%');
// Complete observable
value.complete();
}
// Call async function & return observable
observer();
return observable;
}
It's very readable and works like a charm.

First of all, subscriber passed to observable contructor cannot be async function. There is no support for that.
If you need to create observable from promise, use from:
import { from } from 'rxjs';
const observable = from(promise);
But considering your scenario.
Because there is no way to cancel native js promise, you cannot realy unsubscribe from such created observable, so:
const obs = from(new Promise(resolve => {
setTimeout(() => {
console.log('gonna resolve');
resolve('foo');
}, 1000);
}));
const sub = obs.subscribe(console.log);
setTimeout(() => sub.unsubscribe(), 500);
will print:
gonna resolve
gonna resolve
gonna resolve
(...)
so yeah: gonna resolve will be printed in the cosole all the time, but nothing more - result passed to resolve will be ignored - just not logged.
From the other hand, if you remove that unsubscribtion (setTimeout(() => sub.unsubscribe(), 500);) this time you will see:
gonna resolve
foo
gonna resolve
gonna resolve
gonna resolve
(...)
There is one way that maybe will help you - defer - but it's not strictly related with your question.
import { defer } from 'rxjs';
defer(async () => {
const a = await Promise.resolve(1);
const b = a + await Promise.resolve(2);
return a + b + await Promise.resolve(3);
}).subscribe(x => console.log(x)) // logs 7

run a callback function after forEach loop finish

how to callback timer() function after forEach loop is finished, using the same code. or is there is a better way to loop through each user with delay then after the loop is done, the timer() is called back using forEach.
const usersProfile = () => {
let interval = 1000;
let promise = Promise.resolve();
db.select('*').from('users')
.returning('*')
.then(users => {
users.forEach((user, index) => {
setTimeout(function(){
}, index * 1000)
db.select('*').from(`${user.name}`)
.returning('*')
.then(userdata => {
userdata.forEach(data => {
promise = promise.then(function(){
if(data.status === 'online'){
console.log(`${data.name} ${data.items} ${data.profile} ${data.images}`)
}
return new Promise(function(resolve){
setTimeout(function(){
resolve();
}, interval)
})
})
})
})
})
timer();
})
}
const timer = () => {
setTimeout(usersProfile, 1000)
}
timer();
===============ALL THE ABOVE ARE MY OLD CODE ================
but thanks to https://stackoverflow.com/users/2404452/tho-vu it solved most of the problem but can i do this to serve the purpose of the app
const usersProfile = async () => {
let interval = 1000;
const delayPromise = (data, delayDuration) => {
return new Promise((resolve) => {
setTimeout(() => {
if(data.status === 'online'){
console.log(`${data.name} ${data.items} ${data.profile} ${data.images}`);
resolve();
}
}, delayDuration)
});
};
const userInfo = (data, delayDuration) => {
return new Promise((resolve) => {
setTimeout(() => {
console.log(`${data.info}`);//info about user from his table each user has his own table besides users table that has only the user tables
resolve();
}, delayDuration)
});
};
try {
const userData = await db.select('*').from('users').returning('*');
const promises = userData.map((data, index) => delayPromise(data, index * interval));
const userData = await db.select('*').from(`${data.name}`).returning('*');
const info = userData.map((data, index) => userInfo(data, index * interval));
await Promise.all(promises);
// here you can write your effects
} catch (e) {
console.log(e);
}
}

Another approach using async-await to avoid callback hell.
const usersProfile = async () => {
let interval = 1000;
const delayPromise = (data, delayDuration) => {
return new Promise((resolve) => {
setTimeout(() => {
if(data.status === 'online'){
console.log(`${data.name} ${data.items} ${data.profile} ${data.images}`);
resolve();
}
}, delayDuration)
});
};
try {
const userData = await db.select('*').from('users').returning('*');
const promises = userData.map((data, index) => delayPromise(data, index * interval));
await Promise.all(promises);
// here you can write your effects
} catch (e) {
console.log(e);
}
}

it is hard for me to fully understand what you wanted to accomplish through the code, but I will try to explain what you can do to solve the issue.
First of all, if you want to wait for multiple promises, you should use Promise.all and not promise = promise.then
You can do this as so:
let promises = [];
users.forEach((user, index) => {
let promise = db.select(*).from('users') //should return a promise
promises.push(promise);
});
//promises have already started to execute (in parallel)
Promise.all(promises)
.then(() => console.log('all promises finished successfully'))
.catch((e) => console.error('received error at one of the promises'));
// when the program arrives here we know for a fact that either all promises executed successfully or one of the promises failed
timer();
Explanation: because promises execute asynchronously the forEach() function does not wait for any of the promises created to finish, so the solution is to wait for all of them at then after the entire loop.
This means the promises are created, and are being executed while the forEach() loop executes, in parallel, and when the program reaches Promise.all it stops and waits for all of them to finish.
Second, I would strongly advice you to use functions as a way to simplify your code, that way it would be easier for you to grasp every thing that is happening, even if its complicated.
Good luck !

How to stop the execution of foreach until the end of the API call it has inside?

I'm calling the service sending an array of ids as a parameter. In the function I need to do a foreach of this ids and make an api call for each one to get the full info of this element and after that do something with that info.
so how can I stop the execution of foreach until the end of the API call it has inside?
The function:
addCandidate(ids: any, candidate: string) {
ids.forEach(elem => {
this.getSearch(elem).subscribe(res => {
let currentCandidates = res.candidates;
if(currentCandidates) {
if(!currentCandidates.includes(candidate)){
currentCandidates.push(candidate);
this.itemDoc.update({candidates: currentCandidates});
}
}else{
this.itemDoc.update({candidates: [candidate]});
}
})
});
}
Thanks!!

Here is an Example how to use promise and map.
let doSomthingToWaitFor = (ms) => {
return new Promise(resolve => setTimeout(() => resolve('r-' + ms), ms))
}
(async (ids) => {
await Promise.all(ids.map(async id => {
let res = await doSomthingToWaitFor(id);
console.log("my res: %s from id: %s", res, id);
}));
console.log("Successfully waited");
})([1000, 2000, 3000, 4000]);
You can implement your request in a function like doSomthingToWaitFor and handle the result afterwards.

It sounds like you want to map an array to a set of promises, and then wait for all that concurrent work to finish before doing something else.
I think you will find map, promise and async/await very helpful here, e.g.
const ids = [1, 2];
const work = await Promise.all(
ids.map(ajaxCall)
);
// 'await' means that this won't be called until the Promise.all is finished
printToScreen(work);

Chaining Observables in RxJS

I'm learning RxJS and Angular 2. Let's say I have a promise chain with multiple async function calls which depend on the previous one's result which looks like:
var promiseChain = new Promise((resolve, reject) => {
setTimeout(() => {
resolve(1);
}, 1000);
}).then((result) => {
console.log(result);
return new Promise((resolve, reject) => {
setTimeout(() => {
resolve(result + 2);
}, 1000);
});
}).then((result) => {
console.log(result);
return new Promise((resolve, reject) => {
setTimeout(() => {
resolve(result + 3);
}, 1000);
});
});
promiseChain.then((finalResult) => {
console.log(finalResult);
});
My attempts at doing the same solely using RxJS without the use of promises produced the following:
var observableChain = Observable.create((observer) => {
setTimeout(() => {
observer.next(1);
observer.complete();
}, 1000);
}).flatMap((result) => {
console.log(result);
return Observable.create((observer) => {
setTimeout(() => {
observer.next(result + 2);
observer.complete()
}, 1000);
});
}).flatMap((result) => {
console.log(result);
return Observable.create((observer) => {
setTimeout(() => {
observer.next(result + 3);
observer.complete()
}, 1000);
});
});
observableChain.subscribe((finalResult) => {
console.log(finalResult);
});
It yields the same output as the promise chain. My questions are
Am I doing this right? Are there any RxJS related improvements that I can make to the above code
How do I get this observable chain to execute repeatedly? i.e. Adding another subscription at the end just produces an additional 6 though I expect it to print 1, 3 and 6.
observableChain.subscribe((finalResult) => {
console.log(finalResult);
});
observableChain.subscribe((finalResult) => {
console.log(finalResult);
});
1
3
6
6

About promise composition vs. Rxjs, as this is a frequently asked question, you can refer to a number of previously asked questions on SO, among which :
How to do the chain sequence in rxjs
RxJS Promise Composition (passing data)
RxJS sequence equvalent to promise.then()?
Basically, flatMap is the equivalent of Promise.then.
For your second question, do you want to replay values already emitted, or do you want to process new values as they arrive? In the first case, check the publishReplay operator. In the second case, standard subscription is enough. However you might need to be aware of the cold. vs. hot dichotomy depending on your source (cf. Hot and Cold observables : are there 'hot' and 'cold' operators? for an illustrated explanation of the concept)

Example for clarification:
Top of pipe can emit n values (this answers "How do I get this observable chain to execute repeatedly"), but subsequent chained streams emit one value (hence mimicing promises).
// Emit three values into the top of this pipe
const topOfPipe = of<string>('chaining', 'some', 'observables');
// If any of the chained observables emit more than 1 value
// then don't use this unless you understand what is going to happen.
const firstObservablePipe = of(1);
const secondObservablePipe = of(2);
const thirdObservablePipe = of(3);
const fourthObservablePipe = of(4);
const addToPreviousStream = (previous) => map(current => previous + current);
const first = (one) => firstObservablePipe.pipe(addToPreviousStream(one));
const second = (two) => secondObservablePipe.pipe(addToPreviousStream(two));
const third = (three) => thirdObservablePipe.pipe(addToPreviousStream(three));
const fourth = (four) => fourthObservablePipe.pipe(addToPreviousStream(four));
topOfPipe.pipe(
mergeMap(first),
mergeMap(second),
mergeMap(third),
mergeMap(fourth),
).subscribe(console.log);
// Output: chaining1234 some1234 observables1234
You could also use concatMap or switchMap. They all have subtle differences. See rxjs docs to understand.
mergeMap:
https://www.learnrxjs.io/learn-rxjs/operators/transformation/mergemap
concatMap:
https://www.learnrxjs.io/learn-rxjs/operators/transformation/concatmap
switchMap:
https://www.learnrxjs.io/learn-rxjs/operators/transformation/switchmap