We Keep Coding

RxJs concatMap is emitting only one http stream out of many

I have been trying to get stream of objects in a sequencial order, however concatMap is not working for me. With mergeMap I get the stream but no in a sequencial order.
Below is my code I am trying:
this.result
.pipe(
map((result: Result) => {
return result.contents
.map((content: Content) => this.databaseService
.getResource<Resource>('Type', content.key) // http call in service
)
}),
mergeMap((requests: Observable<Resource>[]) => {
return from(requests)
.pipe(
concatMap((resource: Observable<Resource>) => resource), // ----> Trigger only once.
filter((resource:Resource) => resource.status === 'active'),
skip(this.pageIndex * this.PAGE_SIZE),
take(this.PAGE_SIZE),
)
}),
)
.subscribe({
next: (resource: Resource) => {
console.log(resource) // resource stream
},
complete: () => console.log('completed')
});

concatMap will only process one observable source at a time. It will not process subsequent sources until the current source completes.
In your case the observable from this.databaseService.getResource() is not completing. To force it to complete after the first emission, you could append a take(1) like this:
concatMap((resource: Observable<Resource>) => resource.pipe(take(1))
Alternatively, if the call to databaseService.getResource() is only meant to emit a single value, you could modify your service to emit only a single value.
// http call in service
Note: If you are using the Angular HttpClient, a typical get request will complete when the response is received. So you can probably have your getResource() method return the oservable from the http.get() call.

Implement stream with multicast that doesn't complete the underlying subject

I'm implementing the cache using BehaviorSubject and multicast. The stream returned from the cache should start with HTTP request. I should also be able to force refresh the cache by triggering manually next on subject. The common approach with two subjects outlined by Poul Kruijt is well known and suggested everywhere. My idea is to find a way to achieve the following using only one subject throughout lifecycle of a stream.
It would be easy to achieve with multicast like this
const cache = new BehaviorSubject(null);
const shared = queryThatCompletes.pipe(multicast(cache)) as any;
// sets up subscription, waits for connect
shared.subscribe((values) => console.log(values));
// triggers http request
shared.connect();
setTimeout(() => {
// will only emit COMPLETE from subject
shared.subscribe((values) => console.log(values));
}, 2000);
// force refresh the cache
cache.next();
but since the HTTP query stream completes, the second subscription doesn't get any value, just COMPLETE notification from subject. This behavior is described in detail here.
The other option is to pass a factory function instead of the subject instance like this:
const cache = ()=> new BehaviorSubject(null);
const shared = queryThatCompletes.pipe(multicast(cache)) as any;
This will re-create the subject, that will subscribe to queryThatCompletes and re-trigger HTTP request. But the downsides is the need to call connect multiple times and redundant queries.
const cache = () => new BehaviorSubject(null);
const shared = queryThatCompletes.pipe(multicast(cache)) as any;
// sets up subscription, waits for connect
shared.subscribe((values) => console.log(values));
// triggers http request
shared.connect();
setTimeout(() => {
// sets up subscription, waits for connect
shared.subscribe((values) => console.log(values));
// triggers http request
shared.connect();
}, 2000);
So I simply implemented HTTP stream that doesn't complete by itself and use it like this:
const queryOnceButDontComplete = new Observable((observer) => {
fetch('https://jsonplaceholder.typicode.com/todos/1')
.then(response => response.json())
.then(data => observer.next(data));
return () => {};
});
const cache = new BehaviorSubject(null);
const shared = queryOnceButDontComplete.pipe(multicast(cache)) as any;
// sets up subscription, waits for connect
shared.subscribe((values) => console.log(values));
// triggers http request
shared.connect();
setTimeout(() => {
// sets up subscription, waits for connect
shared.subscribe((values) => console.log(values));
}, 2000);
This works, but I'm wondering if there's a way to achieve what I want without the use of custom observable. Any ideas?

Best would be to just use a shareReplay(1):
const shared = queryThatCompletes.pipe(shareReplay(1));
// sets up subscription, waits for connect
shared.subscribe((values) => console.log(values));
// triggers http request
shared.connect();
setTimeout(() => {
shared.subscribe((values) => console.log(values));
}, 2000);
Not entirely sure what the subscribe and connect are doing there, but if you are just returning a HttpClient get call, then you should just return the shared observable, and whomever subscribes first to it, will trigger the http request. No need for connect. Any subsequent subscriptions will wait for the request to finish or receive the last emitted value from the observable.
Based on your comment, let's wrap this in a service (untested):
#Injectable()
SomeDataService {
readonly refresh$ = new BehaviorSubject(undefined);
readonly get$ = this.httpClient.get(/*url here*/);
readonly shared$ = this.refresh$.pipe(
switchMap(() => this.get$),
shareReplay(1)
);
constructor(private httpClient: HttpClient) {}
getData(): Observable<unknown> {
return this.shared$;
}
refreshData(): void {
this.refresh$.next();
}
}
Does this make sense? Basically you start with a refresh subject, which get mapped to the actual network call. On the first getData(), the network request gets triggered. Any call on getData() after that will get the cached value. Calling refreshData will refresh the data for any subscription

I have made functional approach that I believe is much more simpler
it uses two observables refresh$ and the desired observable,
I hope this solution might give you some thoughts
const makeRestartableCahcedObservable = (ob:Observable<any>)=>{
const refresh$ = new BehaviorSubject(null)
return [
refresh$.pipe(
switchMap(()=>ob),
shareReplay(1)
),
()=>refresh$.next(null)
]
}
const [ob, refreshFun] = makeRestartableCahcedObservable(of('string'))
ob.subscribe(console.log)
setTimeout(()=>{refreshFun()}, 3000)
this basically can refresh your data with much more simpler API,
you can even wrap it with an object to make it a proxy with included refresh function
const makeRestartableCahcedObservable = (ob:Observable<any>)=>{
const refresh$ = new BehaviorSubject(null)
const wrappedOb$ = refresh$.pipe(switchMap(()=>ob),shareReplay(1))
return {
subscribe:(...args)=>wrappedOb$.subscribe(...args),
pipe:(...funs)=>wrappedOb$.pipe(...funs),
refresh:()=>refresh$.next(null)
}
}
const ob = makeRestartableCahcedObservable(of('string'))
ob.subscribe(console.error)
setTimeout(()=>{ob.refresh()}, 3000)

Capture last value of an observable with multiple requests

I have a service layer responsible for handling the data and sending it to components that use the same value.
I need to make requests to the api, as long as the value is isProcessed == false it will do 10 attempts every half a second.
If it fails during the attempts, it will only make 3 more requests.
If isProcessed == true, stopped the requisitions.
All the logic is built in, but I can't output the last observable value. All responses are sent.
All requests are sent by observables, both false and true, but I only need the last one.
Here all requests responses are arriving in the component, not just the last
Service responsible for accessing API:
public getPositionConsolidate(): Observable<PosicaoConsolidada> {
return this.http.get<PosicaoConsolidada>(`${environment.api.basePosicaoConsolidada}/consolidado`)
.pipe(
map(res => res),
retryWhen(genericRetryStrategy()),
shareReplay(1),
catchError(err => {
console.log('Error in Position Consolidate', err);
return throwError(err);
})
)
}
Service responsible for handling the data and sending it to the component:
public positionConsolidate() {
let subject = new BehaviorSubject<any>([]);
this.api.getPositionConsolidate().subscribe(response => {
if(response.hasProcessado == false) {
for (let numberRequest = 0; numberRequest < 10; numberRequest++) {
setTimeout(() => {
//subject.next(this.api.getPosicaoConsolidada().subscribe());
this.api.getPositionConsolidate().subscribe(res => {
subject.next(res)
})
}, numberRequest * 500, numberRequest);
}
} else {
retryWhen(genericRetryStrategy()),
finalize(() => this.loadingService.loadingOff())
}
})
return subject.asObservable()
}
In component:
public ngOnInit() {
this.coreState.positionConsolidate().subscribe(res => console.log(res))
}

The easiest part to answer of your question, is that if you just want the last emission from an observable, then just use the last operator. However, the way you've written things, makes it difficult to incorperate. The following refactors your code as a single stream without any non-rxjs control structures.
public positionConsolidate() {
return this.api.getPositionConsolidate().pipe(
concatMap(res => iif(() => res.hasProcessado,
of(res),
interval(500).pipe(
take(10),
concatMap(() => this.api.getPositionConsolidate())
)
)),
retryWhen(genericRetryStrategy()),
finalize(() => this.loadingService.loadingOff()),
last()
);
}
What's happening
First this executes the initial api call.
Then based on the result, either...
Returns the initial result
Calls the api 10 times every 500ms.
Implements what you have for retryWhen and finalize.
Returns the last emitted result.
Also don't subscribe to observables inside of observables - that's what higher order observables such as concatMap are for.

NGRX state not updated when Firestore changes occur while offline

I have an Angular application using AngularFire, NgRx and Cloud Firestore as db, where I enabled offline persistence.
My problem is that when I change a document while offline the effect function does not trigger the success action, as Firestore promises are not resolved while offline, but only after the request reaches the server.
At the moment I am stuck in trying to find a good way to update the store with the local data when offline.
One idea could be to check the fromCache flag before loading the data, so that if fromCache is true (that is, we are offline) I can load the data from the local db instead of the store, but it looks to me like a dirty workaround.
Effect
//--- Effect triggered to load document in the home page ----
firstSet$ = createEffect(() => {
return this.actions$.pipe(
ofType(PlacesActions.loadFirstPlaces),
switchMap(() => {
return this.placeService.getFirstSet().pipe(
map((places) => {
return PlacesActions.loadFirstPlacesSuccess({ places });
}),
catchError((error) => of(PlacesActions.loadFirstPlacesFailure({ error }))),
takeUntil(this.subService.unsubscribe$)
);
})
);
});
//--- Effect triggered when a document is updated ----
updatePlace$ = createEffect(() => {
return this.actions$.pipe(
ofType(PlacesActions.updatePlace),
concatMap((action) => {
// ----- Below the NEW code, without promise ----
try {
this.placeService.savePlace(action.place);
this.router.navigate(['/home']);
return of(PlacesActions.updatePlaceSuccess({ place: action.place }));
}
catch(error) {
return of(PlacesActions.updatePlaceFailure({ error }))
}
/*
// ----- Below the old code ----
return from(this.placeService.savePlace(action.place))
.pipe(
map((place: Place) => {
this.router.navigate(['/home']);
return PlacesActions.updatePlaceSuccess({ place });
}),
catchError((error) => of(PlacesActions.updatePlaceFailure({ error })))
);
*/
})
);
});
DB service
savePlace(place): void {
this.firestoreRef.doc<Place>(`places/${place.id}`).update(place);
}
/* Old version of savePlace using Promise for the Update
async savePlace(place): Promise<void> {
return await this.firestoreRef.doc<Place>(`places/${place.id}`).update(place);
}
*/
loadFirstPlaces(limit: number = 9,
orderBy: OrderModel = { propName: 'modifiedOn', order: 'desc' }){
const query = (ref: CollectionReference) =>
ref.orderBy(orderBy.propName, orderBy.order)
.limit(limit);
return this.firestoreRef.collection<Place>('Places', query)
.valueChanges()
.pipe(shareReplay(1));
}
Home component
ngOnInit(): void {
// Set the data from the store, if available there
this.places$ = this.store.select(getAllPlaces).pipe(
tap((data) => {
this.places = data;
})
);
/*Dispatch load action to load data from the server or local cache.
If I use the PROMISE approach for the update method,
the data coming from the local cache has the old data.
The edits done while offline are not provided.*/
this.store.dispatch(loadFirstPlaces());
}

The local cache has been updated when the update() call completes. So that's the right moment to update the state of your application too:
async savePlace(place): Promise<void> {
const result = this.firestoreRef.doc<T>(`places/${place.id}`).update(place)
// TODO: update the state here
return await result;
}
I'm not even sure if you should be returning a promise here. If savePlace is meant to return whether the local operation was successful, it should simply be:
savePlace(place): void {
this.firestoreRef.doc<T>(`places/${place.id}`).update(place)
}
If the local write operation fails, update will throw an exception and that will escape from savePlace to signal failure to the caller.

admin-on-rest how to implement a custom saga for auto-refresh

In the example provided in the aor-realtime readme
import realtimeSaga from 'aor-realtime';
const observeRequest = (fetchType, resource, params) => {
// Use your apollo client methods here or sockets or whatever else including the following very naive polling mechanism
return {
subscribe(observer) {
const intervalId = setInterval(() => {
fetchData(fetchType, resource, params)
.then(results => observer.next(results)) // New data received, notify the observer
.catch(error => observer.error(error)); // Ouch, an error occured, notify the observer
}, 5000);
const subscription = {
unsubscribe() {
// Clean up after ourselves
clearInterval(intervalId);
// Notify the saga that we cleaned up everything
observer.complete();
}
};
return subscription;
},
};
};
const customSaga = realtimeSaga(observeRequest);
fetchData function is mentioned, but it's not accessible from that scope, is it just a symbolic/abstract call ?
If I really wanted to refresh the data based on some realtime trigger how could i dispatch the data fetching command from this scope ?

You're right, it is a symbolic/abstract call. It's up to you to implement observeRequest, by initializing it with your restClient for example and calling the client methods accordingly using the fetchType, resource and params parameters.
We currently only use this saga with the aor-simple-graphql-client client