We Keep Coding

RxJs 6 load more pagination stream - javascript

I'm trying to build one of those more modern paginations where there's not dedicated links for the individual pages but one where more results are loaded automatically when the user scrolls to the bottom. On the web page are multiple widgets that allow you to modify the search parameters. When the parameters change more results should be fetched via ajax beginning from the first page again.
I'm fairly new to RxJs and I'm having issues wrapping my head arround how to identify the observables/subjects I need and how to compose them to achive the described behavior.
Here's the specific flow I have in mind:
When the page is first loaded an initial set of parameters is taken and used to load the first page. When a "load more" event is fired the next page should be fetched and rendered to the page.
When the parameters change the page should be loaded starting from page 1 again.
When the server signals that there are no more results to load I should get notified about that via an observable. If further "load more" events are fired after no more pages are available the ajax request should not be made to save bandwidth on mobile devices.
Lastly as long as a network request is open i want to be able to display a loader so i need an observable that informs me about whether there are open requests or not.
As a bonus: Currently I've implemented signaling no more results by returning a 404 from the backend when a page one bigger than last page is requested. I'd like to use catchError on the ajax observable in such a way, that it gracefully stops the ajax request without breaking the subscription.
Here's what I was able to come up with so far, but it has multiple Problems (described below):
import { BehaviorSubject, Subject, fromEvent } from 'rxjs';
import { map, mergeMap, switchMap, takeUntil, tap } from 'rxjs/operators';
import { ajax } from 'rxjs/ajax';
import { stringify } from 'qs';
const paramsEl = document.querySelector<HTMLTextAreaElement>('#params');
const paramsChangedBtn = document.querySelector<HTMLButtonElement>('#paramsSubmit');
const loadNextPageBtn = document.querySelector<HTMLButtonElement>('#loadNextPage');
const getParams = () => JSON.parse(paramsEl.value);
const params$ = new BehaviorSubject(getParams());
const page$ = new BehaviorSubject(1);
const noMoreResults$ = new Subject<void>(); // <- public
const connections$ = new BehaviorSubject(0);
const loading$ = new BehaviorSubject(false); // <- public
// for the sake of this example we're not using an IntersectionObserver etc. but a plain button to fire a "load more" event
const loadNextPage$ = fromEvent(loadNextPageBtn, 'click');
// same for params changed event. In my real app I've got a working stream fed from the widgets
fromEvent(paramsChangedBtn, 'click').subscribe(e => params$.next(getParams()));
// when the params change, reset page to 1
params$.subscribe(() => page$.next(1));
// update loading$ observable for displaying/hiding a loader
connections$.subscribe(connections => {
if(connections > 0 && loading$.getValue() === false) loading$.next(true);
if(connections <= 0 && loading$.getValue() === true) loading$.next(false);
});
// when we need to load the next page, increment the page observable
loadNextPage$.subscribe(e => page$.next(page$.getValue() + 1));
//////////////
// whenever a new page should be requested, get the current parameters and fetch data for this page
page$
.pipe(
takeUntil(noMoreResults$),
tap(() => connections$.next(connections$.getValue() + 1)),
mergeMap(page => {
const qs = stringify({
...params$.getValue(),
page,
});
return ajax.getJSON<any>(`https://httpbin.org/get?${qs}`)
// this doesn't seem to do anything
// furthermore the ajax request would already have been made at this point
// .pipe(
// takeUntil(noMoreResults$)
// );
}),
tap(() => connections$.next(connections$.getValue() - 1)),
)
.subscribe(data => {
console.log(data.args);
// for testing purposes pretend we have no more data at page 5
if(data.args.page === "5") noMoreResults$.next();
});
// for debugging purposes
loading$.subscribe(loading => console.log('loading: ', loading));
noMoreResults$.subscribe(() => console.warn('no more results'));
You can find the running version of this here on stackblitz.
Here's the issues with the code:
Current pace of takeUntil(noMoreResults$) breaks the subscription when noMoreResults$ has been triggered and then params$ emits no further pages are loaded. (See comment in the code for the other location in the ajax pipe).
using params$.getValue() when mergeMapping to the ajax observable feels wrong, however I don't know how to pass down both the page number as well as the parameters in one stream properly.
In general I think I've overused Subjects / BehaviorSubjects quite a bit but I'm not sure. Can you either confirm or deny this?
The composition of the observables feel very messy and hard to follow. Is this based on what I'm trying to do or is there room for improvement for this problem?
Can you please provide a working example as well as elaborating on the biggest mistakes I've made.

I have been keeping this question open in my tabs ever since you created this question, wanting to help, but also, wanting to learn enough of the RxJS so I could create a solution myself.
I'm really sorry, but I haven't looked at your example, but instead, I created my own. I would have to ask you to please forgive me for the extremely large answer that I will provide here.
I was mainly driven by the excellent talk by Ben Lesh, one of the creators of modern RxJS which you can find here. I strongly suggest that you look at this video, even multiple times, to try to understand some of the stuff I used in my solution to this problem.
Like Ben, I also used Angular framework as the basis for this project. You can find my solution at GitHub. Also, just like Ben has been explaining couple of times in his talk some Angular specific stuff, I will try to do it here as well.
What I've got in my app are a simple FeedComponent and a FeedService. The service is being injected using Angular Dependency Injection to the FeedComponent.
Now, I've got an HTML bound to FeedComponent that looks like this:
<div class="loading" *ngIf="loading$ | async">
Loading...
</div>
<div class="filter">
<form #form="ngForm">
...
</form>
</div>
<div #articles class="articles">
<article *ngFor="let article of feed$ | async">
...
</article>
</div>
You can see that I've got three sections: a <div> responsible for displaying a message that the feed is being loaded; another <div> with filtering <form> that displays filtering options; and a third <div> responsible for displaying feed items as a list of <article>s.
Angular specific stuff here include *ngIf and *ngFor directives and async pipe (|). With a single sentence: *ngIf renders certain DOM element if condition stated in attribute value is truthy; *ngFor loops through an array of provided items and renders certain DOM element number of times of the array's length; pipe | transforms items so that items to the left are always input items to the pipe to the right, so does the async pipe do - it transforms an Observable (you can tell that it's an Observable by the $ sign suffix that I and many others are using) to transform items that come from the Observable to what the directive understands. async pipe is also explained in Ben's talk.
Let's get started: you can see that I'm using two Observables in my HTML template, and that's all you need. The one that will give you an array of FeedItems so you can display them on the page, and the other one that will emit boolean values when feed is being fetched from server. If you would not use Angular, but rather some other framework or library, or nothing at all, you could still have only these two streams. You would have to manually subscribe to them (and unsubscribe later, when not needed anymore) and when you'd get results, you should update the DOM accordingly. Angular and async pipe do all of this here for me.
These two are feed$ and loading$ Observable streams, respectively. Both of them are defined in the FeedComponent that is bound to this HTML, very simply, like this:
feed$ = this.feedService.feed$;
loading$ = this.feedService.loading$.pipe(delay(10));
As I said, feedService is injected to FeedComponent through FeedComponent's constructor using Angular DI:
constructor(private feedService: FeedService) {}
You would just have to create new FeedService object if you'd use your own JS framework/lib or no lib at all. I'm adding delay of 10 ms to feedService.loading$ stream because I'm getting some Angular error that I should not explain here. You may not need it at all if not using Angular.
Now, to be able to provide feed items (FeedItems[]) through feed$ stream, you need to listen to the two possible events: a scroll event that would fire when the user has scrolled enough to the bottom of the page and an event that happens when filter form input values change. These two events need to be combined to a single Observable that we will call filterSeed$ - it will emit values contained in the input elements of the used form.
The first event stream can be formed out of these two Observables:
scrollPercent$: Observable<number> = fromEvent(document, 'scroll')
.pipe(
map(() => {
const scrollTop = this.articles.nativeElement.getBoundingClientRect().top;
const docHeight = this.articles.nativeElement.getBoundingClientRect().height;
const winHeight = window.innerHeight;
const scroll = scrollTop / (winHeight - docHeight);
return Math.round(scroll * 100);
})
);
loadMore$: Observable<number> = this.scrollPercent$
.pipe(
filter(percent => percent >= 80),
take(1),
repeatWhen(() => this.feedLoadingStops$)
);
scrollPercent$ is an Observable that emits some numbers. They represent scroll percentage when document's scroll event fires (created using fromEvent). Whatever event it emits, I don't really care about it. I only care about when it emits, so I can map it to percentages using some simple math. this.articles.nativeElement is Angular specific, so if you need another example, please take a look at this Pen about how to achieve it with jQuery. The returning value of the map function is rounded scroll percentage.
loadMore$ is an Observable that fires events only when a user has scrolled enough so that new feed items should be loaded - it fires scroll percentage number, but we don't really care about that, you'll see that we're ignoring these numbers later. The threshold when this should happen is at or after 80% of the scroll. So, I'm using filter here to let only those items that are above the threshold (remember, I need loadMore$ to emit when this threshold is reached and passed). And I'm using take(1) here because I really only need one such item.
Just like Ben has had a problem when the whole stream only worked once, I was having it as well. Because take completes (effectively unsubscribes) from the source when it takes that one item, I need to resubscribe again to the same source, which is, all the way to the top, the stream created by fromEvent.
So, I need to start listening to the scroll events once again, but there's catch here: I don't want to start doing it immediately, but rather when the loading completes. So, I need to use repeatWhen instead of just repeat. repeatWhen takes a factory function that it calls when needed to get an Observable to subscribe to. It listens to the provided Observable (this.feedLoadingStops$) and resubscribes to the source when the this.feedLoadingStops$ emits.
The this.feedLoadingStops$ looks like this:
feedLoadingStops$ = this.loading$.pipe(map(v => !v), filter(v => v));
It inverts false values so they become true, and vice versa, so that emitted true value indicates that loading has stopped (remember, when loading$ emits false, it indicates that loading has stopped). It also filters just true values so that we only get emits when it stops loading.
But, you may wander why. Why did repeat work for Ben and not for me? Why did I have to resubscribe only when the loading stops. It's because we both used higher order mapping operators after events fired by fromEvent to flatten the HTTP requests later on. I will certainly come to that later, but what he used was the exhaustMap operator, and I used switchMap which would always switch to the latest emitted item by the source and subscribe to it. When loadMore$ is resubscribed again (using just repeat), it would start listening to the scroll events again, and since user would certainly continue scrolling more, loadMore$ would start emitting once again and the switchMap would continue to resubscribe to the provided HTTP request all the way until user wouldn't stop scrolling. Which is really not what we want - we don't want to create multiple, exactly the same HTTP requests to fetch a single resource just because user is doing something we're responsible to solve. exhaustMap is different so that it does exactly the opposite to swithcMap - it will wait for the first emitted item to finish (basically, it will exhaust) until it subscribes to the next.
That was the explanation of the first event stream that will help create filterSeed$ Observable. The other one is rather simple one. Angular provides such Observables on its own when it comes to forms. I used this.form.valueChanges and I was automatically subscribed to any form input element value changes. Since I only have a single <select> which I use to fake should I load the feed with all items or only items with text or only items with images, I would like to listen to when a user selects a different option to fire an event.
Since Angular provides this for me, you may want to create your own Observable that would emit changed form input values for you.
And finally, this is what filterSeed$ would look like:
filterSeed$: Observable<FeedFilter> = defer(() => merge(
this.loadMore$.pipe(map(() => this.form.value)),
this.form.valueChanges
));
Here, I want to merge two streams: the one that is created by listening to the loadMore$ events, and the other that is listening to the filter form value changes. And that is exactly what I want: I want to load new feed items only when a user has scrolled enough to the bottom or when it changed a filter. this.form.valueChanges already provides FeedFilter items, but this.loadMore$ does not. Remember, this.loadMore$ emits numbers which I sad I don't really care about, so I'm mapping them to this.form.value. This is yet again Angular specific, so you'd have to implement your own reading of the whole form input elements. Current this.form.value is always the same as the last emitted item from this.form.valueChanges.
The reason to use defer here is yet another Angular specific because by the time filterSeed$ is created, this.form is still undefined, so I need to wait until subscription happens to make sure this.form is available. And I will subscribe to this Observable in ngAfterViewInit() lifecycle hook which is called upon component's view creation. You may not need to use defer here if not using Angular. Now, when everything is ready, it's time to subscribe to this.filterSeed$ and do some data loading. Don't forget to unsubscribe when leaving component/page not to leak memory by not removing all of the event listeners created by fromEvent.
ngAfterViewInit(): void {
this.subscription = this.filterSeed$.subscribe(this.feedService.filter$);
}
ngOnDestroy(): void {
this.subscription.unsubscribe();
}
We now come to the second part of this answer which is FeedService. The FeedService is only a simple JS object that has some Observables and some state. I need this state in order to be able to work with multiple Observables - some might say this is not the true Rx way, but I found this to be easier for me to solve it this way. I'm also injecting Angular's HttpClient (as http variable) to the FeedService which is only an Angular wrapper to XHR. You could use RxJS ajax static creation method instead - both should behave the same.
The FeedService has got three public Observables and you may have seen all of them being used in FeedComponent: filter$, loading$ and feed$ Observables. The later two are used by FeedComponent to render some stuff to the DOM, while filter$ was used to feed it with filterSeed$. Basically, filter$ is just a Subject:
filter$ = new Subject<FeedFilter>();
And since Subjects are both Observables and Observers, I could use it as an Observer, so I passed it to subscribe method when I subscribed to filterSeed$ in FeedComponent. What this means is that filter$ Observer will subscribe to filterSeed$ and any call to next method (basically, any emission) from filterSeed$ will pass through to the filter$ Subject. This means that anyone else using filter$ will get the value emitted by filterSeed$.
And filter$ is used to create feed$ Observable. Here's what it looks like:
feed$: Observable<FeedItem[]> = this.filter$.pipe(
switchMap(filter => {
if (filter !== this.filter) {
this.filter = filter;
this.nextPage = 1;
this.shouldReset = true;
}
return this.getFeed$;
}),
scan((acc, value) => {
if (this.shouldReset) {
this.shouldReset = false;
return value;
}
return acc.concat(value);
}, [])
)
I'm using two operators here: switchMap and scan. I am also having some state that I keep in the class itself in variables this.filter, this.nextPage and this.shouldReset.
I already mentioned higher order mapping operators. I'm using switchMap here. And it is being used after events fired by filter$ Subject which is connected through subscribe method with the FeedComponent's filterSeed$. So, whenever a refresh event is fired (either by user scrolling enough or by user changing a filter in the filter form), I want to map it to getFeed$ Observable which is responsible for creating HTTP requests. The reason to choose switchMap over others (over exhaustMap which Ben used) is that I want to make sure that I always get the result from the latest filter used by the user. I.e. if the user sent a request with one filter and changed a filter in meantime while the first request is still loading, I want to cancel that request and switch to another HTTP request.
Since filter$ Subject is emitting FeedFilter objects, they are passed to switchMap's callback function as filter parameter. This is where I'm checking if filter is actually the same as the filter in FeedService (this.filter). If they are not the same, it means that the filter is changed, so I need to save the new filter to the FeedService's filter (this.filter = filter;). This also means that I have to reset page to page 1 (this.nextPage = 1;) and set this.shouldReset to true. Then I return getFeed$ to which switchMap internally subscribes.
All of these state holding variables are later used by either getFeed$ Observable or the next operator that comes after switchMap: scan. But, how does getFeed$ looks like. Here's how:
getFeed$: Observable<FeedItem[]> = defer(() => {
if (this.nextPage) {
this.loadingSubject$.next(true);
const url = appendQuery('/feed', { page: this.nextPage, feedFilter: this.filter.feedFilter });
return this.http.get<FakeFeedResponse>(url);
} else {
return NEVER;
}
}).pipe(
catchError(() => /* Potentially handle this.nextPage here */EMPTY),
tap(response => {
this.nextPage = response.nextPage;
this.loadingSubject$.next(false);
}),
map(response => response.items),
share()
);
I'm again using defer here. This is because I'm saving some state outside of these streams, so I want values from these state variables to be read when a subscription to getFeed$ is made, not when getFeed$ object is created.
In the defer's callback function body I'm checking if this.nextPage exists. The server returns null if there are no more items to load, so in that case, I'm returning NEVER which is an Observable that never emits. However, if there are items to load (when nextPage is a valid, truthy number), I'm returning an HTTP get request. nextPage is set to 1 by default (or is being reset to 1 in switchMap if filter is changed). I'm constructing url by appending nextPage and feedFilter as query string to '/feed' route.
Also, I'm using loadingSubject$ to emit true indicating that the loading has started. loadingSubject$ looks like this:
private loadingSubject$ = new BehaviorSubject(false);
loading$: Observable<boolean> = this.loadingSubject$.asObservable();
It is a BehaviorSubject with the default value of false. Values emitted by this Subject are offered to FeedComponent through loading$ Observable. When FeedComponent first subscribes to loading$ Observable, it will get false immediately.
The this.http.get<FakeFeedResponse>(url) request returned to defer is using Angular XHR wrapper. I'm injecting http to FeedService, but you should be able to use RxJS's ajax as I already mentioned. It emits objects of FakeFeedResponse type.
After this Observable is constructed using defer, I want to do some more stuff when it emits. First thing is to handle errors using catchError. If an error happens, I want to return an EMPTY Observable which just completes without emitting any item. I added comment here so that you may add some more error handling or handle (re)setting of nextPage or something.
After that, I'm saving nextPage from the response in tap and also emitting false to loadingSubject$ indicating that the loading has stopped. After tap, I'm using map to extract items from the response.
And then I'm shareing it. This is actually not really needed in my case. Why? Because there is only one subscriber to getFeed$ (which is switchMap) so there's really no need to share it across multiple subscribers, but it can stay here if it would ever need - actually, if there would ever exist another subscriber. I added it because Ben added it as well, but he has more than one subscriber to his getFeed$ Observable.
And that's all about getFeed$ which is being returned to switchMap in its callback method. In feed$ Observable, after switchMap, I'm using scan. Actually, scan is here just so that Angular's async pipe could receive already loaded items by concatenating new values to an already loaded ones (acc.concat(value)). I'm using this.shouldReset flag here so that I don't use concat when the filter is changed. If you would not use Angular, you would probably subscribe yourself to feed$ Observable and you would probably handle this case yourself instead of scan, so you wouldn't probably need to have scan here.
After all, I'm using Angular Interceptor feature to fake all of the server responses. Please take a look how.
And that's it. I'm really sorry for the very long answer, if you have questions, please open an Issue on GitHub. I really hope that this answer might help you shape your solution, which I didn't really look at, I'm sorry.
If you'd like to try this example, you can clone the project, run npm install and then ng serve which will compile the whole project and run a dev server so you can try this project on your own.

Related

Working on an app for reporting timesheet information for site resources. For various locations there can be upwards of 500-600 line items, each line item has at least 16 form-fields, though only 8 have an HTML component 'assigned'.
the general form lay-out looks like this:
Each line-item is it's own form component in a FormArray, which is part of the parent form. The form itself is 'managed' by a form service that handles most of the retrieval of data and adding and deleting of line items.
The generation of the FormArry gets done in the formService
//Initiate listener on service
this.service.getDate.subscribe((data: IInterface[]) => {
data.map(lineitem => {
const array = this.getFormArray(lineitem.group);
lineitem.id = array.length + 1;
let iform = this.fb.group(new LineItemForm(lineitem));
array.push(iform);
});
});
The ini for the data gets triggered by a single call from the parent-form on a button click, the above listener distributes the data over 4 individual arrays by means of the lineitem.group.
That all works as supposed to.
In the parent form HTML the lineitem-forms get generated as follows and the Array comes from the formService where it gets filtered by page number into a static Array (i.e. the service doesn't need to call or filter upon retrieving the array in the Parent as that was already done on retrieval of the data. the Array should therefore be 'ready-to-serve')
<lineitem *ngFor="let lineitem of Array?.controls; let i = index"
[index]="i"
[LineItemForm]="lineitem"
(deleteLineItem)="deleteLineItem($event)">
</lineitem>
the lineitem-form component has following spec's and properties.
After the lineitem View is rendered each lineitem-form gets detached from the ChangeDetection process and all subscriptions are pushed into an array so we can unsubscribe from all upon ngOnDestroy. At the moment there are only 3 subscription listeners initiated. (1 for the employee typeahead, one for the employee array (which comes from the service and one for resources array (also from service))
#Component({
selector: 'lineitem',
templateUrl: './lineitem.component.html',
styleUrls: ['./lineitem.component.css', './../form.component.css'],
changeDetection: ChangeDetectionStrategy.OnPush
})
constructor(
private formService: formService,
private modalService: BsModalService,
private cd: ChangeDetectorRef
) {
}
ngAfterViewInit() {
this.cd.detach();
}
ngOnDestroy() {
this.subscriptions.forEach(c => c.unsubscribe());
}
The fields for Monday - Sunday have a (change) binding to a calculation method in the form-component, at the end of the method the ChangeDetectionRef is triggered to run the this.cd.detectChanges() to update each individual lineitem-form.
calculateThings(){
// … do some calculations and form-field sets
//Trigger change detection cycle
this.cd.detectChanges();
}
The Parent-form is set to ChangeDetection.Default (though switching it to OnPush doesn't reduce the quantity of listeners)
To solve (part of) the problem, I have separated the overall quantity of line-items in tabs, each tab gets only generated when actually displayed using an *ngIf binding. This reduces the amount of listeners to about 2000-4000 on the active tab. However, if a user clicks back and forth between two tabs, the listeners quickly start adding up again.
At some stage -though irregular and without clear 'reason'- the listener count resets again to a more manageable 2000-3000 (still a lot but that works) I can display around 35-50 lines (i.e. about 300-400 formfields) before there is noticeable delay in response time.
This leads to a couple of questions:
Is there a method to 'purge' listeners, when moving from tab to tab (i.e. when re-generating the DOM)
Is there any way in which to 'reset' the DOM so the listener count drops to near zero
What is the best (or recommended) way to handle this amount of lines/form-field
(if I look at ag-Grid for example, they seem to be able to generate 10000 lines # 100 columns, without exceeding the 2000 listeners???, though without calculating fields as far as I can see)
Is the only way to keep the amount of listeners in check by means of limiting the actual quantity of form-field in the active DOM?
I would appreciate any hints & tips that could provide some further information.
The current solution has me separating the form-array into tabs and paginating the arrays on each tab that have more than 50 lines. That works, but I am curious if there are 'cleaner' solutions?
PS: I would appreciate if we could avoid comments like why would you need that many form-fields in the first place. Which, although a legitimate question, does not help as this is a design criteria for rolling it out to the users. If you have constructive information on how to input 7 dayvalues for 600 resources in 1 form then I am all ears?

This is not an ideal answer as you are on Angular 6, but I just read about an article that reduces the number of event handlers and change detection cycles in Angular 9. It seems promising, so I hope it works for you, provided you can upgrade your app to v9.
This is using a new option called ngZoneEventCoalescing.
“Reduce Change Detection Cycles with Event Coalescing in Angular” by Netanel Basal https://link.medium.com/jG2bFqx3j2

Here is what I am trying to do: I have a list of users coming from an async data source. The source returns an Observable<User[]>. The list is then displayed in the template via Angular's async pipe like this.
<ng-template #loading>Loading...</ng-template>
<ul *ngIf="users$ | async; let users; else loading">
<li *ngFor="let user of users">{{user.name}}</li>
</ul>
The list of users is searchable. Using reactive forms, I have bound the search field's change event like this (with this.resolve performing the async task, having the signature (text: string) => Observable<User[]>):
search = new FormControl('');
users$ = this.search.valueChanges
.pipe(startWith(this.search.value))
.pipe(debounceTime(250))
.pipe(distinctUntilChanged())
.pipe(switchMap(this.resolve));
This works perfectly fine. The search input is debounced and a new async request to resolve the users is fired when the search input changes.
Now I have a problem. Users may change on the client side and need to be updated in the list without performing the async task again. My attempt to use another pipe operation in the update function results in this.resolve being executed again.
update(user: User): void {
this.users$ = this.users$
.pipe(map(users => {
// replace user in users
return users;
}));
}
However, I do not want to resolve the users again. I just want to locally update the async user list. How can I do this?

You can use Subject and merge it into the users$ chain after switchMap.
otherUsers$ = new Subject();
users$ = this.search.valueChanges
.pipe(
startWith(this.search.value),
debounceTime(250)),
distinctUntilChanged()),
switchMap(this.resolve),
merge(otherUsers$),
);
Then you can call next() and avoid going through the whole chain:
otherUsers$.next([{}, {}, {}, ...]);

Adding on to martin's merge() suggestion, I would also "share" the users result to prevent the resolve from firing again.
RXJS will run through all the pipes every time a subscription is added. Think of them as transformers where the "datastore" is the initial Subject (this.search.valueChanges). So every time you subscribe, RXJS is going to run through your pipes to get the transformed value.
So you want to look at RXJS share() or shareReplay(1). What these will do is internally create a new Subject() (in the case of share()) or a new ReplaySubject(1) (in the case of shareReplay(1)) to hold your transformed value in a new "datastore", if you will.
search = new FormControl('');
otherUsers$ = new Subject();
users$ = merge(
otherUsers$,
this.search.valueChanges.pipe(
startWith(this.search.value)),
debounceTime(250)),
distinctUntilChanged()),
switchMap(this.resolve))
)
).pipe(share());
Subject vs ReplaySubject depends on whether you want to hold a value even when there is no subscription. (This is my interpretation based on my own usage.) I found shareReplay(1) useful in cases where I have subscriptions like obs.pipe(first()).subscribe(...). However one of your subscriptions is the AsyncPipe in the DOM, so you're probably fine with share(), but try it out yourself.

First, I'd like to thank the community for all the support learners like me get while working with new technologies.
I've been using Angular for a while now, and there's something I still don't quite understand nor have I seen asked elsewhere.
Supposing I have a service that returns an Observable with the data I need, how should I use this data properly, in terms of performance?
I know I can use the async pipe and avoid having to sub/unsub, but this only happens in the template. What if I needed to use the same data in the component as well? Wouldn't subscribing again (from the template with the async pipe and from the component with .subscribe())?
How do I keep the observable up to date? For example, I have a table displaying data from an API. If I click on the 2nd page (pagination), I'd like to make another call and have the observable updated.
I'm sorry if this has been asked before, I personally couldn't find if on Stackoverflow. Thanks for your attention!

If you need the data in the component as well, you can just subscribe to it. BUT maybe you should not (see below)...
it's there that you use the operators, you can combine observables to define a custom data flow:
foo$: Observable < Foo[] > ;
randomClickEvent = new Subject < clickEvent > ();
ngOnInit() {
let initialFetch = this.fooService.getData().share()
this.foo$ = Observable.merge(
initialFetch, // need the initial data
initialFetch.flatMap(foos => {
this.randomClickEvent.switchMap(() => { //listen to click events
return this.fooService.getMore().map((moreFoos: Foo[]) => { //load more foos
return foos.concat(...moreFoos) //initial foos values + new ones
})
})
})
);
}
<span *ngFor="let foo of (foo$|async)">{{foo.name}}</span>
<button (click)="randomClickEvent.next($event)">Load More foos !</button>
Most of people just use simple operators like map(),do(), etc and manage their subscription imperatively, but it is usually better to not subscribe, so you avoid many side effects and some "Ooops I forgot to unsubscribe here". usually you can do everything you need without subscribing.
Observables exist to describe a data flow, nothing more, nothing less. It's a paradigm of functional programming: you don't define how things are done, but what they are. Here, this.foo$ is a combination of the initial fooService.getData() and every fooService.fetchMore() that may occur.

This RxJava buffer example (with marble chart!) describes the desired result perfectly:
collect items in buffers during the bursty periods and emit them at the end of each burst, by using the debounce operator to emit a buffer closing indicator to the buffer operator
Edit: having reviewed How to create a RxJS buffer that groups elements in NodeJS but that does not rely on forever running interval?, my issue appears related to using a Subject as opposed to straight Observable.
Using the socket stream to generate window close events (as follows) results in 2 sockets opened and no events streaming out:
ws = Rx.DOM.fromWebSocket(wsURI, null, wsOpenObserver, wsCloseObserver);
var closer = ws.flatMapFirst(Rx.Observable.timer(250));
ws.buffer(closer)
.subscribe(function(e) { console.log(e, 'socket messages');});

Summarizing the findings issue here :
Rx.DOM.fromWebSocket returns a Rx.subject which wraps around the websocket. That subject is made from one observer and one observable (via new Rx.Subject(observer, observable). From what I understood, that observer allows to write to the socket via its onNext method, while the observable allows to read from the socket.
you always read that subjects are hot sources, but apparently here that only means that the observer will immediately push its value to the subject which here pushes it to the socket. In normal cases(new Rx.Subject()), the default observer and observable are so that the observable listens to the observer, hence the default observable is hot. Here however, the observable is a cold source and then any subscription will reexecute the callback creating another websocket. Hence the creation of two sockets.
this does not happen for instance with Rx.dom.fromEvent because the created (cold) observable is shared (via publish().refCount()).
thus by doing the same here, the duplication issue can be solved. That means in this particular case, use in your code ws = Rx.DOM.fromWebSocket(wsURI, null, wsOpenObserver, wsCloseObserver).share();, share being an alias for publish().refCount().
I have to wonder whether that behaviour of Rx.DOM.fromWebSocket should be reported as a bug
Code for both methods:
https://github.com/Reactive-Extensions/RxJS-DOM/blob/master/src/dom/websocket.js
https://github.com/Reactive-Extensions/RxJS-DOM/blob/master/src/events/fromevent.js

You can pass an Observable directly to the buffer operator just like the RxJava version:
source.buffer(source.debounce(150))
is valid. See here.
The alternative syntax using the selector method that you show will invoke that method every time a buffer closes and then subscribe to the Observable it produces.
Also the debounce in the RxJava example is emitting the result of the buffer operator, it does not emit accumulated results by default.

I've got a TypeScript/Angular 2 Observable that works perfectly the first time I call it. However, I'm interested in attaching multiple subscribers to the same observable and somehow refreshing the observable and the attached subscribers. Here's what I've got:
query(): Rx.Observable<any> {
return this.server.get('http://localhost/rawData.json').toRx().concatMap(
result =>
result.json().posts
)
.map((post: any) => {
var refinedPost = new RefinedPost();
refinedPost.Message = post.Message.toLowerCase();
return refinedPost;
}).toArray();
}
Picture that there is a refresh button that when pressed, re-executes this observable and any subscribers that are connected to it get an updated set of data.
How can I accomplish this?

I don't know so much about Angular2 and Typescript, but typescript being a superset of javascript, I made the following hypothesis (let me know if I am wrong) which should make the following javascript code work :
server.get returns a promise (or array or Rx.Observable)
posts is an array of post
You would need to create an observable from the click event on that button, map that click to your GET request and the rest should be more or less as you wrote. I cannot test it, but it should go along those lines:
// get the DOM id for the button
var button = document.getElementById('#xxx');
// create the observable of refined posts
var query$ =
Rx.Observable.fromEvent(button, 'click')
.flapMapLatest(function (ev) {
return this.server.get('http://localhost/rawData.json')
})
.map(function (result){return result.json().posts})
.map(function (posts) {
return posts.map(function(post){
var refinedPost = new RefinedPost();
refinedPost.Message = post.Message.toLowerCase();
return refinedPost;
})
})
.share()
// subscribe to the query$. Its output is an array of refinedPost
// All subscriptions will see the same data (hot source)
var subscriber = query$.subscribe(function(refinedPosts){console.log(refinedPosts)})
Some explanation :
Every click will produce a call to server.get which returns an observable-compatible type (array, promise, or observable). That returned observable is flattened to extract result from that call
Because the user can click many times, and each click generate its flow of data, and we are interested (another hypothesis I make here too) only in the result of the latest click, we use the operator flatMapLatest, which will perform the flatMap only on the observable generated by the latest click
We extract the array of post posts and make an array of refinedPost from it. In the end, every click will produce an array of refinedPost which I assume is what you want
We share this observable as you mention you will have several subscribers, and you want all subscribers to see the same data.
Let me know if my hypotheses are correct and if this worked for you.
In addition I recommend you to have a look at https://gist.github.com/staltz/868e7e9bc2a7b8c1f754
In addition to being a very good reminder of the concepts, it addresses a refresh server call problem very similar to yours.