We Keep Coding

How to create subset of observable basing on input argument?

I know the title might sound confusing, but I'm not sure how to express it accurately.
Let me explain the use case:
We've got a calendar with recurring events (potentially infinite list of events repeating eg every Monday forever)
I've got a bunch of UI elements interested in some part of those events (eg. some needs events from next week, some from next month, some from the previous year - they all might be displayed at once). Note that those requirements might overlap - the eg. current week and current month actually include an overlapping set of days, but those UI components are not aware of each other.
I need to refresh those events on some actions in the app (eg. the user changed calendar permission, list of calendars in his system, user minimized and re-opened the app) - I've got streams of those events like $appStatus $permissionStatus, $selectedCalendars
Fetching events is expensive and I want to re-use fetching and existing data as much as possible. Eg. if 3 components need events from the same week, I don't want to make 3 requests.
Also if one component requests events from March and another from 2nd week of March, I'd like to solve it with one request if possible. Note they might start their request at different moment.
I also don't want to fetch events that are not requested anymore (eg. part of UI showing events from the previous month is not rendered anymore)
I've got a few ideas about how to do it in plain JS, but I wonder if it's possible to create RxJs-ish solution for that
My raw-js-rx-mix ideas:
When a new range is requested, I 'round' it to full weeks. Then I create new observable with shared value for each week and remember it in some map
eg
const observablesMap = new Map();
function getObservableForDate(date: Date) {
const weekStart = getStartOfWeek(date);
// if we have 'cached' version of observable for this week - return cached one
if (observablesMap.has(weekStart.getTime()) {
return observablesMap.get(weekStart.getTime())
}
const weekObservable = // create observable that shares results
// set it to the map
// return it
}
But after a few hours of research, I have no idea if and how would I implement it in RxJS way.
Let's assume the fetching function signature is fetchEvents(startDate, endDate)
ps. I don't expect a working solution code, but just some guide. I've checked most of RxJS documentation and could not find anything promising for such use case

I created a running rxjs solution on stackblitz. I will also add the code here in case stackblitz shuts down one time in the future.
General idea
Save all requests and decide if you need a new request or not
Depending on previous decicion fake or process http request
Depending on the previous kind of request find the existing request or return the newly requested one.
If there need to be further infos please let me know and I try to explain in detail or add comments. I did not implement 100% of your requirements but with the following solution as base it should be possible to expand interfaces and functions within the pipes to implement them. Also if you see any code redundancy or optimization to interfaces, let me know and I will adapt
Interfaces
interface Requests {
action: Action,
currentRequest: number,
accumulatedRequests: number[],
}
interface FulfilledRequest extends Requests{
httpRequest: string
}
interface Response {
accumulatedHttpResponses: {
request: number,
response: string
}[],
response: string
}
Default Values
const defaultRequests: Requests = {
action: Action.IgnoreRequest,
currentRequest: -1,
accumulatedRequests: []
}
const defaultResponseStorage: Response = {
accumulatedHttpResponses: [],
response: ''
}
Enum
enum Action {
IgnoreRequest,
ProcessRequest
}
Functions
const isUpdateAction = (action: Action) => action === Action.ProcessRequest
const fakeHttp = (date: number): Observable<string> => of('http response for: ' + date).pipe(
tap(v => console.warn('fakeHttp called with: ', v))
);
const getResponseForExistingRequest = (storage: Response, request: FulfilledRequest): Response => {
const index = storage.accumulatedHttpResponses.findIndex(response => response.request === request.currentRequest);
return {
accumulatedHttpResponses: storage.accumulatedHttpResponses,
response: storage.accumulatedHttpResponses[index].response
}
}
const getResponseForNewRequest = (storage: Response, request: FulfilledRequest): Response => {
const newEntry = {request: request.currentRequest, response: request.httpRequest};
return {
accumulatedHttpResponses: [...storage.accumulatedHttpResponses, newEntry],
response: request.httpRequest
}
}
const getIgnoredRequest = (date: number, requests: Requests): Requests => ({
currentRequest: date,
action: Action.IgnoreRequest,
accumulatedRequests: requests.accumulatedRequests
})
const getProcessedRequests = (date: number, requests: Requests): Requests => ({
currentRequest: date,
action: Action.ProcessRequest,
accumulatedRequests: [...requests.accumulatedRequests, date]
})
const processRequest = (requests: Requests, date: number): Requests => {
const requestExists = requests.accumulatedRequests.some(request => request === date);
return requestExists
? getIgnoredRequest(date, requests)
: getProcessedRequests(date, requests)
}
const processFulfilledRequest = (storage: Response, request: FulfilledRequest): Response => isUpdateAction(request.action)
? getResponseForNewRequest(storage, request)
: getResponseForExistingRequest(storage, request)
const fulfillFakeRequest = (requests: Requests): Observable<FulfilledRequest> => of('').pipe(
map(response => ({...requests, httpRequest: response})),
)
const fulfillHttpRequest = (requests: Requests): Observable<FulfilledRequest> => fakeHttp(requests.currentRequest).pipe(
map(response => ({...requests, httpRequest: response}))
)
Final Connection via Observables
const date$: Subject<number> = new Subject();
const response$ = date$.pipe(
scan(processRequest, defaultRequests),
switchMap((requests): Observable<FulfilledRequest> => isUpdateAction(requests.action)
? fulfillHttpRequest(requests)
: fulfillFakeRequest(requests)
),
scan(processFulfilledRequest, defaultResponseStorage),
map(response => response.response)
)

RxJS - How to share the output of an expensive observable but rerun that observable if its requested again after N seconds?

Let's say we have this global const:
const isSignedIn = fromPromise(fetch('/api/is-signed-in'))
.pipe(throttleTime(1000), shareReply(1));
After page load, several components will subscribe to this at the same time:
isSignedIn.subscribe(() => console.log('do 1st'));
isSignedIn.subscribe(() => console.log('do 2nd'));
isSignedIn.subscribe(() => console.log('do 3rd'));
The above will only call the API once, however i need it to call the API again (ie after 1 second) if another component subscribes to it.
isSignedIn.subscribe(() => console.log('button press'));
How do i that using RxJS?

I think this is what you want:
A pipeable operator (declare globally somewhere and import it)
export const refreshAfter = (duration: number) => (source: Observable<any>) =>
source.pipe(
repeatWhen(obs => obs.pipe(delay(duration))),
publishReplay(1),
refCount());
Then use it like this:
data$ = fetch('/api/is-signed-in').pipe(refreshAfter(5000)); // refresh after 5000 ms
Note: You actually asked for this:
i need it to call the API again (ie after 1 second) if another component subscribes to
it.
Not quite sure this is what you really meant. I think what you really meant was - you want the data to be refreshed for all components currently subscribed after an expiry time. Anyway my answer sends the new value to all listeners. If you really want what you originally said you'd need to add some kind of alternative repeat trigger.
But if this is for a global constant - the above is what I'm using for the same scenario.
Note: I haven't actually tested the handling of an error condition when the item is repested, but I think the error will propagate to all listeners.

If we reimplement ShareReplay so it:
- will never unsubscribe from source even if it have no more subscribers (remove refCount, potential memory leak).
- accept rerunAfter argument, time passed from last subscribe to source.
import {Subject, of, Observable, ReplaySubject, Subscriber} from 'rxjs';
import {pluck, shareReplay, tap, delay} from 'rxjs/operators';
function shareForeverReplayRerun<T>(bufferSize: number, rerunAfter: number) {
let subject;
let subscription;
let hasError = false;
let isComplete = false;
let lastSubTime = 0;
return source => Observable.create((observer: Subscriber<T>) => {
if (!subject || hasError || (Date.now() - lastSubTime) >= rerunAfter) {
lastSubTime = Date.now();
hasError = false;
subject = new ReplaySubject<T>(bufferSize);
subscription = source.subscribe({
next(value) { subject.next(value); },
error(err) {
hasError = true;
subject.error(err);
},
complete() {
isComplete = true;
subject.complete();
},
});
}
const innerSub = subject.subscribe(observer);
// never unsubscribe from source
return () => {
innerSub.unsubscribe();
};
})
}
const source = of('Initial').pipe(
tap(()=>console.log('COMPUTE')),
delay(200),
shareReplayRerun(1, 1000),
);
source.subscribe(console.log.bind(null, 'syncI:'));
source.subscribe(console.log.bind(null, 'syncII:'));
setTimeout(()=>source.subscribe(console.log.bind(null, 'after500:')), 500);
setTimeout(()=>source.subscribe(console.log.bind(null, 'after900:')), 900);
setTimeout(()=>source.subscribe(console.log.bind(null, 'after1500:')), 1500);
as output we have:
COMPUTE
syncI: Initial
syncII: Initial
after500: Initial
after900: Initial
COMPUTE
after1500:Initial

EDITED: The answer is wrong. BufferSize is how long the last N events are replayed. After this the stream is completed.
signature: shareReplay(
bufferSize?: number,
windowTime?: number,
scheduler?: IIScheduler
):Observable
#param {Number} [bufferSize=Number.POSITIVE_INFINITY] Maximum element count of the replay buffer.
#param {Number} [windowTime=Number.MAX_VALUE] Maximum time length of the replay buffer in milliseconds.
Try to add 1000 as second argument to shareReply:
const isSignedIn = fromPromise(fetch('/api/is-signed-in'))
.pipe(throttleTime(1000), shareReplay(1, 1000));
shareReplay.ts - be care of refCount-- on unsubcribe as it can trigger additional requests.

How can I make withLatestFrom() work with itself?

I have intermediate stream which bound to source but also can fire events from other sources (like user input). At some other place of my program I have derived stream which needs to compare new impulse from intermediate with the last value of source, so it all comes down to such code:
const source = new Rx.Subject;
const derived = new Rx.Subject;
derived.subscribe( () => console.log( "derived" ) );
const intermediate = new Rx.Subject;
//motivation of having "intermediate" is that sometimes it fires on it's own:
source.subscribe( intermediate );
intermediate
.withLatestFrom( source )
.subscribe( derived );
source.next();
<script src="https://cdnjs.cloudflare.com/ajax/libs/rxjs/5.4.3/Rx.min.js"></script>
The problem is that "derived" message is never printed (first event in source is ignored). How can I make some stream that for every message from intermediate stream gets the last value of source even if it's currently the propagation of source?

If I understand correctly, you have a source stream, then an intermediate stream which subscribes to and emits all of source's items and mixes in some others from user input. Then you have two differing requests, which I'll offer suggestions for separately:
Emitting values of source with timings from intermediate: combineLatest will do the job easily:
const source_when_intermediate : Observable<TSource> = intermediate.combineLatest(source, (i, s) => s);
Comparing the latest value from intermediate with source: While this sounds very similar, combineLatest isn't very safe, because if you do something simple like:
const is_different : Observable<bool> = intermediate.combineLatest(source, (i, s) => i !== s);
intermediate might not emit a given value from source before source emits a new one, and you could think that that stale value is unique to intermediate.
Instead, for maximum safety, you'll need to buffer and use a subject:
// untested
const derived : Observable<TSource> = (function() {
const source_buffer = new Queue();
const subj = new Subject();
source.forEach(s => {
source_buffer.enqueue(s);
});
intermediate.forEach(i => {
subj.onNext(source_queue.peek() === i);
if(source_buffer.peek() === i) source_buffer.dequeue();
});
Promise.all([
source.toPromise(),
intermediate.toPromise()
]).then(() => subj.onClose());
return subj.asObservable();
})();

RxJS Observable merge angular

I have a sandbox which subscribes to a stream of messages and I want to filter that stream to find messages that have been sent to or received from a specific user using route params specified in another component.
messages.sandbox.ts:
messages$: Observable<Array<Message>> = this.store.select(state => state.data.messages);
fetchReceived(id: string): Observable<Array<Message>> {
return this.messages$.map((messages: any) => {
return messages.filter((message: Message) => {
return message.recipientId == id;
});
});
}
fetchSent(id: string): Observable<Array<Message>> {
return this.messages$.map((messages: any) => {
return messages.filter((message: Message) => {
return message.userId == id;
})
})
}
messages.detail.container.ts
sentMessages$ = new Observable<Array<Message>>();
receivedMessages$ = new Observable<Array<Message>>();
matchingMessages$ = new Observable<Array<Message>>();
ngOnInit() {
this.route.params.subscribe((params: Params) => {
this.sentMessages$ = this.sb.fetchReceived(params['id']);
this.receivedMessages$ = this.sb.fetchSent(params['id']);
this.matchingMessages$ = Observable.merge(this.sentMessages$, this.receivedMessages$);
});
}
this.matchingMessages$ seems to only include this.receivedMessages$ however I know that this.sentMessages$ is not null as I can use it in my template without a problem.
Am I missing something with merging Observables? Would it be better to create a single fetchMessages method which filters for either the userId or recipientId equalling the route param id? If so how would I go about that?
Thanks!!

You have the right general idea. Just a few flaws.
Never use new Observable<T>(). It does not do what you think it does. It pretty much does not do anything useful. Always construct observables from factory methods or other observables
You need to transform the params observable into a new observable using an operator. Your problem is you subscribe to the params observable, and then construct new observables each time. But other code will have already subscribed to the initial observables so they will never see the changes.
So you want to do something like this:
sentMessages$ : Observable<Array<Message>>;
receivedMessages$ : Observable<Array<Message>>;
matchingMessages$ : Observable<Array<Message>>;
ngOnInit() {
const params$ = this.route.params;
// use switchMap to map the new params to a new sent observable
// each time params change, unsubscribe from the old fetch and subscribe
// to the new fetch. Anyone subscribed to "sentMessages" will see the
// change transparently
this.sentMessages$ = params$.switchMap((params: Params) => this.sb.fetchReceived(params['id']));
// same for received
this.receivedMessages$ = params$.switchMap((params: Params) => this.sb.fetchSent(params['id'])));
// merge the 2 streams together
this.matchingMessages$ = Observable.merge(this.sentMessages$, this.receivedMessages$);
}
Edit:
to answer your other question: is it better to create a single observable that matches senders and receivers: depends upon your use case. But here is how you could go about it:
messages.sandbox.ts:
fetchEither(id: string): Observable<Array<Message>> {
return this.messages$.map((messages: any) => {
return messages.filter((message: Message) => {
return message.recipientId == id || message.userId === id;
});
});
}
container:
matchingMessages$ : Observable<Array<Message>>;
ngOnInit() {
const params$ = this.route.params;
// use switchMap to map the new params to a new either observable
// each time params change, unsubscribe from the old and subscribe
// to the new fetch. Anyone subscribed to "matchingMessages" will see the
// change transparently
this.matchingMessages$ = params$.switchMap((params: Params) => this.sb.fetchEither(params['id']));
}

How do I ensure that a call to subscribe on the observer initially receives the most recent value?

I am using rxjs together with Angular 2 and Typescript. I would like to share a common web-resource (a "project" in the context of my app, essentially a JSON document) between multiple components. To achieve this I introduced a service that exposes an observable, which will be shared by all clients:
/**
* Handed out to clients so they can subscribe to something.
*/
private _observable : Observable<Project>;
/**
* Used to emit events to clients.
*/
private _observer : Observer<Project>;
constructor(private _http: Http) {
// Create observable and observer once and for all. These instances
// are not allowed to changed as they are passed on to every subscriber.
this._observable = Observable.create( (obs : Observer<Project>) => {
this._observer = obs;
});
}
Clients now simply get a reference to that one _observable and subscribe to it.
/**
* Retrieves an observable that always points to the active
* project.
*/
get ActiveProject() : Observable<Project> {
return (this._observable);
}
When some component decides to actually load a project, it calls the following method:
/**
* #param id The id of the project to set for all subscribers
*/
setActiveProject(id : string) {
// Projects shouldn't change while other requests are in progress
if (this._httpRequest) {
throw { "err" : "HTTP request in progress" };
}
this._httpRequest = this._http.get('/api/project/' + id)
.catch(this.handleError)
.map(res => new Project(res.json()));
this._httpRequest.subscribe(res => {
// Cache the project
this._cachedProject = res;
// Show that there are no more requests
this._httpRequest = null;
// Inform subscribers
this._observer.next(this._cachedProject)
console.log("Got project");
});
}
It basically does a HTTP request, transforms the JSON document into a "proper" instance and calls this._observer.next() to inform all subscribers about the change.
But if something subscribes after the HTTP request has already taken place, the see nothing until a new HTTP request is issued. I have found out that there is some kind of caching (or replay?) mechanism in rxjs that seems to adress this, but I couldn't figure out how to use it.
tl;dr: How do I ensure that a call to subscribe on the observer initially receives the most recent value?
Extra question: By "pulling the observer out of the observable" (in the constructor), have I essentially created a subject?

That's what BehaviorSubject does
import { BehaviorSubject } from 'rxjs/subject/BehaviorSubject';
...
obs=new BehaviourSubject(4);
obs.subscribe(); //prints 4
obs.next(3); //prints 3
obs.subscribe(); //prints 3

I usually achieve this with shareReplay(1). Using this operator with 1 as parameter will ensure that the latest value emitted will be kept in a buffer, so when there is a new subscriber that value is immediately passed on to it. You can have a look at the documentation :
var interval = Rx.Observable.interval(1000);
var source = interval
.take(4)
.doAction(function (x) {
console.log('Side effect');
});
var published = source
.shareReplay(3);
published.subscribe(createObserver('SourceA'));
published.subscribe(createObserver('SourceB'));
// Creating a third subscription after the previous two subscriptions have
// completed. Notice that no side effects result from this subscription,
// because the notifications are cached and replayed.
Rx.Observable
.return(true)
.delay(6000)
.flatMap(published)
.subscribe(createObserver('SourceC'));
function createObserver(tag) {
return Rx.Observer.create(
function (x) {
console.log('Next: ' + tag + x);
},
function (err) {
console.log('Error: ' + err);
},
function () {
console.log('Completed');
});
}
// => Side effect
// => Next: SourceA0
// => Next: SourceB0
// => Side effect
// => Next: SourceA1
// => Next: SourceB1
// => Side effect
// => Next: SourceA2
// => Next: SourceB2
// => Side effect
// => Next: SourceA3
// => Next: SourceB3
// => Completed
// => Completed
// => Next: SourceC1
// => Next: SourceC2
// => Next: SourceC3
// => Completed
Extra question: By "pulling the observer out of the observable" (in
the constructor), have I essentially created a subject?
I am not sure what you mean by that, but no. A subject is both an observer and an observable and have specific semantics. It is not enough to 'pull the observer out of the observable' as you say. For subjects semantics, have a look here : What are the semantics of different RxJS subjects?