Is there a way to make sure the order on how subscribers get updated is ensured?
I've got a hot observable and my first subscriber does some sync work to update a variable and my next subscriber then has to initialise a service (only once!), and only after that variable is ensured to be set!
it looks like this:
import App from './App'
var appSource = App.init() // gets the hot observable
// our second subscriber
appSource.take(1).subscribe(() => {
// take 1 to only run this once
nextService.init()
})
where App.init looks like this:
...
init() {
var source = this.createObservable() // returns a hot interval observable that fetches a resource every few minutes
// first subscriber, updates the `myVar` every few minutes
source.subscribe((data) => this.myVar = data)
return source
}
...
this currently works, but I am unsure if it will always follow the order 100%.
EDIT:
As I've heard, subscribers will be invoked FIFO. So the order is somewhat assured.
I don't know if RxJS ever explicitly guarantees that observers are called in order of subscription. But, as you say, it usually works.
However, you might consider modelling your actual workflow instead of relying on implicit observer order.
It sounds like you need to know when your app is initialized so you can take further action. Instead of relying on knowledge of the internal workings of App.init, App could expose an API for this:
One (non-Rx way) is to let the caller supply a callback to init:
//...
init(callback) {
var source = this.createObservable() // returns a hot interval observable that fetches a resource every few minutes
// first subscriber, updates the `myVar` every few minutes
source.subscribe((data) => {
this.myVar = data;
if (callback) {
callback();
callback = undefined;
}
})
return source
}
// elsewhere
App.init(() => nextService.init());
Another option instead of a callback is to just have init return a Promise that your resolve (or an Rx.AsyncSubject that you signal) once initialization is complete.
And yet another option, but requires a bit of a refactor, is to model this.myVar as the observable data that it is. i.e.:
init() {
this.myVar = this.createObservable().replay(1);
this.myVar.connect();
// returns an observable that signals when we are initialized
return this.myVar.first();
}
// elsewhere, you end up with this pattern...
const servicesToInit = [ App, service1, service2, service3 ];
Observable
.of(servicesToInit)
.concatMap(s => Rx.Observable.defer(() => s.init()))
.toArray()
.subscribe(results => {
// all initializations complete
// results is an array containing the value returned by each service's init observable
});
Now, anything that wants to make use of myVar would always need to subscribe to it in someway to get the current and/or future values. They could never just synchronously ask for the current value.
Related
Short backstory: I am trying to create a Readable stream based on data chunks that are emitted back to my server from the client side with WebSockets. Here's a class I've created to "simulate" that behavior:
class DataEmitter extends EventEmitter {
constructor() {
super();
const data = ['foo', 'bar', 'baz', 'hello', 'world', 'abc', '123'];
// Every second, emit an event with a chunk of data
const interval = setInterval(() => {
this.emit('chunk', data.splice(0, 1)[0]);
// Once there are no more items, emit an event
// notifying that that is the case
if (!data.length) {
this.emit('done');
clearInterval(interval);
}
}, 1e3);
}
}
In this post, the dataEmitter in question will have been created like this.
// Our data is being emitted through events in chunks from some place.
// This is just to simulate that. We cannot change the flow - only listen
// for the events and do something with the chunks.
const dataEmitter = new DataEmitter();
Right, so I initially tried this:
const readable = new Readable();
dataEmitter.on('chunk', (data) => {
readable.push(data);
});
dataEmitter.once('done', () => {
readable.push(null);
});
But that results in this error:
Error [ERR_METHOD_NOT_IMPLEMENTED]: The _read() method is not implemented
So I did this, implementing read() as an empty function:
const readable = new Readable({
read() {},
});
dataEmitter.on('chunk', (data) => {
readable.push(data);
});
dataEmitter.once('done', () => {
readable.push(null);
});
And it works when piping into a write stream, or sending the stream to my test API server. The resulting .txt file looks exactly as it should:
foobarbazhelloworldabc123
However, I feel like there's something quite wrong and hacky with my solution. I attempted to put the listener registration logic (.on('chunk', ...) and .once('done', ...)) within the read() implementation; however, read() seems to get called multiple times, and that results in the listeners being registered multiple times.
The Node.js documentation says this about the _read() method:
When readable._read() is called, if data is available from the resource, the implementation should begin pushing that data into the read queue using the this.push(dataChunk) method. _read() will be called again after each call to this.push(dataChunk) once the stream is ready to accept more data. _read() may continue reading from the resource and pushing data until readable.push() returns false. Only when _read() is called again after it has stopped should it resume pushing additional data into the queue.
After dissecting this, it seems that the consumer of the stream calls upon .read() when it's ready to read more data. And when it is called, data should be pushed into the stream. But, if it is not called, the stream should not have data pushed into it until the method is called again (???). So wait, does the consumer call .read() when it is ready for more data, or does it call it after each time .push() is called? Or both?? The docs seem to contradict themselves.
Implementing .read() on Readable is straightforward when you've got a basic resource to stream, but what would be the proper way of implementing it in this case?
And also, would someone be able to explain in better terms what the .read() method is on a deeper level, and how it should be implemented?
Thanks!
Response to the answer:
I did try registering the listeners within the read() implementation, but because it is called multiple times by the consumer, it registers the listeners multiple times.
Observing this code:
const readable = new Readable({
read() {
console.log('called');
dataEmitter.on('chunk', (data) => {
readable.push(data);
});
dataEmitter.once('done', () => {
readable.push(null);
});
},
});
readable.pipe(createWriteStream('./data.txt'));
The resulting file looks like this:
foobarbarbazbazbazhellohellohellohelloworldworldworldworldworldabcabcabcabcabcabc123123123123123123123
Which makes sense, because the listeners are being registered multiple times.
Seems like the only purpose of actually implementing the read() method is to only start receiving the chunks and pushing them into the stream when the consumer is ready for that.
Based on these conclusions, I've come up with this solution.
class MyReadable extends Readable {
// Keep track of whether or not the listeners have already
// been added to the data emitter.
#registered = false;
_read() {
// If the listeners have already been registered, do
// absolutely nothing.
if (this.#registered) return;
// "Notify" the client via websockets that we're ready
// to start streaming the data chunks.
const emitter = new DataEmitter();
const handler = (chunk: string) => {
this.push(chunk);
};
emitter.on('chunk', handler);
emitter.once('done', () => {
this.push(null);
// Clean up the listener once it's done (this is
// assuming the #emitter object will still be used
// in the future).
emitter.off('chunk', handler);
});
// Mark the listeners as registered.
this.#registered = true;
}
}
const readable = new MyReadable();
readable.pipe(createWriteStream('./data.txt'));
But this implementation doesn't allow for the consumer to control when things are pushed. I guess, however, in order to achieve that sort of control, you'd need to communicate with the resource emitting the chunks to tell it to stop until the read() method is called again.
I am new to Angular (just learning) and I am trying to understand the following code. In this code (I simplified it from another app I have seen on Github) subscribe method calls add method with the parameter resolve (from Promise) - so I have a few questions:
Doesn't parameter passed to add mehtod have to be of type `Subscription' ?
What does framework do with passed resolve parameter. I Thought the parameter must be of type Subscription and framework calls <Subscription>.unsubscribe() on it.
const numbers: Observable<number> = interval(5000);
const takeFourNumbers = numbers.pipe(take(4));
const promise$ = new Promise<void>(resolve => {
// attempt to refresh token on app start up to auto authenticate
takeFourNumbers.subscribe()
.add(resolve);
}).then((result) => {
console.log("My result is ", result);
});
That's a bit of an odd piece of code.
Every Subscription has an .add( function, which is used to trigger something else on teardown, this is when that subscription ends (either because the stream errors, completes, or you call .unsubscribe())
The parameter that .add( takes can be a function (in which case it just gets called on teardown), or another subscription, in which case it will call .unsubscribe() to it.
It's not really used too much unless you're building something low-level (such a library)
In this case, takeFourNumbers is a stream that will emit 4 numbers in succession, 1 second between each emission and then complete. promise$ is a Promise that when takeForNumbers ends after 4 seconds it will resolve with void, because the teardown will call the function passed to .add(, which is resolve, and it doesn't give any parameter as far as I know.
Then on that promise it calls .then( to log the result when that happens, but I expect the result to be undefined.
I want to create a class whose duty is to poll data sources, collate information into an array of 'alert' objects, and then deliver a subset of those alerts to any other class that wants them.
Because polling happens asynchronously (I'm requesting data from a web service) then I assume that what I actually need to return is a promise which, when fulfilled, will give the correct subset of Alert objects.
But clearly I don't understand how to do this, because the method that is supposed to return the promise returns something else.
Here's my code so far. As you can see, I'm trying to store the promise in an instance attribute and then retrieve it:
export class AlertCollection {
constructor() {
this.alerts = null;
}
// poll the data sources for alert data; store a promise that resolves
// to an array of alerts
poll() {
this.alerts = this.pollTeapot()
.then( (arr) => {this.pollDeliverance(arr);} );
}
// return a promise that fulfils to an array of the alerts you want
filteredAlerts(filter) {
return this.alerts; // not filtering for now
}
// return a promise that fulfills to the initial array of alerts
pollTeapot() {
let process = (json) => {
json2 = JSON.parse(json);
return json2.map( (a) => new Alert(a) );
};
message = new MessageHandler("teapot", "alerts")
return message.request().then( (json) => {process(json);} );
}
// Modify the alerts based on the response from Deliverance.
// (But for the time being let's not, and say we did.)
pollDeliverance(alerts) {
return alerts;
}
}
message.request() returns a promise from the web service. That works. If I snapshot the process function inside pollTeapot() I get the right data.
But, if I snapshot the return value from filteredAlerts() I don't get that. I don't get null either (which would at at least make sense, although it would be wrong.) I get something like { _45: 0, _81: 0, _65: null, _54: null }.
Any pointers would be very much appreciated at this point. (This is in React Native, by the way, if that helps.)
I am not sure if I understood your problem fully, but I will try to give you an generic solution to chaining promises one after another.
someAsyncFunction().then(dataFromAsync1 => {
return anotherAsyncFunction(dataFromAsync1).then(dataFromAsync2 => {
return doSomethingWithData(dataFromAsync1, dataFromAsync2);
});
});
This is going to be a hard one to describe - I have a working example but it's convoluted in that I've had to "mock up" all of the async parts, and use function classes rather than the class keyword - but the idea is the same!
There are 2 parts to this answer.
It does not make sense to store alerts as an instance variable. They are asynchronous, and wont exist until after the async calls have completed
You'll need to chain all of your behaviour onto the initial call to poll
In general, you chain promises on to one another like this
functionWhichReturnsPromise()
.then(functionPointer)
.then(function(result){
// some functionality, which can return anything - including another promise
});
So your code would end up looking like
var alertCollection = new AlertCollection()
alertCollection.poll().then(function(alerts){
//here alerts have been loaded, and deliverance checked also!
});
The code of that class would look along the lines of:
export class AlertCollection {
constructor() {
}
// poll the data sources for alert data; store a promise that resolves
// to an array of alerts
poll() {
return this.pollTeapot()
.then(filteredAlerts)
.then(pollDeliverance);
}
// return a promise that fulfils to an array of the alerts you want
filteredAlerts(alerts) {
return alerts; // not filtering for now
}
// return a promise that fulfills to the initial array of alerts
pollTeapot() {
let process = (json) => {
json2 = JSON.parse(json);
return json2.map( (a) => new Alert(a) );
};
message = new MessageHandler("teapot", "alerts")
return message.request().then(process);
}
// Modify the alerts based on the response from Deliverance.
// (But for the time being let's not, and say we did.)
pollDeliverance(alerts) {
return alerts;
}
}
A few notes
filteredAlerts can do whatever you like, so long as it returns an array of results
pollDeliverance can also do whatever you like - if it needs to call another async method, remember to return a promise which resolves to an array of alerts - perhaps updated from the result of the async call.
I have created a JSFiddle which demonstrates this - using a simple getJSON call to replicate the async nature of some of this. As I mentioned, it is convoluted, but demonstrates the process:
Live example: https://jsfiddle.net/q1r6pmda/1/
I might be off on the process, but here goes:
I have an angular2 service. The source for the data of this service is going to be localstorage... later optionally updated when a DB call using http returns. Because I'll be wanting to update the data returned as the various sources come back, it appears I want to use an observables. For now, I'm just trying to get the concept down, so I've skipped the localstorage aspect... but I'm including the 'backstory' so it makes (some) sense as to why I'm wanting to do this in multiple methods.
My thought was I would have a "getHTTPEvents()" method that would return an observable with the payload being the events from the DB. (the theory being that at some point in the future I'd also have a 'getLSEvents()' method that would piggy back in there)
To mock that up, I have this code:
private eventsUrl = 'app/mock-events.json';
getHTTPEvents() : Observable<Array<any>> {
return this._http.get(this.eventsUrl)
.map(response => response.json()['events'])
.catch(this.handleError); // handle error is a logging method
}
My goal would be to create a method that allows filtering on the returned events yet still returns an observable to users of the service. That is where my problem is. With that goal, I have a public method which will be called by users of the service. (attempted to use pattern from here https://coryrylan.com/blog/angular-2-observable-data-services)
public getEvents(key:string,value:string) : Observable<Array<any>> {
var allEventsObserve : Observable<Array<any>> = this.getHTTPEvents();
var filteredEventsObserve : Observable<Array<any>>;
allEventsObserve
.subscribe(
events => {
for(var i=0;i<events.length;i++) {
if(events[i][key]==value) {
console.log('MATCH!!!' + events[i][key]); // THIS WORKS!
return new Observable(observer => filteredEventsObserve = observer); // what do I need to return here? I want to return an observable so the service consumer can get updates
}
}
return allEventsObserve
},
error => console.error("Error retrieving all events for filtering: " + error));
}
The above doesn't work. I've watch lots of videos and read lots of tutorials about observables, but nothing I can find seems to go more indepth other than creating and using the http observable.
I further tried this method of making the new observable:
var newObs = Observable.create(function (observer) {
observer.next(events[i]);
observer.complete(events[i]);
});
And while at least that compiles, I'm not sure how to 'return' it at the right time... as I can't "Create" it outside the allEventsObserve.subscribe method (because 'events' doesn't exist) and can't (seem) to "return" it from within the subscribe. I'm also not entirely sure how I'd then "trigger" the 'next'...?
Do I need to modify the data within allEventsObserve and somehow simply still return that? Do I make a new observable (as attempted above) with the right payload - and if so, how do I trigger it? etc... I've checked here: How to declare an observable on angular2 but can't seem to follow how the 'second' observable gets triggered. Perhaps I have the entire paradigm wrong?
It appears that you're misunderstanding what an RxJS operator (like map, filter, etc) actually returns, and I think correcting that will make the solution clear.
Consider this short example:
allEventsObserve
.map(events => {
return 'this was an event';
})
Granted, it's a pretty useless example since all of the data from events is lost, but let's ignore that for now. The result of the code above is not an array of strings or anything else, it's actually another Observable. This Observable will just emit the string 'this was an event' for each array of events emitted by allEventsObserve This is what allows us to chain operators on observables -- each operator in the chain returns a new Observable that emits items that have been modified in some way be the previous operator.
allEventsObserve
.map(events => {
return 'this was an event';
})
.filter(events => typeof events !== 'undefined')
allEventsObserve is obviously an Observable, allEventsObserve.map() evaluates to an Observable, and so does allEventsObserve.map().filter().
So, since you're expecting your function to return an Observable, you don't want to call subscribe just yet, as doing so would return something that isn't really an Observable.
With that in mind, your code can be rewritten in the following way:
public getEvents(key:string,value:string) : Observable<Array<any>> {
var allEventsObserve : Observable<Array<any>> = this.getHTTPEvents();
return allEventsObserve
.map(events => {
var match = events.filter(event => event[key] == value);
if (match.length == 0) {
throw 'no matching event found';
} else {
return match[0];
}
})
.catch(e => {
console.log(e);
return e;
});
}
Since getEvents returns an Observable, somewhere else in your code you would do something like getEvents().subscribe(events => processEvents()) to interact with them. This code also assumes that this.getHTTPEvents() returns an Observable.
Also, notice that I changed your for loop to a call to filter, which operates on arrays. events in this case is a plain-old JavaScript Array, so the filter that is getting called is not the same filter as the RxJS operator filter.
Let's say in an Angular app, I have a controller which outputs a list of data, formatted according to configured rules. Both data should be fetched asynchronously:
list config (for instance, which attributes to be displayed)
data
Alright, since both data (config and data) are retrieved from a backend using $http/ngResource/Restangular, the code may look like this:
angular.module('myApp').controller('ListCtrl', function (Backend) {
var config,
data;
var draw = function() {
Backend.getData(function(retrievedData) {
// data retrieved asynchronously, store them:
data = retrievedData;
// to generate the list, we need both config and data:
if (!config) {
// <---- How can I wait until config has been loaded?
}
$scope.list = generateList(config, data);
}
// let's say, Backend.getConfig would make an asynch call and fetch the config data
Backend.getConfig(function(retrievedConfig) {
// config data retrieved asynchronously, store them:
config= retrievedConfig;
});
}
I hope you get the point: to generate the list and assign it to the scope, both input data is needed. But since the config does not change in the short run, I don't want to use $q.all() to wait for both. So, the config data should only be fetched the first time and then held in the controller (referenced by the variable "config"). This is implemented in the code above, but if somebody fires the 'draw()' function and the config is still loading, how can I make the code "wait" until the necessary config data has been fetched?
$emit/$broadcast come to my mind, but I don't like the idea because it feels like the wrong weapon for this target.
Are there other possibilities?
I would wrap the Backend.getConfig() call with another service that executes the call only when needed and returns a promise. The other times, it returns a promise resolved with a cached value. E.g.:
.service("getCachedConfig", function(Backend, $q) {
var cachedConfig;
return function getCachedConfig(cb) {
var deferred = $q.defer();
deferred.promise.then(cb); // install the callback, to adhere to your API; I would consider using promises all the way...
if( cachedConfig != null ) {
deferred.resolve(cachedConfig);
}
else {
Backend.getConfig(function(retrievedConfig) {
cachedConfig = retrievedConfig;
deferred.resolve(cachedConfig);
});
}
return deferred.promise;
};
})
Now you can do $q.all(Backend.getPromiseForData(...), getCachedConfig()) (assuming the method Backend.getPromiseForData returns a promise). The config will be fetched only once. You can trivially enhance this to add a cache expiration.