I am using rxjs to create a "channel nummber" selector for a remote control on a smart tv. The idea being that as you are entering the numbers you would see them on the screen and after you have finished entering the numbers, the user would would actually be taken to that channel.
I use two observables to achieve this:
A "progress" stream that listens to all number input and emits the concatenated number string out as the numbers are inputed via the scan operator.
A "completed" stream that, after n milliseconds of no number having being entered, would emit the final numeric string as completed. EG: 1-2-3 -> "123".
Here is the code that I use to try and solve this:
channelNumber:
module.exports = function (numberKeys, source, scheduler) {
return function (completedDelay) {
var toNumericString = function (name) {
return numberKeys.indexOf(name).toString();
},
concat = function (current, numeric) {
return current.length === 3 ? current : current + numeric;
},
live = createPress(source, scheduler)(numberKeys)
.map(toNumericString)
.scan(concat, '')
.distinctUntilChanged(),
completed = live.flatMapLatest(function (value) {
return Rx.Observable.timer(completedDelay, scheduler).map(value);
}),
progress = live.takeUntil(completed).repeat();
return {
progress: progress,
completed: completed
};
};
};
createPress:
module.exports = function (source, scheduler) {
return function (keyName, throttle) {
return source
.filter(H.isKeyDownOf(keyName))
.map(H.toKeyName);
};
};
createSource:
module.exports = function (provider) {
var createStream = function (event) {
var filter = function (e) {
return provider.hasCode(e.keyCode);
},
map = function (e) {
return {
type: event,
name: provider.getName(e.keyCode),
code: e.keyCode
};
};
return Rx.Observable.fromEvent(document, event)
.filter(filter)
.map(map);
};
return Rx.Observable.merge(createStream('keyup'), createStream('keydown'));
};
Interestingly the above code, under test conditions (mocking source and scheduler using Rx.TestScheduler) works as expected. But in production, when the scheduler is not passed at all and source is the result of createPress (above), the progress stream only ever emits until complete, and then never again. It's as if the repeat is completely ignored or redundant. I have no idea why.
Am I missing something here?
You can use Window. In this case, I would suggest WindowWithTime. You can also do more interesting things like use Window(windowBoundaries) and then pass the source with Debounce as boundary.
source
.windowWithTime(1500)
.flatMap(ob => ob.reduce((acc, cur) => acc + cur, ""))
Also, since our windows are closed observables, we can use Reduce to accumulate the values from the window and concat our number.
Now, this variant will close after 1,5 second. Rather, we would want to wait x seconds after the last keypress. Naïve we could do source.window(source.debounce(1000)) but now we subscribe to our source twice, that's something we want to avoid for two reasons. First we do not know is subscribing has any side effects, second we do not know the order subscriptions will receive events. That last thing isn't a problem since we use debounce that already adds a delay after the last keypress, but still something to consider.
The solution is to publish our source. In order to keep the publish inside the sequence, we wrap it into observable.create.
Rx.Observable.create(observer => {
var ob = source.publish();
return new Rx.CompositeDisposable(
ob.window(ob.debounce(1000))
.subscribe(observer),
ob.connect());
}).flatMap(ob => ob.reduce((acc, cur) => acc + cur, ""))
Edit: Or use publish like this:
source.publish(ob => ob.window(ob.debounce(1000)))
.flatMap(ob => ob.reduce((acc, cur) => acc + cur, ""))
Related
I am looking on how to make my code after my filter function await the results of my filter function to complete before running. However I am not sure how to do this.
My filter function takes in another function (useLocalCompare) which causes the execution of my filter function to be a little longer than normal, which then leads to my next piece of code (that depends on the results of my filter function) executing before my filter function is complete.....which leads to undefined.
Is there anything similar to a callback I can use to force my subsequent piece of code to wait till the filter is finished?
Relevant code is written below.
if (flatarrayofvalues !== null && genre !== null) {
const filtteredarray = await flatarrayofvalues.filter(
(placeholder) => {
if (useLocalCompare(genre, placeholder.name) == true) {
console.log("HURAY!!!!", placeholder.id, placeholder.name);
placeholder.name == placeholder.name;
}
}
);
console.log("MY FILTERED ARRAY IS", filtteredarray);
console.log("The ID FOR MY MY FILERED ARRAY IS two ID", filtteredarray[0]?.id);
return filtteredarray[0].id;
}
}
}
For those curious, useLocalCompare basically checks to see if the genre parameter pulled down from the URL is the same as a name parameter from the array I am filtering. Reason I have this is due to people having different case sensitivity when putting in URLS. EX: it will pull down "HORrOR" and match it to the object name in the array I am filtering called "horror". I then extract the ID from that object.
you have to return the conditional from filter as it is "explicit return"
const filtteredarray = await flatarrayofvalues.filter(
(placeholder) => {
if (useLocalCompare(genre, placeholder.name) == true) {
console.log("HURAY!!!!", placeholder.id, placeholder.name);
return placeholder.name == placeholder.name; // here
// why not just return true ?? instead of above line
}return false
}
);
Also I'm not sure this makes sense
placeholder.name == placeholder.name; you mean just return true; ?
I am trying to append numbers that I get from an api call (a promise) into an array. When I test the array's length it's always returning 1 as if each api call resets the array and puts in a new number.
here's the code:
The API call
wiki()
.page("COVID-19_pandemic_in_Algeria")
.then((page) => page.fullInfo())
.then((info) => {
(data.confirmed.value = info.general.confirmedCases),
(data.recovered.value = info.general.recoveryCases),
(data.deaths.value = info.general.deaths);
});
const data = {
confirmed: { value: 0 },
deaths: { value: 0 },
recovered: { value: 0 },
};
Now I want to put the deaths count into an array, so that I have a list of numbers over the next days to keep track of.
function countStats() {
const counter = [];
var deathCounter = data.deaths.value;
counter.push(deathCounter);
console.log(counter.length);
return counter;
}
countStats();
every time the functions run (wiki() and countStats()) the counter array's length is always 1. Why is that?
Unless ...
the data source provides multi-day data, or
you are going to run an extremely long javascript session (which is impractical and unsafe),
... then javascript can't, on its own, meet the objective of processing/displaying data arising from multiple days'.
Let's assume that the data source provides data that is correct for the current day.
You will need a permanent data store, in which scraped data can be accumulated, and retreived on demand. Exactly what you choose for your permanent data store is dependant on the environment in which you propose to run your javascript (essentially client-side browser or server-side NODE), and that choice is beyond the scope of this question.
Your master function might be something like this ...
function fetchCurrentDataAndRenderAll() {
return fetchCurrentData()
.then(writeToFile)
.then(readAllFromFile)
.then(data => {
// Here, you have the multi-day data that you want.
return renderData(data); // let's assume the data is to be rendered, say as a graph.
})
.catch(error => {
// something went wrong
console.log(error);
throw error;
});
}
... and the supporting functions might be something like this:
function fetchCurrentData() {
return wiki() // as given in the question ...
.page("COVID-19_pandemic_in_Algeria")
.then(page => page.fullInfo())
.then(info => ({
'timeStamp': Date.now(), // you will most likely need to timestamp the data
'confirmed': info.general.confirmedCases,
'recovered': info.general.recoveryCases,
'deaths': info.general.deaths
}));
}
function writeToFile(scrapedData) {
// you need to write this ...
// return Promise.
}
function readAllFromFile() {
// you need to write this ...
// return Promise.
}
function renderData(data) {
// you need to write this ...
// optionally: return Promise (necessary if rendering is asynchronous).
}
You can use Promise.all(). I take it that you'll not be requesting the same page 10 times but requesting a different page in each call e.g. const Pages = ['COVID-19_pandemic_in_Algeria','page2','page3','page4','page5','page6','page7','page8','page9','page10']. Then you could make the 10 calls as follows:
//const wiki = ......
const Pages = ['COVID-19_pandemic_in_Algeria','page2','page3','page4','page5','page6','page7','page8','page9','page10'];
let counter = [];
Promise.all(
Pages.map(Page => wiki().page(Page))
)
.then(results => {
for (page of results) {
let infoGeneral = page.fullInfo().general;
counter.push(infoGeneral.deaths);
}
console.log( counter.length ); //10
console.log( counter ); //[10 deaths results one for each page]
})
.catch(err => console.log(err.message));
Let's say we have this global const:
const isSignedIn = fromPromise(fetch('/api/is-signed-in'))
.pipe(throttleTime(1000), shareReply(1));
After page load, several components will subscribe to this at the same time:
isSignedIn.subscribe(() => console.log('do 1st'));
isSignedIn.subscribe(() => console.log('do 2nd'));
isSignedIn.subscribe(() => console.log('do 3rd'));
The above will only call the API once, however i need it to call the API again (ie after 1 second) if another component subscribes to it.
isSignedIn.subscribe(() => console.log('button press'));
How do i that using RxJS?
I think this is what you want:
A pipeable operator (declare globally somewhere and import it)
export const refreshAfter = (duration: number) => (source: Observable<any>) =>
source.pipe(
repeatWhen(obs => obs.pipe(delay(duration))),
publishReplay(1),
refCount());
Then use it like this:
data$ = fetch('/api/is-signed-in').pipe(refreshAfter(5000)); // refresh after 5000 ms
Note: You actually asked for this:
i need it to call the API again (ie after 1 second) if another component subscribes to
it.
Not quite sure this is what you really meant. I think what you really meant was - you want the data to be refreshed for all components currently subscribed after an expiry time. Anyway my answer sends the new value to all listeners. If you really want what you originally said you'd need to add some kind of alternative repeat trigger.
But if this is for a global constant - the above is what I'm using for the same scenario.
Note: I haven't actually tested the handling of an error condition when the item is repested, but I think the error will propagate to all listeners.
If we reimplement ShareReplay so it:
- will never unsubscribe from source even if it have no more subscribers (remove refCount, potential memory leak).
- accept rerunAfter argument, time passed from last subscribe to source.
import {Subject, of, Observable, ReplaySubject, Subscriber} from 'rxjs';
import {pluck, shareReplay, tap, delay} from 'rxjs/operators';
function shareForeverReplayRerun<T>(bufferSize: number, rerunAfter: number) {
let subject;
let subscription;
let hasError = false;
let isComplete = false;
let lastSubTime = 0;
return source => Observable.create((observer: Subscriber<T>) => {
if (!subject || hasError || (Date.now() - lastSubTime) >= rerunAfter) {
lastSubTime = Date.now();
hasError = false;
subject = new ReplaySubject<T>(bufferSize);
subscription = source.subscribe({
next(value) { subject.next(value); },
error(err) {
hasError = true;
subject.error(err);
},
complete() {
isComplete = true;
subject.complete();
},
});
}
const innerSub = subject.subscribe(observer);
// never unsubscribe from source
return () => {
innerSub.unsubscribe();
};
})
}
const source = of('Initial').pipe(
tap(()=>console.log('COMPUTE')),
delay(200),
shareReplayRerun(1, 1000),
);
source.subscribe(console.log.bind(null, 'syncI:'));
source.subscribe(console.log.bind(null, 'syncII:'));
setTimeout(()=>source.subscribe(console.log.bind(null, 'after500:')), 500);
setTimeout(()=>source.subscribe(console.log.bind(null, 'after900:')), 900);
setTimeout(()=>source.subscribe(console.log.bind(null, 'after1500:')), 1500);
as output we have:
COMPUTE
syncI: Initial
syncII: Initial
after500: Initial
after900: Initial
COMPUTE
after1500:Initial
EDITED: The answer is wrong. BufferSize is how long the last N events are replayed. After this the stream is completed.
signature: shareReplay(
bufferSize?: number,
windowTime?: number,
scheduler?: IIScheduler
):Observable
#param {Number} [bufferSize=Number.POSITIVE_INFINITY] Maximum element count of the replay buffer.
#param {Number} [windowTime=Number.MAX_VALUE] Maximum time length of the replay buffer in milliseconds.
Try to add 1000 as second argument to shareReply:
const isSignedIn = fromPromise(fetch('/api/is-signed-in'))
.pipe(throttleTime(1000), shareReplay(1, 1000));
shareReplay.ts - be care of refCount-- on unsubcribe as it can trigger additional requests.
I want to buffer events sent to my server. The trigger to flush the buffer is either the buffer size has been reached, buffer period has been reached or the window has been unloaded.
I buffer events sent to my server by creating a Subject and using buffer with a closing notifier. I use race for the closing notifier and race the buffer period with with window.beforeunload event.
this.event$ = new Subject();
this.bufferedEvent$ = this.event$
.buffer(
Observable.race(
Observable.interval(bufferPeriodMs),
Observable.fromEvent(window, 'beforeunload')
)
)
.filter(events => events.length > 0)
.switchMap(events =>
ajax.post(
this.baseUrl + RESOURCE_URL,
{
entries: events,
},
{
'Content-Type': 'application/json',
}
)
);
The question is, how do I now also limit the size of the buffer. ie, I never want the buffer to be flushed when it has 10 items.
This is the working solution I have. Extra console.log()'s are added to show the sequence of events.
The only thing that's a bit bothersome is the .skip(1) in fullBufferTrigger, but it's needed as it will trigger when it's buffer is full (natch), but the buffer in bufferedEvent$ does not seem to have the latest event before it's triggered.
Luckily, with the timeoutTrigger in place, the last event gets emitted. Without timeout, fullBufferTrigger by itself will not emit the final event.
Also, changed buffer to bufferWhen, as the former did not seem to trigger with two triggers, although you'd expect it to from the documentation.
footnote with buffer(race()) the race only completes once, so whichever trigger got there first will thereafter be used and the other triggers dis-regarded. In contrast, bufferWhen(x => race()) evaluates every time an event occurs.
const bufferPeriodMs = 1000
const event$ = new Subject()
event$.subscribe(event => console.log('event$ emit', event))
// Define triggers here for testing individually
const beforeunloadTrigger = Observable.fromEvent(window, 'beforeunload')
const fullBufferTrigger = event$.skip(1).bufferCount(2)
const timeoutTrigger = Observable.interval(bufferPeriodMs).take(10)
const bufferedEvent$ = event$
.bufferWhen( x =>
Observable.race(
fullBufferTrigger,
timeoutTrigger
)
)
.filter(events => events.length > 0)
// output
fullBufferTrigger.subscribe(x => console.log('fullBufferTrigger', x))
timeoutTrigger.subscribe(x => console.log('timeoutTrigger', x))
bufferedEvent$.subscribe(events => {
console.log('subscription', events)
})
// Test sequence
const delayBy = n => (bufferPeriodMs * n) + 500
event$.next('event1')
event$.next('event2')
event$.next('event3')
setTimeout( () => {
event$.next('event4')
}, delayBy(1))
setTimeout( () => {
event$.next('event5')
}, delayBy(2))
setTimeout( () => {
event$.next('event6')
event$.next('event7')
}, delayBy(3))
Working example: CodePen
Edit: Alternative way to trigger the buffer
Since the combination of bufferWhen and race might be a bit inefficient (the race is restarted each event emission), an alternative is to merge the triggers into one stream and use a simple buffer
const bufferTrigger$ = timeoutTrigger
.merge(fullBufferTrigger)
.merge(beforeunloadTrigger)
const bufferedEvent$ = event$
.buffer(bufferTrigger$)
.filter(events => events.length > 0)
One thing that bothers me about the solution using independent triggers is that fullBufferTrigger doesn't know when timeoutTrigger has emitted one of it's buffered values, so given the right event sequence, fullBuffer will trigger early when following timeout.
Ideally, would want fullBufferTrigger to reset when timeoutTrigger fires, but that proves tricky to do.
Using bufferTime()
In RxJS v4 there was an operator bufferWithTimeOrCount(timeSpan, count, [scheduler]), which in RxJS v5 was rolled up into an additional signature of bufferTime() (arguably a mistake from the perspective of clarity).
bufferTime<T>(
bufferTimeSpan: number,
bufferCreationInterval: number,
maxBufferSize: number,
scheduler?: IScheduler
): OperatorFunction<T, T[]>;
The only remaining question is how to incorporate the window.beforeunload trigger. Looking at the source code for bufferTime, it should flush it's buffer when receiving onComplete.
So, we can handle window.beforeunload by sending an onComplete to the buffered event stream.
The spec for bufferTime does not have an explicit test for onComplete, but I think I've managed to put one together.
Notes:
the timeout is set large to take it out of the picture for the test.
the source event stream is not affected, to illustrate event8 is added but never emits because the window is destroyed before it occurs.
to see the output stream without the beforeunloadTrigger, comment out the line that emits onComplete. Event7 is in the buffer, but will not emit.
Test:
const bufferPeriodMs = 7000 // Set high for this test
const bufferSize = 2
const event$ = new Rx.Subject()
/*
Create bufferedEvent$
*/
const bufferedEvent$ = event$
.bufferTime(bufferPeriodMs, null, bufferSize)
.filter(events => events.length > 0)
const subscription = bufferedEvent$.subscribe(console.log)
/*
Simulate window destroy
*/
const destroy = setTimeout( () => {
subscription.unsubscribe()
}, 4500)
/*
Simulate Observable.fromEvent(window, 'beforeunload')
*/
const beforeunloadTrigger = new Rx.Subject()
// Comment out the following line, observe that event7 does not emit
beforeunloadTrigger.subscribe(x=> event$.complete())
setTimeout( () => {
beforeunloadTrigger.next('unload')
}, 4400)
/*
Test sequence
Event stream: '(123)---(45)---6---7-----8--|'
Destroy window: '-----------------------x'
window.beforeunload: '---------------------y'
Buffered output: '(12)---(34)---(56)---7'
*/
event$.next('event1')
event$.next('event2')
event$.next('event3')
setTimeout( () => { event$.next('event4'); event$.next('event5') }, 1000)
setTimeout( () => { event$.next('event6') }, 3000)
setTimeout( () => { event$.next('event7') }, 4000)
setTimeout( () => { event$.next('event8') }, 5000)
Working example: CodePen
I've got an rxjs observer (really a Subject) that tails a file forever, just like tail -f. It's awesome for monitoring logfiles, for example.
This "forever" behavior is great for my application, but terrible for testing. Currently my application works but my tests hang forever.
I'd like to force an observer change to complete early, because my test code knows how many lines should be in the file. How do I do this?
I tried calling onCompleted on the Subject handle I returned but at that point it's basically cast as an observer and you can't force it to close, the error is:
Object # has no method 'onCompleted'
Here's the source code:
function ObserveTail(filename) {
source = new Rx.Subject();
if (fs.existsSync(filename) == false) {
console.error("file doesn't exist: " + filename);
}
var lineSep = /[\r]{0,1}\n/;
tail = new Tail(filename, lineSep, {}, true);
tail.on("line", function(line) {
source.onNext(line);
});
tail.on('close', function(data) {
console.log("tail closed");
source.onCompleted();
});
tail.on('error', function(error) {
console.error(error);
});
this.source = source;
}
And here's the test code that can't figure out how to force forever to end (tape style test). Note the "ILLEGAL" line:
test('tailing a file works correctly', function(tid) {
var lines = 8;
var i = 0;
var filename = 'tape/tail.json';
var handle = new ObserveTail(filename);
touch(filename);
handle.source
.filter(function (x) {
try {
JSON.parse(x);
return true;
} catch (error) {
tid.pass("correctly caught illegal JSON");
return false;
}
})
.map(function(x) { return JSON.parse(x) })
.map(function(j) { return j.name })
.timeout(10000, "observer timed out")
.subscribe (
function(name) {
tid.equal(name, "AssetMgr", "verified name field is AssetMgr");
i++;
if (i >= lines) {
handle.onCompleted(); // XXX ILLEGAL
}
},
function(err) {
console.error(err)
tid.fail("err leaked through to subscriber");
},
function() {
tid.end();
console.log("Completed");
}
);
})
It sounds like you solved your problem, but to your original question
I'd like to force an observer change to complete early, because my test code knows how many lines should be in the file. How do I do this?
In general the use of Subjects is discouraged when you have better alternatives, since they tend to be a crutch for people to use programming styles they are familiar with. Instead of trying to use a Subject I would suggest that you think about what each event would mean in an Observable life cycles.
Wrap Event Emitters
There already exists wrapper for the EventEmitter#on/off pattern in the form of Observable.fromEvent. It handles clean up and keeping the subscription alive only when there are listeners. Thus ObserveTail can be refactored into
function ObserveTail(filename) {
return Rx.Observable.create(function(observer) {
var lineSep = /[\r]{0,1}\n/;
tail = new Tail(filename, lineSep, {}, true);
var line = Rx.Observable.fromEvent(tail, "line");
var close = Rx.Observable.fromEvent(tail, "close");
var error = Rx.Observable.fromEvent(tail, "error")
.flatMap(function(err) { return Rx.Observable.throw(err); });
//Only take events until close occurs and wrap in the error for good measure
//The latter two are terminal events in this case.
return line.takeUntil(close).merge(error).subscribe(observer);
});
}
Which has several benefits over the vanilla use of Subjects, one, you will now actually see the error downstream, and two, this will handle clean up of your events when you are done with them.
Avoid *Sync Methods
Then this can be rolled into your file existence checking without the use of readSync
//If it doesn't exist then we are done here
//You could also throw from the filter if you want an error tracked
var source = Rx.Observable.fromNodeCallback(fs.exists)(filename)
.filter(function(exists) { return exists; })
.flatMap(ObserveTail(filename));
Next you can simplify your filter/map/map sequence down by using flatMap instead.
var result = source.flatMap(function(x) {
try {
return Rx.Observable.just(JSON.parse(x));
} catch (e) {
return Rx.Observable.empty();
}
},
//This allows you to map the result of the parsed value
function(x, json) {
return json.name;
})
.timeout(10000, "observer timed out");
Don't signal, unsubscribe
How do you stop "signal" a stop when streams only travel in one direction. We rarely actually want to have an Observer directly communicate with an Observable, so a better pattern is to not actually "signal" a stop but to simply unsubscribe from the Observable and leave it up to the Observable's behavior to determine what it should do from there.
Essentially your Observer really shouldn't care about your Observable more than to say "I'm done here".
To do that you need to declare a condition you want to reach in when stopping.
In this case since you are simply stopping after a set number in your test case you can use take to unsubscribe. Thus the final subscribe block would look like:
result
//After lines is reached this will complete.
.take(lines)
.subscribe (
function(name) {
tid.equal(name, "AssetMgr", "verified name field is AssetMgr");
},
function(err) {
console.error(err)
tid.fail("err leaked through to subscriber");
},
function() {
tid.end();
console.log("Completed");
}
);
Edit 1
As pointed out in the comments, In the case of this particular api there isn't a real "close" event since Tail is essentially an infinite operation. In this sense it is no different from a mouse event handler, we will stop sending events when people stop listening. So your block would probably end up looking like:
function ObserveTail(filename) {
return Rx.Observable.create(function(observer) {
var lineSep = /[\r]{0,1}\n/;
tail = new Tail(filename, lineSep, {}, true);
var line = Rx.Observable.fromEvent(tail, "line");
var error = Rx.Observable.fromEvent(tail, "error")
.flatMap(function(err) { return Rx.Observable.throw(err); });
//Only take events until close occurs and wrap in the error for good measure
//The latter two are terminal events in this case.
return line
.finally(function() { tail.unwatch(); })
.merge(error).subscribe(observer);
}).share();
}
The addition of the finally and the share operators creates an object which will attach to the tail when a new subscriber arrives and will remain attached as long as there is at least one subscriber still listening. Once all the subscribers are done however we can safely unwatch the tail.