How do I accurately setState on a time delay? - javascript

I am trying to setState on a given time interval (100ms). I have discovered both in practice and through research that setInterval is inaccurate, and will float over time. I need it to be accurate because I'm aiming to have data (from an array) in my app synced up to a video.
Here is what I currently have:
data = this.processCSV(text);
let idx = 0
this.intOne = setInterval(() => {
this.setState(previousState => (
{
data1: parseFloat(data[idx][2]),
data2: parseFloat(data[idx][1]),
data3: parseFloat(data[idx][3])
}
));
idx = idx + 1;
}, 100);
In my research, I've found a few solutions that use Date to accurately keep track of time and account for drifting.
Expected Results: setState is called 10 times per second for an extended amount of time
Current Results: setState is called ~9 times per second and continues to float away from expected time

Related

What is the simplest way of giving Tone.js arrays of notes and durations in seconds to play back?

I would like to give Tone.js a list of notes and corresponding durations for each note and have it play back the sequence. As far as I can see, there is no easy way to do this.
In the following, the corresponding time values are not the ones I entered (i.e 0.25, 0.5, 0.25), as evidenced by the console.log:
var part = new Tone.Part(function(time, note){
console.log(time);
console.log(note);
synth.triggerAttackRelease(note, time);
}, [[0.25, "C2"], [0.5, "C3"], [0.25, "G2"]]);
part.start(0).loop = false;
Tone.Transport.start();
How can I give Tone.js notes and corresponding ms for playback?
I'm not familiar with Tone.js, so there's probably a better way of doing this. The official example for the array shorthand that you're using doesn't seem to work, so it might be a library issue.
As for what you're trying to achieve, I fiddled with it out of curiosity and here's what I've come to:
function timeFromDurations(value, i, arr) {
const prevTime = arr[i - 1]?.time;
value.time = prevTime + arr[i - 1]?.duration || 0;
return value;
}
const notesAndDurations = [
{ note: 'C3', duration: .25 },
{ note: 'C4', duration: .5 },
{ note: 'G2', duration: 1 },
].map(timeFromDurations);
console.log(notesAndDurations);
const synth = new Tone.Synth().toDestination();
// use an array of objects as long as the object has a "time" attribute
const part = new Tone.Part((time, value) => {
// the value is an object which contains both the note and the velocity
synth.triggerAttackRelease(value.note, value.duration, time);
}, notesAndDurations).start(0);
Tone.Transport.start();
The idea is that you need to set start time of each note based on previous note start time + duration. That removes the need to set the start time(not optional) manually.
Edit
For your second case where the durations and the notes come in separate arrays you can use the following reduce function:
const notes = ['C3', 'C4', 'G2'];
const durations = [0.25, 0.5, 1];
const noteDurationTime = notes.reduce((acc, note, i) => {
const prevTime = acc[i - 1]?.time;
const time = prevTime + acc[i - 1]?.duration || 0;
const duration = durations[i];
acc.push({ note, duration, time });
return acc;
}, []);
The idea is the same, you're building an array of objects that have all the needed properties(note, duration, time), but this time from different sources(notes array and durations array).
You want to make sure that both these arrays are the same length.

change state with conditionals inside a UseEffect (simple counter with hooks)

I'm trying to build a simple percentage counter from 0 to 100 that is updating itself using SetInterval() inside useEffect(). I can get the counter work but I would like to restart the counter once it reaches the 100%. This is my code:
const [percentage, setPercentage]=useState(0);
useEffect(() => {
const intervalId= setInterval(() => {
let per = percentage=> percentage+1
if(per>=100){
per=0
}
setPercentage(per)
}, 100);
return () => {
}
}, [])
Inside the console I can see the state is increasing but it will ignore the if statement to reset the state to 0 once it reaches 100. How can I tackle this knowing that if conditionals are not great with hooks setState?
Check the percentage instead of per. per is of type function and will never be greater or equal to 100, percentage is the value that will reach 100.
This will make your effect depend on percentage which you have avoided by using the function. In this situation, if I still don't want to add that dependency, then I might use a reducer instead to manage that state. This way I don't need to depend on percentage inside of the useEffect.
const reducer = (state, action) => state >= 100 ? 0 : state + 1;
The way you would do this while keeping useState is by moving the check into the state setting function.
setPercentage(percentage => percentage >= 100 ? 0 : percentage + 1);
This might be the quicker option for you. Notice how similar these are, in the end useState is implemented using the useReducer code path as far as I know.
Below should work.
const [percentage, setPercentage] = useState(0);
useEffect(() => {
const intervalId = setInterval(() => {
setPercentage(prev => prev >= 100 ? 0 : prev + 1);
}, 100);
return () => clearInterval(intervalId);
}, [])
Note that per in your code is a function and therefore cannot used to compare against numbers. You may also want to clear the interval in destruction of component.

Javascript loop through array recursively

I've run into a bit of an issue trying to map a list of items in javascript. I have the following array that I'm trying to map:
[{"project_id":"EGNL1701","title":"Test Energy Project",
"reservations":
[{"start_time":"1519887600000"},{"start_time":"1519891200000"},
{"start_time":"1519938000000"},{"start_time":"1519898400000"},
{"start_time":"1519902000000"},{"start_time":"1519905600000"},
{"start_time":"1519909200000"},{"start_time":"1529683200000"},
{"start_time":"1529686800000"},{"start_time":"1531893600000"},
{"start_time":"1531897200000"},{"start_time":"1531900800000"},
{"start_time":"1531904400000"}]},
{"project_id":"LENL1701","title":"Vive","reservations":[]}]
Basically it's a list of reservations per project. All the times are in unix code in miliseconds. I have no issue trying to convert the times to the right time, however what I do need is to check for reservation blocks of consecutive hours. A reservation is always an hour, so I want to map reservations that are directly after the previous reservation, to be included into the previous reservation so that it forms 1 reservation of multiple hours. So if the list would contain 3 reservations for 10-07-2018 at 10, 11 and 12 o'clock then it should combine those in one object.
The new array should look like this:
[
{
title: 'Energy Project',
startTime: 1530631437,
endTime: 1530638640
},
{
title: 'HTC VIVE',
startTime: 1530794845,
endTime: 1530797390
}
];
I'm not sure what the best way is to go about this, I was trying to mess with while and for loops to get it done, but I keep getting stuck in an infinite loop.
Here's the code I have now for checking if the next reservation is an hour after the current reservation in the loop:
while (moment.duration(getUnixTime(reservations[index + nextIndex].start_time).diff(getUnixTime(reservation.start_time))).asHours() === 1) {
nextIndex++;
}
I was hoping one of you might have a good idea on how to do this? I feel like I'm going about this all wrong. It's for a React project, I can use ES6 functions etc.
This might not be complete answer so try this
Create an empty reservations object R
Loop through (for each or for ) upto second last item. You can check if next item exists or not.
Check if the criteria for example next one is one hour after the current one then add them together and add it to R.
End time of R will become end time of current reservation or the one next to it.
Move to next one or skip based on what you did in previous step.
When you reach last item simply add it to R if its not being added in previous step.
One of the assumption is that all reservations are in order. You can order them via sorting functions to avoid loop and check where it fits.
I've fixed the issue using #Farrukh Subhani's suggestion.
First had to sort the list, then loop through it twice. Resulting code:
export const mapReservationTimes = reservations => {
const upcomingReservations = [];
reservations.sort((x, y) => {
return getUnixTime(x.start_time).toDate() - getUnixTime(y.start_time).toDate();
});
for (let index = 0; index < reservations.length; index++) {
let newReservation = { startTime: '', endTime: '' };
let reservation = reservations[index];
newReservation.startTime = getUnixTime(reservation.start_time).format('DD-MM-YYYY HH:MM');
newReservation.endTime = getUnixTime(reservation.start_time)
.add(1, 'hours')
.format('DD-MM-YYYY HH:MM');
for (let i = index; i < reservations.length - 1; i++) {
reservation = reservations[index];
const nextReservation = reservations[i + 1];
if (
moment
.duration(
getUnixTime(nextReservation.start_time).diff(
getUnixTime(reservation.start_time)
)
)
.asHours() === 1
) {
index++;
newReservation.endTime = getUnixTime(nextReservation.start_time)
.add(1, 'hours')
.format('DD-MM-YYYY HH:MM');
} else {
break;
}
}
upcomingReservations.push(newReservation);
}
return upcomingReservations;
};
Thanks a lot for the suggestion, it got me on the right track :)
I feel this might be a good fit for an array.reduce, though this might stem from my personal taste. This solution can and should be simplified and cleaned, but I tried to convey the general idea so I made it a little more verbose then I think production code should be.
const input = [{"project_id":"EGNL1701","title":"Test Energy Project","reservations":
[{"start_time":"1519887600000"},{"start_time":"1519891200000"},
{"start_time":"1519938000000"},{"start_time":"1519898400000"},
{"start_time":"1519902000000"},{"start_time":"1519905600000"},
{"start_time":"1519909200000"},{"start_time":"1529683200000"},
{"start_time":"1529686800000"},{"start_time":"1531893600000"},
{"start_time":"1531897200000"},{"start_time":"1531900800000"},
{"start_time":"1531904400000"}]},
{"project_id":"LENL1701","title":"Vive","reservations":[]}]
const hourInMilliseconds = 3600000
console.log(input.reduce((acc, project) => {
const title = project.title
const sortedReservations = project.reservations.map(r => r.start_time).sort()
const reservationCount = sortedReservations.length
const earliest = reservationCount > 0 ? sortedReservations[0] : "none"
const latest = reservationCount > 0 ? sortedReservations[reservationCount - 1] : "none"
const projectObject = {
title,
startTime: earliest,
endTime: isNaN(latest) ? latest : (Number(latest) + hourInMilliseconds).toString()
}
acc.push(projectObject)
return acc
}, [])
)
Disclaimer: This assumes that all the reservations in one array item are consecutive, which seems to be the case in your sample data. If this is not the case the function would have to be altered a bit.

Rate-limiting and count-limiting events in RxJS v5, but also allowing pass-through

I have a bunch of events to send up to a service. But the requests are rate limited and each request has a count limit:
1 request per second: bufferTime(1000)
100 event items per request: bufferCount(100)
The problem is, I am not sure how to combine them in a way that makes sense.
Allowing pass-through
Complicating this further, I need to make sure that events go through instantaneously if we don't hit either limit.
For example, I don't want it to actually wait for 100 event items before letting it go through if it's only one single event during a non-busy time.
Legacy API
I also found that there was a bufferWithTimeOrCount that existed in RxJS v4, although I am not sure how I'd use that even if I had it.
Test playground
Here is a JSBin I made for you to test your solution:
http://jsbin.com/fozexehiba/1/edit?js,console,output
Any help would be greatly appreciated.
The bufferTime() operator takes three parameters which combines the functionality of bufferTime and bufferCount. See http://reactivex.io/rxjs/class/es6/Observable.js~Observable.html#instance-method-bufferTime.
With .bufferTime(1000, null, 3) you can make a buffer every 1000ms or when it reaches 3 items. However, this means that it doesn't guarantee 1000ms delay between each buffer.
So you could use something like this which is pretty easy to use (buffers only 3 items for max 1000ms):
click$
.scan((a, b) => a + 1, 0)
.bufferTime(1000, null, 3)
.filter(buffer => buffer.length > 0)
.concatMap(buffer => Rx.Observable.of(buffer).delay(1000))
.timestamp()
.subscribe(console.log);
See live demo: http://jsbin.com/libazer/7/edit?js,console,output
The only difference to what you probably wanted is that the first emission might be delayed by more than 1000ms. This is because both bufferTime() and delay(1000) operators make a delay to ensure that there's always at least 1000ms gap.
I hope this works for you.
Operator
events$
.windowCount(10)
.mergeMap(m => m.bufferTime(100))
.concatMap(val => Rx.Observable.of(val).delay(100))
.filter(f => f.length > 0)
Doc
.windowCount(number) : [ Rx Doc ]
.bufferTime(number) : [ Rx Doc ]
Demo
// test case
const mock = [8, 0, 2, 3, 30, 5, 6, 2, 2, 0, 0, 0, 1]
const tInterval = 100
const tCount = 10
Rx.Observable.interval(tInterval)
.take(mock.length)
.mergeMap(mm => Rx.Observable.range(0, mock[mm]))
// start
.windowCount(tCount)
.mergeMap(m => m.bufferTime(tInterval))
.concatMap(val => Rx.Observable.of(val).delay(tInterval))
.filter(f => f.length > 0)
// end
.subscribe({
next: (n) => console.log('Next: ', n),
error: (e) => console.log('Error: ', e),
complete: (c) => console.log('Completed'),
})
<script src="https://unpkg.com/rxjs/bundles/Rx.min.js"></script>
Updated
After more testing. I found the answer above has some problem in extreme condition. I think they are caused by .window() and .concat(), and then I find a warning in the doc#concatMap.
Warning: if source values arrive endlessly and faster than their corresponding inner Observables can complete, it will result in memory issues as inner Observables amass in an unbounded buffer waiting for their turn to be subscribed to.
However, I thought the right way to limit the request rate possibly is, that we could limit the cycle time of requests. In your case, just limit there is only 1 request per 10 milliseconds. It is simpler and may be more efficient to control the requests.
Operator
const tInterval = 100
const tCount = 10
const tCircle = tInterval / tCount
const rxTimer = Rx.Observable.timer(tCircle).ignoreElements()
events$
.concatMap(m => Rx.Observable.of(m).merge(rxTimer)) // more accurate than `.delay()`
// .concatMap(m => Rx.Observable.of(m).delay(tCircle))
or
events$
.zip(Rx.Observable.interval(tCircle), (x,y) => x)
I've modified the answer I gave to this question to support your use case of adding a limited number of values (i.e. events) to pending requests.
The comments within should explain how it works.
Because you need to keep a record of the requests that have been made within the rate limit period, I don't believe that it's possible to use the bufferTime and bufferCount operators to do what you want - a scan is required so that you can maintain that state within the observable.
function rateLimit(source, period, valuesPerRequest, requestsPerPeriod = 1) {
return source
.scan((requests, value) => {
const now = Date.now();
const since = now - period;
// Keep a record of all requests made within the last period. If the
// number of requests made is below the limit, the value can be
// included in an immediate request. Otherwise, it will need to be
// included in a delayed request.
requests = requests.filter((request) => request.until > since);
if (requests.length >= requestsPerPeriod) {
const leastRecentRequest = requests[0];
const mostRecentRequest = requests[requests.length - 1];
// If there is a request that has not yet been made, append the
// value to that request if the number of values in that request's
// is below the limit. Otherwise, another delayed request will be
// required.
if (
(mostRecentRequest.until > now) &&
(mostRecentRequest.values.length < valuesPerRequest)
) {
mostRecentRequest.values.push(value);
} else {
// until is the time until which the value should be delayed.
const until = leastRecentRequest.until + (
period * Math.floor(requests.length / requestsPerPeriod)
);
// concatMap is used below to guarantee the values are emitted
// in the same order in which they are received, so the delays
// are cumulative. That means the actual delay is the difference
// between the until times.
requests.push({
delay: (mostRecentRequest.until < now) ?
(until - now) :
(until - mostRecentRequest.until),
until,
values: [value]
});
}
} else {
requests.push({
delay: 0,
until: now,
values: [value]
});
}
return requests;
}, [])
// Emit only the most recent request.
.map((requests) => requests[requests.length - 1])
// If multiple values are added to the request, it will be emitted
// mulitple times. Use distinctUntilChanged so that concatMap receives
// the request only once.
.distinctUntilChanged()
.concatMap((request) => {
const observable = Rx.Observable.of(request.values);
return request.delay ? observable.delay(request.delay) : observable;
});
}
const start = Date.now();
rateLimit(
Rx.Observable.range(1, 250),
1000,
100,
1
).subscribe((values) => console.log(
`Request with ${values.length} value(s) at T+${Date.now() - start}`
));
.as-console-wrapper { max-height: 100% !important; top: 0; }
<script src="https://unpkg.com/rxjs#5/bundles/Rx.min.js"></script>

How do I rate limit requests losslessly using RxJS 5

I would like to use make a series of requests to a server, but the server has a hard rate limit of 10 request per second. If I try to make the requests in a loop, it will hit the rate limit since all the requests will happen at the same time.
for(let i = 0; i < 20; i++) {
sendRequest();
}
ReactiveX has lots of tools for modifying observable streams, but I can't seem to find the tools to implement rate limiting. I tried adding a standard delay, but the requests still fire at the same time, just 100ms later than they did previously.
const queueRequest$ = new Rx.Subject<number>();
queueRequest$
.delay(100)
.subscribe(queueData => {
console.log(queueData);
});
const queueRequest = (id) => queueRequest$.next(id);
function fire20Requests() {
for (let i=0; i<20; i++) {
queueRequest(i);
}
}
fire20Requests();
setTimeout(fire20Requests, 1000);
setTimeout(fire20Requests, 5000);
The debounceTime and throttleTime operators are similar to what I'm looking for as well, but that is lossy instead of lossless. I want to preserve every request that I make, instead of discarding the earlier ones.
...
queueRequest$
.debounceTime(100)
.subscribe(queueData => {
sendRequest();
});
...
How do I make these requests to the server without exceeding the rate limit using ReactiveX and Observables?
The implementation in the OP's self answer (and in the linked blog) always imposes a delay which is less than ideal.
If the rate-limited service allows for 10 requests per second, it should be possible to make 10 requests in, say, 10 milliseconds, as long as the next request is not made for another 990 milliseconds.
The implementation below applies a variable delay to ensure the limit is enforced and the delay is only applied to requests that would see the limit exceeded.
function rateLimit(source, count, period) {
return source
.scan((records, value) => {
const now = Date.now();
const since = now - period;
// Keep a record of all values received within the last period.
records = records.filter((record) => record.until > since);
if (records.length >= count) {
// until is the time until which the value should be delayed.
const firstRecord = records[0];
const lastRecord = records[records.length - 1];
const until = firstRecord.until + (period * Math.floor(records.length / count));
// concatMap is used below to guarantee the values are emitted
// in the same order in which they are received, so the delays
// are cumulative. That means the actual delay is the difference
// between the until times.
records.push({
delay: (lastRecord.until < now) ?
(until - now) :
(until - lastRecord.until),
until,
value
});
} else {
records.push({
delay: 0,
until: now,
value
});
}
return records;
}, [])
.concatMap((records) => {
const lastRecord = records[records.length - 1];
const observable = Rx.Observable.of(lastRecord.value);
return lastRecord.delay ? observable.delay(lastRecord.delay) : observable;
});
}
const start = Date.now();
rateLimit(
Rx.Observable.range(1, 30),
10,
1000
).subscribe((value) => console.log(`${value} at T+${Date.now() - start}`));
<script src="https://unpkg.com/rxjs#5/bundles/Rx.min.js"></script>
This blog post does a great job of explaining that RxJS is great at discarding events, and how they came to the answer, but ultimately, the code you're looking for is:
queueRequest$
.concatMap(queueData => Rx.Observable.of(queueData).delay(100))
.subscribe(() => {
sendRequest();
});
concatMap adds concatenates the newly created observable to the back of the observable stream. Additionally, using delay pushes back the event by 100ms, allowing 10 request to happen per second. You can view the full JSBin here, which logs to the console instead of firing requests.
Actually, there's an easier way to do this with the bufferTime() operator and its three arguments:
bufferTime(bufferTimeSpan, bufferCreationInterval, maxBufferSize)
This means we can use bufferTime(1000, null, 10) which means we'll emit a buffer of max 10 items or after max 1s. The null means we want to open a new buffer immediately after the current buffer is emitted.
function mockRequest(val) {
return Observable
.of(val)
.delay(100)
.map(val => 'R' + val);
}
Observable
.range(0, 55)
.concatMap(val => Observable.of(val)
.delay(25) // async source of values
// .delay(175)
)
.bufferTime(1000, null, 10) // collect all items for 1s
.concatMap(buffer => Observable
.from(buffer) // make requests
.delay(1000) // delay this batch by 1s (rate-limit)
.mergeMap(value => mockRequest(value)) // collect results regardless their initial order
.toArray()
)
// .timestamp()
.subscribe(val => console.log(val));
See live demo: https://jsbin.com/mijepam/19/edit?js,console
You can experiment with different initial delay. With only 25ms the request will be sent in batches by 10:
[ 'R0', 'R1', 'R2', 'R3', 'R4', 'R5', 'R6', 'R7', 'R8', 'R9' ]
[ 'R10', 'R11', 'R12', 'R13', 'R14', 'R15', 'R16', 'R17', 'R18', 'R19' ]
[ 'R20', 'R21', 'R22', 'R23', 'R24', 'R25', 'R26', 'R27', 'R28', 'R29' ]
[ 'R30', 'R31', 'R32', 'R33', 'R34', 'R35', 'R36', 'R37', 'R38', 'R39' ]
[ 'R40', 'R41', 'R42', 'R43', 'R44', 'R45', 'R46', 'R47', 'R48', 'R49' ]
[ 'R50', 'R51', 'R52', 'R53', 'R54' ]
But with .delay(175) we'll emit batches of less than 10 items because we're limited by the 1s delay.
[ 'R0', 'R1', 'R2', 'R3', 'R4' ]
[ 'R5', 'R6', 'R7', 'R8', 'R9', 'R10' ]
[ 'R11', 'R12', 'R13', 'R14', 'R15' ]
[ 'R16', 'R17', 'R18', 'R19', 'R20', 'R21' ]
[ 'R22', 'R23', 'R24', 'R25', 'R26', 'R27' ]
[ 'R28', 'R29', 'R30', 'R31', 'R32' ]
[ 'R33', 'R34', 'R35', 'R36', 'R37', 'R38' ]
[ 'R39', 'R40', 'R41', 'R42', 'R43' ]
[ 'R44', 'R45', 'R46', 'R47', 'R48', 'R49' ]
[ 'R50', 'R51', 'R52', 'R53', 'R54' ]
There's however one difference to what you might need. This solution starts initially starts emitting values after 2s delay because of the .bufferTime(1000, ...) and delay(1000). All other emissions happen after 1s.
You could eventually use:
.bufferTime(1000, null, 10)
.mergeAll()
.bufferCount(10)
This will always collect 10 items and only after that it'll perform the request. This would be probably more efficient.
I wrote a library to do this, you set up the maximum number of requests per interval and it rate limits observables by delaying subscriptions. It's tested and with examples: https://github.com/ohjames/rxjs-ratelimiter
Go with Adam’s answer. However, bear in mind the traditional of().delay() will actually add a delay before every element. In particular, this will delay the first element of your observable, as well as any element that wasn’t actually rate limited.
Solution
You can work around this by having your concatMap return a stream of observables that immediately emit a value, but only complete after a given delay:
new Observable(sub => {
sub.next(v);
setTimeout(() => sub.complete(), delay);
})
This is kind of a mouthful, so I’d create a function for it. That said, since there’s no use for this outside of actual rate limiting, you’d probably be better served just writing a rateLimit operator:
function rateLimit<T>(
delay: number,
scheduler: SchedulerLike = asyncScheduler): MonoTypeOperatorFunction<T> {
return concatMap(v => new Observable(sub => {
sub.next(v);
scheduler.schedule(() => sub.complete(), delay);
}));
}
Then:
queueRequest$.pipe(
rateLimit(100),
).subscribe(...);
Limitation
This will now create a delay after every element. This means that if your source observable emits its last value then completes, your resulting rate-limited observable will have a little delay between itself between its last value, and completing.
Updated cartant's answer as pipe-able operator for newer rxjs versions:
function rateLimit(count: number, period: number) {
return <ValueType>(source: Observable<ValueType>) => {
return source.pipe
( scan((records, value) => {
let now = Date.now();
let since = now - period;
// Keep a record of all values received within the last period.
records = records.filter((record) => record.until > since);
if (records.length >= count) {
// until is the time until which the value should be delayed.
let firstRecord = records[0];
let lastRecord = records[records.length - 1];
let until = firstRecord.until + (period * Math.floor(records.length / count));
// concatMap is used below to guarantee the values are emitted
// in the same order in which they are received, so the delays
// are cumulative. That means the actual delay is the difference
// between the until times.
records.push(
{ delay: (lastRecord.until < now) ?
(until - now) :
(until - lastRecord.until)
, until
, value });
} else {
records.push(
{ delay: 0
, until: now
, value });
}
return records;
}, [] as RateLimitRecord<ValueType>[])
, concatMap((records) => {
let lastRecord = records[records.length - 1];
let observable = of(lastRecord.value);
return lastRecord.delay ? observable.pipe(delay(lastRecord.delay)) : observable;
}) );
};
}
interface RateLimitRecord<ValueType> {
delay: number;
until: number;
value: ValueType;
}

Categories