We Keep Coding

Why are two setTimeouts not in the same event loop

setTimeout(() => new Promise((r) => {console.log(1);r();}).then(() => console.log('1 mic')))
setTimeout(() => new Promise((r) => {console.log(2);r();}).then(() => console.log('2 mic')))
1
1 mic
2
2 mic
Why are two setTimeouts not in the same event loop
I didn't find an understandable answer after Google

As I stated in the Community Wiki answer, the simple reason is that calling setTimeout(fn) will, after the timeout expires, queue a new task to execute the callback.
So each call to setTimeout will queue its own task. [Specs]
But I have to note that your test wouldn't tell you if two callbacks are ran in the same event loop iteration anyway.
Indeed, every time* a callback is invoked, a microtask checkpoint is performed in the "cleanup after running a script" algorithm.
So even in a case where we'd have multiple callbacks firing in the same event loop iteration, we'd have a microtask checkpoint in between each callback. For instance, requestAnimationFrame() callbacks are all fired in the same event-loop iteration, as part of the "update the rendering" step of the event loop. But even there, microtasks will be fired:
requestAnimationFrame(() => {
console.log(1); Promise.resolve().then(() => console.log(1, "micro"));
// Try to include a new task in between
setTimeout(() => { console.log("a new task"); });
// block the event loop a bit so we ensure our timer should fire already
const t1 = performance.now();
while(performance.now() - t1 < 100) {}
});
requestAnimationFrame(() => {
console.log(2); Promise.resolve().then(() => console.log(2, "micro"));
});
So using microtasks is not a good way of checking if two callbacks are fired in the same event loop iteration. To do so, you could try to schedule a new task, like I did in that example. But even this isn't bullet proof, because browsers do have quite complex task prioritization system in place and we can't be sure when they'll fire different types of task.
The best, in supporting browsers, is to use the still under development Prioritized task scheduler.postTask method. This allows us to post tasks with the highest priority, and thus we can check if two callbacks are indeed in the same event loop iteration or not. In browsers that don't support this API we have to resort to using a MessageChannel object which should be the closest way of post high priority tasks:
const postTask = globalThis.scheduler
? (cb) => scheduler.postTask(cb, { priority: "user-blocking" })
: postMessageTask;
const test = (fn, label) => {
return new Promise((res) => {
let sameIteration = true;
fn(() => {
console.log(label, "first callback");
// If this task is executed before the next callback
// it means both callbacks weren't executed in the same iteration
postTask(() => sameIteration = false);
// block the event-loop a bit between both callbacks
const t1 = performance.now();
while(performance.now() - t1 < 100) {}
});
fn(() => {
console.log(label, "second callback");
console.log({ label, sameIteration });
res();
});
});
};
test(setTimeout, "setTimeout")
.then(() => test(requestAnimationFrame, "requestAnimationFrame") );
// If scheduler.postTask isn't available, use a MessageChannel to post high priority tasks
function postMessageTask(cb) {
const { port1, port2 } = (postMessageTask.channel ??= new MessageChannel());
port1.start();
port1.addEventListener("message", () => cb(), { once: true });
port2.postMessage("");
}
* That is, if the callstack is empty, which is the case most of the time, with one of the only exceptions being callbacks for events dispatched programmatically.

Because each timer callback has its own task scheduled.
[Specs]

Loops in RxJS Observable don't stop after calling complete or error

When calling observer.complete() or observer.error() an observer stops sending data and is considered done. However, If there is a for loop in the observer, the loop will continue running even after observer.complete() is called. Can someone explain to me this behaviour? I expect that the loop would be cut short. The current behaviour means that an interval or a while loop will forever run in an Observable unless I unsubscribe in the code.
In the below snippet I added a console.log to see if the log will be called after observer.complete().
const testObservable = new Observable( observer => {
for (let count = 0; count < 11; count++){
observer.next(count);
if (count > 5) {
observer.complete()
console.log("test")
}
if (count > 7) {
observer.error(Error("This is an error"))
}
}}
);
let firstObservable = testObservable.subscribe(
next => {console.log(next)},
error => { alert(error.message)},
() => {console.log("complete")}
)

It is a responsibility of the create function to properly destroy resources. In your case there could be simply return statement after observer.complete().
But in general if needed, the create function should return the teardown function (TeardownLogic), that is called when the observable is finalised.
new Observable( observer => {
// observable logic, e.g. ajax request
return () => {
// teardown function, e.g. cancel ajax request
};
});
Observable just ignore future calls of next(...), complete() and error(...) once the observable is finalized. It means that complete() or error(...) function was called internally or the observable was unsubscribed externally.
new Observable( observer => {
observer.next('a');
observer.complete();
// next calls will be ignored
observer.next('b');
});
const observable = new Observable( observer => {
setTimeout(
() => observer.next('a'), // this will be ignored due to unsubscribe
10
);
});
const subscription = observable.subscribe(...);
subscription.unsubscribe();

Async type ahead input buffer or queue

I'm working on a type ahead input component and part of my implementation is a queue or buffer FIFO of cancelable promises.
Because the typing can happen much faster than the async requests on each letter, I have to cancel the previous request, so that state is not inadvertently changed if the previous promise is slower than the current.
My current implementation works well, but I'm wondering if its necessary to maintain a queue vs. a single reference. The queue should never really grow to retain more than one promise at a time, right?
_onChangeEvent(str) {
var self = this;
var str = String(str) || '';
var toBeCanceled = null;
// set state return false if no str
// and shift off last request
if (!str) {
if (this.requestQueue.length > 0) {
toBeCanceled = this.requestQueue.shift();
toBeCanceled.cancel();
}
this.setState({results: [], loading: false, value: ''});
return false;
} else {
this.setState({loading: true, value: str});
}
// setup a cancelable promise and add it to the queue
// this API spoofer should eventually be a dispatch prop
var cancelable = CancelablePromise(APISpoofer(str));
cancelable
.promise
.then((res) => {
var sorted = res.found.sort((a, b) => {
var nameA = a.toLowerCase(), nameB = b.toLowerCase();
if(nameA < nameB) return -1;
if(nameA > nameB) return 1;
return 0;
}).slice(0, 4);
// set state and shift queue when resolved
return self.setState({
results: sorted,
loading: false,
value: str
}, () => {
self.requestQueue.shift();
});
})
.catch((err) => {
return err;
});
// check len of requestQueue and push
if(this.requestQueue.length === 0) {
this.requestQueue.push(cancelable);
} else {
// cancel previous and push
toBeCanceled = this.requestQueue.shift();
toBeCanceled.cancel();
this.requestQueue.push(cancelable);
}
}
Is there a problem with maintaining a queue of one value? Also, is there a name for this kind of problem?

As your code immediately cancels any in-flight request before sending a new one, your queue will never contain more than one element. So it's useless to use a queue, a simple currentRequest field is enough to ensure you can handle only the most recent request, no matter in what order they are processed.
A common practice in type-ahead controls is to throttle the input events, i.e. wait a short amount of time for another change before actually sending an AJAX request. This avoids sending too many requests when users type several letters in a quick fashion.
Both of those problems can be abstracted from your code if you are willing to use reactive programming techniques, e.g. using RxJS http://reactivex.io/.
In fact somewhere down on the following RxJS page you will find an example that does exactly that: observing input changes, requiring at least 2 chars, debouncing, querying a webservice and then handling the results: https://github.com/Reactive-Extensions/RxJS

RxJs flatMapLatest/switchMap cancel callback. Where is onCancel()?

I've got 2 nested Observable Streams which do HTTP requests. Now I'd like to display a loading indicator, but can't get it working correctly.
var pageStream = Rx.createObservableFunction(_self, 'nextPage')
.startWith(1)
.do(function(pageNumber) {
pendingRequests++;
})
.concatMap(function(pageNumber) {
return MyHTTPService.getPage(pageNumber);
})
.do(function(response) {
pendingRequests--;
});
Rx.createObservableFunction(_self, 'search')
.flatMapLatest(function(e) {
return pageStream;
})
.subscribe();
search();
nextPage(2);
nextPage(3);
search();
This will trigger pendingRequests++ 4 times, but pendingRequests-- only once, because flatMapLatest will cancel the inner observable before the first 3 HTTP responses arrive.
I couldn't find anything like an onCancel callback. I also tried onCompleted and onError, but those too won't get triggered by flatMapLatest.
Is there any other way to get this working?
Thank you!
EDIT: Desired loading indicator behavior
Example: Single search() call.
search() -> start loading indicator
when search() response comes back -> disable loading indicator
Example: search() and nextPage() call. (nextPage() is called before search() response came back.)
search() -> start loading indicator
nextPage() -> indicator is already started, though nothing to do here
stop loading indicator after both responses arrived
Example: search(), search(). (search() calls override each other, though the response of the first one can be dismissed)
search() -> start loading indicator
search() -> indicator is already started, though nothing to do here
stop loading indicator when the response for the second search() arrived
Example: search(), nextPage(), search(). (Again: Because of the second search(), the responses from the previous search() and nextPage() can be ignored)
search() -> start loading indicator
nextPage() -> indicator is already started, though nothing to do here
search() -> indicator is already started, though nothing to do here
stop loading indicator when response for the second search() arrived
Example: search(), nextPage(). But this time nextPage() is called after search() response came back.
search() -> start loading indicator
stop loading indicator because search() response arrived
nextPage() -> start loading indicator
stop loading indicator because nextPage() response arrived
I tried using pendingRequests counter, because I can have multiple relevant requests at the same time (for example: search(), nextPage(), nextPage()). Then of course I'd like to disable the loading indicator after all those relevant requests finished.
When calling search(), search(), the first search() is irrelevant. Same applies for search(), nextPage(), search(). In both cases there's only one active relevant request (the last search()).

With switchMap aka flatMapLatest you want to trim asap execution of the current inner-stream as new outer-items arrive. It is surely a good design decision as otherwise it would bring a lot of confusion and allow some spooky actions. If you really wanted do do something onCancel you can always create your own observable with custom unsubscribe callback. But still I would recommend not to couple unsubscribe with changing state of the external context. Ideally the unsubscribe would only clean up internally used resources.
Nevertheless your particular case can be solved without accessing onCancel or similar. The key observation is - if I understood your use case correctly - that on search all previous / pending actions may be ignored. So instead of worry about decrementing the counter we can simply start counting from 1.
Some remarks about the snippet:
used BehaviorSubject for counting pending requests - as it is ready to be composed with other streams;
checked all cases you posted in your question and they work;
added some fuzzy tests to demonstrate correctness;
not sure if you wanted to allow nextPage when a search is still pending - but it seems to be just a matter of using concatMapTo vs merge;
used only standard Rx operators.
PLNKR
console.clear();
const searchSub = new Rx.Subject(); // trigger search
const nextPageSub = new Rx.Subject(); // triger nextPage
const pendingSub = new Rx.BehaviorSubject(); // counts number of pending requests
const randDurationFactory = min => max => () => Math.random() * (max - min) + min;
const randDuration = randDurationFactory(250)(750);
const addToPending = n => () => pendingSub.next(pendingSub.value + n);
const inc = addToPending(1);
const dec = addToPending(-1);
const fakeSearch = (x) => Rx.Observable.of(x)
.do(() => console.log(`SEARCH-START: ${x}`))
.flatMap(() =>
Rx.Observable.timer(randDuration())
.do(() => console.log(`SEARCH-SUCCESS: ${x}`)))
const fakeNextPage = (x) => Rx.Observable.of(x)
.do(() => console.log(`NEXT-PAGE-START: ${x}`))
.flatMap(() =>
Rx.Observable.timer(randDuration())
.do(() => console.log(`NEXT-PAGE-SUCCESS: ${x}`)))
// subscribes
searchSub
.do(() => console.warn('NEW_SEARCH'))
.do(() => pendingSub.next(1)) // new search -- ingore current state
.switchMap(
(x) => fakeSearch(x)
.do(dec) // search ended
.concatMapTo(nextPageSub // if you wanted to block nextPage when search still pending
// .merge(nextPageSub // if you wanted to allow nextPage when search still pending
.do(inc) // nexpage started
.flatMap(fakeNextPage) // optionally switchMap
.do(dec) // nextpage ended
)
).subscribe();
pendingSub
.filter(x => x !== undefined) // behavior-value initially not defined
.subscribe(n => console.log('PENDING-REQUESTS', n))
// TEST
const test = () => {
searchSub.next('s1');
nextPageSub.next('p1');
nextPageSub.next('p2');
setTimeout(() => searchSub.next('s2'), 200)
}
// test();
// FUZZY-TEST
const COUNTER_MAX = 50;
const randInterval = randDurationFactory(10)(350);
let counter = 0;
const fuzzyTest = () => {
if (counter % 10 === 0) {
searchSub.next('s' + counter++)
}
nextPageSub.next('p' + counter++);
if (counter < COUNTER_MAX) setTimeout(fuzzyTest, randInterval());
}
fuzzyTest()
<script src="https://npmcdn.com/rxjs#5.0.0-beta.11/bundles/Rx.umd.js"></script>

One way: use the finally operator (rxjs4 docs, rxjs5 source). Finally triggers whenever the observable is unsubscribed or completes for any reason.
I'd also move the counter logic to inside the concatMap function since you are really counting the getPage requests, not the number of values that have gone through. Its a subtle difference.
var pageStream = Rx.createObservableFunction(_self, 'nextPage')
.startWith(1)
.concatMap(function(pageNumber) {
++pendingRequests;
// assumes getPage returns an Observable and not a Promise
return MyHTTPService.getPage(pageNumber)
.finally(function () { --pendingRequests; })
});

I wrote a solution for your problem from scratch.
For sure it might be written in a more functional way but it works anyway.
This solution is based on reqStack which contains all requests (keeping the call order) where a request is an object with id, done and type properties.
When the request is done then requestEnd method is called.
There are two conditions and at least one of them is enough to hide a loader.
When the last request on the stack was a search request then we can hide a loader.
Otherwise, all other requests have to be already done.
function getInstance() {
return {
loaderVisible: false,
reqStack: [],
requestStart: function (req){
console.log('%s%s req start', req.type, req.id)
if(_.filter(this.reqStack, r => r.done == false).length > 0 && !this.loaderVisible){
this.loaderVisible = true
console.log('loader visible')
}
},
requestEnd: function (req, body, delay){
console.log('%s%s req end (took %sms), body: %s', req.type, req.id, delay, body)
if(req === this.reqStack[this.reqStack.length-1] && req.type == 'search'){
this.hideLoader(req)
return true
} else if(_.filter(this.reqStack, r => r.done == true).length == this.reqStack.length && this.loaderVisible){
this.hideLoader(req)
return true
}
return false
},
hideLoader: function(req){
this.loaderVisible = false
console.log('loader hidden (after %s%s request)', req.type, req.id)
},
getPage: function (req, delay) {
this.requestStart(req)
return Rx.Observable
.fromPromise(Promise.resolve("<body>" + Math.random() + "</body>"))
.delay(delay)
},
search: function (id, delay){
var req = {id: id, done: false, type: 'search'}
this.reqStack.push(req)
return this.getPage(req, delay).map(body => {
_.find(this.reqStack, r => r.id == id && r.type == 'search').done = true
return this.requestEnd(req, body, delay)
})
},
nextPage: function (id, delay){
var req = {id: id, done: false, type: 'nextPage'}
this.reqStack.push(req)
return this.getPage(req, delay).map(body => {
_.find(this.reqStack, r => r.id == id && r.type == 'nextPage').done = true
return this.requestEnd(req, body, delay)
})
},
}
}
Unit tests in Moca:
describe('animation loader test:', function() {
var sut
beforeEach(function() {
sut = getInstance()
})
it('search', function (done) {
sut.search('1', 10).subscribe(expectDidHideLoader)
testDone(done)
})
it('search, nextPage', function (done) {
sut.search('1', 50).subscribe(expectDidHideLoader)
sut.nextPage('1', 20).subscribe(expectDidNOTHideLoader)
testDone(done)
})
it('search, nextPage, nextPage', function(done) {
sut.search('1', 50).subscribe(expectDidHideLoader)
sut.nextPage('1', 40).subscribe(expectDidNOTHideLoader)
sut.nextPage('2', 30).subscribe(expectDidNOTHideLoader)
testDone(done)
})
it('search, nextPage, nextPage - reverse', function(done) {
sut.search('1', 30).subscribe(expectDidNOTHideLoader)
sut.nextPage('1', 40).subscribe(expectDidNOTHideLoader)
sut.nextPage('2', 50).subscribe(expectDidHideLoader)
testDone(done)
})
it('search, search', function (done) {
sut.search('1', 60).subscribe(expectDidNOTHideLoader) //even if it takes more time than search2
sut.search('2', 50).subscribe(expectDidHideLoader)
testDone(done)
})
it('search, search - reverse', function (done) {
sut.search('1', 40).subscribe(expectDidNOTHideLoader)
sut.search('2', 50).subscribe(expectDidHideLoader)
testDone(done)
})
it('search, nextPage, search', function (done) {
sut.search('1', 40).subscribe(expectDidNOTHideLoader) //even if it takes more time than search2
sut.nextPage('1', 30).subscribe(expectDidNOTHideLoader) //even if it takes more time than search2
sut.search('2', 10).subscribe(expectDidHideLoader)
testDone(done)
})
it('search, nextPage (call after response from search)', function (done) {
sut.search('1', 10).subscribe(result => {
expectDidHideLoader(result)
sut.nextPage('1', 10).subscribe(expectDidHideLoader)
})
testDone(done)
})
function expectDidNOTHideLoader(result){
expect(result).to.be.false
}
function expectDidHideLoader(result){
expect(result).to.be.true
}
function testDone(done){
setTimeout(function(){
done()
}, 200)
}
})
Part of the output:
JSFiddle is here.

I think there's a much simpler solution, to explain it I'd like to "re-phrase" the examples you gave in your edit:
The status is "pending" as long as there are un-closed requests.
A response closes all previous requests.
Or, in stream/marbles fashion
(O = request [open], C = response [close], p = pending, x = not pending)
http stream: ------O---O---O---C---O---C---O---O---C---O---
------ Status: x----P--------------x---P----x----P---------x---P---
You can see that the count doesn't matter, we have a flag which is actually on (pending) or off (a response was returned). This true because of you switchMap/flatMap, or as you said at the end of your edit, at each time there's only one active request.
The flag is actually a boolean observable/oberver or simply a subject.
So, what you need is to first define:
var hasPending: Subject<boolean> = BehaviorSubject(false);
The BehaviorSubject is relevant for 2 reasons:
You can set an initial value (false = nothing pending).
New subscribers get the last value, hence, even components that are created later will know if you have a pending request.
Than the rest becomes simple, whenever you send out a request, set the pending to 'true', when a request is done, set the pending flag to 'false'.
var pageStream = Rx.createObservableFunction(_self, 'nextPage')
.startWith(1)
.do(function(pageNumber) {
hasPending.next(true);
})
.concatMap(function(pageNumber) {
return MyHTTPService.getPage(pageNumber);
})
.do(function(response) {
hasPending.next(false);
});
Rx.createObservableFunction(_self, 'search')
.flatMapLatest(function(e) {
return pageStream;
})
.subscribe();
This is rxjs 5 syntax, for rxjs 4 use onNext(...)
If you don't need your flat as an observable, just a value, just declare:
var hasPending: booolean = false;
Then in the .do before the http call do
hasPending = true;
and in the .do after the http call do
hasPending = false;
And that's it :-)
Btw, after re-reading everything, you can test this by an even simpler (though somewhat quick and dirty) solution:
Change your post http 'do' call to:
.do(function(response) {
pendingRequests = 0;
});

Looking for one or two Node modules related to locking and update scheduling

Some people are going to think this is too trivial to look for modules or almost put it in a leftPad category perhaps, but there are so many things in npm I have a feeling these two requirements are covered and so am interested to see what other people have done.
I inherited this codebase that has this sort of long updating function with a number of steps. It has sort of a custom locking flag that is half-set up to expire but doesn't actually have the expiration code. Anyway the main thing is to prevent the update from happening while something else is happening. Except sometimes we know we definitely need to update as soon as possible, but not right in the middle of a current update. This stuff is not actually quite working.
So I am looking for one or two modules to do two things:
a lock with an expiration that two different long-running functions (that call other functions with callbacks) can use to make sure they don't step on eachother
a simple module/function that says 'do this function now, or run it again after it finishes its current run'.
Or possibly something that handles both of those might actually make sense in this case, even though from my description it may not be obvious what they have to do with eachother.
The expiration is if there is some case that I don't anticipate that causes the lock not to be removed for a very long time, we don't want to be stuck forever.
Here is an example:
function update() {
// if we are trading, wait until that is done
// if currently updating, skip it or schedule another one right
// after, but only need one more, not a queue of updates
// make a bunch of callbacks or a promise chain, takes a few seconds
// in some cases we will find that we need to do one more update
// immediately after all of this finishes, but not until the end
}
// ...some other places that want to call update
function trade() {
// trade happens sort of sporadically when the tick event fires
// if we are updating, wait until we are done
// make a bunch of callbacks or a promise chain, takes a few seconds
// at the end we need to call update
// but it can't update if it is already updating
}

var Promise = require('bluebird');
// note using a bluebird type of promise
const tradeQueue = [];
var currentlyTrading = false;
var currentlyUpdating = false;
var pendingUpdate = null;
var queuedUpdatePromiseDetails = null;
// returns a promise that returns void, the promise will return
// when either the currently executing update finishes, or a fresh one
// does
function update() {
var test = Math.floor(Math.random() * 1000);
console.log(test, 'update call');
if (!pendingUpdate) {
pendingUpdate = new Promise(function (resolve, reject) {
queuedUpdatePromiseDetails = { resolve: resolve, reject: reject, test: test };
if (currentlyTrading) {
console.log(test, 'setting update to pending');
} else {
// perform update, in callback call resolve() if no err, otherwise reject(err)
console.log(test, 'running update');
currentlyUpdating = true;
setTimeout(resolve, 100);
}
}).finally(function () {
currentlyUpdating = false;
console.log(test, 'update call complete');
pendingUpdate = null;
queuedUpdatePromiseDetails = null;
runPendingTradeOrUpdate();
});
} else {
console.log(test, 'returning existing update promise', queuedUpdatePromiseDetails.test);
}
return pendingUpdate;
}
// returns a promise that completes when this
// new trade finishes
function trade(param) {
var test = Math.floor(Math.random() * 1000);
console.log(test, 'trade call');
return new Promise(function (resolve, reject) {
if (currentlyTrading || currentlyUpdating) {
console.log(test, 'queued trade')
var newTrade = { param: param, resolve: resolve, reject: reject, test: test };
tradeQueue.push(newTrade);
} else {
currentlyTrading = true;
console.log(test, 'beginning trade run');
// perform trade, in callback call resolve() if no error, otherwise reject(err)
setTimeout(resolve, 100);
}
}).finally(function () {
console.log(test, 'trade call complete');
// note that this bit is run for every single trade
currentlyTrading = false;
runPendingTradeOrUpdate();
});
}
// dequeue next trade and run it
function runPendingTradeOrUpdate() {
if (queuedUpdatePromiseDetails && !currentlyUpdating && !currentlyTrading) {
// perform update, in callback call resolve() if no err, otherwise reject(err)
console.log(queuedUpdatePromiseDetails.test, 'running pending update');
currentlyUpdating = true;
setTimeout(queuedUpdatePromiseDetails.resolve, 100);
} else {
if (tradeQueue.length > 0 && !currentlyTrading) {
var nextTrade = tradeQueue.shift();
console.log(nextTrade.test, 'calling queued trade');
trade(nextTrade.param).then(nextTrade.resolve).catch(nextTrade.reject);
}
}
}
var runRandomly = function () {
setTimeout(function () {
update();
}, Math.floor(Math.random() * 300))
setTimeout(function () {
trade(null);
}, Math.floor(Math.random() * 300))
setTimeout(function () {
runRandomly();
}, Math.floor(Math.random() * 300))
}
runRandomly();

OK, thank you so much for your help guys.
This is what I decided I need to do: I created a really simple module now-or-again and am using the existing lock-lock with its maxPending feature instead of an expiration time.
The way I am using those two modules is identical to the examples in the READMEs.