We Keep Coding

JavaScript: trying to create a well-behaved background job, but it gets too little time to run while rest of the system is mostly idle?

In a browser, I am trying to make a well-behaved background job like this:
function run() {
var system = new System();
setInterval(function() { system.step(); }, 0);
}
It doesn't matter what that System object is or what the step function does [except it needs to interact with the UI, in my case, update a canvas to run Conway's Game of Life in the background], the activity is performed slowly and I want it to run faster. But I already specified no wait time in the setInterval, and yet, when I check the profiling tool in Chrome it tells me the whole thing is 80% idle:
Is there a way to make it do less idle time and perform my job more quickly on a best effort basis? Or do I have to make my own infinite loop and then somehow yield back time to the event loop on a regular basis?
UPDATE: It was proposed to use requestIdleCallback, and doing that makes it actually worse. The activity is noticably slower, even if the profiling data isn't very obvious about it, but indeed the idle time has increased:
UPDATE: It was then proposed to use requestAnimationFrame, and I find that once again the slowness and idleness is the same as the requestIdleCallback method, and both run at about half the speed that I get from the standard setInterval.
PS: I have updated all the timings to be comparable, all three now timing about 10 seconds of the same code running. I had the suspicion that perhaps the recursive re-scheduling might be the cause for the greater slowness, but I ruled that out, as the recursive setTimeout call is about the same speed as the setInterval method, and both are about twice as fast as these new request*Callback methods.
I did find a viable solution for what I'm doing in practice, and I will provide my own answer later, but will wait for a moment longer.
OK, unless somebody comes with another answer this here would be my FINAL UPDATE: I have once again measured all 4 options and measured the elapsed time to complete a reasonable chunk of work. The results are here:
setTimeout - 31.056 s
setInterval - 23.424 s
requestIdleCallback - 68.149 s
requestAnimationFrame - 68.177 s
Which provides objective data to my impression above that the two new methods with request* will perform worse.
I also have my own practical solution which allows me to complete the same amount of work in 55 ms (0.055 s), i.e., > 500 times faster, and still be relatively well behaved. Will report on that in a while. But wonder what anybody else can figure out here?

I think this is really dependent on what exactly you are trying to achieve though.
For example, you could initialize your web-worker on loading the page and make it run the background-job, if need be, then communicate the progress or status of the job to the main thread of your browser. If you don't like the use of post-message for communication between the threads, consider user Comlink
Web worker
Comlink
However, if the background job you intend to do isn't something worth a web-worker. You could use the requestIdleCallback API. I think it fits perfectly with what you mentioned here since you can already make it recursive. You would not need a timer anymore and the browser can help you schedule the task in such a way that it doesn't affect the rendering of your page (by keeping everything with 60fps).
Something like =>
function run() {
// whatever you want to keep doing
requestIdleCallback(run)
}
You can read more about requestIdleCallback on MDN.

OK, I really am not trying to prevent others to get the bounty, but as you can see from the details I added to my question, none of these methods allow high rate execution of the callback.
In principle the setInterval is the most efficient way to do it, as we already do not need to re-schedule the next call back all the time. But it is a small difference only. Notably requestIdleCallback and requestAnimationFrame are the worst when you want to be rapidly called back.
So, what needs to be done is instead of executing only a tiny amount of work and then expect to be called back quickly, we need to batch up more work. Problem is we don't know exactly how much work we should batch up before it is too much. That can probably in most cases be figured out with trial and error.
Dynamically one might take timing probes to find out how quickly we are being called back again and preemptively exit the work (loop of some kind) when the time between the call-backs is expired.

How to Long Poll in NodeJS / Javascript?

Just to be clear my understanding of long polling is that you make request to a server on a time interval.
I am trying to implement a bitcoin purchasing system that checks the blockchain for change in my wallets balance. I know there are websockets that do this but I have to wait for 1 confirmation to receive an update and the REST API offers more flexibility, so I would just prefer to make a request to the server every 5 seconds or so and check each response for a change in my balance then go from there.
The issue is I can't seem to figure out how to do this in NodeJS. Functionially this is how I imagine my code.
Get current balance (make request)
Get current balance again (make request)
Check if there is a difference
**If not**
wait 5 seconds
Get current balance
Check for difference
repeat till different (or till timeout or something)
If different
do some functions and stop checking balance.
I've been trying to do each step but I've gotten stuck at figuring out how to create a loop of checking the balance, and stopping the loop if it changes.
My original thought was to use promises and some for loops but that doesn't materialize.
So now I am asking for your help, how should I go about this?

One way to do this would be to setup a setInterval timer to kickoff a request every x seconds. By setting some logic after the response you can then choose to de-reference the timer and trigger another function. Here's a snippet. You'll notice I set a variable to reference the timer, and then de-reference it by setting it to null, where then the GC is smart enough to release. You may also use the 'clearTimeout' function, which is perhaps the better way to go.

A Reliable timer for Javascript

I have a web site and i am using a javscript timer to swap images about.
I am using the timer like this:
var myTimer = window.setTimeout(MyFunction, MyInterval);
function MyFunction
{
//do something
//recalll timer
}
Now, the problem I have is not that the interval does not fire off at regular intervals as I can accept that in my application and I understand why it can vary.
The issue I have is that every now and then the timer stops for a few seconds and then resumes.
What I am trying ascertain is what is the main cause of this random suspension of the timer?
Is it due to the resources being diverted to another process on the hosting browser PC OR is it just the nature of using a JavaScript timer?
If the latter should I look to do an eternal loop? Everywhere I read and have practised elsewhere indicates that an infinite loop will grab all the resources and it would be a greater evil than the timer random suspension.
Are there any alternatives to using a javascript timer when a regular quick execution of code is paramount?
Thanks

The code you run inside MyFunction takes some time to execute (depending on what you are doing). When you recall the timer at the end of that function, the interval is not exactly MyInterval, because of the code execution time being added.
If you use setInterval() instead of setTimeout(), the given function will be executed exactly every MyInterval milliseconds rather than (MyInterval + execution time) milliseconds.
To answer your question, the random suspension happens because of the execution time of your code.

I had a similar issue on a website I was working on and ultimately found the culprit in another timer-triggered job in a jQuery plugin that was occasionally delaying execution of my own function. If you're using external code in your site, you could do some debugging to see if this is your case too.
As a possible remedy, you could give a look at web workers. Since worker tasks are executed in a separated thread, they are not subject to delay when something in your main thread is taking too long to complete.
Your code would then look like this:
var worker = new Worker('worker.js');
And in another file called "worker.js" you would write:
var myTimer = setTimeout(MyFunction, MyInterval);
function MyFunction
{
//do something
//recalll timer
}
Just note that there is no window. anymore before setTimeout. This is because web workers don't have direct access to the DOM.
It's not guaranteed to solve your problem, but it's worth a test.

Batch Backbone.js events?

My app's framework is built around collapsing backbone models sending the data via websockets and updating models on other clients with the data. My question is how should I batch these updates for times when an action triggers 5 changes in a row.
The syncing method is set up to update on any change but if I set 5 items at the same time I don't want it to fire 5 times in a row.
I was thinking I could do a setTimeout on any sync that gets cleared if something else tries to sync within a second of it. Does this seem like the best route or is there a better way to do this?
Thanks!

i haven't done this with backbone specifically, but i've done this kind of batching of commands in other distributed (client / server) apps in the past.
the gist of it is that you should start with a timeout and add a batch size for further optimization, if you see the need.
say you have a batch size of 10. what happens when you get 9 items stuffed into the batch and then the user just sits there and doesn't do anything else? the server would never get notified of the things the user wanted to do.
timeout generally works well to get small batches. but if you have an action that generates a large number of related commands you may want to batch all of the commands and send them all across as soon as they are ready instead of waiting for a timer. the time may fire in the middle of creating the commands and split things apart in a manner that causes problems, etc.
hope that helps.

Underscore.js, the utility library that Backbone.js uses, has several functions for throttling callbacks:
throttle makes a version of a function that will execute at most once every X milliseconds.
debounce makes a version of a function that will only execute if X milliseconds elapse since the last time it was called
after makes a version of a function that will execute only after it has been called X times.
So if you know there are 5 items that will be changed, you could register a callback like this:
// only call callback after 5 change events
collection.on("change", _.after(5, callback));
But more likely you don't, and you'll want to go with a timeout approach:
// only call callback 30 milliseconds after the last change event
collection.on("change", _.debounce(30, callback));

Reducing Javascript CPU Usage

I'm planning on writing some code for encrypting files in javascript locally. For large files and large key sizes, the CPU usage (naturally) is pretty high. In a single script design, this often hangs the browser until the task is complete.
In order to improve responsiveness and allow users to do other things in the mean time I want to try make the script 'friendlier' to the user's PC. The encryption process will be reading a file as a binary string and then encrypting the string in chunks (something like 1KB/chunk - needs testing). I want to try and user HTML5-based workers to make the whole thing as incremental as possible. Something like:
Spawn worker
Send worker a binary data chunk
Worker completes encryption, passes back new chunk
Worker dies.
This might also help with multicore processors, by having multiple workers alive at once.
Anyway, has anybody looked at deliberately slowing down a script in order to reduce CPU usage? Something like splitting the worker's encryption task into single operations, and introducing a delay between them.
Interval timer callback every 100ms (example).
Is worker busy?
Yes - Wait for another interval
No - Start encrypting the next letter
Advice/thoughts?
Does anyone have experience using workers? If you seperate the main UI from intensieve work by making it a worker, does the responsiveness increase?

This doesn't utilize anything HTML5, but here's an example for calling a function every N milliseconds, assuming you can determine an appropriate wait time. Basically, I'm trying to help you by showing you a way to enforce stalling some amount of time before doing more processing.
function doSomething(){
clearTimeout(timeout);
// do your "expensive" processing
// ...
// decide the job is done and return here or else
// call doSomething again in sleepMS milliseconds
timeout = setTimeout(doSomething,sleepMS);
}
var timeout;
var sleepMS = 1000;
doSomething();
EDIT
changed last line from
var timeout = setTimeout(doSomething,1000);
to just this
doSomething()
EDIT 2
Changed 1000 to sleepMS in setTimeout call, duh :)