I have a somewhat bigger computation (~0.5sec) in my node app, and I want to make it non-blocking without using a webworker. (I think a webworker would be a little overkill in this situation)
Is there a way to force a return to the main loop, in order to give node the chance to process another request?
It sounds like you're saying you want to do the calculation in bite-sized chunks, on the main thread, rather than spinning it off to its own thread (e.g., using webworker-threads or child processes or the like).
I can think of three options:
ES6 Generator functions.
A function that returns its state and lets you call it again with a state object (which is basically what ES6 generator functions are, but they give you much nicer syntax).
A function that continues running on its own by using nextTick, without your being in control of how it runs.
The third option offers the calling code the least control; the first and third are probably simplest to implement.
ES6 Generator Function
You can use ES6's generator functions in recent versions of NodeJS via the --harmony_generators flag. Generator functions can "yield" back to the calling code, which can then tell them to pick up where they left off later.
Here's an example of a simple generator that counts to a limit, adding one to the count each time you call it:
function* counter(start, inc, max) {
while (start < max) {
yield start;
start += inc;
}
}
var x = counter(1, 1, 10);
console.log(x.next().value); // 1
console.log(x.next().value); // 2
console.log(x.next().value); // 3
Note that that does not spin the calculation off to a different thread, it just lets you do a bit of it, do something else, then come back and do a bit more of it.
A function that returns and accepts its state
If you can't use generators, you can implement the same sort of thing, making all your local variables properties of an object, having the function return that object, and then having it accept it again as an argument:
function counter(start, inc, max) {
var state;
if (typeof start === "object") {
state = start;
if (!state.hasOwnProperty("value")) {
state.value = state.start;
} else if (state.value < state.max) {
state.value += state.inc;
} else {
state.done = true;
}
} else {
state = {
start: start,
inc: inc,
max: max,
done: false
};
}
return state;
}
var x = counter(1, 1, 10);
console.log(counter(x).value); // 1
console.log(counter(x).value); // 2
console.log(counter(x).value); // 3
You can see how generators simplify things a bit.
A function that runs without the calling code controlling it
Here's an example of a function that uses nextTick to perform its task in bite-sized pieces, notifying you when it's done:
function counter(start, inc, max, callback) {
go();
function go() {
var done = start >= max;
callback(start, done);
if (!done) {
++start;
process.nextTick(go);
}
}
}
counter(1, 1, 10, function(value, done) {
console.log(value);
});
Right now, you can't use generators at global scope (you probably don't want to anyway), you have to do it within a function in strict mode. (Even a global "use strict" won't do it.) This is because V8 is still getting its ES6 features...
Complete sample script for a current version of node (allowing for the above):
"use strict";
(function() {
function* counter(start, inc, max) {
while (start < max) {
yield start;
start += inc;
}
}
var x = counter(1, 1, 10);
console.log(x.next().value); // 1
console.log(x.next().value); // 2
console.log(x.next().value); // 3
})();
Related
I've written a simple function which triggers a callback function once it is done printing out a certain string. Are there any caveats I should be aware of when structuring my callbacks the way I did?
Also, what would be the best approach if the original function were to be subjected to asynchronicity?
Code:
// Output via console
var message = "hello there";
function typeOut(message, callback = null, i = 0) {
var interval = setInterval(function() {
if (i < message.length) {
console.log(message.substring(0, i + 1));
i++;
} else {
clearInterval(interval);
callback();
}
}, 150);
//callback;
}
function postDialog() {
console.log('this is postdialog');
}
typeOut(message, postDialog);
Fiddle
Here
Two caveats:
Don't use null as a default value. This will inevitably throw an exception when called. Either use no default value, requiring the caller to provide a function, or use a function that does nothing (e.g. () => {}) for the default value.
The callback should always be the last parameter by convention. This makes calling a function with a long callback nicer, as all the arguments to the call are placed in the same spot, above the continuation.
Given that your i parameter is optional as well, this might not be trivial. Potential workarounds I can think of:
Don't make i a parameter at all - you're not using it anyway. Also in a real-world use case where you "animate" a DOM node it's trivial to prepend a constant prefix to the animated node.
Overload your function to have multiple signatures, and decide depending on the typeof the second parameter whether its i or callback. This does get tedious though.
And in general, the advise for writing new code in a modern code base is of course to use promises instead of callbacks! They will dispose of both the above problems:
function delay(ms) {
return new Promise(res => setTimeout(res, ms));
}
async function typeOut(message, i = 0) {
while (i < message.length) {
await delay(150);
i++;
console.log(message.slice(0, i));
}
}
var message = "hello there";
typeOut(message).then(function postDialog() {
console.log('this is postdialog');
});
I require a long loop (long enough to make the browser hang if I used it naively). The contents of this loop requires other javascript files/libraries and I dont like the idea of concatonating them all into one file as this would make it harder to maintain. So (as far as I understand) this rules out web workers, Are there any other solutions out there?
I need the contents of the loop to run as fast as possible so any delay that can be measured in ms is unacceptable, any fixed delay at all would not be ideal.
I might be able to concatonate them all into a blob at runtime and then feed that into a web worker, I havn't really looked into this to far b/c it doesn't seem like a good solution.
You will have to use an asynchronous approach for this.
An example is to use a "dispatcher" function which invokes the work for each count and keep tracks of current count. It will make sure the next call is called asynchronous. When done the given callback is invoked.
Example
function startLoop(lFrom, lTo, workFunc, callback) {
var i = lFrom - 1;
(function loop() {
if (++i <= lTo) {
setTimeout(function() {
workFunc(i, loop); // start worker for this count
}, 9); // 9ms delay here - tweak as needed
}
else callback(); // done
})(); // self-invokes loop
}
Then in the worker function:
function worker(i, callback) {
// use i for what is needed, then
callback(); // will trigger next count
}
And of course, you can do batches instead of invoking a worker function for each count.
For block based approach for more "heavy" data, see also my answer here.
Simple demo below
startLoop(0, 20, worker, function() {alert("done")});
function startLoop(lFrom, lTo, workFunc, callback) {
var i = lFrom - 1;
(function loop() {
if (++i <= lTo) {
setTimeout(function() {
workFunc(i, loop); // start worker for this count
}, 9); // 9ms delay here - tweak as needed
}
else callback(); // done
})(); // self-invokes loop
}
function worker(i, callback) {
// use i for what is needed, then
document.querySelector("div").innerHTML += (i + "...");
callback(); // will trigger next count
}
<div></div>
Try to wrap loop body into setTimeout with zero time:
while (some_condition) {
setTimeout(function(){
// ... place your code here
}, 0);
}
Context.
Memoization is a functional technique operating over recursive functions with overlapping invocations aiming to optimize time performance by using an internal cache that remembers previous results with already used parameters. A typical use case is the fibonacci function. Below, it is shown a non memoized and memoized version of that function and an assisting function for timing purposes:
function time (fn) {
return function () {
var before = Date.now();
var result = fn.apply(this, arguments);
var after = Date.now();
return {
value : result,
time : after - before
};
};
}
var fib = function (n) {
if (n < 2) return n;
else return fib(n-1) + fib(n-2);
};
var mfib = function (n) {
var cache = {};
var memoizefib = function (n) {
if (n < 2) return n;
else {
var k1 = JSON.stringify(n-1);
var k2 = JSON.stringify(n-2);
var v1 = k1 in cache ? cache[k1] : (cache[k1] = memoizefib(n-1));
var v2 = k2 in cache ? cache[k2] : (cache[k2] = memoizefib(n-2));
return v1 + v2;
}
};
return memoizefib (n);
};
If now we test our functions we realize that memoization dramatically reduces execution time:
(function test (n) {
var tfib = time(fib);
var tmfib = time(mfib);
console.log(tfib(n)); // -> { value: 433494437, time: 5780 }
console.log(tmfib(n)); // -> { value: 433494437, time: 1 }
})(43);
Problem.
As it frequently happens in functional programming, memoization becomes a useful tool when applied at the higher order to allow defining a memoize function that can transform over a generic function fn. Typical solutions similar to the next one can be found on the Web [1][2][3]:
function memoize (fn) {
var cache = {};
return function () {
var args = [].slice.call (arguments);
var key = JSON.stringify(args);
return key in cache ?
cache[key] :
cache[key] = fn.apply(this, args); (1)
};
}
fn.js - http://eliperelman.com/fn.js
underscore - http://underscorejs.org
Addy Osmani on Memoization - http://addyosmani.com/blog/faster-javascript-memoization
Question.
Nevertheless, surprisingly none of these solutions works!!! After revolving around the code. I think that the problem is in (1) because the recursion it is not applied over the memoized version of fn but over primitive fn and hence memoization is only applied once. Here are my results:
(function test (n) {
var tfib = time(fib);
var tmfib = time(memoize(fib));
console.log (tfib(n)); // -> { value: 433494437, time: 5768 }
console.log (tmfib(n)); // -> { value: 433494437, time: 5723 } :(
})(43);
It seems that in Javascript it is not possible to apply this technique at higher order. Am I right? Does anybody have any solution or alternative code to get a higher order memoization function?
Interesting problem. Why not just memoize the function onto itself?
function factorial(n) { return n ? n * factorial(n-1) : 1; }
// simple memoization with one argument and console reporting
function memoize(fn) {
var cache = {};
return function(x) {
if (x in cache) { console.log('retrieved value from cache for', x); }
return x in cache ? cache[x] : cache[x] = fn.apply(this, arguments);
};
}
// redefine factorial to be its memoized version
factorial = memoize(factorial);
Once you do this, factorial will now be calling its memoized version.
> factorial(6)
720
> factorial(7)
retrieved value from cache for 6
5040
Applying this to your case (no need for mfib):
(function test (n) {
var tfib = time(fib);
console.log(tfib(n));
fib = memoize(fib); // <-- memoize on top of itself
var tmfib = time(fib);
console.log(tmfib(n));
})(30);
Results:
Object {value: 832040, time: 714}
Object {value: 832040, time: 22}
Note that this solution very much applies to "internal memoization" used within a single recursive computation, not just additional external calls to the function as in the factorial case above. By redefining the function with its memoized version, the internal recursive calls are now made to the memoized function. That accounts for the dramatic time improvement from 714 to 22.
If you are looking for "internal" memoization within a single recursive computation, the factorial function (which is given in the currently accepted answer - which BTW I also think is wrong) is a bad candidate and can't be used to showcase it. This is because, there's a single chain of recursion, so once you compute, e.g. the value for 5! you are not going to need it again within the same computation. To showcase what you are asking for, you would indeed need to use an example like the Fibonnaci sequence (which you are yourself using in the question).
The higher-order memoize function you wrote will not work internally within the same computation as it is calling the non-memoized version of it. The currently accepted answer suggests "redefining" the function to be its memoized version. However I think this is wrong. The memoize function closes over the function it was provided with as an argument (functions are values). Changing the variable that used to point to that function value to some other function value (the memoized version) accomplishes nothing.
So, I don't think it is possible to generalize it. That is, I don't believe it is possible to implement a memoize function that externally memoizes a function and makes it work also for a single recursive computation.
Here's what would work for Fibonnaci (like I said, not generalizable):
var fib = (function() {
function fib(n) {
if ((n===1) || (n==2))
return 1;
else
return memoizedFibonacci(n-1)+memoizedFibonacci(n-2);
}
var cache = {};
function memoizedFibonacci(n) {
if (n in cache) { console.log('retrieved value from cache for', n); }
return n in cache ? cache[n] : cache[n] = fib(n);
};
return fib;
})();
console.log(fib(10));
console.log(fib(10));
The above code produces on the output:
retrieved value from cache for 2
retrieved value from cache for 3
retrieved value from cache for 4
retrieved value from cache for 5
retrieved value from cache for 6
retrieved value from cache for 7
retrieved value from cache for 8
55
retrieved value from cache for 9
retrieved value from cache for 8
55
... which is consistent with the expectation for "internal memoization" for the first recursive computation. That the value for 10 is not cached is insignificant for this example and could be trivially fixed but it would add very little (basically you would need to access the cache from the fib function as well).
I have this code that uses an inefficientProcess() that consumes plenty of memory:
My goal is to use some sort of setTimeout(function(){...},0) technique so the browser will not get stuck while executing the code.
How do I change the code so it will work with setTimeout?
function powerOfTwo(num) {
inefficientProcess();
if (num > 0) {
return powerOfTwo(num-1)*2;
} else {
return 1;
}
}
function inefficientProcess() {
var sum;
for (var i=0; i < 500000; i++) {
sum+=10;
}
}
powerOfTwo(1000);
My goal is ofcourse to learn how to avoid browser crush when executing heavy calculations.
Javascript is single-threaded, and all your code is blocking.
There is a new standard in HTML5, WebWorkers API, that will allow you to delegate your task to a different thread. You can then pass a callback function to be executed with the result.
https://developer.mozilla.org/en-US/docs/DOM/Using_web_workers
Simple example:
function powerOfTwo(num, callback) {
var worker = new Worker('inneficient.js');
worker.postMessage('runTask');
worker.onmessage = function(event) {
var num = event.data.result;
var pow;
if (num > 0) {
pow = Multiply(num-1)*2;
} else {
pow = 1;
}
callback(pow);
};
}
powerOfTwo(1000, function(pow){
console.log('the final result is ' + pow);
});
in inneficient.js you have something like:
//inneficient.js
function inefficientProcess() {
var sum;
for (var i=0; i < 500000; i++) {
sum+=10;
}
postMessage({ "result": sum});
}
inefficientProcess();
As was mentioned in Andre's answer, there's a new HTML5 standard that will allow you to set off a task on a different thread. Otherwise, you can call setTimeout with a time of 0 to allow the current execution path to finish (and perhaps some UI changes to render) before the inefficientProcess is called.
But whether you can use HTML5 or not, the powerOfTwo function has to be changed to be asynchronous - whoever calls it needs to provide a callback method that will be called when (a) a thread spun up via WebWorkers returns, or (b) the setTimeout method finishes.
Edited to add example:
function powerOfTwo(num, callback)
{
setTimeout(function ()
{
inefficientProcess();
if (num > 0)
callback(Multiply(num-1)*2);
else
callback(1);
}, 0);
}
function inefficientProcess() { ... }
The HTML element allows you to define when the JavaScript
code in your page should start executing. The “async” and “defer”
attributes were added to WebKit early September. Firefox has been
supporting them quite a while already.
Saw that on this Site
I asked a question a couple weeks ago about using setTimeout for a factorial function, but it was unfortunately with an unregistered account and I never got a complete answer.
My main issue is that I'd like to write a function that calculates the factorial of a number but uses the setTimeout or setInterval commands. The impetus behind this is to reset a counter the IE uses so as to avoid a long-running script warning. Currently, the factorial function I have is:
function factorial(n) {
return 0 === n || 1 === n ? 1 : n * factorial(n - 1)
}
In my other post, jsumners was kind enough to provide me with code that tried to use setTimeout periodically when calculating a factorial:
function factorial(x) {
executions++;
if (x > 1) {
if (executions % 20 === 0) {
return (function() {
var y = x;
setTimeout(function(y) { return y*factorial(y-1); }, 1);
});
} else {
return x*factorial(x-1);
}
} else {
executions = 0;
return 1;
}
}
In the above code, it should theoretically use the setTimeout command to perform the next multiplication when the number of elapsed executions is a factor of 20 (mod 20). Unfortunately, the code does not work, and what happens is that if trying to calculate the factorial of a number greater than 20, then the result is NaN. If the number is less than 20, then the answer is correct.
Does anyone know of a solution to this or another way to calculate a factorial by using the setTimeout or setInterval commands?
Thanks!
It's because you're specifying y as a parameter which is undefined when executed because it's not passed in, you can fix it by change this:
setTimeout(function(y) { return y*factorial(y-1); }, 1);
To this:
setTimeout(function() { return y*factorial(y-1); }, 1);
However, it'll still be NaN because here:
return (function() {
var y = x;
setTimeout(function() { return y*factorial(y-1); }, 1);
});
You're still returning a function, not a number that can be multiplied, so you still can't use a setTimeout() in this manner. You could pass a callback that executes when everything's done, but you can't have it recurse and return to a caller like this.
The callback style factorial with each recurrent step scheduled with setTimeout is:
// private helper function (recurrency with accumulation)
function _factorial(acc, n, callback){
if(n==0){
callback(acc);
}else{
var callback_wrapper = function(result){
callback(result);
};
setTimeout(function(){_factorial(acc * n, n-1, callback_wrapper)}, 10);
}
}
// public function
function factorial(n, callback){
_factorial(1, n, callback);
}
// usage example
factorial(10, function(result){console.log(result)});
--
Cheers,
Lambder
http://lambder.com/
http://vanadiumJS.com/