Reading the book "Node.js Design Patterns" , on the page 78, I found the following code:
var tasks = [...];
var concurrency = 2, running = 0, completed = 0, index = 0;
function next() {
while(running < concurrency && index < tasks.length) {
task = tasks[index++];
task(function() { // THE ISSUE IS HERE!!!
if(completed === tasks.length) {
return finish();
}
completed++, running--;
next();
}); // THE END OF MY ISSUE
running++;
}
}
next();
function finish() {
//all tasks finished
}
So I can't understand what the snippet task(function(){...}); does mean. Intersted the syntax as itself. How this style is called? May be some shorthand or, for example, "passing function as a parameter (or method) to the object"? And where I can read about this syntax? I found nothing.
A function that accepts another function as an argument is known as a Higher Order Function.
I wouldn't call it a "syntax". It's more of a technique to create more generalized code.
In this case, the function being passed is likely a callback that will be executed at a later time. This is fairly common in (but not specific to) Javascript due to how often asynchronous code is used.
Related
I've written a simple function which triggers a callback function once it is done printing out a certain string. Are there any caveats I should be aware of when structuring my callbacks the way I did?
Also, what would be the best approach if the original function were to be subjected to asynchronicity?
Code:
// Output via console
var message = "hello there";
function typeOut(message, callback = null, i = 0) {
var interval = setInterval(function() {
if (i < message.length) {
console.log(message.substring(0, i + 1));
i++;
} else {
clearInterval(interval);
callback();
}
}, 150);
//callback;
}
function postDialog() {
console.log('this is postdialog');
}
typeOut(message, postDialog);
Fiddle
Here
Two caveats:
Don't use null as a default value. This will inevitably throw an exception when called. Either use no default value, requiring the caller to provide a function, or use a function that does nothing (e.g. () => {}) for the default value.
The callback should always be the last parameter by convention. This makes calling a function with a long callback nicer, as all the arguments to the call are placed in the same spot, above the continuation.
Given that your i parameter is optional as well, this might not be trivial. Potential workarounds I can think of:
Don't make i a parameter at all - you're not using it anyway. Also in a real-world use case where you "animate" a DOM node it's trivial to prepend a constant prefix to the animated node.
Overload your function to have multiple signatures, and decide depending on the typeof the second parameter whether its i or callback. This does get tedious though.
And in general, the advise for writing new code in a modern code base is of course to use promises instead of callbacks! They will dispose of both the above problems:
function delay(ms) {
return new Promise(res => setTimeout(res, ms));
}
async function typeOut(message, i = 0) {
while (i < message.length) {
await delay(150);
i++;
console.log(message.slice(0, i));
}
}
var message = "hello there";
typeOut(message).then(function postDialog() {
console.log('this is postdialog');
});
Although I usually enjoy the callback-soup that is Node.JS, I found that a certain part of my code needs to be run in a blocking manner because of an SQLite issue. (Yes, I know I could try and address the SQLite part, it actually makes more sense to ensure blocking.)
I like using the async module, and though I have a feeling that module has a feature which can be used here, I can't seem to find it. Or, maybe there is a better module out there. Anyway, without further ado:
func = function(callback) {
let i = 0;
arr.forEach(val => {
if (val.trim().length > 0) {
console.log(`Starting for user ${val}.`);
let mylc = new lcapp(val);
////// TODO this needs to be made sycnronous. /////
async.series({
getMyDetails: callback => getMyDetails(mylc, callback)
}, (err, results) => handleResults(err, results, mylc, callback));
/////////////
}
});
};
The section of code surrounded by //// I would like to block until the handleResults function returns. I understand it will require reprogramming the callback in handleResults, or maybe I need to write a parent function around func but I'd like to see if StackOverflow people have some good ideas.
You could turn it into a function that recursively calls itself when the handleResults callback is hit.
You can do this by following the example below.
fun()
function fun() {
console.time("fun")
var arr = [1, 2, 3, 4, 5]
var i = arr.length - 1;
doStuff(doStuffCallback)
function doStuffCallback() {
if (i > 0) {
i--
doStuff(doStuffCallback)
} else {
console.timeEnd("fun")
}
}
function doStuff(callback) {
setTimeout(function() {
logIt()
callback()
}, 25)
}
function logIt() {
console.log(arr[i])
}
}
// Output:
// 5
// 4
// 3
// 2
// 1
// fun: about 160ms
PS: I'm assuming you only need to be synchronous within this method and the loop therein. Other code might still be running elsewhere in your application while this runs.
Yes, I know I could try and address the SQLite part, it actually makes more sense to ensure blocking.
No it doesn't, because you can't. You need to resolve whatever issue you have with it being async because there is no way to turn asynchronous code into synchronous code.
I am new to NodeJS and I am not really sure how should the following function be declared. The function contains only a for loop which generates a string. There are no "heavy-weight" calculations done.
Variant 1:
function getRandomString(arg) {
for (var i = 0; i < 100; i++) {
// ...
}
return string;
}
var randomString = getRandomString(arg);
// ... an async code which will use the string
Variant 2: Or should I make it async (async-style)? It would look something like this:
function getRandomString(arg, callback) {
for (var i = 0; i < 100; i++) {
// ...
}
callback(string);
}
getRandomString(arg, function(randomString) {
// Async code...
});
Or should I make it async? So something like this:
function getString(arg, callback) {
for(var i = 0;i<100;i++) {
// ...
}
callback(string);
}
No. That code does still run synchronousls, only with an odd callback style for returning the result. Since JS does no tail call optimisation or has continuation support, it just introduces the pyramid of doom without any benefit.
Do not use this unless you really make it asynchronous (setTimeout, nextTick etc), for example to defer the single loop iterations.
There is no benefit from making it "asynchronous". In fact, the code will run synchronously, with the getString method only exiting after the callback is complete.
The only decision you really have to make is one of coding style (Do you want it to seem asynchronous or not.)
function getString(arg, callback)
{
for(var i = 0;i<100;i++)
{
// ...
}
//Callback will execute synchronously, and you will wait here until it is complete
callback(string);
}
I have a JavaScript function like the following.
function changeTheDom(var1, var2, var3) {
// Use DWR to get some server information
// In the DWR callback, add a element to DOM
}
This function is called in a couple of places in the page. Sometimes, in a loop. It's important that the elements be added to the DOM in the order that the changeTheDom function is called.
I originally tried adding DWREngine.setAsync(false); to the beginning of my function and DWREngine.setAsync(true); to the end of my function. While this worked, it was causing utter craziness on the rest of the page.
So I am wondering if there is a way to lock the changeTheDom function. I found this post but I couldn't really follow the else loop or how the lockingFunction was intended to be called.
Any help understanding that post or just making a locking procedure would be appreciated.
Don't try to lock anything. The cleanest way is always to adapt to the asynchronous nature of your code. So if you have an asynchronous function, use a callback. In your particular case I would suggest that you split your function up in one part that is executed before the asych call and one part that is executed afterwards:
function changeTheDomBefore(var1, var2, var3) {
//some code
//...
asyncFunction(function(result){
//this will be executed when the asynchronous function is done
changeTheDomAfter(var1, var2, var2, result);
});
}
function changeTheDomAfter(var1, var2, var3, asynchResult) {
//more code
//...
}
asyncFunction is the asynchronous function which, in this example, takes one argument - the callback function, which then calls your second changeTheDom function.
I think I finally got what you mean and I decided to create another answer, which is hopefully more helpful.
To preserve order when dealing with multiple asynchronous function calls, you could write a simple Queue class:
function Queue(){
var queue = [];
this.add = function(func, data) {
queue.push({func:func,data:data});
if (queue.length === 1) {
go();
}
};
function go() {
if (queue.length > 0) {
var func = queue[0].func,
data = queue[0].data;
//example of an async call with callback
async(function() {
func.apply(this, arguments);
queue.shift();
go();
});
}
}
};
var queue = new Queue();
function doit(data){
queue.add(function(result){
console.log(result);
}, data);
}
for (var i = 0; i < 10; i++) {
doit({
json: JSON.stringify({
index: i
}),
delay: 1 - i / 10.0
});
}
FIDDLE
So everytime you invoke your async function, you call queue.add() which adds your function in the queue and ensures that it will only execute when everything else in the queue is finished.
My specific problem is that I need to execute a (potentially) large number of Javascript functions to prepare something like a batch file (each function call adds some information to the same batch file) and then, after all those calls are completed, execute a final function to send the batch file (say, send it as an HTML response). I'm looking for a general Javascript programming pattern for this.
Generalize problem:
Given the Javascript functions funcA(), funcB(), and funcC(), I would to figure out the best way to order execution so that funcC is only executed after after funcA and funcB have executed. I know that I could use nested callback functions like this:
funcA = function() {
//Does funcA stuff
funcB();
}
funcB = function() {
//Does funcB stuff
funcC();
}
funcA();
I could even make this pattern a little more general by passing in callback parameters, however, this solution becomes quite verbose.
I am also familiar with Javascript function chaining where a solution might look like:
myObj = {}
myObj.answer = ""
myObj.funcA = function() {
//Do some work on this.answer
return this;
}
myObj.funcB = function() {
//Do some more work on this.answer
return this;
}
myObj.funcC = function() {
//Use the value of this.answer now that funcA and funcB have made their modifications
return this;
}
myObj.funcA().funcB().funcC();
While this solution seems a little cleaner to me, as you add more steps to the computation, the chain of function executions grows longer and longer.
For my specific problem, the order in which funcA, funcB, etc. are executed DOES NOT matter. So in my solutions above, I am technically doing more work than is required because I am placing all the functions in a serial ordering. All that matters to me is that funcC (some function for sending the result or firing off a request) is only called after funcA and funcB have ALL completed execution. Ideally, funcC could somehow listen for all the intermediate function calls to complete and THEN would execute? I hoping to learn a general Javascript pattern to solve such a problem.
Thanks for your help.
Another Idea:
Maybe pass a shared object to funcA and funcB and when they complete execution mark the shared object like sharedThing.funcA = "complete" or sharedThing.funcB = "complete" and then somehow? have funcC execute when the shared object reaches a state where all fields are marked complete. I'm not sure how exactly you could make funcC wait for this.
Edit:
I should note that I'm using server-side Javascript (Node.js) and I would like to learn a pattern to solve it just using plain old Javascript (without the use of jQuery or other libraries). Surely this problem is general enough that there is a clean pure-Javascript solution?
If you want to keep it simple, you can use a counter-based callbacks system. Here's a draft of a system that allows when(A, B).then(C) syntax. (when/then is actually just sugar, but then again the whole system arguably is.)
var when = function() {
var args = arguments; // the functions to execute first
return {
then: function(done) {
var counter = 0;
for(var i = 0; i < args.length; i++) {
// call each function with a function to call on done
args[i](function() {
counter++;
if(counter === args.length) { // all functions have notified they're done
done();
}
});
}
}
};
};
Usage:
when(
function(done) {
// do things
done();
},
function(done) {
// do things
setTimeout(done, 1000);
},
...
).then(function() {
// all are done
});
If you don't use any asynchronous functions and your script doesn't break the order of execution, then the most simple solution is, as stated by Pointy and others:
funcA();
funcB();
funcC();
However, since you're using node.js, I believe you're going to use asynchronous functions and want to execute funcC after a async IO request has finished, so you have to use some kind of counting mechanisms, for example:
var call_after_completion = function(callback){
this._callback = callback;
this._args = [].slice.call(arguments,1);
this._queue = {};
this._count = 0;
this._run = false;
}
call_after_completion.prototype.add_condition = function(str){
if(this._queue[str] !== undefined)
throw new TypeError("Identifier '"+str+"' used twice");
else if(typeof str !== "String" && str.toString === undefined)
throw new TypeError("Identifier has to be a string or needs a toString method");
this._queue[str] = 1;
this._count++;
return str;
}
call_after_completion.prototype.remove_condition = function(str){
if(this._queue[str] === undefined){
console.log("Removal of condition '"+str+"' has no effect");
return;
}
else if(typeof str !== "String" && str.toString === undefined)
throw new TypeError("Identifier has to be a string or needs a toString method");
delete this._queue[str];
if(--this._count === 0 && this._run === false){
this._run = true;
this._callback.apply(null,this._args);
}
}
You can simplify this object by ignoring the identifier str and just increasing/decreasing this._count, however this system could be useful for debugging.
In order to use call_after_completion you simply create a new call_after_completion with your desired function func as argument and add_conditions. func will only be called if all conditions have been removed.
Example:
var foo = function(){console.log("foo");}
var bar = new call_after_completion(foo);
var i;
bar.add_condition("foo:3-Second-Timer");
bar.add_condition("foo:additional function");
bar.add_condition("foo:for-loop-finished");
function additional_stuff(cond){
console.log("additional things");
cond.remove_condition("foo:additional function");
}
for(i = 0; i < 1000; ++i){
}
console.log("for loop finished");
bar.remove_condition("foo:for-loop-finished");
additional_stuff(bar);
setTimeout(function(){
console.log("3 second timeout");
bar.remove_condition("foo:3-Second-Timer");
},3000);
JSFiddle Demo
If you don't want to use any helper libraries, than you need to write some helper yourself, there's no simple one line solution for this.
If you'd like to end with something that looks as readable as it would in synchronous case, try some deferred/promise concept implementation (it's still plain JavaScript), e.g. using deferred package you may end up with something as simple as:
// Invoke one after another:
funcA()(funcB)(funcC);
// Invoke funcA and funcB simultaneously and afterwards funcC:
funcA()(funcB())(funcC);
// If want result of both funcA and funcB to be passed to funcC:
deferred(funcA(), funcB())(funcC);
Have a look into jQuery's deferred objects. This provides a sophisticated means of controlling what happens when in an asynchronous environment.
The obvious use-case for this is AJAX, but it is not restricted to this.
Resources:
jQuery docs: deferred object
good introduction to deferred object patterns
Non-AJAX use for jQuery's deferred objects
I was looking for the same kind of pattern. I am using APIs that interrogate multiple remote data sources. The APIs each require that I pass a callback function to them. This means that I cannot just fire off a set of my own functions and wait for them to return. Instead I need a solution that works with a set of callbacks that might be called in any order depending on how responsive the different data sources are.
I came up with the following solution. JS is way down the list of languages that I am most familiar with, so this may not be a very JS idiom.
function getCallbackCreator( number_of_data_callbacks, final_callback ) {
var all_data = {}
return function ( data_key ) {
return function( data_value ) {
all_data[data_key] = data_value;
if ( Object.keys(all_data).length == number_of_data_callbacks ) {
final_callback( all_data );
}
}
}
}
var getCallback = getCallbackCreator( 2, inflatePage );
myGoogleDataFetcher( getCallback( 'google' ) );
myCartoDataFetcher( getCallback( 'cartodb' ) );
Edit: The question was tagged with node.js but the OP said, "I'm looking for a general Javascript programming pattern for this," so I have posted this even though I am not using node.
Nowadays, one can do something like this:
Let's say we have both funcA, funcB and funcC:
If one's want funcA and funcB results to be passed to funcC:
var promiseA = new Promise((resolve, reject) => {
resolve(await funcA());
});
var promiseB = new Promise((resolve, reject) => {
resolve(await funcB());
});
var promise = Promise.all([ promiseA, promiseB ]).then(results => {
// results = [result from funcA, result from funcB]
return funcC(results);
});
If one's want funcA, then funcB and then funcC:
var promise = (
new Promise(async resolve => resolve( await funcA() ))
).then(result_a => funcB(result_a)).then(result_b => funcC(result_b));
And finally:
promise.then(result_c => console.log('done.'));
how about:
funcC(funcB(funcA)));
I think the questions is because some of functions run longer and there might be a situation when we run funcC when funcA or funcB did not fininsh executing.