I started learning node.js by reading Node JS in Action book recently. This is probably a newbie question, but after reading several posts of callback functions and javascript scope of variables, I still have problem understand the idea behind this code in chapter 5 of the book.
function loadOrInitializeTaskArray(file, cb) {
fs.exists(file, function(exists) {
var tasks = [];
if (exists) {
fs.readFile(file, 'utf8', function(err, data) {
if (err) throw err;
var data = data.toString();
var tasks = JSON.parse(data || '[]');
cb(tasks);
});
} else {
cb([]);
}
});
}
function listTasks(file) {
loadOrInitializeTaskArray(file, function(tasks) {
for(var i in tasks) {
console.log(tasks[i]);
}
});
}
It includes three callback functions divided in two functions. listTasks(..) is called first and it calls loadorInitializeTaskArray(.. ) later. My problem starts here, how is this call handle by node? loadOrInitializeTaskArray takes two arguments and the second one is the callback function which shouldn't accept any parameters according to is signature but it does!!
when does cb(..) get called in loadorInitializeTaskArray and what is that (the same function that calls helper function)?
"tasks" is an array declared inside function loadOrInitializeTaskArray, how do we have access to it in listTasks(..) function?
I know in Javascript, scope of a variable is inside the function it define and all nested functions. But I have a hard time understanding it here. So can someone explain what is going on here?
Thank you
You really are having a hard time in understanding the scope of variable inside the nested functions. So, lets start with it.
Let's consider this code
function foo(){
var a = 3;
function bar(){
console.log(a);
}
return bar;
}
var baz = foo();
baz(); // the value of a i.e 3 will appear on console
If you know langauges like C, C++... Your brain would be interpreting this code like this. If you don't know any of them just ignore thiese points.
When foo() is called, variable a will be declared and assigned value 3.
When foo is returned the stack containing a will be destroyed. And hence a would be destroyed too.
Then how in the world baz() is outputting 3. ???
Well, in javascript when a function is called the things that happen are different than in C. So, first let your all C things go out from your mind before reading further.
In javascript scope resolution is done by travelling down a chain of objects that defines variables that are "in scope" for that code. Let's see how?
When you declare a global JavaScript variable, what you are actually doing is defining a property of the global object.
In top-level JavaScript code (i.e., code not contained within any function definitions), the scope chain consists of a single object, the global object.
In a non-nested function, the scope chain consists of two objects. The first is the object that defines the function’s parameters and local variables, and the second is the global object.
In a nested function, the scope chain has three or more objects. When a function is defined, it stores the scope chain then in effect. When that function is invoked, it creates a new object to store its local variables, and adds that new object to the stored scope chain to create a new, longer, chain that represents the scope for that function invocation.
So, when foo() is executed. It creates a new object to store it's local variables. So, variable a will be stored in that object. And also, bar is defined. When bar is defined it stores the scope chain in effect. So, the scope chain of bar now contains the object that has variable a in it. So, when bar is returned it has a reference to it's scope chain. And hence it knows a.
So, I guess it answers your question how node handles the code.
You wrote:
loadOrInitializeTaskArray takes two arguments and the second one is the callback function which shouldn't accept any parameters according to is signature but it does!!
The callback function is
function(tasks) {
for(var i in tasks) {
console.log(tasks[i]);
}
}
And it accepts an argument tasks. So, you are wrong here.
And When loadOrIntializeTaskArray is called cb refers to this call back function. And cb(arg) basically does this tasks = arg where tasks is the argument in callback function.
I guess you still will be having a lot of questions. Let me know in comments. And I highly recommend you to go through the Core Javascript before diving into node.
First, there is not really any such thing as a function signature in javascript. You can pass as many or as few arguments to a function as you like. The second argument to loadOrInitialiseTaskArray is simply assigned to the local variable cb when the function is called. The line cb(tasks) then invokes this value, so the second argument had better have been a function.
When you call loadOrInitializeTaskArray from listTasks, the second argument is indeed a function, and the first argument to this function is named tasks in its own scope. It isn't reaching into loadOrInitializeTaskArray and using the variable declared in that function's scope. You explicitly passed in that value when you invoked cb.
The code would work the same and perhaps be easier to understand if we renamed the variables:
function loadOrInitializeTaskArray(file, cb) {
fs.exists(file, function(exists) {
var taskArray = [];
if (exists) {
fs.readFile(file, 'utf8', function(err, data) {
if (err) throw err;
var data = data.toString();
var taskArray = JSON.parse(data || '[]');
cb(taskArray);
});
} else {
cb([]);
}
});
}
function listTasks(file) {
loadOrInitializeTaskArray(file, function(listOfTasks) {
for(var i in listOfTasks) {
console.log(listOfTasks[i]);
}
});
}
I'm not sure what you mean by "the second one is the callback function which shouldn't accept any parameters according to is signature". The callback signature (function(tasks)) certainly does expect an argument (tasks).
cb is the callback function that was passed in. In this case it is literally the anonymous function:
function(tasks) {
for(var i in tasks) {
console.log(tasks[i]);
}
}
tasks is the name of two different variables (but they point to the same object because arrays are passed by reference) in different scopes. One is defined in loadOrInitializeTaskArray() as you noted, the other is a parameter for the callback passed to loadOrInitializeTaskArray().
Also, on an unrelated note, typically callbacks follow the "error-first" format. This allows for handling of errors upstream. Modifying the code to follow this convention would result in:
function loadOrInitializeTaskArray(file, cb) {
fs.exists(file, function(exists) {
var tasks = [];
if (exists) {
fs.readFile(file, 'utf8', function(err, data) {
if (err)
return cb(err);
var data = data.toString();
// JSON.parse() throws errors/exceptions on parse errors
try {
var tasks = JSON.parse(data || '[]');
} catch (ex) {
return cb(ex);
}
cb(null, tasks);
});
} else {
cb(null, []);
}
});
}
function listTasks(file) {
loadOrInitializeTaskArray(file, function(err, tasks) {
if (err) throw err;
for (var i in tasks) {
console.log(tasks[i]);
}
});
}
Related
I have had trouble understanding the rules concerning callbacks in javascript. I understand that callbacks run after function x finishes however I find there is ambiguity when defining them.
In the node.js docs : https://nodejs.org/en/knowledge/getting-started/control-flow/what-are-callbacks/
The code
function processData () {
var data = fetchData ();
data += 1;
return data;
}
is changed to
function processData (callback) {
fetchData(function (err, data) {
if (err) {
console.log("An error has occurred. Abort everything!");
return callback(err);
}
data += 1;
callback(data);
});
}
when the anonymous function is created why can we use parameters, where do these arguments come from, what rules regard these parameters?
The context of this question comes from the sockets.io library
Specifically:
var io = socket(server);
io.on('connection', function(socket){}
Why can we reference the socket, can I just add in function(random_param, socket)? What tells the function to reference when passing random_param?
I was told read the docs, which I had already done but that didn't make things any clearer.
Thanks in advance.
I understand that callbacks run after function x finishes...
Not necessarily. The JavaScript standard library (and various others) have lots of non-asynchronous callbacks. Think of the callbacks on the array functions forEach, map, sort...
...when the anonymous function is created why can we use parameters, where do these arguments come from, what rules regard these parameters?
They come from the code calling the callback. In your case, code in the socket.io library you're using calls the callback with two arguments, which your callback receives in the parameters err and data.
Here's a somewhat silly example: A function that calls a callback with a random number:
// This is analogous to the code in the socket.io library
function getARandomNumberWithRandomDelay(callback) {
setTimeout(() => {
callback(Math.random());
}, Math.random() * 1000);
}
// This is analogous to your code using the socket.io library
getARandomNumberWithRandomDelay(function(num) {
console.log("I got the number " + num);
});
I'm sure the answer to this question exists somewhere, but unsure how to phrase the question and don't find what I'm looking for when I try to research callbacks.
Ok, so I've just started dabbling with Gulp for compiling my CSS, and I think it's great!.. despite being useless with JavaScript.
My code is below, but I don't understand how these callbacks are working. How can callbacks be set as a parameter and then be called from inside the function? I don't get it.. is the function basically expecting something to be in there? What value is being set or what's expected to run? I can't seem to make sense of it.
I see this quite frequently in JavaScript, but unsure how it works. I've looked up videos and tutorials of functions and callbacks, they makes sense, but I never seem to be able to find anywhere where this concept is explained which makes me believe I'm not looking for the right thing.
I see the same sort of thing with Promises as well where 'resolve' or 'reject' parameters are set, but unsure what's going on when a condition is met, or what values are set and where.. hope this makes sense and appreciate any help in understanding this better.
const gulp = require('gulp');
const sass = require('gulp-sass');
const browserSync = require('browser-sync');
// TASKS
// Compile SASS
gulp.task('sass-compile', (callback) => {
gulp.src('scss/*.scss')
.pipe(sass().on('error', sass.logError))
.pipe(gulp.dest('css'))
.pipe(browserSync.stream());
console.log('******************** SCSS > CSS successful ********************');
callback();
});
// Live reload
gulp.task('browser-sync', (callback) => {
browserSync.init({
proxy: 'http://localhost/test1',
port: 80
});
callback();
});
// WATCHER
gulp.task('default', gulp.series('browser-sync', (callback) => {
gulp.watch('scss/*.scss', gulp.series('sass-compile'));
callback();
}));
function foo(baz) {
console.log("Foo: " + baz);
}
function other(bar) {
const arg = "Bar";
bar(arg);
}
other(foo);
I can see how this is useful when it comes to performing asynchronous tasks. But its quite counter intuitive and I can immediately see the potential for type errors.
I've since changed my coding style to use ASYNC/AWAIT instead as much as possible. But it still a pain to navigate legacy code. Hence, the reason why I ended up looking up this thread. Haha
Functions are a type of object. Objects are a type of value.
Values can be stored in variables, in object properties, and passed as arguments to functions.
So:
Here is a simple function, which is called.
function foo() {
console.log("Foo");
}
foo();
Here is a simple function, copied to a different variable, which is called.
function foo() {
console.log("Foo");
}
const bar = foo;
bar();
And now passed as an argument to another function.
function foo() {
console.log("Foo");
}
function other(bar) {
bar();
}
other(foo);
And the same, but with arguments passed to the original function.
function foo(baz) {
console.log("Foo: " + baz);
}
function other(bar) {
const arg = "Bar";
bar(arg);
}
other(foo);
You've just been looking at examples where the function responsible for calling the callback is not code you've written (and you aren't looking at the source code of the function which calls the callback).
In javascript, functions are first class objects, which means they can be passed around to other functions. There are many cases where this can be used, but the cases you're referring to (gulp.task and new Promise) are essentially using it to do two-way communication between your code and their code. When you use gulp.task, basically the following steps are happening
1) You call gulp.task, saying "i'd like to do some work please". You pass in a function saying what you'd like to do
2) at the appropriate time, gulp.task calls your function, but passes in to it another function saying "when you're done, call this to let me know".
A simplified version of gulp.task would look like this:
const task = (name, workToDo) => {
const done = () => {
console.log('task', name, 'has let me know its done');
}
console.log('ive been asked to start task', name);
setTimeout(() => {
console.log('500ms elapsed; starting task', name);
workToDo(done);
}, 500);
}
task('sample', (done) => {
console.log('doing work');
setTimeout(() => {
console.log('1 second has elapsed. calling back to let it know i am done');
done();
}, 1000)
});
The constructor for a promise has a similar purpose. You say "i'd like to create a promise, using the following code", and then it calls your code passing in two functions that it just created (usually named resolve and reject). These are used to tell the promise "i'm done successfully" or "i'm done with an error", and thus move the promise to a resolved state or a rejected state.
I am attempting to dynamically load a series of anonymous functions and execute them, passing the results of the previous to the next.
Here is an example function:
module.exports = function (data) {
// do something with data
return (data);
}
When the functions are loaded (they all sit in separate files) they are returned as an object:
{ bar: [Function], foo: [Function] }
I would like to execute these functions using async.waterfall. This takes an array of functions, not an object of functions, so I convert as follows:
var arr =[];
for( var i in self.plugins ) {
if (self.plugins.hasOwnProperty(i)){
arr.push(self.plugins[i]);
}
}
This gives:
[ [Function], [Function] ]
How can I now execute each function using each async.waterfall passing the result of the previous function to the next?
SOLUTION
Thanks to the comments from #piergiaj I am now using next() in the functions. The final step was to make sure that a predefined function was put first in the array that could pass the incoming data:
var arr =[];
arr.push(function (next) {
next(null, incomingData);
});
for( var i in self.plugins ) {
if (self.plugins.hasOwnProperty(i)){
arr.push(self.plugins[i]);
}
}
async.waterfall(arr,done);
If you want them to pass data on to the next one using async.waterfall, instead of returning at the end of each function, you need to call the next() method. Additionally, you need to have next be the last parameter to each function. Example:
module.exports function(data, next){
next(null, data);
}
The first parameter to next must be null because async.waterfall treats it as an error (if you do encounter an error in one of the methods, pass it there and async.waterfall will stop execute and finish passing the error to the final method).
You can then convert it like you already to (object to array), then call it like so:
async.waterfall(arrayOfFunctions, function (err, result) {
// err is the error pass from the methods (null if no error)
// result is the final value passed from the last method run
});
As a js/node newcomer, I'm having some problems understanding how I can get around this issue.
Basically I have a list of objects that I would like to save to a MongoDB database if they don't already exist.
Here is some code:
var getDataHandler = function (err, resp, body) {
var data = JSON.parse(body);
for (var i=0; i < data.length; i++) {
var item = data[i];
models.Entry.findOne({id: item.id}, function(err, res) {
if (err) { }
else if (result === null) {
var entry = new models.Entry(item);
feedbackEntry.save(function(err, result) {
if (err) {}
});
}
});
}
}
The problem I have is that because it is asynchronous, once the new models.Entry(item) line is executed the value of item will be equal to the last element in the data array for every single callback.
What kind of pattern can I use to avoid this issue ?
Thanks.
Two kinds of patterns are available :
1) Callbacks. That is you go on calling functions from your functions by passing them as parameters. Callbacks are generally fine but, especially server side when dealing with database or other asynchronous resources, you fast end in "callback hell" and you may grow tired of looking for tricks to reduce the indentation levels of your code. And you may sometimes wonder how you really deal with exceptions. But callbacks are the basis : you must understand how to deal with that problem using callbacks.
2) Promises. Using promises you may have something like that (example from my related blog post) :
db.on(userId) // get a connection from the pool
.then(db.getUser) // use it to issue an asynchronous query
.then(function(user){ // then, with the result of the query
ui.showUser(user); // do something
}).finally(db.off); // and return the connection to the pool
Instead of passing the next function as callback, you just chain with then (in fact it's a little more complex, you have other functions, for example to deal with collections and parallel resolution or error catching in a clean way).
Regarding your scope problem with the variable evolving before the callback is called, the standard solution is this one :
for (var i=0; i<n; i++) {
(function(i){
// any function defined here (a callback) will use the value of i fixed when iterating
})(i);
});
This works because calling a function creates a scope and the callback you create in that scope retains a pointer to that scope where it will fetch i (that's called a closure).
This question already has answers here:
How do I access previous promise results in a .then() chain?
(17 answers)
Closed 8 years ago.
I need to chain some asynch actions. I'm trying to write Parse.com cloud code which uses express. (I think express is where the promises support originates). I see why promises are valuable, but am still unsure about a couple things. The first is how to collect the results of sequenced actions. The code below illustrates:
function doAsnychThingsInSequence(params) {
return doThing0(params).then(function(resultOfThing0) {
// thing1 depends on resultOfThing0
doThing1(resultOfThing0);
}).then(function(resultOfThing1) {
// here is where I am confused.
// thing2 depends on the results of thing0 and thing1
doThing2(resultOfThing0 /* out of scope?? */, resultOfThing1);
}, function(error) {
// handle error
});
}
After thing1 is done, I need the results of both actions. I think I can allocate a variable at the top of the function and assign it to the first result in the first callback, is that the right way? I guess, at the heart of my confusion is question two...
return doThing0(params0).then(function(resultOfThing0) {
doThing1(resultOfThing0);
// what does return mean here? does what I return here relate to the
// parameters of the next function?
return "foo";
}).then(function(param) {
// what is in param? is it "foo"?
}, function(error) {
});
Promises can be imagined as a stream processing: one function gets input, does something with it and pass it to the next function in the chain.
So if you need to pass input (for the chain) parameters further you should include them into
the output data so next one in the chain can use them:
function doThing1(params1) {
...
return [params1, result1];
}
function doThing2(params1, params2) {
...
}
As you've mentioned you can use some variable outside doThing1 and doThing2 but it will make these functions statefull that may lead to side effects of various kinds. Not desired in general for async processing.
The function supplied to then() must return a value, whether that value is the value to be used, or a promise for an upcoming value. Either way (per the promises spec) then() will return a new promise. That's the first issue I see with your code.
Next is that you have to store thing1 in a higher scope so that you can access it later. So something like this might be in order:
// example
var thing1;
getThing1()
.then(function(value){
thing1 = value; // <-- store in higher scope
return getThing2(); // <-- return
})
.then(function(value){
thing2 = value;
// now you have both thing1 and thing2 in scope
})
I'd like to summarize what I've learned from the two helpful answers from #greim and #c-smile. +1 both for the kind help. But If I'm understanding correctly, then this is a disappointment for me about Promises.
#greim's answer will make my callbacks refer to local variables in the containing function. But this is disappointing because it makes the callback dependent those locals. It would be hard, for example, to pass in a callback.
#c-smile's answer will make my functions return an array (or some collection) of their parameters and their result. All callers of that function would then be expected to dig through the return value for the "natural" result as opposed to the params that it used to get the result.
Restating the answers in terms of my original post:
// #greim
function doAsnychThingsInSequence(params) {
var theResultOfThing0; // yuck, for needing to make up a non-colliding variable name
return doThing0(params).then(function(resultOfThing0) {
theResultOfThing0 = resultOfThing0;
return doThing1(resultOfThing0);
}).then(function(resultOfThing1) {
return doThing2(theResultOfThing0, resultOfThing1);
}, function(error) {
// handle error
});
}
// #c-smile
function doThing1(params) { // params are resultOfThing0
// do stuff
// the "natural" answer is just resultOfThing1, but ...
return [params, resultOfThing1];
}
// if this function was doThingN, then we'd have
return [params0, params1 ... paramsN, resultOfThingN]; // yikes!
// at least this method looks great now, how I want it to look...
function doAsnychThingsInSequence(params) {
return doThing0(params).then(function(resultOfThing0) {
return doThing1(resultOfThing0);
}).then(function(resultOfThing0, resultOfThing1) {
return doThing2(resultOfThing0, resultOfThing1);
}, function(error) {
// handle error
});
}