Today is my first foray into nodejs and I am particularly stumped trying to understand the way the following piece of logic flows. The logic is as follows:
request({ uri: db.createDbQuery('identifier:abcd1234') },
function(err, response, body) {
response.should.have.status(200);
var search = JSON.parse(body);
search.response.numFound.should.equal(1);
done();
});
});
At a higher level I do understand is that an http request is being made and the function is being called at some juncture that is taking the response and doing something to it. What I am trying to understand is the proper order of the calls and how does the binding of variables take place in the above given logic. How does the compiler know how to bind the return values from the request to the anonymous function? Basically, I want to gain an understanding on how things work under the hood for this snippet.
Thanks
Your question isnt specific to node.js, this is basically a feature of javascript.
Basically you are calling request() which is defined like function request(obj, callback)
Internally, the http request is being called, and once its completed, it calls callback which is actually a function pointer.
function request(obj, callback){
//http request logic...
var err = request_logic_internal_function();
var response = ...
var body = ...
callback(err, response, body)
}
Your code can actually be restructured as :
var options = { uri: db.createDbQuery('identifier:abcd1234') };
var request_callback = function(err, response, body) {
response.should.have.status(200);
var search = JSON.parse(body);
search.response.numFound.should.equal(1);
done();
};
request(options, request_callback);
What you're basically doing is sending in a function pointer as a variable.
I don't know what library(ies) you're using, and it looks like you may have anonymized them by assigning methods into your code's global scope like request, done, and db.
What I can say is this:
That indentation is horrible and initially misled me on what it was doing, please gg=G (vim syntax) your code so it's properly indented.
request takes two arguments, a configuration object and a callback.
db.createDbQuery must be a blocking method or the anonymous object you're creating won't have the proper value.
request uses that configuration value, makes a non-blocking I/O request of some kind, and later will call the callback function you provide. That means that the code immediately after that request call will execute before the callback you provide will execute.
Some time later the request data will come back, Node.js's event loop will provide the data to the library's registered event handler (which may or may not be your callback directly -- it could do something to it and then call your event handler afterwards, you don't know or really care).
Then the function does some checks that will throw errors if they fail, and finally calls a done function in its scope (defined somewhere else) that will execute and continue the logical stream of execution.
Related
I understand the essence of callback functions in that the function is executed again after being passed as the parameter to another function. However, I'm confused as to where the variables inside the callback function come from as shown in the following node.js example:
router.get('/', function(req, res){
res.render('index', {});
});
How do the variables req and res get populated? An example explaining how I can just call res.render(...) without declaring res myself would be greatly appreciated.
They come from the same place they come from when a normal non callback function is invoked, at invocation time.
If you have this function,
function add (a, b) {
return a + b
}
You're fine with knowing that a and b come from when you invoke add,
add(1,2)
and it's the same principle with callbacks, don't let your brain get all twisted just because it's getting invoked later.
At some point the function you pass to router.get is going to be invoked, and when it does, it will receive req and res.
Let's pretend the definition for router.get looks like this
router.get = function(endpoint, cb){
//do something
var request = {}
var response = {}
cb(request, response) // invocation time
}
In the case of your example, it's just up to node to pass your function request and response whenever .get is invoked.
The whole point of the callback is that the invoked function calls it back.
In the case of router.get, it will insert the route (path, method, callback) in a lookup table; when a request comes in, Express will construct the response object, match the request's path and method against all the entries in the lookup table, take the callback from the matching entry and invoke callback(request, response) (passing the detected request and created response).
They get populated by whatever code is calling the callback. In your example, this is something inside the Express framework, though Express uses the node http library under the hood and adds additional functionality to the request and response objects provided by it.
But in code you write you can create a callback function signature that takes whatever params you want.
I understand the essence of callback functions in that the function is executed again after being passed as the parameter to another function. However, I'm confused as to where the variables inside the callback function come from as shown in the following node.js example:
router.get('/', function(req, res){
res.render('index', {});
});
How do the variables req and res get populated? An example explaining how I can just call res.render(...) without declaring res myself would be greatly appreciated.
They come from the same place they come from when a normal non callback function is invoked, at invocation time.
If you have this function,
function add (a, b) {
return a + b
}
You're fine with knowing that a and b come from when you invoke add,
add(1,2)
and it's the same principle with callbacks, don't let your brain get all twisted just because it's getting invoked later.
At some point the function you pass to router.get is going to be invoked, and when it does, it will receive req and res.
Let's pretend the definition for router.get looks like this
router.get = function(endpoint, cb){
//do something
var request = {}
var response = {}
cb(request, response) // invocation time
}
In the case of your example, it's just up to node to pass your function request and response whenever .get is invoked.
The whole point of the callback is that the invoked function calls it back.
In the case of router.get, it will insert the route (path, method, callback) in a lookup table; when a request comes in, Express will construct the response object, match the request's path and method against all the entries in the lookup table, take the callback from the matching entry and invoke callback(request, response) (passing the detected request and created response).
They get populated by whatever code is calling the callback. In your example, this is something inside the Express framework, though Express uses the node http library under the hood and adds additional functionality to the request and response objects provided by it.
But in code you write you can create a callback function signature that takes whatever params you want.
I run into an issue when trying to use the request method in javascript where I can't save a value. I'll run a block of code like:
let savedData;
request({
url: url,
json: true
}, function (err, resp, body) {
if (err) {
return;
}
savedData = body.data;
});
console.log(savedData);
I know that request doesn't block or something, so I think it's run after the console.log or something like that? I just need to know how I can save the desired data for use later in the method.
Your code is working correctly, you're just neglecting the fact that the callback provided as a second parameter to request() is executed asynchronously.
At the time when your console.log() is executed, the network request may or may not have successfully returned the value yet.
Further Explanation
Take a look at the documentation for the request() function.
It states that the function call takes the following signature,
request(options, callback);
In JavaScript, a callback performs exactly as the name perscribes; it calls back by executing the provided function, after it has done what it first needs to.
This asynchronous behavior is especially prominent in making networking requests, since you wouldn't want your program to freeze and wait for the network request to retrieve or send what you've requested.
Example
function callback() {
console.log('I finished doing my asynchronous stuff!');
console.log('Just calling you back as I promised.');
}
console.log('Running some asynchronous code!');
request({...options}, callback);
console.log('Hi! I'm being called since I'm the next line of code; and the callback will be called when its ready.');
Output
Running some asynchronous code!
Hi! I'm being called since I'm the next line of code; and the callback
will be called when its ready.
I finished doing my asynchronous stuff!
Just calling you back as I promised.
You'll either need to do the rest of the code in the callback of the request function, or use a promise. Anything outside of that callback is going to execute before saveedData ever shows up.
When run, the code shown below logs "[]". I believe this is because the request function is not complete by the time the console.log method is called.
var urlArray = []
request('http://www.example.com', function (error, response, body) {
urlArray.push("example.com")
}
console.log(urlArray)
How do i put a conditional to make sure the request function is complete. the pseudo code would be:
if request function is complete{
console.log(urlArray)
}
If this isn't the best way, what is?
First, if you run it non-async, the line right after the request line will be executed when the request finishes, but thats not recommendable. Second, running async, ussually you get a callback or a promise depending the library you are using in order to perform the request, even if you use pure JS, you have the option to set callbacks for error and success states, that will be executed only after the request has received a response.
I recommend reading a bit about ajax and promises.
http://www.w3schools.com/ajax/ajax_xmlhttprequest_send.asp
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/then
Having a look at the code you've provided, it seems that is just a concept or pseudocode, that will never run and throw all kind of errors and exceptions.
I have this code:
var resources = myFunc();
myFunc2(resources);
The problem is that JavaScript calls myFunc() asynchronous, and then myFunc2(), but I don't have the results of myFunc() yet.
Is there a way to block the first call? Or a way to make this work?
The reason why this code doesn't work represents the beauty and pitfalls of async javascript. It doesn't work because it is not supposed to.
When the first line of code is executed, you have basically told node to go do something and let you know when it is done. It then moves on to execute the next line of code - which is why you don't have the response yet when you get here. For more on this, I would study the event-loop in greater detail. It's a bit abstract, but it might help you wrap your head around control flow in node.
This is where callbacks come in. A callback is basically a function you pass to another function that will execute when that second function is complete. The usual signature for a callback is (err, response). This enables you to check for errors and handle them accordingly.
//define first
var first = function ( callback ) {
//This function would do something, then
// when it is done, you callback
// if no error, hand in null
callback(err, res);
};
//Then this is how we call it
first ( function (err, res) {
if ( err ) { return handleError(err); }
//Otherwise do your thing
second(res)
});
As you might imagine, this can get complicated really quickly. It is not uncommon to end up with many nested callbacks which make your code hard to read and debug.
Extra:
If you find yourself in this situation, I would check out the async library. Here is a great tutorial on how to use it.
myFunc(), if asynchronous, needs to accept a callback or return a promise. Typically, you would see something like:
myFunc(function myFuncCallback (resources) {
myFunc2(resources);
});
Without knowing more about your environment and modules, I can't give you specific code. However, most asynchronous functions in Node.js allow you to specify a callback that will be called once the function is complete.
Assuming that myFunc calls some async function, you could do something like this:
function myFunc(callback) {
// do stuff
callSomeAsyncFunction(callback);
}
myFunc(myFunc2);