Coming from the .NET world I am having a hard time consuming documentation for javascript frameworks. I will take "Node.js MongoDB Driver API" as an example. There is a Collection object which has the count() method. Here is the link for it: http://mongodb.github.io/node-mongodb-native/2.0/api/Collection.html#count
From there I see that count() takes three parameters:
count(query, options, callback)
At first I thought I need to provide all three of them in order to use this method. However, in the example code for this method I see that it sometimes uses only one and sometimes only one parameter:
// Perform a total count command
collection.count(function(err, count) {
test.equal(null, err);
test.equal(4, count);
// Peform a partial account where b=1
collection.count({b:1}, function(err, count) {
test.equal(null, err);
test.equal(1, count);
db.close();
});
In the first case it calls count() with only callback parameter while in the second it provides options and callback. There is no example of using all three parameters as stated in the original method description.
My question is how can one know these kind of things? I mean, if there were no examples in this documentation how would I know that would be possible? Also how can I be sure that there are some other possible usages of the method if that is not covered with examples like this?
I know it is perfectly legal to call javascript functions with only some parameters provided and function should be able to handle it based on the implementation. But as a consumer of API I don't want to look into function implementation to figure out what kind of parameter combination I can pass. It feels to me like this documentation is not complete (I took this mongodb driver just as an example but I came across to similar problem with other js framework docs).
Is there some kind of reasoning that I should have when reading javascript documentation, how to think about it when trying to understand API of different frameworks, how to know what is possible and what is not etc...?
In JavaScript and specially Node.JS world there is a common pattern on asynchronous functions that their last argument should be a callback function.
count(query, options, callback)
When you call a async function, you must foresee that the result of the call will come in a callback otherwise how could get the results of your async function call? so
based on the above statement you should at least provide a callback function to the count method and other async functions that you call in JavaScript.
Another thing that could help you when you see methods of MongoDB is that it is common that if you are going to process some data with the database, you need to provide at least a query object and a callback function but if you need additional information for your query you will provide, the options object.
I know that the documentation is not good explaining if the query and options parameters are optionals but if you see the source code of the count method and read my comments added on the code you could understand:
Collection.prototype.count = function(query, options, callback) {
// args represents the three parameters
var args = Array.prototype.slice.call(arguments, 0);
// extract your last argument and assumes that it is the callback function
callback = args.pop();
// extract the first argument that is query
query = args.length ? args.shift() || {} : {};
// extract the second argument but this time shift already extracted query
// so the first parameter this time will be options.
options = args.length ? args.shift() || {} : {};
...
}
if you pass the following values to the call, you will see the order commented in how the count method is going to grab the arguments:
count({}, {}, function callback() {}); // count(query, options, callback)
count({}, function callback() {}); // count(query, callback)
count(function callback() {}); // count(callback)
In fact in your example that you provided in the nested call to count method, you are passing a query and a callback and not options and a callback to the count method:
collection.count(function(err, count) {
test.equal(null, err);
test.equal(4, count);
// You are passing the values in this order (query, callback)
// {b: 1} === query
collection.count({b:1}, function(err, count) {
test.equal(null, err);
test.equal(1, count);
db.close();
});
I hope this explanation and code helps you to understand this common pattern of callbacks around JavaScript world.
Related
I have had trouble understanding the rules concerning callbacks in javascript. I understand that callbacks run after function x finishes however I find there is ambiguity when defining them.
In the node.js docs : https://nodejs.org/en/knowledge/getting-started/control-flow/what-are-callbacks/
The code
function processData () {
var data = fetchData ();
data += 1;
return data;
}
is changed to
function processData (callback) {
fetchData(function (err, data) {
if (err) {
console.log("An error has occurred. Abort everything!");
return callback(err);
}
data += 1;
callback(data);
});
}
when the anonymous function is created why can we use parameters, where do these arguments come from, what rules regard these parameters?
The context of this question comes from the sockets.io library
Specifically:
var io = socket(server);
io.on('connection', function(socket){}
Why can we reference the socket, can I just add in function(random_param, socket)? What tells the function to reference when passing random_param?
I was told read the docs, which I had already done but that didn't make things any clearer.
Thanks in advance.
I understand that callbacks run after function x finishes...
Not necessarily. The JavaScript standard library (and various others) have lots of non-asynchronous callbacks. Think of the callbacks on the array functions forEach, map, sort...
...when the anonymous function is created why can we use parameters, where do these arguments come from, what rules regard these parameters?
They come from the code calling the callback. In your case, code in the socket.io library you're using calls the callback with two arguments, which your callback receives in the parameters err and data.
Here's a somewhat silly example: A function that calls a callback with a random number:
// This is analogous to the code in the socket.io library
function getARandomNumberWithRandomDelay(callback) {
setTimeout(() => {
callback(Math.random());
}, Math.random() * 1000);
}
// This is analogous to your code using the socket.io library
getARandomNumberWithRandomDelay(function(num) {
console.log("I got the number " + num);
});
I have doubts on selecting async.js method for probing several alternatives and stop when the first of them is successful.
For example:
async.probeSeries([
function (callback) {
// try something and call callback
// without arguments - probing fails
callback();
},
function (callback) {
// try something and call callback
// with arguments - probing successful
callback(null, ok);
},
function (callback) {
// will be not executed, because
// the second one is successful
callback();
}
], function (err, result) {
// handle the result returned by the second probe
});
I think that using series and return the result as error way may be a workaround but is there a better way?
Maybe you are looking for detectSeries? It works a littlebit different than your example as it checks values from an array with the same function and then callbacks with one of these values, but maybe you can apply it to your problem.
Btw, this looks like the perfect use case for a promise library like Q, where you'd write
probe1().fail(probe2).fail(probe3).done(resulthandler, errhandler);
Probably asked before, but after the serious searching I'm still not able to find a proper solution. Please consider something like this:
function compute() {
asyncCall(args, function(err, result) {
});
/* 'join thread here' */
}
Even though asyncCall is asynchronous I'd like to use the result and return it from the function compute synchronously. asyncCall is a library call and I can't modify it in any way.
How to wait properly for the asynchronous result without setTimeout and watching a conditional variable? This is possible but suboptimal.
not sure how you can really use something that doesn't exist yet, but it's easy enough to return a slot where the result will be:
function compute() {
var rez=[];
asyncCall(args, function(err, result) {
rez[0]=result;
if(rez.onchange){ rez.onchange(result); }
});
/* 'join thread here' */
return rez;
}
now, you can refer to the [0] property of the return, and once the callback comes in, compute()[0] will have the result. It will also fire an event handler you can attach to the returned array that will fire when the data updates inside the callback.
i would use something more formal like a promise or secondary callback, but that's me...
EDIT: how to integrate a callback upstream:
// sync (old and busted):
function render(){
var myView=compute();
mainDiv.innerHTML=myView;
}
//async using my re-modified compute():
function render(){
var that=compute();
that.onchange=function(e){ mainDiv.innerHTML=e; }
}
see how making it wait only added a single wrapper in the render function?
There's no await syntax in browsers that is widely available. Your options are generally limited to Callback patterns or Promises.
NodeJS follows a callback pattern for most async methods.
function someAsyncMethod(options, callback) {
//callback = function(error, data)
// when there is an error, it is the first parameter, otherwise use null
doSomethingAsync(function(){
callback(null, response);
});
}
....
someAsyncMethod({...}, function(err, data) {
if (err) return alert("OMG! FAilZ!");
// use data
});
Another common implementation is promises, such as jQuery's .ajax() method...
var px = $.ajax({...});
px.data(function(data, xhr, status){
//runs when data returns.
});
px.fail(function(err,xhr, status){
//runs when an error occurs
});
Promises are similar to events...
Of the two methods above, the callback syntax tends to be easier to implement and follow, but can lead to deeply nested callback trees, though you can use utility patterns, methods like async to overcome this.
How does futures handle a callback that has more than one argument? This is critical to just about every use I could have for futures. The github example shows it only handleing one argument.
The example from the Github Readme is
var fileNames = readdir('.').wait();
But what about something a mysql call like
client.query("select * from employees", function(err, results, fields) {
// callback function returns employees array
callback(results);
});
How would I get those three (err, results, and fields) using the futures wait() method?
Edit
Experimentation has taught me that the first argument of the callback, in this case err, is always treated as an error and is thrown if the value is truthy. The second argument is assigned. Any further arguments are ignored as best as I can tell.
Create a wrapper for client.query, returning one data parameter with both results.
client.prototype.query2 = function ( sql, callback ) {
this.query(sql,
function(err, results, fields) {
var data={result:result, fields:fields};
callback(err,data);
}
};
I've recently created simpler abstraction than 'futures', also based on Fibers.
I haven't tested my code with a real world DB example yet. Please try it and help me test it if you can: https://github.com/luciotato/waitfor
Please could I ask for some advice on a control flow issue with node and redis? (aka Python coder trying to get used to JavaScript)
I don't understand why client.smembers and client.get (Redis lookups) need to be callbacks rather than simply being statements - it makes life very complicated.
Basically I'd like to query a set, and then when I have the results for the set, I need to carry out a get for each result. When I've got all the data, I need to broadcast it back to the client.
Currently I do this inside two callbacks, using a global object, which seems messy. I'm not even sure if it's safe (will the code wait for one client.get to complete before starting another?).
The current code looks like this:
var all_users = [];
// Get all the users for this page.
client.smembers("page:" + current_page_id, function (err, user_ids ) {
// Now get the name of each of those users.
for (var i = 0; i < user_ids.length; i++) {
client.get('user:' + user_ids[i] + ':name', function(err, name) {
var myobj = {};
myobj[user_ids[i]] = name;
all_users.push(myobj);
// Broadcast when we have got to the end of the loop,
// so all users have been added to the list -
// is this the best way? It seems messy.
if (i === (user_ids.length - 1)) {
socket.broadcast('all_users', all_users);
}
});
}
});
But this seems very messy. Is it really the best way to do this? How can I be sure that all lookups have been performed before calling socket.broadcast?
scratches head Thanks in advance for any advice.
I don't understand why client.smembers and client.get (Redis lookups) need to be callbacks rather than simply being statements - it makes life very complicated.
That's what Node is. (I'm pretty sure that this topic was discussed more than enough times here, look through other questions, it's definitely there)
How can I be sure that all lookups have been performed before calling socket.broadcast?
That's what is err for in callback function. This is kinda Node's standard - first parameter in callback is error object (null if everything fine). So just use something like this to be sure no errors occurred:
if (err) {
... // handle errors.
return // or not, it depends.
}
... // process results
But this seems very messy.
You'll get used to it. I'm actually finding it nice, when code is well formatted and project is cleverly structured.
Other ways are:
Using libraries to control async code-flow (Async.js, Step.js, etc.)
If spaghetti-style code is what you think mess is, define some functions to process results and pass them as parameters instead of anonymous ones.
If you totally dislike writing stuff callback-style, you might want to try streamlinejs:
var all_users = [];
// Get all the users for this page.
var user_ids = client.smembers("page:" + current_page_id, _);
// Now get the name of each of those users.
for (var i = 0; i < user_ids.length; i++) {
var name = client.get('user:' + user_ids[i] + ':name', _);
var myobj = {};
myobj[user_ids[i]] = name;
all_users.push(myobj);
}
socket.broadcast('all_users', all_users);
Note that a disadvantage of this variant is that only one username will be fetched at a time. Also, you should still be aware of what this code really does.
Async is a great library and you should take a look. Why ? Clean code / process / easy to track .. etc
Also, keep in mind that all your async function will be processed after your for loop. In you exemple, it may result in wrong "i" value. Use closure :
for (var i = 0; i < user_ids.length; i++) { (function(i) {
client.get('user:' + user_ids[i] + ':name', function(err, name) {
var myobj = {};
myobj[user_ids[i]] = name;
all_users.push(myobj);
// Broadcast when we have got to the end of the loop,
// so all users have been added to the list -
// is this the best way? It seems messy.
if (i === (user_ids.length - 1)) {
socket.broadcast('all_users', all_users);
}
});
})(i)}
What you should do to know when it's finish is use a recursive pattern like async ( i think ) do. It's much simple then doing it yourself.
async.series({
getMembers: function(callback) {
client.smembers("page:" + current_page_id, callback);
}
}, function(err, results) {
var all_users = [];
async.forEachSeries(results.getMembers, function(item, cb) {
all_users.push(item);
cb();
}, function(err) {
socket.broadcast('all_users', all_users);
});
});
This code may not be valid, but you should be able to figure out how to do it.
Step library is good too ( and only 30~ line of code i think)
I don't understand why client.smembers and client.get (Redis lookups)
need to be callbacks rather than simply being statements - it makes
life very complicated.
Right, so everyone agrees callback hell is no bueno. As of this writing, callbacks are a dying feature of Node. Unfortunately, the Redis library does not have native support for returning Promises.
But there is a module you can require in like so:
const util = require("util");
This is a standard library that is included in the Node runtime and has a bunch of utility functions we can use, one of them being "promisify":
https://nodejs.org/api/util.html#util_util_promisify_original
Now of course when you asked this question seven years ago, util.promisify(original) did not exist as it was added in with the release of -v 8.0.0, so we can now update this question with an updated answer.
So promisify is a function and we can pass it a function like client.get() and it will return a new function that take the nasty callback behavior and instead wraps it up nice and neat to make it return a Promise.
So promisify takes any function that accepts a callback as the last argument and makes it instead return a Promise and it sounds like thats the exact behavior that you wanted seven years ago and we are afforded today.
const util = require("util");
client.get = util.promisify(client.get);
So we are passing a reference to the .get() function to util.promisify().
This takes your function, wraps it up so instead of implementing a callback, it instead returns a Promise. So util.promisify() returns a new function that has been promisified.
So you can take that new function and override the existing one on client.get().
Nowadays, you do not have to use a callback for Redis lookup. So now you can use the async/await syntax like so:
const cachedMembers = await client.get('user:' + user_ids[i]);
So we wait for this to be resolved and whatever it resolves with will be assigned to cachedMembers.
The code can be even further cleaned up to be more updated by using an ES6 array helper method instead of your for loop. I hope this answer is useful for current users, otherwise the OP was obsolete.