I'm writing my first tests for a Mongodb database using should.js, and need a way to force errors in various functions that make database calls.
I've spent a little time exploring how Rewire.js might help, but am still new enough to Mongo that I'm not sure what I should 'rewire' to force database errors.
Perhaps there is a better way to write the functions to trigger errors during testing.
I appreciate any ideas.
For what it's worth, this is what I've come up with and would appreciate any comments. I didn't have long to write this up, so please forgive any typos.
The typical layout of a function with an embedded async call is usually something like:
function my_function(callback) {
async_call(function(err, doc) {
if (err) {
//Do error prep such as with VError
callback(err, null);
} else {
//Do success prep such as:
var result = doc.property;
callback(null, result);
}
});
}
The problem is having a great way to test the code in the error path. There are solutions like Rewire which do well if, by changing the value of a global variable you can trigger the desired error, but its hard to find a way to break Mongodb calls by rewriting a global.
My solution was to:
1) rewrite my async functions that I need to "break" in order to test
my_function = {
get: function(callback) {
async_call(function(err, doc) {
my_function.manage_return(err, doc, callback);
});
},
manage_return: function(err, doc, callback) {
if (err) {
//Do error prep such as with VError
callback(err, null);
} else {
//Do success prep such as:
var result = doc.property;
callback(null, result);
}
});
}
2) During testing, if I need to force an error, I just rewrite get:
var original_get = my_lib.my_function.get; //NO parenthesis!!!
my_lib.my_function.get = function(callback) {
my_lib.my_function.manage_return("This is a forced error", null, callback);
};
Later if I need to restore the function:
my_lib.my_function.get = original_get;
I've actually written a constructor that makes the rewrite/restore procedure much more streamlined, that I'd be glad to post if anyone is interested.
I did try using Rewire to handle the rewrite/restore procedure without success. Worse, "rewritten" code can't be single-stepped.
PROS
*Doesn't use eval
*Single step debugging (at least in WebStorm) works
*Can force any error at will
*Doesn't alter the function signature
*Gives ability to force a specific function to fail, which is a huge help when testing error handling of larger functions that rely on several subordinate functions
CONS
*I have to rewrite all my async functions as objects with get and mange_return
*If the function is in a module, I have to export the function in order to rewrite/restore it
Thoughts?
Related
This question already has answers here:
Why is my variable unaltered after I modify it inside of a function? - Asynchronous code reference
(7 answers)
Closed 4 years ago.
I'm trying to send back results data from my API call in Node. I have an HTML form with a search bar set up that takes the search term and fires off a search with that term.
Right now the data gets sent back and is empty on the first click, but when I click again, the data from the previous call is sent back to my HTML page, so I know there's something going on in regards to callbacks. I could use a setTimeout on the res.json() but that seems a little...juvenile.
I am pretty new to Node and I know the solution is rather simple...but I can't quite seem to figure it out.
Promises, callbacks, async. Maybe I'm just a little confused on where they go...
var result;
var Questrade = require("questrade");
var qt = new Questrade("env.KEY")
qt.account = "env.ACCOUNT";
qt.on("ready", function(err) {
if (err) {
console.log(err);
}
});
function searchQt() {
qt.search(searchTerm, function(err, symbols) {
if (err) {
console.log(err);
}
result = symbols;
});
}
app.get("/search", function(req, res) {
searchTerm = req.query.symbol;
searchQt();
res.json(result);
});
I'm expecting to receive my data back on time.
Easiest and most incremental solution would be to move res.json(result) to the callback where the value of result is received.
function searchQt(res) { // <-- note added argument
qt.search(searchTerm, function(err, symbols) {
if (err) {
console.log(err);
}
result = symbols;
res.json(result); // here it is!
});
}
app.get("/search", function(req, res) {
searchTerm = req.query.symbol;
searchQt(res); <-- note added argument
// res.json(result); <-- commented out and moved above
});
You could also make things work with promises or async/await, but since you're already using callbacks, this is the most incremental way to get to working code.
While you're in there, you should remove result from the outer scope. You don't need or want that value preserved between requests. So just res.json(symbols); and get rid of the results variable entirely.
And passing res as an argument like that above is a bit of a code smell. Let's move the call to qt.search() directly into the callback for app.get() to clean things up a tiny bit. This eliminates the searchQt() function entirely:
app.get("/search", function(req, res) {
searchTerm = req.query.symbol;
qt.search(searchTerm, function(err, symbols) {
if (err) {
// handle error here somehow. maybe something like this?
console.log(`An error occurred: ${err}`);
res.status(500).end('An error occurred');
return;
}
res.json(symbols);
});
});
There are still some other improvements possible here. (I'm not sure if the ready event needs to fire before a search is tried, for example. So there might be a race condition there.) Promises or async/await might make this more readable. (People seem to like those better than deeply-nested callbacks. I personally don't mind the nesting, but I get it.) But that hopefully gets you going in the right direction. (And hopefully I didn't add any bugs that weren't there before!)
I'm playing with the NodeJS REPL console and following this tutorial.
http://www.tutorialspoint.com/nodejs/nodejs_file_system.htm
I'm focusing on the File System(FS) module. Let's look at the mkdir function used for creating directories.
According to TutorialsPoint, this is how you create a directory with FS
var fs = require("fs");
console.log("Going to create directory /tmp/test");
fs.mkdir('/tmp/test',function(err){
if (err) {
return console.error(err);
}
console.log("Directory created successfully!");
});
They specifically say you need this syntax
fs.mkdir(path[, mode], callback)
Well I just tried using less code without the callback and it worked.
var fs = require('fs');
fs.mkdir('new-directory');
And the directory was created. The syntax should just be
fs.mkdir(path);
I have to ask, what is the purpose of the callback and do you really need it? For removing a directory I could understand why you would need it, in case the directory didn't exist. But I can't see what could possibly go wrong with the mkdir command. Seems like a lot of unnecessary code.
As of node v10.0, the callback to fs.mkdir() is required. You must pass it, even if you just pass a dummy function that does nothing.
The point of the callback is to let you know if and when the call succeeded and if it didn't succeed, what the specific error was.
Remember, this type of function is asynchronous. It completes some unknown time in the future so the only way to know when it is done or if it completed successfully is by passing a callback function and when the callback is called, you can check the error and see that it has completed.
As it turns out, there are certainly things that can go wrong with mkdir() such as a bad path, a permissions error, etc... so errors can certainly happen. And, if you want to immediately use that new directory, you have to wait until the callback is called before using it.
In response to one of your other comments, the fs.mkdir() function is always asynchronous whether you pass the callback or not.
Here's an example:
var path = '/tmp/test';
fs.mkdir(path, function (err) {
if (err) {
console.log('failed to create directory', err);
} else {
fs.writeFile(path + "/mytemp", myData, function(err) {
if (err) {
console.log('error writing file', err);
} else {
console.log('writing file succeeded');
}
});
}
});
Note: Modern versions of nodejs, include fs.promises.mkdir() which returns a promise that resolves/rejects instead of using plain callbacks. This allows you to use await with try/catch or .then() and .catch() instead of the plain callback to know when it's done and promises make it typically easier to sequence in with other asynchronous operations and to centralize error handling.
Because mkdir is async.
Example:
If you do:
fs.mkdir('test');
fs.statSync('test').isDirectory();//might return false cause it might not be created yet
But if you do:
fs.mkdir('test', function() {
fs.statSync('test').isDirectory();//will be created at this point
});
You can still use mkdirSync if you need a sync version.
Many things could go wrong by using mkdir, and you should probably handle exceptions and errors and return them back to the user, when possible.
e.g. mkdir /foo/bar could go wrong, as you might need root (sudo) permissions in order to create a top-level folder.
However, the general idea behind callbacks is that the method you're using is asynchronous, and given the way Javascript works you might want to be notified and continue your program execution once the directory has been created.
Update: bare in mind that if you need — let's say — to save a file in the directory, you'll need to use that callback:
fs.mkdir('/tmp/test', function (err) {
if (err) {
return console.log('failed to write directory', err);
}
// now, write a file in the directory
});
// at this point, the directory has not been created yet
I also recommend you having a look at promises, which are now being used more often than callbacks.
Because it's an async call, it may be that further execution of the program depends on the outcome of the operation (dir created sucessfully). When the callback executes is the first point in time when this can be checked.
However, this operation is really fast, it may seem as it's happening instantly, but really (because it's async), the following line after fs.mkdir(path); will be executed without waiting for any feedback from the fs.mkdir(path); thus w/o any guarantee that the directory creation finished already, or if it failed.
As the following code shows:
var fs = require('fs');
fs.readFile('somefile.txt', function(err, data) {
if(err) {
return console.error(err);
}
console.log(data.toString('utf8'));
});
I understand that we can easily catch the err info if it happens.
But I wonder how this approach is evolved little by little.
I would appreciate any pointer/article that would explain it well.
It's a design choice to force programmers to not be lazy and ignore the error.
Javascript allows you to call functions with fewer or more arguments than specified. For example:
function foo (a,b) {
console.log(a,b);
}
foo(100); // not an error
foo(100,200,300); // also not an error
This means, if the error is defined as the second parameter:
async(function (data,err) {/*...*/});
lazy programmers can simply be lazy and ignore the error by doing:
async(function (data) {/*...*/});
Of course, this doesn't prevent people form ignoring the error anyway. But at least it forces them to write down the parameter so that future programmers who maintains the code will be reminded of the missing error handler.
Disclaimer: I know that synchro stuff is to be avoided, and that promises and callbacks are preferrable, but I'm stuck writing a system that needs a small amount of backwards compatability, and need to write this for a temporary stop gap.
Writing an express.js app, I have a function that takes the request object from a .get or .post etc function, and confirms whether the session key is valid (after checking this with a remote API server). The main version of the function is like this:
module.exports.isValidSession(req, cb) {
// REST API Calls and callbacks, error handling etc, all works fine.
}
I need a version of the above to be compatible with an older system (that will soon be phased out). I'm quite new to Node, although not to JS, so I'm wondering if there's a good convention on how to do this?
Additional Info
The main problem I hit is returning a synchronous function from some kind of 'watcher' - See below for one approach I considered, which doesn't work, but I figure maybe someone knows an approach to. (It's essentialy polling the value of a variable until the async function sets it to indicate it's done.)
function isValidSession(req) {
var running = 1, ret = -1;
var looper = setInterval(function () {
if (!running) {
clearInterval(looper);
return ret; // This is the problem bit. Returns an async interval function, not the parent.
}
}, 500);
request(requestOpts, function (error, response, body) {
if (error) {
ret = new Error('Some error went on');
running = -1;
}
if (response.statusCode === 200) {
ret = true;
running = -1;
}
});
}
Might well not be possible, or more likely, not viable, but it'd be really useful if I could include this for backwards compat for a week or so. It's a dev project atm, so if it's not good practice, it'll do as long as it doesn't comprimise everything. I do realise, though, that it basically goes against the whole point of Node (although Node itself includes several functions that are available in both sync and async versions).
All help gratefully received.
Lets pretend there is a strong reason why author desires this.
There is a rather popular (so, you are not alone) module called synchronize.js
Use case:
Add callback to your isValidSession
function isValidSession(req, cb) {
request(requestOpts, function (error, response, body) {
if (error) {
cb(new Error('Some error went on'));
}
if (response.statusCode === 200) {
cb(null, true);
}
});
}
Use it with sync module:
var ret = sync.await(isValidSession(req, sync.defer()))
P.S. Might require testing, refer to the documentation.
I am working with Node and I have a "class" that takes a directory as a parameter. It tries to create that directory and if it fails, then it throws an error:
function Config(dir) {
fs.mkdir(dir, function(err) {
if(err) throw new Error('Error', err);
}
}
My question is, is this an approved way of doing this? If I were to use a callback, then the rest of my program would have to reside in that callback, which seems odd to me.
This issue manifested itself when I tried to write a test using mocha which won't work since the exception is thrown in an async call:
it('should throw an error on a bad directory', function() {
var fn = function() {
var badConfig = new Config('/asdf');
};
assert.throws(fn, Error);
});
I've investigated domains as a way to solve the unit test issue, but that didn't seem to solve my problem (or I didn't implement them correctly).
var d = domain.create().on('error', function(err) { throw err; }
d.run(function() {
function Config(dir) {
fs.mkdir(dir, function(err) {
if(err) throw err;
}
}
});
Ultimately, I'm looking for a best practice that allows me to indicate to the application that something bad happened, and allows me to create tests for that solution.
You have three possibilities:
Using a synchronous call. As AsolBerg explained, your case suits exactly why some fs functions have their synchronous equivalent. It's ok because in your case, all your application depends on one Config instance to be loaded. but there are cases
Using a callback as constructor argument.
If constructor callback sounds really too odd for you, put your initialization code into an init() method, that takes a callback. It's a matter of personnal preference, but rather use this technic.
Last option, you can returns a Future in your init() method. There are several future libraries in NodeJS, that are an elegant alternative to callback parameter. But you can't use it in your constructor... as the constructor's return is the created object.
It sounds like in this case you might actually want to make a synchronous call (e.g. the rest of your application depends on this call being finished before proceeding). So although its normally not the way you want to think about building your node apps you could use the synchronous version mkdirSync().
http://nodejs.org/api/fs.html#fs_fs_mkdirsync_path_mode
Then if the call fails you can catch the error and return it and (probably) exit the app.