Whats the proper way to wrap a mongo query/insert in a function to make it reusable? I have an update operation that may take place in various places and want to only write it once and reuse it. I'm using MongoJS to interface w/ the MongoDb API.
When I take something like the following:
mongo.myt.update(
{'_id': req._id},
{
$addToSet: {
"aggregate.clientIds": req.myt.clientIds
},
$inc: {"aggregate.seenCount": 1},
$set: {
"headers": req.myt.headers,
"ip": req.myt.ip
},
$setOnInsert: {
'_id': req.myt._id,
'derived': req.myt.derived
}
},
{upsert: true},
function (err, savedId) {
if (err || !savedId) console.log("failed to save :" + req.myt + " because of " + err);
else console.log("successfully saved :" + req.myt);
});
And wrap it with a simple function like:
function mongoInsert(req) {
//same query as above
}
Then call it using:
mongoInsert(req);
I don't see any impact to speed when profiling. Should I be adding a callback to the wrapper function, is that needed? I was expecting it would have some performance impact and need to be done differently.
So a few questions.
Does the approach above calling mongoInsert() get called synchronously and block until the async mongo update is done?
If it does become sync, I would expect a performance impact which I didnt see. So is the approach I took ok to use?
And if not what would be the correct way to do this?
mongoInsert() is still asynchronous because it's calling an asynchronous function (mongo.myt.update()). Even if you don't add a callback function, it won't magically become synchronous.
The way you wrote it now is "fire-and-forget": somewhere in your code you call mongoInsert(), your code will continue to run while the update is taking place, and since you don't pass a callback the calling code cannot get informed about the result of the update (right now, you're just logging the result to console).
It's a design decision whether or not this is acceptable.
You could make the callback optional for situations where you do want to get informed about the update result:
function mongoInsert(req, callback) {
callback = callback || function() {}; // dummy callback when none is provided
mongo.myt.update(..., function(err, savedId) {
...log status here...
callback(err, savedId);
});
}
Related
I am trying to implement a function to check if a doc exists in my FireStore database, given an id. The problem is my fire_PatronExists function always return undefined.
const patronsRef = db.collection("patrons");
alert(fire_PatronExists(user.uid));
function fire_PatronExists(id) {
patronsRef.doc(id).get().then(function(doc) {
// return doc.exists();
if (doc){return true}else {return false}
}).catch(function(error) {
console.log("Error getting document:", error);
});
}
The fact that your function returns undefined is totally normal: get() is an asynchronous method, so the return you put inside the then will not be executed inside fire_PatronExists; it will be executed a bit later. There is a great SO article that explains the difference between synchronous and asynchronous execution.
There are different solutions, depending on the version of JavaScript you are using. One that will work for sure is passing a callback function to fire_PatronExists and pass the result to that function.
This would look like that (not tested):
const patronsRef = db.collection("patrons");
fire_PatronExists(user.uid, function(exists) {
alert(exists);
});
// Remember though, if you put code here, it will be executed in parallel
// with the code you put inside your callback.
function fire_PatronExists(id, callback) {
patronsRef.doc(id).get().then(function(doc) {
}).catch(function(error) {
console.log("Error getting document:", error);
});
}
Using callbacks can get really messy though. If you are using a recent version of JavaScript, you might want to read about the async and await keywords, they can improve your code readability a lot.
I'm writing my first tests for a Mongodb database using should.js, and need a way to force errors in various functions that make database calls.
I've spent a little time exploring how Rewire.js might help, but am still new enough to Mongo that I'm not sure what I should 'rewire' to force database errors.
Perhaps there is a better way to write the functions to trigger errors during testing.
I appreciate any ideas.
For what it's worth, this is what I've come up with and would appreciate any comments. I didn't have long to write this up, so please forgive any typos.
The typical layout of a function with an embedded async call is usually something like:
function my_function(callback) {
async_call(function(err, doc) {
if (err) {
//Do error prep such as with VError
callback(err, null);
} else {
//Do success prep such as:
var result = doc.property;
callback(null, result);
}
});
}
The problem is having a great way to test the code in the error path. There are solutions like Rewire which do well if, by changing the value of a global variable you can trigger the desired error, but its hard to find a way to break Mongodb calls by rewriting a global.
My solution was to:
1) rewrite my async functions that I need to "break" in order to test
my_function = {
get: function(callback) {
async_call(function(err, doc) {
my_function.manage_return(err, doc, callback);
});
},
manage_return: function(err, doc, callback) {
if (err) {
//Do error prep such as with VError
callback(err, null);
} else {
//Do success prep such as:
var result = doc.property;
callback(null, result);
}
});
}
2) During testing, if I need to force an error, I just rewrite get:
var original_get = my_lib.my_function.get; //NO parenthesis!!!
my_lib.my_function.get = function(callback) {
my_lib.my_function.manage_return("This is a forced error", null, callback);
};
Later if I need to restore the function:
my_lib.my_function.get = original_get;
I've actually written a constructor that makes the rewrite/restore procedure much more streamlined, that I'd be glad to post if anyone is interested.
I did try using Rewire to handle the rewrite/restore procedure without success. Worse, "rewritten" code can't be single-stepped.
PROS
*Doesn't use eval
*Single step debugging (at least in WebStorm) works
*Can force any error at will
*Doesn't alter the function signature
*Gives ability to force a specific function to fail, which is a huge help when testing error handling of larger functions that rely on several subordinate functions
CONS
*I have to rewrite all my async functions as objects with get and mange_return
*If the function is in a module, I have to export the function in order to rewrite/restore it
Thoughts?
Check my updated answer below the other answers. The question turned out to be quite simple, I just didn't understand how async worked or should be handled at the time of posting it
I have the following code:
function qcallb(err,result){
console.log("result.insertID:"+result.insertId);
return result.insertId;
}
var createRecord= function (tableName,record){
try{
queryStatment="INSERT INTO " + tableName + " SET ?";
var result='';
var query = connection.query(queryStatment, record, qcallb,false);
}
catch(err){
return 0;
}
return iid;
}
I want to return the variable result.insertId from the callback function, to the function that calls create record, but I have not found a way to do this.
I tried to set a global variable in the callback function, but when I try to access it, its value is still unchanged.
Is there a way for me to either access the value directly in the callback function, or to have callback function notify me when it's called then return the value?
EDIT:
After testing every proposed solution here, and those that I found by searching elsewhere, the only thing that worked for me (without using external modules to make a piece of code run in 'sync' rather than 'async'), was to implement the logic I wanted inside the callback itself, which is what I wanted to avoid in the first place, but I couldn't work around it no matter what I did.
2 years later:
After getting much more experience with node.js and async code, I see how naive this question was, although it was beyond me at the time, which is also obvious in my using a try/catch block around an async call which is useless with async code, where handling errors is done by checking for err in the callback and acting upon it if it exists.
Since it seems to have quite a number of views, this seems to be a recurring issue for those newly introduced to async development.
This can't be done in the sense that was asked in the question, for an operation to occur after an async operation and use a value that is a result of the async operation, it has to be called from the callback of the async operation with that value.
To resolve the issue, the code can be modified to:
function doSomethingWithResult(result) {
// query is done, and this would only be called if no error occurred
console.log('Result is: ', result)
// do whatever you want
}
function queryCallback(err,result) {
if (err) {
throw err
}
doSomethingWithResult(result)
}
function createRecord(tableName, record) {
queryStatment='INSERT INTO `' + tableName + '` SET ?'
connection.query(queryStatment, record, queryCallback)
}
---
Old Answer: After testing every proposed solution here, and those that I found by searching elsewhere, the only thing that worked for me (without using external modules to make a piece of code run in 'sync' rather than 'async'), was to implement the logic I wanted inside the callback itself, which is what I wanted to avoid in the first place, but I couldn't work around it no matter what I did.
Yes, 'notify me' is the way to go:
var createRecord= function (tableName,record, cb){
var query = "INSERT INTO " + connection.escape(tableName) + " SET ?";
connection.query(query, record, function(err, res) {
if (err) return cb(err);
cb(null, res.insertId); // this is "notify me" part
});
}
// now use it
createRecord('blah', { foo: "bar" }, function(err, id) {
console.log(id);
});
Also, try/catch won't help much here as callback function is not executed down stack frame but called outside from event loop. You need to use domains/zones or some similar sort of async error handling
i think there is a way with waiting for the response with an endless loop, maybe you can also use the context to assign the variable like so :
var sleep = require('sleep');
var createRecord= function (tableName,record){
try{
queryStatment="INSERT INTO " + tableName + " SET ?";
var insertId=0;
var query = connection.query(queryStatment, record, function(err,result){
console.log("result.insertID:"+result.insertId);
insertId = result.insertId;
},false);
}
catch(err){
return 0;
}
while(insertId==0){
sleep.sleep(0.1); # sleep for 0.1 second
}
return insertId;
}
i haven't tested this code on my machine, please test it and let me know if there is any error.
I'm using node.js and the async package.
Here's the code I have:
async.waterfall(
[
function(callback) {
var data = getSomeData();
callback(null, data);
},
function(data, callback) {
someFunctionThatNeedsData(data);
callback(null, 'done');
}
],
function(err, result) {
}
);
getSomeData has an asynchronous HTTP request that grabs some data from a web service. I'd like to wait until I get a response, and then return that data and pass it to someFunctionThatNeedsData.
What I expected was that getSomeData -- including the callback inside of it -- would have to complete before moving on to invoke someFunctionThatNeedsData.
The problem is that, despite using the waterfall function here, data is undefined by the time it gets to someFunctionThatNeedsData.
Additionally, from console.log I can see that the end of getSomeData is reached before the callback inside of getSomeData even begins.
Am I using waterfall incorrectly, or is it just not the right tool here? If it's just not right, what can I use to achieve the desired effect?
Or do I have to resign to having deeply nested callbacks (which, with future work, I will) and have to just mitigate it by extracting inline code into named functions?
getSomeData() has an asynchronous http request that grabs some data from a web service.
This is the issue. The execution flow already continued to the callback and executed it. This is how asynchronous functions work!
You have to pass the callback to getSomeData, which calls it once the HTTP request finished. So yes: You may need to nest the callbacks.
If you have async operation. You don't necessary to use async.waterfall. You could just do that in a promise chain style.
getSomeData().then(function(data)
{
var changeData = changeYourData(data);
return changeData;
}).then(function(changedData)
{
// some more stuff with it. You can keep on forwarding to the next `then`
}).catch(function(err)
{
// if any error throw at any point will get catch here
}).finally(function()
{
// this one will guarantee get call no matter what,
// exactly the same like async.waterfall end of chain callback
});
This example will work with Q, When, and any promise lib that follow standard.
If you need to use async.waterfall (because you could drive it with an Array.map)
You just need to callback in your then
async.waterfall(
[
function(callback) {
// A
getSomeData().then(function(data)
{
callback(null, data);
});
// B - just throw the whole thing in
callback(null , getSomeData());
},
function(data, callback) {
// A
someFunctionThatNeedsData(data);
// B
data.then(function(resolvedData)
{
someFunctionThatNeedsData(resolvedData);
callback(null, 'done');
});
}
],
function(err, result) {
});
Hope this help.
Probably asked before, but after the serious searching I'm still not able to find a proper solution. Please consider something like this:
function compute() {
asyncCall(args, function(err, result) {
});
/* 'join thread here' */
}
Even though asyncCall is asynchronous I'd like to use the result and return it from the function compute synchronously. asyncCall is a library call and I can't modify it in any way.
How to wait properly for the asynchronous result without setTimeout and watching a conditional variable? This is possible but suboptimal.
not sure how you can really use something that doesn't exist yet, but it's easy enough to return a slot where the result will be:
function compute() {
var rez=[];
asyncCall(args, function(err, result) {
rez[0]=result;
if(rez.onchange){ rez.onchange(result); }
});
/* 'join thread here' */
return rez;
}
now, you can refer to the [0] property of the return, and once the callback comes in, compute()[0] will have the result. It will also fire an event handler you can attach to the returned array that will fire when the data updates inside the callback.
i would use something more formal like a promise or secondary callback, but that's me...
EDIT: how to integrate a callback upstream:
// sync (old and busted):
function render(){
var myView=compute();
mainDiv.innerHTML=myView;
}
//async using my re-modified compute():
function render(){
var that=compute();
that.onchange=function(e){ mainDiv.innerHTML=e; }
}
see how making it wait only added a single wrapper in the render function?
There's no await syntax in browsers that is widely available. Your options are generally limited to Callback patterns or Promises.
NodeJS follows a callback pattern for most async methods.
function someAsyncMethod(options, callback) {
//callback = function(error, data)
// when there is an error, it is the first parameter, otherwise use null
doSomethingAsync(function(){
callback(null, response);
});
}
....
someAsyncMethod({...}, function(err, data) {
if (err) return alert("OMG! FAilZ!");
// use data
});
Another common implementation is promises, such as jQuery's .ajax() method...
var px = $.ajax({...});
px.data(function(data, xhr, status){
//runs when data returns.
});
px.fail(function(err,xhr, status){
//runs when an error occurs
});
Promises are similar to events...
Of the two methods above, the callback syntax tends to be easier to implement and follow, but can lead to deeply nested callback trees, though you can use utility patterns, methods like async to overcome this.