We Keep Coding

Callback in Node.js and Database variable [duplicate]

This question already has answers here:
Returning a value from callback function in Node.js [duplicate]
(4 answers)
How do I return the response from an asynchronous call?
(41 answers)
Closed 5 years ago.
I don't understand callbacks in nodejs.
I need to get a podcast number from the database and use it in my code
I get right now from console.log
[Function: index]
Is there any solution in node.js to get variable from a database and reuse it later in the code?
var index = function( callback ) {
var podcast = 0;
User.findOne({ sessionID: this.event.session.sessionId }, function(err, user) {
if (err ||!user){
}
else {
console.log(user);
podcast = user.podcast;
}
});
callback( podcast );
};
index();
var callback = function(data) {
return data;
}
var iUseMyAnywhere = callback;

It look like it is impossible what I want to do in Node.js
It's completely possible. Only, you need to use asynchronous API's, which makes it ankward.
Is there any solution in node.js to get variable from a database and reuse it later in the code?
Not exactly. When you connect to a database -or, by the way, do anything asynchronously, like fetching something over http or reading from disc- you can't assign that thing right over:
var myUserFromDb = User.find('john doe', function(err, res){...}); //this will fail
Because that function you're passing as the second parameter will execute sometime in the future. User.find() itself doesn't return the user.
So, sadly, you can't just get the user in the user var and pass it to another module -let's say a podcast module-.
However, let's say you have a 'user.js' module, with exposes a withUser method than aks the database for a user and then calls a provided function with the user, when the db call is resolved.
And let's say you have a 'podcast.js' file/module with a getPodcast method that needs a user.
getPodcast can't just ask 'user.js' a user. However, it can ask for a function that will run with the user passed as parameter:
User.js
function withUser(callback){
User.find({_id: 1}, (err, user)=> {
callback(user);
})
}
podcast.js
function getPodcast(){
withUser( function(user){
//now we really have the user inside podcast.js, and we can work with it.
//Sadly, that will surely involve more asynchronous api's, which is painful.
})
}
Now getPodcast will have access to the user inside its parameter callback.
Is there any easier method rather than callback?
Yes, you should read about promises. When using promises, things are -a little less- painful. A promise api would work as:
User.js
function getUser(id){
//let's say we return a promise that wraps the `User.find` database request
}
podcast.js
getUser(userId).then(user => getPodcast(user)).then(podcastResult => ...)
This don't see really better. However, when you are working with promise api's, you can then start using async/await.
podcast.js
async function getPodcast(userId){
const user = await User.getUser(uesrId);
const otherAsyncThing = await ...someAsyncApiCall;
doAnythingWithUser(user); //this line won't execute until user is resolved, even if there aren't callbacks involved :-D
}
A final, unasked word of advice: when working with node.js, be sure you understand how callback api's and async things really work before writing a ton of code. Otherwise, you'll get really coupled and brittled code, where objects get passed through mountains of callbacks and code is unreadable and undebuggable :-D

PS. Edited to follow question.
Think like this, you get to a restaurant, sit down and ask a waitress for a coffee. But in the meanwhile you are not frozen, you are moving, doing things, talking, so once your coffee is ready, the waitress will bring it to you and you will stop other things that you are doing and drink your coffee.
So, it would become something like this:
User.findOne({ sessionID: this.event.session.sessionId }).exec().then(data => {
console.log(data);
}).catch(err => {
console.log(err);
});

How do I use Zapier's StoreClient to set and retrieve a single value?

Following the documentation, I am using this in my JS:
var store = StoreClient('my secret');
store.set('active', true);
var status = store.get('active');
The variable status never has a value. I'm clearly not using the library correctly.
For context, this is inside a switch statement that does something like this for many of the cases, where some of them need to set or get a value from the StoreClient.
The documentation uses this example:
var store = StoreClient('your secret here');
store
.set('hello', 'world')
.then(function() {
return store.get('hello');
})
.then(function(value) {
// value === 'world'
return store.delete('hello');
})
.then(function() {
callback();
})
.catch(callback);
Because I'm on the amateur side, I'm not super familiar with promises. In that example, it's unclear to me which parts of the are required in order to [a] set, and eventually, [b] get a value. I suggest including an example that doesn't have set/get/delete combined into one.
I tried this:
var store = StoreClient('my secret');
store
.set('active', true)
.then(function() {
return store.get('active');
})
.then(function() {
callback();
})
.catch(callback);
... but then I get an error that there is no output variable, even though I haven't touched the output variable at the bottom of the script.

David from Zapier's Platform team here.
Sorry about the confusion in the docs. I'll give you a quick answer on how to fix your code and a long one as to why. In the meantime, I'll make a note to update the docs with more sensical examples.
Short
Two big things:
A. Promises pick whatever was returned in the last function. If you don't bring them along, they're lost. Your code should read:
.then(function(storedVal) { // <- that variable is missing in your code
console.log('stored val is', storedVal);
})
B. You need to provide a value to the second argument of callback. There's a better example here.
.then(function(storedVal) {
callback(null, {active: storedVal});
})
Long
Here's some of the nitty gritty on how to make all Zapier code work great.
Callback
Your code runs inside AWS Lambda, which always needs to know when you're finished. It executes all of your code in a special function with a certain set of arguments. The pertinent one here is callback, a function that you can call when you're ready to exit (or have an error). You can read more about that setup here.
Like most node callbacks, the callback has the function signature callback (error, result). To throw an error, you pass something in the first spot:
callback({msg: 'thing went wrong'});
To pass a result, use the second (and nothing in the first)
callback(null, {myData: 4});
So, not passing anything there is why the zap result isn't seeing any data.
Promises
In general, callbacks suck and are confusing to work with, so we designed StoreClient to return promises. There's a lot of materials about promises online so I won't go into the details here. The important thing is that whatever gets returned from a promise's function is the argument in the next one. For example:
Promise.resolve(1)
.then(function(val) {
// val === 1
return Promise.resolve(val + 1)
})
.then(function(val) {
// val === 2
})
There's a more practical example in these docs:
var store = StoreClient('your secret here');
var outCount;
store
.get('some counter')
.then(function(count) {
count = (count || 0) + 1;
outCount = count;
return store.set('some counter', count);
})
.then(function() {
callback(null, {'the count': outCount});
})
.catch(callback);
Hopefully that clears things up a bit!
Also, if you want to give Python a try, you can do the same code, but much simpler (example here).
Either way, let us know if there's anything else we can do to help!

AngularJs: Have method return synchronously when it calls $http or $resource internally

Is there a way to wait on a promise so that you can get the actual result from it and return that instead of returning the promise itself? I'm thinking of something similar to how the C# await keyword works with Tasks.
Here is an example of why I'd like to have a method like canAccess() that returns true or false instead of a promise so that it can be used in an if statement. The method canAccess() would make an AJAX call using $http or $resource and then somehow wait for the promise to get resolved.
The would look something like this:
$scope.canAccess = function(page) {
var resource = $resource('/api/access/:page');
var result = resource.get({page: page});
// how to await this and not return the promise but the real value
return result.canAccess;
}
Is there anyway to do this?

In general that's a bad idea. Let me tell you why. JavaScript in a browser is basically a single threaded beast. Come to think of it, it's single threaded in Node.js too. So anything you do to not "return" at the point you start waiting for the remote request to succeed or fail will likely involve some sort of looping to delay execution of the code after the request. Something like this:
var semaphore = false;
var superImportantInfo = null;
// Make a remote request.
$http.get('some wonderful URL for a service').then(function (results) {
superImportantInfo = results;
semaphore = true;
});
while (!semaphore) {
// We're just waiting.
}
// Code we're trying to avoid running until we know the results of the URL call.
console.log('The thing I want for lunch is... " + superImportantInfo);
But if you try that in a browser and the call takes a long time, the browser will think your JavaScript code is stuck in a loop and pop up a message in the user's face giving the user the chance to stop your code. JavaScript therefore structures it like so:
// Make a remote request.
$http.get('some wonderful URL for a service').then(function (results) {
// Code we're trying to avoid running until we know the results of the URL call.
console.log('The thing I want for lunch is... " + results);
});
// Continue on with other code which does not need the super important info or
// simply end our JavaScript altogether. The code inside the callback will be
// executed later.
The idea being that the code in the callback will be triggered by an event whenever the service call returns. Because event driven is how JavaScript likes it. Timers in JavaScript are events, user actions are events, HTTP/HTTPS calls to send and receive data generate events too. And you're expected to structure your code to respond to those events when they come.
Can you not structure your code such that it thinks canAccess is false until such time as the remote service call returns and it maybe finds out that it really is true after all? I do that all the time in AngularJS code where I don't know what the ultimate set of permissions I should show to the user is because I haven't received them yet or I haven't received all of the data to display in the page at first. I have defaults which show until the real data comes back and then the page adjusts to its new form based on the new data. The two way binding of AngularJS makes that really quite easy.

Use a .get() callback function to ensure you get a resolved resource.
Helpful links:
Official docs
How to add call back for $resource methods in AngularJS

You can't - there aren't any features in angular, Q (promises) or javascript (at this point in time) that let do that.
You will when ES7 happens (with await).
You can if you use another framework or a transpiler (as suggested in the article linked - Traceur transpiler or Spawn).
You can if you roll your own implementation!

My approach was create a function with OLD javascript objects as follows:
var globalRequestSync = function (pUrl, pVerbo, pCallBack) {
httpRequest = new XMLHttpRequest();
httpRequest.onreadystatechange = function () {
if (httpRequest.readyState == 4 && httpRequest.status == 200) {
pCallBack(httpRequest.responseText);
}
}
httpRequest.open(pVerbo, pUrl, false);
httpRequest.send(null);
};

I recently had this problem and made a utility called 'syncPromises'. This basically works by sending what I called an "instruction list", which would be array of functions to be called in order. You'll need to call the first then() to kick things of, dynamically attach a new .then() when the response comes back with the next item in the instruction list so you'll need to keep track of the index.
// instructionList is array.
function syncPromises (instructionList) {
var i = 0,
defer = $q.defer();
function next(i) {
// Each function in the instructionList needs to return a promise
instructionList[i].then(function () {
var test = instructionList[i++];
if(test) {
next(i);
}
});
}
next(i);
return defer.promise;
}
This I found gave us the most flexibility.
You can automatically push operations etc to build an instruction list and you're also able to append as many .then() responses handlers in the callee function. You can also chain multiple syncPromises functions that will all happen in order.

Chaining Promises recursively

I'm working on a simple Windows 8 app in which I need to fetch a set of data from a web site. I am using WinJS.xhr() to retrieve this data, which returns a Promise. I then pass a callback into this Promise's .then() method, which supplies my callback with the returned value from the asynchronous call. The .then() method returns another Promise, giving it the value that my callback returns. The basic structure of such a query would be as follows:
WinJS.xhr({ url: "http://www.example.com/" }).then(
function callback( result_from_xhr )
{
//do stuff
return some_value;
}).then(
function secondcallback( some_value )
{
//do stuff
});
In my situation, however, I may need to make additional queries for data depending on the data returned by the first query, and possibly more queries depending on THAT data... and so on, recursively.
I need a way to code this such that the final .then() is not executed until ALL the recursions have completed, similar to this:
function recurse() {
return WinJS.xhr({ url: "http://www.example.com/" }).then(
function callback( result_from_xhr )
{
if( result_from_xhr == something )
{
recurse();
}
});
}
recurse().then(
function final()
{
//finishing code
});
The problem is that, of course, the finishing code is called as soon as the first level of recursion completes. I need some way to nest the new promise and the old promise from within the callback.
I hope my question is clear enough, I'm really not sure how to explain it and frankly the idea of asynchronous recursive code makes my head hurt.

What I would do here is to create a whole new, standalone promise that you can complete manually, and return that from the recurse() function. Then, when you hit the point that you know you're done doing async work, complete that promise.
Promise.join works when you've got a known set of promises - you need the entire array of promises available before you call join. If I followed the original question, you have a variable number of promises, with more possibly popping up as part of async work. Join isn't the right tool in these circumstances.
So, what does this look like? Something like this:
function doSomethingAsync() {
return new WinJS.Promise(function (resolve, reject) {
function recurse() {
WinJS.xhr({ url: "http://www.example.com/" })
.then(function onResult(result_from_xhr) {
if (result_from_xhr === something) {
recurse();
} else {
// Done with processing, trigger the final promise
resolve(whateverValue);
},
function onError(err) {
// Fail everything if one of the requests fails, may not be
// the right thing depending on your requirements
reject(err);
});
}
// Kick off the async work
recurse();
});
}
doSomethingAsync().then(
function final()
{
//finishing code
});
I rearranged where the recursion is happening so that we aren't recreating a new promise every time, thus the nested recurse() function instead of having it at the outer level.

I solved this problem in perhaps a different way (I think it's the same problem) while making my own Windows 8 app.
The reason I came across this problem is because, by default, a query to a table will paginate, and only return 50 results at a time, so I created a pattern to get the first 50, and then the next 50, etc, until a response comes back with less than 50 results, and then resolve the promise.
This code isn't going to be real code, just for illustration:
function getAllRows() {
return new WinJS.Promise(function(resolve, reject){
var rows = [];
var recursivelyGetRows = function(skipRows) {
table.skip(skipRows).read()
.then(function(results){
rows = rows.concat(results);
if (results.length < 50) {
resolve(rows);
} else {
recursivelyGetRows(skipRows + 50);
}
})
}
recursivelyGetRows(0);
});
}
so getAllRows() returns a promise that resolves only after we get a result back with less than 50 results (which indicates it's the last page).
Depending on your use case, you'll probably want to throw an error handler in there too.
In case it's unclear, 'table' is a Mobile Services table.

You will need to use Promise.join().done() pattern. Pass an array of Promises to join(), which in your case would be a collection of xhr calls. The join() will only call done() when all of the xhr Promises have completed. You will get an array of results passed to the done(), which you can then iterate over and start again with a new Promise.join().done() call. The thing to be away of, when using this approach, is that if one of the Promises passed to join() fail, the entire operation is treated as an error condition.
Sorry I don't have time right now to try and stub out the code for you. If I get a chance, I will try to later. But you should be able to insert this into your recursive function and get things to work.

Well, I solved my problem; my recursive function was misinterpreting the data and thus never stopped recursing. Thank you for your help, and I'll be sure to watch those screencasts, as I still don't quite fully grasp the Promise chaining structure.

Control flow issue with node/redis and callbacks?

Please could I ask for some advice on a control flow issue with node and redis? (aka Python coder trying to get used to JavaScript)
I don't understand why client.smembers and client.get (Redis lookups) need to be callbacks rather than simply being statements - it makes life very complicated.
Basically I'd like to query a set, and then when I have the results for the set, I need to carry out a get for each result. When I've got all the data, I need to broadcast it back to the client.
Currently I do this inside two callbacks, using a global object, which seems messy. I'm not even sure if it's safe (will the code wait for one client.get to complete before starting another?).
The current code looks like this:
var all_users = [];
// Get all the users for this page.
client.smembers("page:" + current_page_id, function (err, user_ids ) {
// Now get the name of each of those users.
for (var i = 0; i < user_ids.length; i++) {
client.get('user:' + user_ids[i] + ':name', function(err, name) {
var myobj = {};
myobj[user_ids[i]] = name;
all_users.push(myobj);
// Broadcast when we have got to the end of the loop,
// so all users have been added to the list -
// is this the best way? It seems messy.
if (i === (user_ids.length - 1)) {
socket.broadcast('all_users', all_users);
}
});
}
});
But this seems very messy. Is it really the best way to do this? How can I be sure that all lookups have been performed before calling socket.broadcast?
scratches head Thanks in advance for any advice.

I don't understand why client.smembers and client.get (Redis lookups) need to be callbacks rather than simply being statements - it makes life very complicated.
That's what Node is. (I'm pretty sure that this topic was discussed more than enough times here, look through other questions, it's definitely there)
How can I be sure that all lookups have been performed before calling socket.broadcast?
That's what is err for in callback function. This is kinda Node's standard - first parameter in callback is error object (null if everything fine). So just use something like this to be sure no errors occurred:
if (err) {
... // handle errors.
return // or not, it depends.
}
... // process results
But this seems very messy.
You'll get used to it. I'm actually finding it nice, when code is well formatted and project is cleverly structured.
Other ways are:
Using libraries to control async code-flow (Async.js, Step.js, etc.)
If spaghetti-style code is what you think mess is, define some functions to process results and pass them as parameters instead of anonymous ones.

If you totally dislike writing stuff callback-style, you might want to try streamlinejs:
var all_users = [];
// Get all the users for this page.
var user_ids = client.smembers("page:" + current_page_id, _);
// Now get the name of each of those users.
for (var i = 0; i < user_ids.length; i++) {
var name = client.get('user:' + user_ids[i] + ':name', _);
var myobj = {};
myobj[user_ids[i]] = name;
all_users.push(myobj);
}
socket.broadcast('all_users', all_users);
Note that a disadvantage of this variant is that only one username will be fetched at a time. Also, you should still be aware of what this code really does.

Async is a great library and you should take a look. Why ? Clean code / process / easy to track .. etc
Also, keep in mind that all your async function will be processed after your for loop. In you exemple, it may result in wrong "i" value. Use closure :
for (var i = 0; i < user_ids.length; i++) { (function(i) {
client.get('user:' + user_ids[i] + ':name', function(err, name) {
var myobj = {};
myobj[user_ids[i]] = name;
all_users.push(myobj);
// Broadcast when we have got to the end of the loop,
// so all users have been added to the list -
// is this the best way? It seems messy.
if (i === (user_ids.length - 1)) {
socket.broadcast('all_users', all_users);
}
});
})(i)}
What you should do to know when it's finish is use a recursive pattern like async ( i think ) do. It's much simple then doing it yourself.
async.series({
getMembers: function(callback) {
client.smembers("page:" + current_page_id, callback);
}
}, function(err, results) {
var all_users = [];
async.forEachSeries(results.getMembers, function(item, cb) {
all_users.push(item);
cb();
}, function(err) {
socket.broadcast('all_users', all_users);
});
});
This code may not be valid, but you should be able to figure out how to do it.
Step library is good too ( and only 30~ line of code i think)

I don't understand why client.smembers and client.get (Redis lookups)
need to be callbacks rather than simply being statements - it makes
life very complicated.
Right, so everyone agrees callback hell is no bueno. As of this writing, callbacks are a dying feature of Node. Unfortunately, the Redis library does not have native support for returning Promises.
But there is a module you can require in like so:
const util = require("util");
This is a standard library that is included in the Node runtime and has a bunch of utility functions we can use, one of them being "promisify":
https://nodejs.org/api/util.html#util_util_promisify_original
Now of course when you asked this question seven years ago, util.promisify(original) did not exist as it was added in with the release of -v 8.0.0, so we can now update this question with an updated answer.
So promisify is a function and we can pass it a function like client.get() and it will return a new function that take the nasty callback behavior and instead wraps it up nice and neat to make it return a Promise.
So promisify takes any function that accepts a callback as the last argument and makes it instead return a Promise and it sounds like thats the exact behavior that you wanted seven years ago and we are afforded today.
const util = require("util");
client.get = util.promisify(client.get);
So we are passing a reference to the .get() function to util.promisify().
This takes your function, wraps it up so instead of implementing a callback, it instead returns a Promise. So util.promisify() returns a new function that has been promisified.
So you can take that new function and override the existing one on client.get().
Nowadays, you do not have to use a callback for Redis lookup. So now you can use the async/await syntax like so:
const cachedMembers = await client.get('user:' + user_ids[i]);
So we wait for this to be resolved and whatever it resolves with will be assigned to cachedMembers.
The code can be even further cleaned up to be more updated by using an ES6 array helper method instead of your for loop. I hope this answer is useful for current users, otherwise the OP was obsolete.