This question already has answers here:
How do I return the response from an asynchronous call?
(41 answers)
Closed 8 years ago.
I am in the process of relearning Javascript and last week when writing this code for a university assignment I think that there is probably a much better way of executing this code
app.get('/member/all', function(req, res) {
connection.query('CALL GetAllMembers()', function(err,rows){
connection.query('CALL CountMembers()', function(err, allMembers){
console.log(err);
connection.query('CALL CountAllIndMembers()', function(err,indMembers){
console.log(err);
connection.query('CALL CountInactiveMembers()', function(err,inactiveMembers){
console.log(err);
connection.query('CALL CountAllMembersInGroups()', function(err,groupMembers){
console.log(err);
res.render('members', {members : rows[0], title : "All Members", groupMembers : groupMembers[0][0].AllGrpMembers,
inactiveMembers : inactiveMembers[0][0].AllInactiveMembers, indMembers : indMembers[0][0].AllIndMembers,
allMembers : allMembers[0][0].AllMembers, statistics : true});
});
});
});
});
});
});
});
When I was trying to declare variables under the app.get such as var allMembers... when the callback was executed I was unable to set allMembers = rowsFromTheCallback. It seemed that it was a local variable to that callback. I'm sure this is something to do with the variable scope and/or hoisting. Just wanted to ask you guys if there would be a better way to do this as even though this function works. It is very ugly to look at haha!
Thanks in advance
Jack
As far as scope goes, all the inner functions should be able to read and write to the outer variable unless it is shadowed by an inner variable declaration or function parameter.
The problem you are having might be related to the async-ness of the code. See this code:
function delay(n, cb){
setTimeout(function(){ bs(delay) }, delay);
}
function main(){
var allMembers = 17;
delay(500, function(){
console.log(allMembers); // This looks at the outer "allMembers"
allMembers = 18;
delay(200, function(allMembers){ // <-- SHADOW
console.log(allMembers); // This looks at the allMembers from "delay 200"'s callback
allMembers = 42;
});
delay(300, function(){
console.log(allMembers); //This is the outside "allMembers" again
});
});
return allMembers; // Still 17!
}
main();
main will return before the setTimeouts have even fired so its going to return the original value of that variable. In order to wait for the inner callbacks to run, the only way is to make main take a callback to signa when its done, instead of just returning.
function main(onResult){
delay(500, function(){
//...
onResult(allMembers);
});
// <-- no return value
});
main(function(allM){
console.log(allM);
});
See async library: https://github.com/caolan/async
async.series([
getAllMembers,
countMembers,
...
], function(err, results) {
// err contains an error if any of the functions fails. No more functions will be run.
// results is an array containing results of each function if all the functions executed without errors
}));
function getAllMembers(callback) {
connection.query('CALL CountMembers()', callback);
}
function countMembers(callback) {
...
}
If the execution order of the functions does not matter, async.parallel can be used instead of async.series.
There is power in using a library to handle and encapsulate "Continuation Passing Style" (CPS) interactions with your asynchronous calls. The following code isn't from a library, but I'm going to walk through it and use it as an example of one way to implement CPS.
Setting up a scope appropriate queue is the first step. This example uses about the most simple method for doing so:
var nextList = [];
After that we need a method to handle our first case, the need to queue tasks to be performed in the future. In this case I was focused on performing them in order so I named it next.
function next() {
var todo,
current,
task,
args = {};
if (arguments.length > 0) { // if called with parameters process them
// if parameters aren't in an array wrap them
if (!Array.isArray(arguments['0'])) {
todo = [arguments];
} else { // we were passed an array
todo = [];
arguments['0'].forEach(function (item) {
// for each item we were passed add it to todo
todo.push(item);
});
}
nextList = todo.concat(nextList);
// append the new items to the end of our list
}
if (nextList.length > 0) { // if there are still things to do
current = Array.prototype.slice.apply(nextList.shift());
task = current[0];
args = current.slice(1);
task.apply(null, args); // execute the next item in the list
}
}
This allows us to make calls like:
.map(function (filepath) {
tasks.push(
[
handleAsset,
{
'path': filepath,
}
]
);
});
tasks.push([done]);
next(tasks);
This will call handleAsset, which is async, once for each file, in order. This will allows you to take your code and change each of the nested calls into a separate function in the form:
function memberAll() {
app.get('/member/all', function(req, res) {
if (err) {
handleError(err, 'memberAll');
} else {
next(getAllMembers, 'parameters to that call if needed');
}
});
}
where handleError is a common error handler, and the next call allows you to pass on relevant parameters to the next function that is needed. Importantly in the success side of the if statement you could either:
conditionally call one of several functions
call next with an array of calls to make, for instance if you had functions for processFolder and processFile you could expect that processing a folder might involve processing other folders and files and that the number would vary
do nothing except call next() with no parameters and end the current branch
Embellishments can include writing a clean function for emptying the nextList, adding items to nextList without calling an item from the list, etc. The alternative at this point is to either use an existing library for this or to continue writing your own.
Related
so I am having some issues with scopes when it comes to nodejs. I am wondering how you can initialize a variable in the global space, initialize it in a function (private scope) then have it equal to what you initialized it to in the private scope when you call it anywhere in the class. One thing I have noticed is that although you can use the variable in a private scope, when it is called again outside that function it just turns undefined. How can I return the same functions initialized when it was private? This is not my issued code but an example in case the picture isn't clear
let name;
class blah {
static blablah() {
name = josh;
}
console.log(name);
//this will return undefined and not josh
}
What I need in my context:
let genesis;
let jsonChain;
class Blockchain {
constructor() {
//this.chain = [Blockchain.getGenesis()];
console.log(Blockchain.getGenesis());
}
//down the file we find the issued function...
static getGenesis() {
fs.readFile(jsonRoute, 'utf-8', function(err, data) {
if (err) throw err;
jsonChain = JSON.parse(data);
genesis = jsonChain.blocks[0].GENESIS_DATA;
return genesis;
});
//returning here instead inside the callback also yields undefined
//I want to be able to access my contents from json file through
//the genesis variable which is global when I return this function
//but I cannot reach the contents of the callback
}
}
SOLUTION: Make sure to return the promise so that the async function runs later in your program when you call it.
let genesis;
let jsonChain;
class Blockchain {
constructor() {
this.chain = [Blockchain.getGenesis()];
console.log(Blockchain.getGenesis().then(function(genesisData) {
console.log(genesisData); // Genesis block data is here.
}, function(err) {
// This only runs if there was an error.
console.log(err);
}));
}
//down the file we find the solved function...
static getGenesis() {
return new Promise(function(resolve, reject) {
fs.readFile(jsonRoute, 'utf-8', function(err, data) {
if(err) return reject(err);
const jsonChain = JSON.parse(data);
resolve(jsonChain.blocks[0].GENESIS_DATA);
});
});
}
}
module.exports = Blockchain;
Now that you've added the real code, there are a number of issues illustrated there.
if (err) throw err inside a plain asynchronous callback does nothing useful. Nobody can catch that error and thus this should never be used. You need real error handling there. IMO, one should never write that line of code in that circumstance.
return genesis; does nothing useful. That just returns back from the fs.readFile() callback where the internals of fs.readFile() are ignoring any return value. That does not return from the higher level function.
jsonChain = JSON.parse(data); does successfully assign to the higher scoped variable, but any code that is trying to use the jsonChain variable will have no idea when it was assigned a value because fs.readFile() is asynchronous and calls its callback at some future, indeterminate time. So, if you're seeing that jsonChain has no value, that's because you're looking at the variable BEFORE it gets assigned. Basically, this is an anti-pattern. You can't program with asynchronous code that way in node.js. Instead, you need to USE the asynchronous value inside the callback or you need to call some function from within the callback and pass it the value. ONLY in that circumstance do you know WHEN the value is available.
If the top level problem here is that you're trying to return some value you retrieved asynchronously, then you cannot directly do that in Javascript. You can read all about that here where it explains that you have to communicate that asynchronous value back to the caller with a callback, a promise or you can also use an event.
The first problem with your code is that josh is not in quotes. josh is undefined; did you mean 'josh' (in quotes)?
One way is to make your getGenesis() function async, so that you can wait for it to finish assigning the correct value to genesis. This requires wrapping everything inside an anonymous async function:
(async () => {
const fs = require('fs');
let genesis;
let jsonChain;
class Blockchain {
constructor() {
this.chain = [Blockchain.getGenesis()];
console.log(Blockchain.getGenesis());
}
static async getGenesis() {
jsonChain = JSON.parse(await fs.promises.readFile(jsonRoute, 'utf-8'));
genesis = jsonChain.blocks[0].GENESIS_DATA;
}
}
console.log(genesis); // undefined D:
await Blockchain.getGenesis();
console.log(genesis); // not undefined :D
})();
I'm setting up a scraper using NodeJS, and I'm having a hard time figuring out the right way to pass data around when using async.parallel.
Here's the batch function, which receives the list of zip codes in an array inside of the zip_results object. I'm trying to setup the array asyncTasks as an array of functions to be run by async. The function I want called for each zip code is Scraper.batchOne, and I want to pass it a zip code and the job version. Right now, the function is called immediately. I tried wrapping the call to Scraper.batchOne in an anonymous function, but that lost the scope of the index variable i and always sent in undefined values.
How can make it so that the function is passed to the array, along with some parameters?
// zip_results: {job_version: int, zip_codes: []}
Scraper.batch = function (zip_results) {
//tasks - An array or object containing functions to run, each function
//is passed a callback(err, result) it must call on completion with an
//error err (which can be null) and an optional result value.
var asyncTasks = [], job_version = zip_results.job_version;
for (var i=0; i < zip_results['zip_codes'].length; i++) {
asyncTasks.push(Scraper.batchOne(zip_results['zip_codes'][i], job_version));
}
// Call async to run these tasks in parallel, with a max of 2 at a time
async.parallelLimit(asyncTasks, 2, function(err, data) { console.log(data); });
};
Why don't you use async.eachLimit instead? (With async.parallel you would need to use bind / apply techniques)
async.eachLimit(zip_results['zip_codes'], 2, function(zip, next) {
Scraper.batchOne(zip, zip_results.job_version));
return next();
}, function(err) {
// executed when all zips are done
});
You can do a self invoking anonymous function and pass the parameters that you want to retain after the method is called like this:
(function(asyncTasksArr, index, zipResults, jobVersion){
return function(){
asyncTasksArr.push(Scraper.batchOne(zipResults['zip_codes'][index], jobVersion));
}
}(asyncTasks, i, zip_results, job_version));
Hope this helps.
I have a JavaScript function like the following.
function changeTheDom(var1, var2, var3) {
// Use DWR to get some server information
// In the DWR callback, add a element to DOM
}
This function is called in a couple of places in the page. Sometimes, in a loop. It's important that the elements be added to the DOM in the order that the changeTheDom function is called.
I originally tried adding DWREngine.setAsync(false); to the beginning of my function and DWREngine.setAsync(true); to the end of my function. While this worked, it was causing utter craziness on the rest of the page.
So I am wondering if there is a way to lock the changeTheDom function. I found this post but I couldn't really follow the else loop or how the lockingFunction was intended to be called.
Any help understanding that post or just making a locking procedure would be appreciated.
Don't try to lock anything. The cleanest way is always to adapt to the asynchronous nature of your code. So if you have an asynchronous function, use a callback. In your particular case I would suggest that you split your function up in one part that is executed before the asych call and one part that is executed afterwards:
function changeTheDomBefore(var1, var2, var3) {
//some code
//...
asyncFunction(function(result){
//this will be executed when the asynchronous function is done
changeTheDomAfter(var1, var2, var2, result);
});
}
function changeTheDomAfter(var1, var2, var3, asynchResult) {
//more code
//...
}
asyncFunction is the asynchronous function which, in this example, takes one argument - the callback function, which then calls your second changeTheDom function.
I think I finally got what you mean and I decided to create another answer, which is hopefully more helpful.
To preserve order when dealing with multiple asynchronous function calls, you could write a simple Queue class:
function Queue(){
var queue = [];
this.add = function(func, data) {
queue.push({func:func,data:data});
if (queue.length === 1) {
go();
}
};
function go() {
if (queue.length > 0) {
var func = queue[0].func,
data = queue[0].data;
//example of an async call with callback
async(function() {
func.apply(this, arguments);
queue.shift();
go();
});
}
}
};
var queue = new Queue();
function doit(data){
queue.add(function(result){
console.log(result);
}, data);
}
for (var i = 0; i < 10; i++) {
doit({
json: JSON.stringify({
index: i
}),
delay: 1 - i / 10.0
});
}
FIDDLE
So everytime you invoke your async function, you call queue.add() which adds your function in the queue and ensures that it will only execute when everything else in the queue is finished.
I've been developing in JavaScript for quite some time but net yet a cowboy developer, as one of the many things that always haunts me is synching JavaScript's callbacks.
I will describe a generic scenario when this concern will be raised: I have a bunch of operations to perform multiple times by a for loop, and each of the operations has a callback. After the for loop, I need to perform another operation but this operation can only execute successfully if all the callbacks from the for loop are done.
Code Example:
for ... in ... {
myFunc1(callback); // callbacks are executed asynchly
}
myFunc2(); // can only execute properly if all the myFunc1 callbacks are done
Suggested Solution:
Initiate a counter at the beginning of the loop holding the length of the loop, and each callback decrements that counter. When the counter hits 0, execute myFunc2. This is essentially to let the callbacks know if it's the last callback in sequence and if it is, call myFunc2 when it's done.
Problems:
A counter is needed for every such sequence in your code, and having meaningless counters everywhere is not a good practice.
If you recall how thread conflicts in classical synchronization problem, when multiple threads are all calling var-- on the same var, undesirable outcomes would occur. Does the same happen in JavaScript?
Ultimate Question:
Is there a better solution?
The good news is that JavaScript is single threaded; this means that solutions will generally work well with "shared" variables, i.e. no mutex locks are required.
If you want to serialize asynch tasks, followed by a completion callback you could use this helper function:
function serializeTasks(arr, fn, done)
{
var current = 0;
fn(function iterate() {
if (++current < arr.length) {
fn(iterate, arr[current]);
} else {
done();
}
}, arr[current]);
}
The first argument is the array of values that needs to be passed in each pass, the second argument is a loop callback (explained below) and the last argument is the completion callback function.
This is the loop callback function:
function loopFn(nextTask, value) {
myFunc1(value, nextTask);
}
The first argument that's passed is a function that will execute the next task, it's meant to be passed to your asynch function. The second argument is the current entry of your array of values.
Let's assume the asynch task looks like this:
function myFunc1(value, callback)
{
console.log(value);
callback();
}
It prints the value and afterwards it invokes the callback; simple.
Then, to set the whole thing in motion:
serializeTasks([1,2, 3], loopFn, function() {
console.log('done');
});
Demo
To parallelize them, you need a different function:
function parallelizeTasks(arr, fn, done)
{
var total = arr.length,
doneTask = function() {
if (--total === 0) {
done();
}
};
arr.forEach(function(value) {
fn(doneTask, value);
});
}
And your loop function will be this (only parameter name changes):
function loopFn(doneTask, value) {
myFunc1(value, doneTask);
}
Demo
The second problem is not really a problem as long as every one of those is in a separate function and the variable is declared correctly (with var); local variables in functions do not interfere with each other.
The first problem is a bit more of a problem. Other people have gotten annoyed, too, and ended up making libraries to wrap that sort of pattern for you. I like async. With it, your code might look like this:
async.each(someArray, myFunc1, myFunc2);
It offers a lot of other asynchronous building blocks, too. I'd recommend taking a look at it if you're doing lots of asynchronous stuff.
You can achieve this by using a jQuery deferred object.
var deferred = $.Deferred();
var success = function () {
// resolve the deferred with your object as the data
deferred.resolve({
result:...;
});
};
With this helper function:
function afterAll(callback,what) {
what.counter = (what.counter || 0) + 1;
return function() {
callback();
if(--what.counter == 0)
what();
};
}
your loop will look like this:
function whenAllDone() { ... }
for (... in ...) {
myFunc1(afterAll(callback,whenAllDone));
}
here afterAll creates proxy function for the callback, it also decrements the counter. And calls whenAllDone function when all callbacks are complete.
single thread is not always guaranteed. do not take it wrong.
Case 1:
For example, if we have 2 functions as follows.
var count=0;
function1(){
alert("this thread will be suspended, count:"+count);
}
function2(){
//anything
count++;
dump(count+"\n");
}
then before function1 returns, function2 will also be called, if 1 thread is guaranteed, then function2 will not be called before function1 returns. You can try this. and you will find out count is going up while you are being alerted.
Case 2: with Firefox, chrome code, before 1 function returns (no alert inside), another function can also be called.
So a mutex lock is indeed needed.
There are many, many ways to achieve this, I hope these suggestions help!
First, I would transform the callback into a promise! Here is one way to do that:
function aPromise(arg) {
return new Promise((resolve, reject) => {
aCallback(arg, (err, result) => {
if(err) reject(err);
else resolve(result);
});
})
}
Next, use reduce to process the elements of an array one by one!
const arrayOfArg = ["one", "two", "three"];
const promise = arrayOfArg.reduce(
(promise, arg) => promise.then(() => aPromise(arg)), // after the previous promise, return the result of the aPromise function as the next promise
Promise.resolve(null) // initial resolved promise
);
promise.then(() => {
// carry on
});
If you want to process all elements of an array at the same time, use map an Promise.all!
const arrayOfArg = ["one", "two", "three"];
const promise = Promise.all(arrayOfArg.map(
arg => aPromise(arg)
));
promise.then(() => {
// carry on
});
If you are able to use async / await then you could just simply do this:
const arrayOfArg = ["one", "two", "three"];
for(let arg of arrayOfArg) {
await aPromise(arg); // wow
}
// carry on
You might even use my very cool synchronize-async library like this:
const arrayOfArg = ["one", "two", "three"];
const context = {}; // can be any kind of object, this is the threadish context
for(let arg of arrayOfArg) {
synchronizeCall(aPromise, arg); // synchronize the calls in the given context
}
join(context).then(() => { // join will resolve when all calls in the context are finshed
// carry on
});
And last but not least, use the fine async library if you really don't want to use promises.
const arrayOfArg = ["one", "two", "three"];
async.each(arrayOfArg, aCallback, err => {
if(err) throw err; // handle the error!
// carry on
});
I'm creating a Phonegap application that will perform differently on first run. The way that I am detecting the first run is by seeing of one of the database tables exists. As you can probably tell from the code below, I am checking for the error that is (probably) indicating that the table already exists, thus proving that this is not the application's first run.
function databaseExists(){
var exists;
database.transaction(function(tx){
tx.executeSql('CREATE TABLE GLOBAL (uid, property, value)');
}, function(err){
exists = true;
}, function(){
exists = false;
});
return exists;
}
My problem, however, is that the asynchronous execution of the Javascript code means that the function returns its value before the success (or error) function has set it's value.
This function is called in the initialising stage of the application:
if (databaseExists()){
// Do Something
}
And therefore must return the value rather than execute the function in the success callback of the transaction.
Is there a way to force the execution to wait until the database transaction is complete or return the value through the database.transaction object?
Thanks in advance,
Jon
You need to write it in callback form:
var dataBaseExists(yep, nope) {
database.transaction(function(tx) {
tx.executeSql('CREATE TABLE GLOBAL (uid, property, value)');
}, function(){
if (yep) {
yep.apply(this, arguments);
}
}, function(){
if (nope) {
nope.apply(this, arguments);
}
});
};
var itDoes = function() {
console.log("great");
};
var itDoesNot = function() {
console.log("what a pity");
};
databaseExists(itDoes, itDoesNot);
You need callbacks, but if don't need checking existment of your tables, you can do that easily with localStorage.
e.g.
if(localStorage.getItem('init') === null){
//init
localStorage.setItem('init', true);
}
You will avoid dealing with database.
and maybe this gonna be helpful "CREATE TABLE IF NOT EXISTS..."
I know there's gonna be programmers don't like my solution, but I love it!
var myfEspereti=false;
function Espereti(pStatus)
{
if (pStatus==="wait")
{
myfEspereti = true;
while(myfEspereti)
{
}
}
else if (pStatus==="go")
{
myfEspereti=false;
}
}
and then call Espereti ("wait") when you want to wait for an async call. Inside the async call, when it's finish, call Espereti ("go") and that's it!