I am trying to append the results of a db query to a table like so:
function foo() {
var result = {};
pool.getConnection(function(err, connection) {
if(err) {
console.log("Problem establishing connection with the database");
return;
}
var tables = ["first", "second", "third"];
_.forEach(tables, function(table) {
var query = "SELECT * FROM " + table;
connection.query(query, function(err, data) {
if(!err) {
result[table] = data;
} else {
console.log("Problem performing query \"%s\"", query);
}
});
});
});
return result;
}
The returned result is empty, but inside of the forEach, it is being populated. I can prove this by adding a log message in the loop.
I can only think this is a scoping problem but when I changed result to a property (this.result) and tried to assign to that from in the loop I get the same results. I made sure I was using the correct instance of this by adding var that = this; at the start of the function and assigning to that.result inside of the loop.
It's not a scoping problem, it's a timing problem.
You're trying to return result before result is filled in, because your code inside the anonymous function doesn't run until after your foo function returns. It's asynchronous. I expect the query calls are asynchronous as well.
Since foo relies on something asynchronous to do its work, it cannot return the result. It has to do what getConnection and query do: Accept a callback function that it will call, later, with the result.
Here's one way you might do that, see the code comments:
function foo(callback) { // <== Note `callback`
pool.getConnection(function(err, connection) {
if(err) {
console.log("Problem establishing connection with the database");
callback(null); // Do the callback, with a flag value for failure
return;
}
var tables = ["first", "second", "third"];
var result = {};
var results = 0; // Remember how many results we've seen
_.forEach(tables, function(table) {
var query = "SELECT * FROM " + table;
connection.query(query, function(err, data) {
if(!err) {
result[table] = data;
} else {
console.log("Problem performing query \"%s\"", query);
result[table] = null; // Flag value for failure
}
if (++results === tables.length) {
// We're done, issue the callback
callback(results);
}
});
});
});
}
Because the query functions are asynchronous, when the code first runs through, it sends off the queries, then returns your blank object because the queries have not returned yet.
When the queries do return, the properties of that object should be filled out, but you will need to have some code to run at that time in order to "see" it. If you are expecting your code to return the populated object from that function, you will be disappointed. :-)
Related
I'm trying to query a database, then make an array of objects from the results, turn them into a JSON object.
I'm not used to Javascript's asynchronous nature and I'm really confused about how to implement something that needs to work synchronously. I know I probably need to use callbacks, but after looking at many tutorials I'm only left more confused.
This is the code without callbacks:
var foreignTable = (tablename,idArr)=>{
var dataArray = [];
//call a query for each of the ids
var objectToAdd;
for(var id of idArr){
objectToAdd = queryForeignTable(tablename,id);
dataArray.push(objectToAdd);
}
return dataArray;
connection.end();
};
var queryForeignTable = (tablename,id)=>{
connection.query("SELECT * FROM "+tablename+" WHERE id="+id, function (error, results, fields) {
if(error)throw error;
var objectToAddToArray={};
//Go through each field in a result and make the object
for(packet of fields){
var label = packet.name;
objectToAddToArray[label] = results[0][label];
}
return objectToAddToArray;
});
};
var arrayOfDrivers = foreignTable("driver",[1,2]);
outputJson["drive"]=arrayOfDrivers;
console.log(outputJson); // { drive: [ undefined, undefined ] }
I attempted foreignTable(tablename, idArr, callback) with the callback calling queryForeignTable with no luck.
Can someone explain how I can get this piece of code working with callbacks?
A callback function is a function passed into another function as an argument, which is then invoked inside the outer function to complete some kind of routine or action.
from MDN: Callback function
Callbacks are a way of telling a function what to do next, as in "after you're done, run this function".
For example:
first = function (callback) {
console.log("First!");
callback();
}
second = function () {
console.log("Second!");
}
first(second);
Will produce:
First!
Second!
You can also use anonymous function to product the same result:
first(function() {
console.log("Second!")
});
Regarding the specific example from your question, you are correct that you need to use callbacks a bit differently. Rather than using a return statement in each of your two functions, you'll need to use a callback. connection.query is asynchronously coming back with your results as results, but you can't return them to your queryForeignTable function. Instead, give queryForeignTable a callback function to run. The same idea goes for your foreignTable function.
I'm not connected to your database, obviously, so I stubbed a DB connection and simplified what you're trying to do, but it should look something like this:
// Stubbed DB connection
var connection = {};
connection.query = (id, cb) => {
var results = [
{
id: id,
name: 'Name of ' + id,
},
];
cb(null, results);
};
var foreignTable = (ids, cb) => {
var data = [];
for (var i = 0; i < ids.length; i++) {
queryForeignTable(ids[i], (error, obj) => {
data.push(obj);
if (i == ids.length - 1) {
cb(null, data);
}
});
}
};
var queryForeignTable = (id, cb) => {
connection.query(id, (error, results) => {
if (error) {
cb(error, null);
}
cb(null, results[0]);
});
};
foreignTable([1, 2], (error, data) => {
if (error) {
console.error(error);
}
console.log(data);
});
That produces:
[ { id: 1, name: 'Name of 1' }, { id: 2, name: 'Name of 2' } ]
In essence, when you have an urge to return some value(s) from a function in an asynchronous way, give the function a callback parameter, then invoke that callback with your return values.
You can run the code here: https://repl.it/K0YI/3
When you have an asynchronous call, like connection.query(statement, callback), then whatever you want to do with the results of that call needs to be done within the callback.
Bear in mind that async functions don't return the final value that you generally want (usually they return undefined). Instead of using a return value, you pass a callback as a way of saying, "when you're finished, carry on and do this with the results", aka. continuation-passing style.
One of the challenges in your code is that you're issuing separate queries for each ID, and then aggregating the results into an array of responses. This means you'll need to check when all the queries have completed, and only then proceed to display the final results.
Here's your example, re-written, commented, and simplified. Hopefully this helps to explain how the control flow works.
// Read row IDs 1 and 2 from the "driver" table.
readTable("driver", [1, 2], displayData);
// Print the results.
function displayData (arrayOfDrivers) {
console.log(arrayOfDrivers);
}
// Read all rows matching IDs in `idArray` from `tableName`,
// put results into an array, and finally invoke `callback`.
function readTable (tablename, idArray, callback) {
var resultsArray = [];
// Queue up all the async queries.
for (var id of idArray){
queryTable(table, id, handleResponse);
}
// A query finished, so handle the result.
function handleResponse (error, results, fields) {
if (error) {
throw error;
}
// Add the query result to array of results.
resultsArray.push(results[0]);
// Check if all queries are done.
if (resultsArray.length === idArray.length) {
// Invoke the callback with the resultsArray.
// The callback is in fact the `displayData` function.
callback(resultsArray);
}
}
}
// Execute a query, using the `cb` callback to handle the response.
function queryForeignTable (tablename, id, cb) {
var query = "SELECT * FROM " + tablename + " WHERE id=" + id;
connection.query(query, cb);
}
Note that the handleResponse function is defined within the scope of the readTable function, so it can access the variables in readTables scope, such as resultsArray and callback.
Hope that helps.
I'm trying to populate an array using a MYSQL query on a table that takes all rows and pushes the rows to WordList.
I can print each line within the method fine, but when I go out of the scope of that method it doesn't push anything to Wordlist.
function getParrotMessage() {
wordList = [];
console.log(wordList);
// Implementation
getWord('result', function (err, result) {
console.log(result); // works, prints the row data in MySQL table
wordList.push(result); // doesn't work
});
console.log(wordList);
return parrot_message;
}
// Method
function getWord(word, callback) {
var query = con.query('SELECT * FROM word_table');
query.on('result', function (row) {
callback(null, row.word);
});
};
wordlist: []
wordlist shows up as an empty array.
Any help would be greatly appreciated, just beginning with javascript and node.js
Your method getWord is asynchronous!
So the second console.log(wordList); is printed before any results are returned (before you even call wordList.push(result); for the first time)
Also since you query db(which is asynchronous) in getParrotMessage function you need to use callback (or Promise or whatever else there is that can be used) instead of return statement.
function getParrotMessage(callback) {
getWord('result', function (err, result) {
if(err || !result.length) return callback('error or no results');
// since result is array of objects [{word: 'someword'},{word: 'someword2'}] let's remap it
result = result.map(obj => obj.word);
// result should now look like ['someword','someword2']
// return it
callback(null, result);
});
}
function getWord(word, callback) {
con.query('SELECT * FROM word_table', function(err, rows) {
if(err) return callback(err);
callback(null, rows);
});
};
now use it like this
getParrotMessage(function(err, words){
// words => ['someword','someword2']
});
I ran into an issue whilst attempting to create the logic to add rows to a new table I made on my MySql database. When adding a row I need to query the database 4 times to check other rows and to then add the correct value to the new row. I am using node.js and the mysql module to accomplish this. While coding I ran into a snag, the code does not wait for the 4 queries to finish before inserting the new row, this then gives the values being found a value of 0 every time. After some research I realize a callback function would be in order, looking something like this:
var n = 0;
connection.query("select...", function(err, rows){
if(err) throw err;
else{
if(rows.length === 1) ++n;
}
callback();
});
function callback(){
connection.query("insert...", function(err){
if(err) throw err;
});
}
Note: The select queries can only return one item so the if condition should not effect this issue.
A callback function with only one query to wait on is clear to me, but I become a bit lost for multiple queries to wait on. The only idea that I had would be to create another variable that increments before the callback is called, and is then passed in the callback function's arguments. Then inside the callback the query could be encapsulated by an if statement with a condition of this being the variable equaling the number of queries that need to be called, 4 for my purposes here. I could see this working but wasn't sure if this sort of situation already has a built in solution or if there are other, better, solutions already developed.
You need async (https://github.com/caolan/async). You can do a very complex logic with this module.
var data = {} //You can do this in many ways but one way is defining a global object so you can add things to this object and every function can see it
firstQueryFunction(callback){
//do your stuff with mysql
data.stuff = rows[0].stuff; //you can store stuff inside your data object
callback(null);
}
secondQueryFunction(callback){
//do your stuff with mysql
callback(null);
}
thirdQueryFunction(callback){
//do your stuff with mysql
callback(null);
}
fourthQueryFunction(callback){
//do your stuff with mysql
callback(null);
}
//This functions will be executed at the same time
async.parallel([
firstQueryFunction,
secondQueryFunction,
thirdQueryFunction,
fourthQueryFunction
], function (err, result) {
//This code will be executed after all previous queries are done (the order doesn't matter).
//For example you can do another query that depends of the result of all the previous queries.
});
As per Gesper's answer I'd recommend the async library, however, I would probably recommend running in parallel (unless the result of the 1st query is used as input to the 2nd query).
var async = require('async');
function runQueries(param1, param2, callback) {
async.parallel([query1, query2, query3(param1, param2), query4],
function(err, results) {
if(err) {
callback(err);
return;
}
var combinedResult = {};
for(var i = 0; i < results.length; i++) {
combinedResult.query1 = combinedResult.query1 || result[i].query1;
combinedResult.query2 = combinedResult.query2 || result[i].query2;
combinedResult.query3 = combinedResult.query3 || result[i].query3;
combinedResult.query4 = combinedResult.query4 || result[i].query4;
}
callback(null, combinedResult);
});
}
function query1(callback) {
dataResource.Query(function(err, result) {
var interimResult = {};
interimResult.query1 = result;
callback(null, interimResult);
});
}
function query2(callback) {
dataResource.Query(function(err, result) {
var interimResult = {};
interimResult.query2 = result;
callback(null, interimResult);
});
}
function query3(param1, param2) {
return function(callback) {
dataResource.Query(param1, param2, function(err, result) {
var interimResult = {};
interimResult.query3 = result;
callback(null, interimResult);
});
}
}
function query4(callback) {
dataResource.Query(function(err, result) {
var interimResult = {};
interimResult.query4 = result;
callback(null, interimResult);
});
}
Query3 shows the use of parameters being 'passed through' to the query function.
I'm sure someone can show me a much better way of combining the result, but that is the best I have come up with so far. The reason for the use of the interim object, is that the "results" parameter passed to your callback is an array of results, and it can be difficult to determine which result is for which query.
Good luck.
I am running a MySQL query inside a .js file running on Node JS. I have the connection setup ok and the query works but when I try returning the result back to the original call it doesn't seem to work.
function sqlQuery(query){
console.log("Running query");
var conn = connection();
var result = false;
conn.query(query, function(err, rows, fields) {
conn.end();
if (!err){ result = rows; } else { result = err; }
});
return result;
}
var loginResult = sqlQuery("SELECT * FROM `players`");
console.log(loginResult);
If I use the following code it does write the result to the console inside the query but not the final "loginResult". I am not getting any errors so my question is - is there an error in the way I am getting the returned result?
if (!err){ result = rows; console.log(rows); } else { result = err; }
Virtually everything in Node.js is asynchronous, and SQL query functions definitely are. You're calling conn.query(query, callback), which means that query is called, and then once there is a result at some point in the future, your callback function gets called with the result for you to work with. So:
conn.query(query, function runThisEventually(err, rows, fields) {
if (err) {
console.error("One or more errors occurred!");
console.error(err);
return;
}
processResults(rows, fields);
});
You won't get the result immediately after calling conn.query(...), so your code gets to do "other things" in the mean time, and at some point, your callback will be triggered and you can pick up result processing there.
I'm beggining with javascript. I think this question only about javascript but it envolves PhoneGap and WebSQL. My problem and what I want to do are in the code comments.
var MyDatabase = function() {
if (!(this instanceof MyDatabase)) return new MyDatabase();
}
MyDatabase.prototype = {
db: window.openDatabase("my_database", "1.0", "My Database", 5000000),
getAllPosts: function(callback) {
var query = "SELECT * FROM posts",
that = this,
result;
function onSuccess (transaction, resultSet) {
console.log('get posts with success.');
result = resultSet.rows; // I think this should work, but it doesn't
if (typeof callback === 'function') callback.call(that, result);
}
function onError(transaction, error) {
console.log(error);
}
this.db.transaction(function(t){ t.executeSql(query, [], onSuccess, onError) });
return result; // result still undefined
}
};
// Imagine that the posts table are created and has some rows seted.
var database = MyDatabase();
// The callback works fine.
database.getAllPosts(function(result) {
// do something with result.
console.log(result);
// SQLResultSetRowList
});
// But in some cases I want to do this and I get result as undefined =(
var result = database.getAllPosts();
Any ideia?
Thanks.
You have to use a callback. You can't return result in your code, since onSuccess won't yet have been called, so nothing will have set result.