We Keep Coding

Javascript promise with MySQL

I am implementing an API to manage a MySQL database from Nodejs, and I want to keep my code clean as the work I intend to do can take me to have a lot of nested connection.query functions all over the place, so I've decided to separate as many as the queries I could into separate functions.
To start I wanted to do a function to check if there is an existing element already on the table, like this:
function validator(item) {
return new Promise(function(err,done) {
connection.query('select * from items where item_id = ?', [item.id] , function(err,rows){
console.log(rows);
console.log("above row object");
if (err) {
return done(false);
}
if (rows.length) {
console.log('That item is already in the DB.');
return done(false);
} else {
console.log('That item is new.');
return done(true);
}
});
});
}
And then I want to use it like this:
var insert = function (item, done) {
validator(item).then(function(err,done) {
console.log('here I go: '+done);
if(done) {
console.log('good');
} else{
console.log('bad')
}
);
}
But it does not work like I expected, the first function accurately detects if an item exists or not, but the second one never never reaches the "here I go" print.
I have checked Use promise to process MySQL return value in node.js, Javascript Promise node.js? and Javascript & promises with Q - closure problems in promises but the conclussion I reach is that this is a classic PEBCAK problem, such as that I'm a very incompetent javascript programmer and can't wrap my head around the callbacks and promises all that well so I'm managing them the wrong way.

The first callback you pass to the Promise constructor isn't the failing one.
Change
return new Promise(function(err,done) {
into
return new Promise(function(done, reject) {
and call it like this:
validator(item).then(function(done) {
(by the way it's very confusing to call done both the resolve callback and the result).
Handle the incoming error: Change
if (err) {
return done(false);
}
into
if (err) {
reject(err);
}
and don't forget to catch those errors.
Also, it's probably outside the scope of this QA but you usually take and release connections. You should probably have a look at promises ready mysql drivers.

Why does fs.readFile need a wrapper when it includes a callback as an argument?

I'm pretty new to Node.js and I had a confusing interaction with fs.readFile().
Initially, I thought that I could read a file using
fs.readFile("file.txt",function(err,data) {
if(err) {throw err;}
console.log(data);
});
However, this printed null. I consulted the documentation and it gives the exact same example and claims that it works fine!
Then I consulted stack overflow and found in this post and this post that a solution is to wrap fs.readFile() in your own function that takes a callback:
function read(file,callback) {
fs.readFile("file.txt",function(err,data) {
if(err) {throw err;}
callback(data);
});
}
read(file, function(data) {
console.log(data);
});
Alternatively, it's possible to just assign data to a new variable:
var content;
fs.readFile("file.txt",function(err,data) {
if(err) {throw err;}
content = data;
console.log(content);
});
My understanding is that when an asynchronous function completes and returns some value (here the contents of the file) then the callback runs on the returned data.
If fs.readFile(file,callback) expects to be passed a callback function, then why does it seemingly run the callback before fs.readFile() has completed?
Why does assigning the data to another variable change the way it behaves?
Thanks.

fs.readFile("file.txt",function(err,data) {
if(err) {throw err;}
console.log(data);
});
Would actually work.
What wouldn't work is:
var content;
fs.readFile("file.txt",function(err,data) {
if(err) {throw err;}
content = data;
});
console.log(content);
(which is the example that was in the post you had referenced)
And the reason is that fs.readLine is asynchronous to your code, meaning that the execution will carry on immediately without waiting for fs.readLine's response.
So in the case of the latter example,
console.log(content);
would execute before fs.readLine returns with an answer (i.e. the callback is triggered).
In general, you cannot run asynchronous methods and expect them to have the answer right away, the whole idea of asynchronous is that it doesn't block, the execution/program carries on before the asynchronous method returns with an answer.
The whole purpose of the callback in asynchronous method is to provide a way for the method to notify you when it's done and feed you with the result.
wrapping fs.readLine doesn't solve the problem, it just provides a more clean interface for reading files (instead of calling "fs.readLine" you would just call "read")

Lines getting run in strange order

I'll try to keep this as simple as possible. The following function seems to run in a very strange manner. The output I'm getting from my test-prints is in the order of 1, 4, 3, 2 (with an empty returnList).
This seems to suggest that the code within the interior of these code-blocks are getting executed last, which as you might guess creates some problem when I want to return the returnList.
var server = new mongo.Server('localhost', 27017);
var db = new mongo.Db('tdp013', server);
app.get('/getall', function (req, res) {
var returnList = [];
console.log("1");
db.open(function(err, client){
client.collection("messages", function(err, col){
col.find({}, function(err, cursor){
cursor.each(function(err, item){
if(item!=null){
console.log("2");
returnList.push(item);
}
});
console.log("3");
});
});
});
console.log("4");
console.log(returnList);
res.sendStatus(200);
});
My question is simply this, is there a good explanation(in lack of a better word) as to why/how these lines are not getting executed in the written order?
Or alternatively, is there a way to return the returnList without doing it in the iteration?

This is simply the asynchronous nature that node.js further fortifies through the use of JavaScript.
You're trying to read this code and reason through its execution sequentially (i.e. synchronously), but that is not how it works.
console.log('1');
db.open(function (err, db) {
// ...
});
console.log('4');
Take the above snippet, for instance. You expect the function in db.open() to complete before it moves on to write 4 to the console, but that is not the case at all. What happens is that node will asynchronously run that open() method and continue on to the next code (here, that'd be console.log('4');). When that open() method has completed running, it'll kick off the callback function defined in the parameter list.
This is async JavaScript. You can't expect it to run the code in your editor synchronously/sequentially written like this.
Developing logic that will flow the way (it appears) you want it to flow, it would have to be refactored like the following:
var server = new mongo.Server('localhost', 27017);
var db = new mongo.Db('tdp013', server);
app.get('/getall', function (req, res) {
var returnList = [];
console.log("1");
db.open(function(err, client){
client.collection("messages", function(err, col){
col.find({}, function(err, cursor){
cursor.each(function(err, item){
if(item!=null){
console.log("3");
returnList.push(item);
}
else {
console.log("4");
console.log(returnList);
res.sendStatus(200);
}
});
console.log("2");
});
});
});
});

Node.js is asynchronous in nature and runs on single threaded event loop.
Example:
Let’s say you have triggered a database query and you are waiting for query to execute and then proceed further, but in a same time what will happen to the piece of code which are not dependent on that database query, they won’t execute and wait for their turn.
Asynchronous is exactly opposite of this. While you are doing database query Node.js will continue executing code and when database query will be done, it will come back to process it.
So in your case you have added console.log("2"); and console.log("3"); in callback function, so those lines are executing when those method executed. And console.log("4"); will not wait for db.open method to be finished. Hence the results are 1, 4, 3, 2.
So to make sure returnList returning correct data you have to add that in the callback method itself.
Hope this will help you to understand the problem you are facing.

NodeJS MySQL How to get the result outside of the query function

I can't seem to figure out how the get the result outside of a NodeJS MySQL Pool query. Here is some sample code to better explain what I mean.
var result = 'Hello world!';
var mysql = require('mysql');
var pool = mysql.createPool({
connectionLimit : 100,
host : process.env.DB_HOST,
user : process.env.DB_USERNAME,
password : process.env.DB_PASSWORD,
database : process.env.DB_DATABASE
});
pool.query('SELECT * from user LIMIT 10', function (err, rows) {
result = rows;
});
res.send(result);
The above will return 'Hello world!' instead of the query.
If I console.log(result) inside the pool.query function it returns the rows from the query but I can't seem to get the data outside of the function. I've logged the function and checked all the associated functions and I think I'm just missing something basic.

You're sending the results back before the query finishes (and the callback is called). Sending the results in your callback will fix the problem:
pool.query('SELECT * from user LIMIT 10', function (err, rows) {
result = rows;
res.send(result);
});
As Aaron pointed out, this is a common problem. A more thorough answer can be found here.

The pool query function is asynchronous. It means that your code won't execute in the same sequence you declared the statements.
You start a query and you specify a callback function which should run when the query async operation completes.
The res.send will run immediately after the query and callback declaration and the callback will run much later. By the time you set the result you already send it.
Try to move your res.send inside the callback.
pool.query('SELECT * from user LIMIT 10', function (err, rows) {
res.send(rows);
});

Sorry if my English is bad,
I have facing the same problem with you and I found a solution where you can extract the result of your query function by mixing the async-await function with the promise
Look at my solution code here :
async () => {
const sqlQuery = `select * from contact;`
const promise = await new Promise((resolve, reject) => {
pool.query(sqlQuery, (err, result) => {
resolve(result)
})
})
console.log(promise)
})
With this code, your console definitely show the result of sqlQuery. How's this can be happen? The promise that you make will always has a pending status if the resolve/reject aren't being executed. With the mixing of asynchronus function and await in the promise that you make, your promise code has to be in resolve/reject status, if it's in pending status, they will not execute the next code, and always waiting until the resolve/reject has been executed. As you know, the result of your query statement shows in the callback function in pool.query. So why we don't execute the resolve/reject in the callback function of pool.query.
You can pass the result of query into the variable of your promise by adding it to the parameter of resolve function.

Meteor.call() callbacks not returning values to template

I am having difficulty working with Meteor.call callbacks. I have defined a function which retrieves values from the server side; however, it does not return them to the template, in order to loop over them with an {{#each}} loop.
Here is the function:
search: function() {
Meteor.call('mySearchFunction', arg1, function(err, res) {
if (err) console.log(err);
else if(res) {
console.log(res);
return res;
}
});
}
The console.log(res) shows me the data that I need, which is properly fetched with mySearchFunction, yet I am unable to pass it to the template handler, despite it being an array which may be iterated over. So I tried the following:
search: function() {
var s = Meteor.call('mySearchFunction', arg1, function(err, res) {
if (err) console.log(err);
else if(res) {
console.log(res);
return res;
}
});
console.log(s);
return s;
}
And console.log(res) continues to display the requisite data, yet console.log(s) appears as undefined. I think this is because the async nature of meteor returns s before res gets a chance to be evaluated server-side. Either way, its strange that I cannot return data from a helper that I have stored in the helper function.

On the client, Meteor.call is asynchronous - it returns undefined and its return value can only be accesses via a callback. Helpers, on the other hand, execute synchronously. See the answers to this question for how you can call methods from helpers. Here's a quick solution:
$ meteor add simple:reactive-method
Template.showPost.helpers({
search: function () {
arg = Session.get('currentSearch');
return ReactiveMethod.call('mySearchFunction', arg);
}
});
I'm not sure what arg1 is in your original post so I'm using a session variable in my solution, but that should get you on the right track.
The problem with this package is that it can encourage bad behavior, but it should be fine for method calls which don't change state (such as a search).
Also see the section about helpers in this post.