We Keep Coding

Trying to retrieve information from a db [node js, mysql, promise]

This is supposed to be a simple connection to access a localhost database.
Important information: I've tried very similar code, if not the exact same, and it works. The difference is I didn't put the connection in a class variable (this.connection) but because it's not async I didn't think much of it. So maybe it's simply a just javascript syntax related issue. Any tips would be greatly appreciated.
class Request {
constructor(connection) {
this.connection = mysql.createConnection(connection);
} // The parameter that gets passed is a dictionary { host: 'xxx', user: "xxx" ...
sendMessage(message) {
let arr = message.content.substring(1,message.length).toLowerCase().split(' '); // unimportant
arr = arr.filter(function (el) {return el != '';}); // unimportant
const promise = new Promise((resolve, reject) => {
this.connection.connect(function(err) {
console.log(this.connection); // This returns a description of the connection just fine
if (err) reject(err); // No error fires here
console.log(this.connection); // WHERE THINGS GO WRONG: Nothing gets printed on the console
this.connection.query('SELECT * FROM categories;', function (err, rows, fields) {
if (err) reject(err); // No error fires here
resolve(rows);
});
});
});
promise.then((result) => console.log(result)); // Nothing fires here either

There were two problems in your code:
When you reject with an error, you don't return, but continue the function
You were accessing this in anonymous non-lambda functions, which would overwrite this
This version has fixes for the above two issues applied:
sendMessage(message) {
let arr = message.content.substring(1, message.length).toLowerCase().split(' '); // unimportant
arr = arr.filter(function (el) { return el != ''; }); // unimportant
const promise = new Promise((resolve, reject) => {
this.connection.connect((err) => { // Using arrow function to keep original `this`
if (err) return reject(err); // If there's an error, return!
this.connection.query('SELECT * FROM categories;', function (err, rows, fields) {
if (err) return reject(err); // Again, if there's an error, return!
resolve(rows);
});
});
});
promise.then((result) => console.log(result)); // Nothing fires here either
}

How to test a "throwing an error" in Mocha having it in asyncronous code (read from db) using only callbacks (no promises, async/await)?

I want to write some tests for a method which reads from a JSON file (simulating a db) and returns the correct name, given that exists.
This is the code I have written for my method. It does throw an error when the id is not valid.
const getOne = (id, callback) => {
...
fs.readFile('db.json', (err, data) => {
if (err) {
throw new Error('Error reading file');
}
const person = JSON.parse(data)
.filter(el => el.id === id)
.map(el => el.name);
if (person.length === 0) {
throw new Error('It does not match DB entry');
}
callback(person);
});
...
The test I have written is:
it('Should reject an invalid id', (done) => {
api.getOne(100, (person) => {
try {
personFromDB = person;
} catch (error) {
assert.throws(() => {
}, new Error('It does not match DB entry'));
//done();
}
But it doesn't seem to pass the test. When I have the 'done()' uncommented, it passes the test, but I don't think it is because I pass the actual test, but rather because the test gets in the catch and executes the done() callback.
Any help, guidance or recommendation is much appreciated.

You won't be able to catch an Error being thrown in the fs.readFile callback.
Instead, pass any errors to the callback you pass to getOne.
Then you can check if an Error got passed to your callback in your test.
Here is a working example to get you started:
const fs = require('fs');
const assert = require('assert');
const api = {
getOne: (id, callback) => {
// ...
fs.readFile('db.json', (err, data) => {
if (err) return callback(err); // <= pass err to your callback
const person = JSON.parse(data)
.filter(el => el.id === id)
.map(el => el.name);
if (person.length === 0) return callback(new Error('It does not match DB entry')); // <= pass the Error to your callback
callback(null, person); // <= call the callback with person if everything worked
})
}
}
it('Should reject an invalid id', done => {
api.getOne(100, (err, person) => {
assert.strictEqual(err.message, 'It does not match DB entry'); // Success!
done();
});
});

Error handling with Node.js, Async and Formidable

In the following snippet I would like to validate the fields in the first async method.
If they are not valid I would like to return an error to the user immediately.
How do I do that?
var form = new formidable.IncomingForm();
async1.series([
function (callback) {
form.parse(req);
form.on('field', function (name, val) {
// Get the fields
});
form.on('fileBegin', function (name, file) {
if (file.name !== "") {
file.path = __dirname + '/upload/' + file.name;
}
});
callback();
},
function (callback) {
form.on('file', function (name, file) {
try {
// Do something with the file using the fields retrieved from first async method
}
catch (err) {
logger.info(err);
}
});
callback();
}
], function (err) {
//the upload failed, there is nothing we can do, send a 500
if (err === "uploadFailed") {
return res.send(500);
}
if (err) {
throw err;
}
return res.status(200);
});

I would extract the form checking into a function:
var form = new formidable.IncomingForm();
function check(name, cb, err) {
return new Promise((res,rej) => {
form.on('field', function(n, val) {
if(n !== name) return;
if(cb(val)){
res(val);
}else{
rej(err);
}
});
});
}
form.parse(req);
So now we can implement the checks and use Promise.all to summarize them:
Promise.all(
check("username", val => val.length > 4, "username isnt valid"),
check("password", val => true, "we need a password")
).then(_ => res.json({status:200}))
.catch(err => res.json({err}));
If not all all parameters have been passed, this will wait endlessly. So lets terminate if it was ended:
const ended = new Promise((_,rej) => form.on("end", () => rej("params required"));
Promise.race(
ended,
Promise.all(
check("username", val => val.length > 4, "username isnt valid"),
check("password", val => true, "we need a password")
)
).then(_ => res.json({status:200}))
.catch(err => res.json({err}));
So with that we can create a good flow of data. e.g.:
const login = Promise.all(
//usable as one liners
check("username", val => val.length >= 8, "username invalid"),
//or more extensible
check("password", val => {
if( val.length < 8 ) return false;
//other checks
console.log(password);
return true;
}, "password invalid")
//the field values are resolved by the promises so we can summarize them below
).then(([username,password]) =>
//a random (maybe async) call to evaluate the credentials
checkAgainstDB(username,password)
//we can directly fail here, err is "password invalid" or "username invalid"
).catch(err => res.json({error:"login failed",details:err}));
//another parameter can be extra handled
const data = check("something", val => val.length);
//we need to summarize all the possible paths (login /data in this case) to one that generates the result
Promise.race(
//here we join them together
Promise.all(login, data)
.then((l, d) => res.json(whatever),
//and we use the ended promise ( from above ) to end the whole thing
ended
//and at last the errors that can occur if the response ended or that have not been canceled early
).catch(e => res.json(e));

var form = new formidable.IncomingForm();
async1.series([
function (callback) {
form.parse(req);
form.on('field', function (name, val) {
if (!name || !val) {
// the moment callback is called with an error, async will stop execution of any of the steps
// in the series and execute the function provided as the last argument
// idimoatic node, when calling the callback with instance of Error
return callback(new Error('InvalidParams'));
}
/**
* This is from async documentation: https://caolan.github.io/async/docs.html#series
* Run the functions in the tasks collection in series, each one running once the previous function
* has completed. If any functions in the series pass an error to its callback, no more functions are
* run, and callback is immediately called with the value of the error. Otherwise, callback receives
* an array of results when tasks have completed.
*/
});
form.on('fileBegin', function (name, file) {
if (file.name !== "") {
file.path = __dirname + '/upload/' + file.name;
}
});
form.on('end', function () {
// call callback with null to specify there's no error
// if there are some results, call it like callback(null, results);
return callback(null);
});
// if you call the callback immediately after registering event handlers for on('field') etc,
// there will be no time for those events to be triggered, by that time, this function will be
// done executing.
//callback();
},
function (callback) {
form.on('file', function (name, file) {
try {
// Do something with the file using the fields retrieved from first async method
}
catch (err) {
logger.info(err);
return callback(err);
}
});
// This should also not be called immediately
//callback();
}
], function (err) {
//the upload failed, there is nothing we can do, send a 500
if (err === "uploadFailed") {
return res.send(500);
}
if (err.message === 'InvalidParams') {
// This will be immediately returned to the user.
return res.sendStatus(400);
}
if (err) {
// I'm not sure if this was just for the example, but if not, you should not be throwing an error
// at run time.
throw err;
}
return res.status(200);
});
I added some comments in the code where I needed to show where and how to create an error and how it's bubbled up to the user immediately.
Reference: Async Documentation
P.S. Code Snippet is not runnable, but it has a better representation of the code.
-- edit --
After knowing more from the comment, adding another snippet. You are unreasonably mixing callback and event handling. You can just pass a callback to form.parse and the callback is called when all fiels are collected. You can do validation, return error immediately or just handle the form fields right away.
form.parse(req, function(err, fields, files) {
if (err) return res.sendStatus(400);
if (fields.areNotValid()) return res.sendStatus(400);
// parse fields
});
Or, you can register event handlers for it. All events as they flow in will be handled concurrently, like async.series.
var form = new formidable.IncomingForm();
form.parse(req);
form.on('field', (name, val) => {
if (!name || val) {
console.log('InvalidParams')
return res.sendStatus(400);
}
});
form.on('fileBegin', (name, file) => {
if (file.name !== "") {
file.path = __dirname + '/upload/' + file.name;
}
});
form.on('file', (name, file) => {
});
form.on('error', (err) => {
console.log('ParsingError');
return res.sendStatus(400);
})
form.on('end', () => {
if (res.headersSent) {
console.log('Response sent already')
} else {
// handle what you want to handle at the end of the form when all task in series are finished
return res.sendStatus(200);
}
});

I'm assuming that this is a good place to validate since this is when the fields are coming in:
form.on('field', function (name, val) {
//if values are null
if (!name || !val) {
//pass the callback an error
return callback("Values are null")
}
// Get the fields
});
Please let me know if this helps.

how to trigger a async process one after another

How I should modify the following code, so I can make sure Process3 is triggered after Process2.update or Process2.create completed?
The main purpose for following code is I want to makeProcess1 finished. Then check if id exist, if yes, Process2.update is triggered. if not, Process2.create is triggered.Once Process2 finished, check if cmd existed. if yes,triggered Process3.
run: function (req, res) {
if (req.session) {
const values = req.params.all();
const id = values.id;
const cmd = values.cmd;
const param = _.omit(values, ['cmd', 'id']);
const cb1 = (e, d) => {
if (e) {
console.log(e);
res.status(400).send({ e });
} else {
Process1(values);
res.status(200).send({ d });
}
};
const cd2 = (id, param, cb1) => {
if (id) {
Process2.update({ id }, param, cb1);
} else {
Process2.create(param, cb1);
}
};
if (cmd) {
cd2(id, param, cb1, Process3(values, cmd));
}
else {
cd2(id, param, cb1);
}
} else {
res.status(403).send({ e: 'Forbidden access.' });
}
}
try approach by following, but not sure how I can pass argument id, params to Process2 and process3
let async = require('async');
const Process1 = (value, cb) => {
console.log("Process1()");
console.log(value);
cb(null, value + 1);
};
const Process2 = (value, cb) => {
console.log("value(): wait 5 sec");
console.log(value);
cb(null, value+10);
};
const Process3 = (value, cb) => {
console.log(value);
console.log("Process3(): wait 5 sec");
cb(null, value+100);
};
let Pro_1_2 = async.compose(Process2, Process1);
let Pro_2_3 = async.compose(Process3, Process2);
Pro_1_2(1, (e, r) => {
Pro_2_3(r, (error, result) => {
console.log(result);
});
});

The code you posted in your original question seems pretty twisted up, so I'm not going to attempt to rewrite it, but in general if you want to perform asynchronous calls which depend on each other, async.auto is a good way to go. Rather than declaring variables at the top that you attempt to mutate via some function calls, it's better to make Process1, Process2 and Process3 asynchronous functions that call their callbacks with a new values object. Something like:
async.auto({
doProcess1: function(cb) {
// Assuming Process1 calls `cb(undefined, newValues)` when done.
Process1(values, cb);
return;
},
doProcess2: ['doProcess1', function(results, cb) {
if (results.doProcess1.id) {
Process2.update({id: results.doProcess1.id}, cb);
return;
} else {
Process2.create(_.omit(results.doProcess1, ['cmd', 'id']), cb);
return;
}
}],
doProcess3: ['doProcess2', function(results, cb) {
if (results.doProcess2.cmd) {
Process3(results.doProcess2, cb);
return;
}
else {
cb(undefined, results.process2);
return;
}
}]
}, function afterProcess3(err, results) {
// Handler err or process final results.
});
Note all the return calls. They're not strictly necessary, but good practice to avoid accidentally running more code after calling your asynchronous functions.

Have you considered using "compose", from async.js?
const a = (data, cb) => {
var result = 'a';
cb(null, result);
};
const b = (data, id, cb) => {
var result = 'b';
cb(null, result);
};
const c = (data, cb) => {
// stuff to do with result
};
var aThenC = async.compose(c, a);
var bThenC = async.compose(c, b);
if (useA) {
aThenC(data, (result) => {
// result from c
res.status(200).send(result);
});
} else {
bThenC(data, id, (result) => {
// result from c
res.status(200).send(result);
});
}
In this scenario, a and b are your Process2 create and update, respectively, and c is the callback to Process3, if I understood correctly.
EDIT: You'll only have to enter the initial parameters (e.g. register ID) on the composed function. What composes really do is this: a(b(c(param))). That param is basically everything you need to start the process. The parameters for the following functions will be set inside the function before that.
I'll add code to support it as soon as I'm on a keyboard.

Promise chains and anonymous promise returns

Here I have a chain of promises that works fine. All the *.destroy's are promises that return promises:
function callDBDestroy() {
var db;
DB_Categories.destroy().then(function () {
return DB_Equipment.destroy();
}).catch(function (err) {
showMsg("Error in callDBDestroy: " + err);
}).then(function () {
return DB_Certificates.destroy();
}).catch(function (err) {
showMsg("Error in callDBDestroy: " + err);
}).then(function () {
return DB_Locations.destroy();
}).catch(function (err) {
showMsg("Error in callDBDestroy: " + err);
});
}
But I want to add an if statement into each one to check to see if the PouchDB database exists (which it doesn't if the DB_* is null).
If it exists, I want to destroy it then return (and these all return promises).
If it doesn't exist, I want to return an anonymous promise which returns nothing as none of the promises have any data I am concerned with.
In the example, I added in some sample code to do the if statement and I was wondering what I would put in the null instance that would pass a promise (resolve) value.
function callDBDestroy() {
var db;
DB_Categories.destroy().then(function () {
if( DB_Equipment != null) {
return DB_Equipment.destroy();
}
else {
Anonymous empty promise - something like:
new Promise().resolve();
}
}).then(function () {
return DB_Certificates.destroy();
}).then(function () {
return DB_Locations.destroy();
}).catch(function (err) {
showMsg("Error in callDBDestroy: " + err);
});
}
Thanks,
Tom

It looks like you are just wondering how to manually resolve/reject a Promise. If that is the case you can just call Promise.resolve(optionalValue) or Promise.reject(optionalValue) if you want to go to the catch handler:
function callDBDestroy() {
var db;
DB_Categories.destroy()
.then(function () {
if( DB_Equipment != null) {
return DB_Equipment.destroy();
} else {
return Promise.resolve();
}
}).then(function () {
return DB_Certificates.destroy();
}).then(function () {
return DB_Locations.destroy();
}).catch(function (err) {
showMsg("Error in callDBDestroy: " + err);
});
}

You could wrap it:
function withDb(db, handler) {
return function onFulfilled(value) {
if(db === null) throw new Error("DB Null");
return handler(value);
});
}
Which would let you do:
function callDBDestroy() {
var db;
var w = withDb(db); // or whatever instance
DB_Categories.destroy().then(w(function () {
// do stuff
}))); // .then(w( to chain calls here.
...
}

I want to return an anonymous promise which returns nothing as none of the promises have any data I am concerned with. Something like:
new Promise().resolve();
You are looking for Promise.resolve(undefined). Though you can omit the undefined, that's implicit.
….then(function () {
if (DB_Equipment != null) {
return DB_Equipment.destroy();
} else {
return Promise.resolve(undefined);
}
}).…
And you don't even have to return a promise from a then callback, simply returning undefined (or not returning) will have the same effect.
….then(function () {
if (DB_Equipment != null) {
return DB_Equipment.destroy();
}
}).…
In your case, I'd recommend a wrapper function:
function destroyDatabase(db, name = "db") {
if (db != null)
return db.destroy().catch(err => {
showMsg(`Error in destroying ${name}: ${err}`);
});
else
return Promise.resolve();
}
function callDBDestroy() {
return destroyDatabase(DB_Categories, "categories")
.then(() => destroyDatabase(DB_Certificates, "certificates"))
.then(() => destroyDatabase(DB_Locations, "locations"))
}
// or even in parallel:
function callDBDestroy() {
return Promise.all([
destroyDatabase(DB_Categories, "categories"),
destroyDatabase(DB_Certificates, "certificates"),
destroyDatabase(DB_Locations, "locations")
]);
}

How about using an Array, since you do the very same task, and only the DB changes:
//serial
function callDBDestroy() {
var databases = [
DB_Categories,
DB_Equipment,
DB_Certificates,
DB_Locations
];
function errorMessage(err){ showMsg("Error in callDBDestroy: " + err) };
databases.reduce(
(prev, db) => db == null?
prev:
prev.then(() => db.destroy().catch(errorMessage)),
Promise.resolve()
)
}
//parallel
function callDBDestroy() {
var databases = [
DB_Categories,
DB_Equipment,
DB_Certificates,
DB_Locations
];
function errorMessage(err){ showMsg("Error in callDBDestroy: " + err) };
databases.forEach( db => db && db.destroy().catch(errorMessage) );
}
I've added a serial and a paralell version.

It seems that you can DRY this out and replace a lot of redundant code by using an array of databases and then just loop through the array:
function callDbDestroy();
var dbs = [DB_Categories, DB_Equipment, DB_Certificates, DB_Locations];
// chain all the destroys together
return dbs.reduce((p, db) => {
return p.then(() => {
if (db) {
return db.destroy().catch(err => {
showMsg("Error in callDBDestroy: " + err);
});
}
});
}, Promise.resolve());
}
You do not have to return a promise from a .then() handler. If you just have no return value, then it's just like doing return undefined which just means that no value will be passed to the next .then() handler, but the promise chain will continue just fine. Conceptually, it works the same as return Promise.resolve(), but there's no need to make an extra promise there.
Since you aren't passing a value from one .then() to the next in the chain, you have nothing to pass there so you can just not return anything if there's no db value to call destroy on.
FYI, using .reduce() to loop through an array is with the return p.then(...) structure is a common design pattern for sequencing async operations on an array.
FYI, using the Bluebird promise library (which has some useful helpers), this could be done like this:
function callDbDestroy();
var dbs = [DB_Categories, DB_Equipment, DB_Certificates, DB_Locations];
return Promise.mapSeries(dbs, db => {
if (db) {
return db.destroy().catch(err => {
showMsg("Error in callDBDestroy: " + err);
});
}
});
}
For more info on why the Bluebird (or other promise libraries) are still useful even with ES6, see Are there still reasons to use promise libraries like Q or BlueBird now that we have ES6 promises?
Since it appears that these databases might all be independent, I'm wondering why you are forcing them to be executed in sequence. If they don't have to be forced into sequence, then you could do this:
function callDbDestroy();
var dbs = [DB_Categories, DB_Equipment, DB_Certificates, DB_Locations];
return Promise.all(dbs.map(db => {
if (db) {
return db.destroy().catch(err => {
showMsg("Error in callDBDestroy: " + err);
});
}
}));
}
Since this runs the operations in parallel, it has the opportunity for faster end-to-end execution time vs. strict serialization.