We Keep Coding

ExpressJS Middleware - Execution order of Promises not correct

I want to create an Express Middleware to perform a basic check if the user/password pair in the authorization header exists in a JSON file (educational purpose). I added it on a very simple unit converter app.
The problem is that I receive a 403 instead of the resource when the username/password are correct.
I found that when I perform a request, the Promise.then in the middleware is executed before the Promise is fulfilled in my function findUserByCredentials. See an illustration of the problem in the third code snippet below.
index.js
const express = require('express')
const app = express()
const port = process.env.PORT || 3000
const findUserByCredentials = require("./lib/find-user");
app.use(function (req, res, next) {
if (req.headers) {
let header = req.headers.authorization || '';
let [type, payload] = header.split(' ');
if (type === 'Basic') {
let credentials = Buffer.from(payload, 'base64').toString('ascii');
let [username, password] = credentials.split(':');
findUserByCredentials({username, password}).then(() => {
console.log("next")
next();
}).catch(() => {
console.log("403")
res.sendStatus(403);
});
}
} else {
next();
}
});
app.get('/inchtocm', (req, res) => {
const cm = parseFloat(req.query.inches) * 2.54;
res.send({"unit": "cm", "value": cm});
});
app.listen(port, () => {
console.log(`Example app listening at http://localhost:${port}`)
})
module.exports = app;
./lib/find-user.js
const bcrypt = require('bcrypt');
const jsonfile = require('../users.json');
let findUserByCredentials = () => (object) => {
const username = object.username;
const password = object.password;
return new Promise((resolve, reject) => {
jsonfile.forEach(user => {
if (user.username === username) {
bcrypt.compare(password, user.password).then((buffer) => {
if (buffer) {
console.log("resolve")
resolve();
} else {
console.log("reject")
reject();
}
});
}
});
reject();
});
};
module.exports = findUserByCredentials();
Server console after sending a request
Example app listening at http://localhost:3000
403
resolve
How can I force Express to wait for the first Promise to finish before performing the second operation ?

To have a better control over the order of your promises and also have a less nested code, you should use the async/await syntax. You can read more about it here.
What it essentially does is lets you...well, await for an async operation to finish before proceeding. If you use await before something that returns a Promise (like your findUserByCredentials), it will assign to the variable what you resolve your promise with, for example:
const myPromise = () => {
return new Promise(resolve => resolve(3));
}
const myFunc = async () => {
const number = await myPromise();
console.log(number); // Output: 3
}
I would rephrase your findUserByCredentials function like so:
const bcrypt = require('bcrypt');
const jsonfile = require('../users.json');
const findUserByCredentials = async (object) => {
const username = object.username;
const password = object.password;
// This assumes there's only a single unique user with a specific username
const potentialUser = jsonfile.find(user => user.username === username);
if (!potentialUser) {
throw new Error('wrong credentials')
}
const passHashCompare = await bcrypt.compare(password, potentialUser.password);
if (!passHashCompare) {
throw new Error('wrong credentials')
}
};
module.exports = findUserByCredentials;
This way it's less nested, more readable and works in the order you need.
You can go even further with this principle and make your middleware (the function you're passing to app.use) an async function as well and use the await keyword instead of .then() and .catch()

I would switch the code to use the latest features of the javascript language related to asynchronous code async/await in that way you can have better control of your execution flow.
I will modify your code in the following way:
Firstly for the findUserByCredentials function:
const bcrypt = require('bcrypt');
const jsonfile = require('../users.json');
const findUserByCredentials = async (object) => {
const username = object.username;
const password = object.password;
const user = jsonfile.find(user => user.username == username);
if (user)
await bcrypt.compareSync(user.password, password);
return false;
};
module.exports = findUserByCredentials;
Secondly for the index.js
const express = require('express')
const app = express()
const port = process.env.PORT || 3000
const findUserByCredentials = require("./lib/find-user");
app.use(async (req, res, next) => {
if (req.headers) {
let header = req.headers.authorization || '';
let [type, payload] = header.split(' ');
if (type === 'Basic') {
const credentials = Buffer.from(payload, 'base64').toString('ascii');
const [username, password] = credentials.split(':');
const result = await findUserByCredentials({username, password})
if(result) {
return next()
}
return res.sendStatus(403);
}
return res.sendStatus(403);
}
return next();
});
app.get('/inchtocm', (req, res) => {
const cm = parseFloat(req.query.inches) * 2.54;
res.send({"unit": "cm", "value": cm});
});
app.listen(port, () => {
console.log(`Example app listening at http://localhost:${port}`)
})
module.exports = app;
It's very important learn how to use asynchronous code in Nodejs because it's a single thread and if it's not used correctly you can block it and your performance of your app won't be optimal, also async/await is syntactical sugar of javascript languages which allows to write clean asynchronous code, try to use the latest things of the specification of language because they are created to ease our lives.

How to rewrite this request using async/await syntax?

Here's the task:
You need to make a GET request for the resource: https://jsonplaceholder.typicode.com/posts using fetch method
Save the response to response.json file
Save only those items, where id < 20
What I wrote:
const fetch = require('node-fetch');
const fs = require('fs');
const path = require('path');
const filePath = path.join(__dirname, 'response.json');
fetch('https://jsonplaceholder.typicode.com/posts')
.then(res => res.json())
.then(data => {
const refined = data.filter(item => item.id < 20);
const stringified = JSON.stringify(refined);
fs.appendFile(filePath, stringified, err => {
if (err) {
throw err;
}
});
});
How to write the same fetch, but with async/await syntax?

await keyword can only be used inside an async function, so you need to write an async function that makes the API request to fetch the data
async function fetchData() {
const response = await fetch('https://jsonplaceholder.typicode.com/posts');
const data = await response.json();
const refined = data.filter(item => item.id < 20);
const stringified = JSON.stringify(refined);
// promise version of appendFile function from fs.promises API
await fs.appendFile(filePath, stringified);
}
fs module of nodeJS has functions that use promises instead of callbacks. if you don't want to use callback version, you will need to use promise version of appendFile function.
You can import the promise version of fs module as require('fs').promises or require('fs/promises').
To handle errors, make sure that code that calls this function has a catch block to catch and handle any errors that might be thrown from this function. You could also wrap the code in this function with try-catch block to handle the errors inside this function.
Side tip: If you want to write data in the file in easily readable format, change
const stringified = JSON.stringify(refined);
to
const stringified = JSON.stringify(refined, null, 4);

Below snippet could help you (tested in node v14)
const fetch = require("node-fetch")
const fs = require("fs")
const path = require("path")
const filePath = path.join(__dirname, "response.json")
async function execute() {
const res = await fetch("https://jsonplaceholder.typicode.com/posts")
const data = await res.json()
const refined = data.filter((item) => item.id < 20)
const stringified = JSON.stringify(refined)
fs.appendFile(filePath, stringified, (err) => {
if (err) {
throw err
}
})
}
execute()

Extract function from javascript promise (expressjs + neDB)

The following code is working and retrieving all users from my neDB-promisses:
const getUsers = (res) => {
db.find({})
.sort({ name: 1 })
.exec()
.then(
(content) => {
res.status(200).json(content);
},
(err) => {
res.status(400).json(err);
}
);
};
What I'm trying to do: optimize this piece of code in order to avoid future repetitions on CRUD functions, something like this:
...
.then(successFunctionCall, failureFunctionCall)
...
I thought about creating a separate module called successFunctionCall/failureFunctionCall but I need to call res inside it to set the response JSON and status code. Is that a better way to achieve this ?
Thank you.

You can curry the functions. When you use them as handlers, pass res, and get a new function that waits for the content or err:
const successFunctionCall = res => content =>
res.status(200).json(content);
const failureFunctionCall = res => err =>
res.status(400).json(err);
const getUsers = (res) => {
db.find({})
.sort({ name: 1 })
.exec()
.then(
successFunctionCall(res),
failureFunctionCall(res)
);
};

In order to avoid repetitions on CRUD functions, your concerns could be separated a little differently. Below is a basic idea of what I mean.
const usersDb = {} // your users db instance here
const findAllSortedBy = db => (...args) => db.find({}).sort(...args).exec()
const findAllUsersSortedBy = findAllSortedBy(usersDb)
const success = res => content => res.status(200).json(content)
const failure = res => err => res.status(400).json(err)
const getUsers = res => {
findAllUsersSortedBy({ name: 1 })
.then(success(res))
.catch(failure(res))
}
here is getUsers in context of an express route handler
const getUsers = (req, res) => {
findAllUsersSortedBy({ name: 1 })
.then(success(res))
.catch(failure(res))
}
router.get('/users', getUsers)

Execute code after fs.writeFile using async/await

I have a function, startSurvey, which, when run, checks if there are questions in a .json file. If there are no questions, it fetches some questions from Typeform and writes them to the .json file using saveForm. After it writes, I would like to continue executing some code that reads the .json file and logs its contents. Right now, await saveForm() never resolves.
I have promisified the fs.readFile and fs.writeFile functions.
//typeform-getter.js
const fs = require('fs')
const util = require('util')
const fetch = require('cross-fetch')
require('dotenv').config()
const conf = require('../../private/conf.json')
const typeformToken = conf.tokens.typeform
const writeFile = util.promisify(fs.writeFile)
const getForm = async () => {
const form = await fetch(`https://api.typeform.com/forms/${process.env.FORM_ID}`, {
headers: {
"Authorization": `bearer ${typeformToken}`
}
}).then(res => res.json())
const fields = form.fields
return fields
}
const saveForm = async () => {
const form = await getForm()
return writeFile(__dirname + '/../data/questions.json', JSON.stringify(form))
.then((e) => {
if (e) console.error(e)
else console.log('questions saved')
return
})
}
module.exports = saveForm
//controller.js
const fs = require('fs')
const util = require('util')
const request = require('request')
require('dotenv').config()
const typeformGetter = require('./functions/typeform-getter')
const readFile = util.promisify(fs.readFile)
const saveForm = util.promisify(typeformGetter)
let counter = 1
const data = []
const getQuestions = async() => {
console.log('called')
try {
let data = await readFile(__dirname + '/data/questions.json')
data = JSON.parse(data)
return data
} catch (e) {
console.error('error getting questions from read file', e)
}
}
const startSurvey = async (ctx) => {
try {
const questions = await getQuestions()
if (!questions) await saveForm()
console.log(questions) //NEVER LOGS
} catch (error) {
console.error('error: ', error)
}
}
startSurvey() //function called

I don't know your exact error, but there are multiple things wrong with your code:
You're using incorrectly the promisified version of fs.writeFile, if an error occurs, the promise will be rejected, you won't get a resolved promise with an error as the resolved value, which is what you're doing.
Use path.join instead of concatenating paths.
In startSurvey, you're using console.log(questions) but that wont have any data if questions.json doesn't exists, which should happen the first time you run the program, since it's filled by saveForm, so you probably want to return the questions in saveForm
So saveForm should look something like this:
const saveForm = async () => {
const form = await getForm();
const filePath = path.join(path.__dirname, '..', 'data', 'questions.json');
await writeFile(filePath, JSON.stringify(form));
console.log('questions saved');
return form;
}
And startSurvey
const startSurvey = async (ctx) => {
try {
const questions = await getQuestions() || await saveForm();
// This will be logged, unless saveForm rejects
// In your code getQuestions always resolves
console.log(questions);
} catch (error) {
console.error('error: ', error)
}
}
In your controller.js you're using util.promisify on saveForm when it is already a promise.
So it should be:
const saveForm = require('./functions/typeform-getter')

How to stub a function in a module?

I build express app, there is a route A use many middlewares:
// fblogin.js
const saveUser = require('./middlewares').saveUser;
const issueJWT = require('./middlewares').issueJWT;
const exchangeLongTimeToken = (a) => { //return promise to call API };
const retrieveUserInfo = (b) => { //return promise to call API };
const service = {
exchangeLongTimeToken,
retrieveUserInfo,
};
const asyncAll = (req, res) => {
// use Promise.all() to get service.exchangeLongTimeToken
// and service.retrieveUserInfo
};
router.post('/', [asyncAll, saveUser, issueJWT], (req, res) => {
//some logic;
});
module.exports = { router, service };
And this is my middlewares.js:
const saveUser = (req, res, next) => { //save user };
const issueJWT = (req, res, next) => { //issue jwt };
module.exports = { saveUser, issueJWT };
It works well. But got problem when I tried to write the test.
This is my test, I use mocha, chai, supertest and sinon:
const sinon = require('sinon');
const middlewares = require('../../../../src/routes/api/auth/shared/middlewares');
const testData = require('../../testdata/data.json');
let app = require('../../../../src/config/expressapp').setupApp();
const request = require('supertest');
let service = require('../../../../src/routes/api/auth/facebook/fblogin').service;
describe('I want to test', () => {
context('Let me test', function () {
const testReq = {name: 'verySad'};
beforeEach(() => {
sinon.stub(middlewares, 'saveUser').callsFake((req, res, next)=>{
console.log(req);
});
sinon.stub(service, 'exchangeLongTimeToken').callsFake((url) => {
return Promise.resolve(testData.fbLongTimeToken);
});
sinon.stub(service, 'retrieveUserInfo').callsFake((url) => {
return Promise.resolve(testData.fbUserInfo);
});
});
it('Should return 400 when bad signedRequest', () => {
return request(app).post(facebookAPI).send(testReq).then((response) => {
response.status.should.be.equal(400);
});
});
});
What is the problem
You could see that there are 3 stubs, 1 for middlewares.saveUser and 2 for services.XXXX which is in the same file of the route.
The problem is that, the 2 stubs works while the 1 for middlewares.saveUser not work, always trigger the original one.
I think it maybe that when I call the setupApp(), the express will load all the routers it needs, so mock it afterwards won't have a effect, but it
is strange that route.service could be mocked...
How to get the stub work?
The only way to get it work, is to put the stub at the top of test file, just after that middleware require.
I tried:
1. Use 3rd party modules like proxyquire, rewire
2. Use node's own delete require.cache[middlewares] and 'app' and re-require them.
3. Many other tricks.
4. Use jest's mock, but still only works if I put it at the top of the file.
What is the way of solving this problem without putting the stub at the top of the test file? Thanks!

The solution in the question is a bit restricted, since the mock has polluted the whole test suites.
I end up by doing this, The logic is simple, we still need to mock the saveUser first, but then we require all the other variables into the test function rather than require them at the top of the file, more flexible this time. And I add a checkIfTheStubWorks method to check if the stub works, to make sure the whole test works.
const middlewares = require('../../../../src/routes/api/auth/shared/middlewares');
const removeEmptyProperty = require('../../../../src/utils/utils').removeEmptyProperty;
let app;
let service;
let request;
/*
* The reason we need this is:
* If the mock not works,
* the whole test is meaningless
*/
const checkIfTheStubWorks = () => {
expect(spy1).toHaveBeenCalled();
expect(spy2).toHaveBeenCalled();
expect(spy3).toHaveBeenCalled();
};
const loadAllModules = () => {
service = require('../../../../src/routes/api/auth/facebook/fblogin').service;
app = require('../../../../src/config/expressapp').setupApp();
request = require('supertest')(app);
};
describe('Mock response from facebook', () => {
let spy1 = {};
let spy2 = {};
let spy3 = {};
const testReq = testData.fbShortToken;
beforeAll(() => {
spy1 = jest.spyOn(middlewares, 'saveUser').mockImplementation((req, res, next) => {
userToSaveOrUpdate = removeEmptyProperty(res.locals.user);
next();
});
// It must be load here, in this order,
// otherwise, the above mock won't work!
loadAllModules();
spy2 = jest.spyOn(service, 'exchangeLongTimeToken').mockImplementation((url) => {
// mock it
});
spy3 = jest.spyOn(service, 'retrieveUserInfo').mockImplementation((url) => {
// mock it
});
});
afterAll(() => {
spy1.mockRestore();
spy2.mockRestore();
spy3.mockRestore();
});
test('Return a JWT should have same length as facebook one', async () => {
const response = await request.post(facebookAPI).send(testReq);
// assert this, assert that
checkIfTheStubWorks();
});
});