javascript promise after foreach loop with multiple mongoose find - javascript

I'm trying to have a loop with some db calls, and once their all done ill send the result. - Using a promise, but if i have my promise after the callback it dosent work.
let notuser = [];
let promise = new Promise((resolve, reject) => {
users.forEach((x) => {
User.find({
/* query here */
}, function(err, results) {
if(err) throw err
if(results.length) {
notuser.push(x);
/* resolve(notuser) works here - but were not done yet*/
}
})
});
resolve(notuser); /*not giving me the array */
}).then((notuser) => {
return res.json(notuser)
})
how can i handle this ?

Below is a function called findManyUsers which does what you're looking for. Mongo find will return a promise to you, so just collect those promises in a loop and run them together with Promise.all(). So you can see it in action, I've added a mock User class with a promise-returning find method...
// User class pretends to be the mongo user. The find() method
// returns a promise to 'find" a user with a given id
class User {
static find(id) {
return new Promise(r => {
setTimeout(() => r({ id: `user-${id}` }), 500);
});
}
}
// return a promise to find all of the users with the given ids
async function findManyUsers(ids) {
let promises = ids.map(id => User.find(id));
return Promise.all(promises);
}
findManyUsers(['A', 'B', 'C']).then(result => console.log(result));

I suggest you take a look at async it's a great library for this sort of things and more, I really think you should get used to implement it.
I would solve your problem using the following
const async = require('async')
let notuser = [];
async.forEach(users, (user, callback)=>{
User.find({}, (err, results) => {
if (err) callback(err)
if(results.length) {
notUser.push(x)
callback(null)
}
})
}, (err) => {
err ? throw err : return(notuser)
})
However, if you don't want to use a 3rd party library, you are better off using promise.all and await for it to finish.
EDIT: Remember to install async using npm or yarn something similar to yarn add async -- npm install async

I used #danh solution for the basis of fixing in my scenario (so credit goes there), but thought my code may be relevant to someone else, looking to use standard mongoose without async. I want to gets a summary of how many reports for a certain status and return the last 5 for each, combined into one response.
const { Report } = require('../../models/report');
const Workspace = require('../../models/workspace');
// GET request to return page of items from users report
module.exports = (req, res, next) => {
const workspaceId = req.params.workspaceId || req.workspaceId;
let summary = [];
// returns a mongoose like promise
function addStatusSummary(status) {
let totalItems;
let $regex = `^${status}$`;
let query = {
$and: [{ workspace: workspaceId }, { status: { $regex, $options: 'i' } }],
};
return Report.find(query)
.countDocuments()
.then((numberOfItems) => {
totalItems = numberOfItems;
return Report.find(query)
.sort({ updatedAt: -1 })
.skip(0)
.limit(5);
})
.then((reports) => {
const items = reports.map((r) => r.displayForMember());
summary.push({
status,
items,
totalItems,
});
})
.catch((err) => {
if (!err.statusCode) {
err.statusCode = 500;
}
next(err);
});
}
Workspace.findById(workspaceId)
.then((workspace) => {
let promises = workspace.custom.statusList.map((status) =>
addStatusSummary(status)
);
return Promise.all(promises);
})
.then(() => {
res.status(200).json({
summary,
});
})
.catch((err) => {
if (!err.statusCode) {
err.statusCode = 500;
}
next(err);
});
};

Related

save results of a promise to a variable in controller?

I am stuck on promise resolution, can anyone explain to me how to store the promise result in a variable? I am working with mongoDB and I want to get some data from it.
here is my example.
const checkRelated = new Promise((resolve, reject) => {
return RelatedProducts.find({
parent_id: { $in: product_id.split(',') },
})
.then((res) => reject(res))
.catch(err => reject(err))
})
after getting the data I want to do some validation. For example.
if(checkRelated.length > 0) {
do smth....
}
but to my surprise I get "promise {pending}"
You're overcomplicating Mongoose, if you want to do an async function you can but I've never seen a new Promise being used with Mongoose.
If you want to create an async method with mongoose do the following
exports.getRelatedProducts = async (req, res) => {
try {
const checkRelated = await RelatedProducts.find({parent_id: {
$in: product_id.split(',') }})
if (checkRelated.length > 0){
do smth...
}
} catch (error) {
reject(error)
}
}
but keep in mind you must create the async method first. then you can use await.

Testing promise chains vs async await [mocha/chai/sinon]

So I'm new to testing and have set up this basic test of a method that mocks the failure of a database call (sorry if my terminology isn't quite right)
I'm using sequelize, so Job is a model and findAndCountAll a related method.
it('Should throw a 500 error if accessing the DB fails', () => {
sinon.stub(Job, 'findAndCountAll');
Job.findAndCountAll.throws();
const req = {
query: {
index: 0,
limit: 10,
orderField: 'createdAt',
order: 'DESC'
}
};
adminController.getJobs(req, {}, () => {}).then(result => {
expect(result).to.be.an('error');
expect(result).to.have.property('statusCode', 500);
done();
})
Job.findAndCountAll.restore();
})
My problem is that most of my code is written using promise chaining:
exports.getJobs = (req, res, next) => {
const index = req.query.index || 0;
const limit = req.query.limit || 10;
const orderField = req.query.orderField || 'createdAt';
const order = req.query.orderDirection || 'DESC';
Job.findAndCountAll({
// ...arguments
})
.then(results => {
res.status(200).json({ jobs: results.rows, total: results.count });
return // Attempted to add a return statement to enter the .then() block in the test
})
.catch(err => {
if(!err.statusCode) err.statusCode = 500;
next(err);
return err; // Attempted to return the error to enter the .then() block in the test
});
This doesn't work, and my (unecessary) return statements don't help either.
However, rewriting the method using async await does work (see below). However I'd like to avoid rewriting all my code, and it would be nice to understand the difference here.
My best guess is that rather than getting the sinon stub to throw an error, I should have it reject the promise? I'm just not entirely sure whether that's along the right lines or not, or how to achieve it. I'm kind of stumbling round the docs not really knowing
Any help appreciated,
Thanks,
Nick
exports.getJobs = async(req, res, next) => {
const index = req.query.index || 0;
const limit = req.query.limit || 10;
const orderField = req.query.orderField || 'createdAt';
const order = req.query.orderDirection || 'DESC';
try {
const results = await Job.findAndCountAll({ //...params });
// ...
res.status(200).json({ jobs: results.rows, total: results.count });
return;
} catch(err) {
if(!err.statusCode) err.statusCode = 500;
next(err);
return err;
}
};
So I think I found the answer:
The stub in sinon needs to return a rejected promise, rather than throwing an error:
sinon.stub(Job, 'findAndCountAll');
Job.findAndCountAll.rejects();
Afaik this is because you can't really throw an error in async code.
And the Promise chain in the method you're testing (in my case 'getJobs'), needs to return that promise.
So instead of
Job.findAndCountAll({
// ...arguments
})
.then(results => {
res.status(200).json({...});
return;
})
.catch(err => {
if(!err.statusCode) err.statusCode = 500;
next(err);
return err;
});
Use
const results = Job.findAndCountAll({
// ...arguments
})
.then(results => {
res.status(200).json({...});
return;
})
.catch(err => {
if(!err.statusCode) err.statusCode = 500;
next(err);
return err;
});
return results;
Also, the async function in the test needs to either be returned, or awaited, so that mocha knows to wait. Using done() didn't work for me:
const result = await adminController.getJobs(req, {}, () => {});
expect(result).to.be.an('error');
expect(result).to.have.property('statusCode', 500);
Job.findAndCountAll.restore();
Hope that helps someone
**edit: as mentioned below, I forgot to pass done as an argument, which is why that method didn't work

Sequelize Update Returning Before and After Results

I'm trying to create an Update API route using Sequelize that will:
Capture the record before the update (beforeRec)
Perform the update
Capture the updated record (updatedRec)
Return both the beforeRec and updatedRec
I'm having trouble with my promise chain, which is executing the before and after select queries before executing the update. I've tried several different ways of chaining and capturing results, but here's the latest code:
router.put('/:id', (req, res) => {
const pk = req.params.id;
const getBeforeRec = Master.findByPk(pk)
.then(rec => {return rec})
const updateRec = getBeforeRec
.then(
Master.update(
req.body,
{ where: {id: pk} }
)
)
const getUpdatedRec = updateRec
.then(
Master.findByPk(pk)
.then(rec => {return rec})
);
return Promise.all([getBeforeRec, updateRec, getUpdatedRec])
.then( ([beforeRec, updateRes, afterRec]) => {
return res.json({beforeRec, afterRec})
})
.catch(err => {
return res.status(400).json({'error': err});
});
});
Here's a sanitized example of how the results look:
{
"beforeRec": {
"id": 100,
"updated_col_name": false,
},
"afterRec": {
"id": 100,
"updated_col_name": false,
}
}
In the console, I can see that the update is executing last:
Executing (default): SELECT [id], [updated_col_name] FROM [master] WHERE [master].[id] = N'100';
Executing (default): SELECT [id], [updated_col_name] FROM [master] WHERE [master].[id] = N'100';
Executing (default): UPDATE [master] SET [updated_col_name]=1 WHERE [id] = N'106'
What's the best way to make the second select statement wait for the update?
Any help in clarifying how to chain promises while capturing results along the way will be greatly appreciated! Thanks.
After trying a number of ways, it finally works with nesting:
router.put('/:id', (req, res) => {
const pk = req.params.id;
let beforeRec;
Master.findByPk(pk)
.then(rec => { beforeRec = rec; })
.then(() => {
Master.update(
req.body,
{ where: {id: pk} }
)
.then(() => {
Master.findByPk(pk)
.then(rec => { return rec; })
.then((afterRec) => {
return res.json({beforeRec, afterRec})
})
})
})
.catch(err => {
return res.status(400).json({'error': err});
});
});
If I don't nest the second Master.findByPk, then Master.update() ends up executing last. Also, while I can set beforeRec outside of the promise chain, it didn't work for afterRec.
I don't love it, since I'm still confused by promises, but it's returning the desired results. However, with this nesting mess, I'm not sure where the catch() belongs. Will it catch errors within the nested then()s? Only further testing will tell.
You can do that with , previous method of the instance that returned by update query :
Master.update( req.body , { where: {id: pk} }).then(master => {
console.log(master.get()); // <---- Will give you latest values
console.log(master.previous()); // <---- returns the previous values for all values which have changed
})
For More Detail :
http://docs.sequelizejs.com/class/lib/model.js~Model.html#instance-method-previous
https://github.com/sequelize/sequelize/issues/1814
Give this a shot:
router.put('/:id', (req, res) => {
const pk = req.params.id;
let beforeRec, afterRec;
Master.findByPk(pk)
.then(rec => { beforeRec = rec; })
.then(() => {
Master.update(
req.body,
{ where: {id: pk} }
)
})
.then(() => {
Master.findByPk(pk)
.then(rec => { afterRec = rec; })
})
.then(() => {
res.json({beforeRec, afterRec})
})
.catch(errror => {
res.status(400).json({error});
});
});
Resurrecting an old question to help people in the future...
I've been using sequelize v6 with MySQL. I can't speak to other variances but assuming you just want the snapshot of the "previous" values, you can use the following method to create a copy the properties and their values before updating them
// then catch method
router.put('/:id', (req, res) => {
const pk = req.params.id;
let beforeRecord;
const updateRec = Master.findByPk(pk).then(rec => {
// .get() method is synchronous
beforeRecord = rec.get({ plain: true });
// calling .update on the model instance will also
// call .reload on the instance as well.
// Same thing happens when calling .save on the instance
return rec.update(req.body);
});
updateRec.then(rec => {
const afterRec = rec.get({ plain: true });
return res.json({beforeRec, afterRec})
}).catch(err => {
return res.status(400).json({'error': err});
});
});
// Async await method
router.put('/:id', async (req, res) => {
const pk = req.params.id;
try {
/** #type{import('sequelize').Model} */ // rec equals a sequelize model instance
const rec = await Master.findByPk(pk)
// .get() method is synchronous and returns an object (NOT a sequelize model instance)
const beforeRecord = rec.get({ plain: true });
// calling .update on the model instance will also
// call .reload on the instance as well.
// Same thing happens when calling .save on the instance
await rec.update(req.body); // after this call, rec contains the new updated values
const afterRec = rec.get({ plain: true });
return res.json({beforeRec, afterRec})
} catch (err) {
return res.status(400).json({'error': err});
}
});

delayed returning of array of collections from mongodb nodejs

I need to retrieve a list of collections using express and mongodb module.
First, ive retrieved a list of collection names which works, I then retrieve the data of those given collections in a loop. My problem is in getColAsync():
getColAsync() {
return new Promise((resolve, reject) => {
this.connectDB().then((db) => {
var allCols = [];
let dbase = db.db(this.databaseName);
dbase.listCollections().toArray((err, collectionNames) => {
if(err) {
console.log(err);
reject(err);
}
else {
for(let i = 0; i < collectionNames.length; i++) {
dbase.collection(collectionNames[i].name.toString()).find({}).toArray((err, collectionData) => {
console.log("current collection data: " + collectionData);
allCols[i] = collectionData;
})
}
console.log("done getting all data");
resolve(allCols);
}
})
})
})
}
connectDB() {
if(this.dbConnection) {
// if connection exists
return this.dbConnection;
}
else {
this.dbConnection = new Promise((resolve, reject) => {
mongoClient.connect(this.URL, (err, db) => {
if(err) {
console.log("DB Access: Error on mongoClient.connect.");
console.log(err);
reject(err);
}
else {
console.log("DB Access: resolved.");
resolve(db);
}
});
});
console.log("DB Access: db exists. Connected.");
return this.dbConnection;
}
}
In the forloop where i retrieve every collection, the console.log("done getting all data") gets called and the promise gets resolved before the forloop even begins. for example:
done getting all data
current collection data: something
current collection data: something2
current collection data: something3
Please help
The Problem
The problem in your code is this part:
for (let i = 0; i < collectionNames.length; i++) {
dbase.collection(collectionNames[i].name.toString()).find({}).toArray((err, collectionData) => {
console.log("current collection data: " + collectionData);
allCols[i] = collectionData;
})
}
console.log("done getting all data");
resolve(allCols);
You should notice that resolve(allCols); is called right after the for loop ends, but each iteration of the loop doesn't wait for the toArray callback to be called.
The line dbase.collection(collectionNames[i].name.toString()).find({}).toArray(callback) is asynchronous so the loop will end, you'll call resolve(allCols);, but the .find({}).toArray code won't have completed yet.
The Solution Concept
So, basically what you did was:
Initialize an array of results allCols = []
Start a series of async operations
Return the (still empty) array of results
As the async operations complete, fill the now useless results array.
What you should be doing instead is:
Start a series of async operations
Wait for all of them to complete
Get the results from each one
Return the list of results
The key to this is the Promise.all([/* array of promises */]) function which accepts an array of promises and returns a Promise itself passing downstream an array containing all the results, so what we need to obtain is something like this:
const dataPromises = []
for (let i = 0; i < collectionNames.length; i++) {
dataPromises[i] = /* data fetch promise */;
}
return Promise.all(dataPromises);
As you can see, the last line is return Promise.all(dataPromises); instead of resolve(allCols) as in your code, so we can no longer execute this code inside of a new Promise(func) constructor.
Instead, we should chain Promises with .then() like this:
getColAsync() {
return this.connectDB().then((db) => {
let dbase = db.db(this.databaseName);
const dataPromises = []
dbase.listCollections().toArray((err, collectionNames) => {
if (err) {
console.log(err);
return Promise.reject(err);
} else {
for (let i = 0; i < collectionNames.length; i++) {
dataPromises[i] = new Promise((res, rej) => {
dbase.collection(collectionNames[i].name.toString()).find({}).toArray((err, collectionData) => {
console.log("current collection data: " + collectionData);
if (err) {
console.log(err);
reject(err);
} else {
resolve(collectionData);
}
});
});
}
console.log("done getting all data");
return Promise.all(dataPromises);
}
});
})
}
Notice now we return a return this.connectDB().then(...), which in turn returns a Promise.all(dataPromises); this returning new Promises at each step lets us keep alive the Promise chain, thus getColAsync() will itself return a Promise you can then handle with .then() and .catch().
Cleaner Code
You can clean up your code a bit as fallows:
getColAsync() {
return this.connectDB().then((db) => {
let dbase = db.db(this.databaseName);
const dataPromises = []
// dbase.listCollections().toArray() returns a promise itself
return dbase.listCollections().toArray()
.then((collectionsInfo) => {
// collectionsInfo.map converts an array of collection info into an array of selected
// collections
return collectionsInfo.map((info) => {
return dbase.collection(info.name);
});
})
}).then((collections) => {
// collections.map converts an array of selected collections into an array of Promises
// to get each collection data.
return Promise.all(collections.map((collection) => {
return collection.find({}).toArray();
}))
})
}
As you see the main changes are:
Using mondodb functions in their promise form
Using Array.map to easily convert an array of data into a new array
Below I also present a variant of your code using functions with callbacks and a module I'm working on.
Promise-Mix
I'm recently working on this npm module to help get a cleaner and more readable composition of Promises.
In your case I'd use the fCombine function to handle the first steps where you select the db and fetch the list of collection info:
Promise.fCombine({
dbase: (dbURL, done) => mongoClient.connect(dbURL, done),
collInfos: ({ dbase }, done) => getCollectionsInfo(dbase, done),
}, { dbURL: this.URL })
This results in a promise, passing downstream an object {dbase: /* the db instance */, collInfos: [/* the list of collections info */]}. Where getCollectionNames(dbase, done) is a function with callback pattern like this:
getCollectionsInfo = (db, done) => {
let dbase = db.db(this.databaseName);
dbase.listCollections().toArray(done);
}
Now you can chain the previous Promise and convert the list of collections info into selected db collections, like this:
Promise.fCombine({
dbase: ({ dbURL }, done) => mongoClient.connect(dbURL, done),
collInfos: ({ dbase }, done) => getCollectionsInfo(dbase, done),
}, { dbURL: this.URL }).then(({ dbase, collInfos }) => {
return Promise.resolve(collInfos.map((info) => {
return dbase.collection(info.name);
}));
})
Now downstream we have a the list of selected collections from our db and we should fetch data from each one, then merge the results in an array with collection data.
In my module I have a _mux option which creates a PromiseMux which mimics the behaviour and composition patterns of a regular Promise, but it's actually working on several Promises at the same time. Each Promise gets in input one item from the downstream collections array, so you can write the code to fetch data from a generic collection and it will be executed for each collection in the array:
Promise.fCombine({
dbase: ({ dbURL }, done) => mongoClient.connect(dbURL, done),
collInfos: ({ dbase }, done) => getCollectionsInfo(dbase, done),
}, { dbURL: this.URL }).then(({ dbase, collInfos }) => {
return Promise.resolve(collInfos.map((info) => {
return dbase.collection(info.name);
}));
})._mux((mux) => {
return mux._fReduce([
(collection, done) => collection.find({}).toArray(done)
]).deMux((allCollectionsData) => {
return Promise.resolve(allCollectionsData);
})
});
In the code above, _fReduce behaves like _fCombine, but it accepts an array of functions with callbacks instead of an object and it passes downstream only the result of the last function (not a structured object with all the results). Finally deMux executes a Promise.all on each simultaneous Promise of the mux, putting together their results.
Thus the whole code would look like this:
getCollectionsInfo = (db, done) => {
let dbase = db.db(this.databaseName);
dbase.listCollections().toArray(done);
}
getCollAsync = () => {
return Promise.fCombine({
/**
* fCombine uses an object whose fields are functions with callback pattern to
* build consecutive Promises. Each subsequent functions gets as input the results
* from previous functions.
* The second parameter of the fCombine is the initial value, which in our case is
* the db url.
*/
dbase: ({ dbURL }, done) => mongoClient.connect(dbURL, done), // connect to DB, return the connected dbase
collInfos: ({ dbase }, done) => getCollectionsInfo(dbase, done), // fetch collection info from dbase, return the info objects
}, { dbURL: this.URL }).then(({ dbase, collInfos }) => {
return Promise.resolve(collInfos.map((info) => {
/**
* we use Array.map to convert collection info into
* a list of selected db collections
*/
return dbase.collection(info.name);
}));
})._mux((mux) => {
/**
* _mux splits the list of collections returned before into a series of "simultaneous promises"
* which you can manipulate as if they were a single Promise.
*/
return mux._fReduce([ // this fReduce here gets as input a single collection from the retrieved list
(collection, done) => collection.find({}).toArray(done)
]).deMux((allCollectionsData) => {
// finally we can put back together all the results.
return Promise.resolve(allCollectionsData);
})
});
}
In my module I tried to avoid most common anti-pattern thought there still is some Ghost Promise which I'll be working on.
Using the promises from mongodb this would get even cleaner:
getCollAsync = () => {
return Promise.combine({
dbase: ({ dbURL }) => { return mongoClient.connect(dbURL); },
collInfos: ({ dbase }) => {
return dbase.db(this.databaseName)
.listCollections().toArray();
},
}, { dbURL: this.URL }).then(({ dbase, collInfos }) => {
return Promise.resolve(collInfos.map((info) => {
return dbase.collection(info.name);
}));
}).then((collections) => {
return Promise.all(collections.map((collection) => {
return collection.find({}).toArray();
}))
});
}

Get Knex.js transactions working with ES7 async/await

I'm trying to couple ES7's async/await with knex.js transactions.
Although I can easily play around with non-transactional code, I'm struggling to get transactions working properly using the aforementioned async/await structure.
I'm using this module to simulate async/await
Here's what I currently have:
Non-transactional version:
works fine but is not transactional
app.js
// assume `db` is a knex instance
app.post("/user", async((req, res) => {
const data = {
idUser: 1,
name: "FooBar"
}
try {
const result = await(user.insert(db, data));
res.json(result);
} catch (err) {
res.status(500).json(err);
}
}));
user.js
insert: async (function(db, data) {
// there's no need for this extra call but I'm including it
// to see example of deeper call stacks if this is answered
const idUser = await(this.insertData(db, data));
return {
idUser: idUser
}
}),
insertData: async(function(db, data) {
// if any of the following 2 fails I should be rolling back
const id = await(this.setId(db, idCustomer, data));
const idCustomer = await(this.setData(db, id, data));
return {
idCustomer: idCustomer
}
}),
// DB Functions (wrapped in Promises)
setId: function(db, data) {
return new Promise(function (resolve, reject) {
db.insert(data)
.into("ids")
.then((result) => resolve(result)
.catch((err) => reject(err));
});
},
setData: function(db, id, data) {
data.id = id;
return new Promise(function (resolve, reject) {
db.insert(data)
.into("customers")
.then((result) => resolve(result)
.catch((err) => reject(err));
});
}
Attempt to make it transactional
user.js
// Start transaction from this call
insert: async (function(db, data) {
const trx = await(knex.transaction());
const idCustomer = await(user.insertData(trx, data));
return {
idCustomer: idCustomer
}
}),
it seems that await(knex.transaction()) returns this error:
[TypeError: container is not a function]
I couldn't find a solid answer for this anywhere (with rollbacks and commits) so here's my solution.
First you need to "Promisify" the knex.transaction function. There are libraries for this, but for a quick example I did this:
const promisify = (fn) => new Promise((resolve, reject) => fn(resolve));
This example creates a blog post and a comment, and rolls back both if there's an error with either.
const trx = await promisify(db.transaction);
try {
const postId = await trx('blog_posts')
.insert({ title, body })
.returning('id'); // returns an array of ids
const commentId = await trx('comments')
.insert({ post_id: postId[0], message })
.returning('id');
await trx.commit();
} catch (e) {
await trx.rollback();
}
Here is a way to write transactions in async / await.
It is working fine for MySQL.
const trx = await db.transaction();
try {
const catIds = await trx('catalogues').insert({name: 'Old Books'});
const bookIds = await trx('books').insert({catId: catIds[0], title: 'Canterbury Tales' });
await trx.commit();
} catch (error) {
await trx.rollback(error);
}
Async/await is based around promises, so it looks like you'd just need to wrap all the knex methods to return "promise compatible" objects.
Here is a description on how you can convert arbitrary functions to work with promises, so they can work with async/await:
Trying to understand how promisification works with BlueBird
Essentially you want to do this:
var transaction = knex.transaction;
knex.transaction = function(callback){ return knex.transaction(callback); }
This is because "async/await requires the either a function with a single callback argument, or a promise", whereas knex.transaction looks like this:
function transaction(container, config) {
return client.transaction(container, config);
}
Alternatively, you can create a new async function and use it like this:
async function transaction() {
return new Promise(function(resolve, reject){
knex.transaction(function(error, result){
if (error) {
reject(error);
} else {
resolve(result);
}
});
});
}
// Start transaction from this call
insert: async (function(db, data) {
const trx = await(transaction());
const idCustomer = await(person.insertData(trx, authUser, data));
return {
idCustomer: idCustomer
}
})
This may be useful too: Knex Transaction with Promises
(Also note, I'm not familiar with knex's API, so not sure what the params are passed to knex.transaction, the above ones are just for example).
For those who come in 2019.
After I updated Knex to version 0.16.5. sf77's answer doesn't work anymore due to the change in Knex's transaction function:
transaction(container, config) {
const trx = this.client.transaction(container, config);
trx.userParams = this.userParams;
return trx;
}
Solution
Keep sf77's promisify function:
const promisify = (fn) => new Promise((resolve, reject) => fn(resolve));
Update trx
from
const trx = await promisify(db.transaction);
to
const trx = await promisify(db.transaction.bind(db));
I think I have found a more elegant solution to the problem.
Borrowing from the knex Transaction docs, I will contrast their promise-style with the async/await-style that worked for me.
Promise Style
var Promise = require('bluebird');
// Using trx as a transaction object:
knex.transaction(function(trx) {
var books = [
{title: 'Canterbury Tales'},
{title: 'Moby Dick'},
{title: 'Hamlet'}
];
knex.insert({name: 'Old Books'}, 'id')
.into('catalogues')
.transacting(trx)
.then(function(ids) {
return Promise.map(books, function(book) {
book.catalogue_id = ids[0];
// Some validation could take place here.
return knex.insert(book).into('books').transacting(trx);
});
})
.then(trx.commit)
.catch(trx.rollback);
})
.then(function(inserts) {
console.log(inserts.length + ' new books saved.');
})
.catch(function(error) {
// If we get here, that means that neither the 'Old Books' catalogues insert,
// nor any of the books inserts will have taken place.
console.error(error);
});
async/await style
var Promise = require('bluebird'); // import Promise.map()
// assuming knex.transaction() is being called within an async function
const inserts = await knex.transaction(async function(trx) {
var books = [
{title: 'Canterbury Tales'},
{title: 'Moby Dick'},
{title: 'Hamlet'}
];
const ids = await knex.insert({name: 'Old Books'}, 'id')
.into('catalogues')
.transacting(trx);
const inserts = await Promise.map(books, function(book) {
book.catalogue_id = ids[0];
// Some validation could take place here.
return knex.insert(book).into('books').transacting(trx);
});
})
await trx.commit(inserts); // whatever gets passed to trx.commit() is what the knex.transaction() promise resolves to.
})
The docs state:
Throwing an error directly from the transaction handler function automatically rolls back the transaction, same as returning a rejected promise.
It seems that the transaction callback function is expected to return either nothing or a Promise. Declaring the callback as an async function means that it returns a Promise.
One advantage of this style is that you don't have to call the rollback manually. Returning a rejected Promise will trigger the rollback automatically.
Make sure to pass any results you want to use elsewhere to the final trx.commit() call.
I have tested this pattern in my own work and it works as expected.
Adding to sf77's excellent answer, I implemented this pattern in TypeScript for adding a new user where you need to do the following in 1 transaction:
creating a user record in the USER table
creating a login record in the LOGIN table
public async addUser(user: User, hash: string): Promise<User> {
//transform knex transaction such that can be used with async-await
const promisify = (fn: any) => new Promise((resolve, reject) => fn(resolve));
const trx: knex.Transaction = <knex.Transaction> await promisify(db.transaction);
try {
let users: User [] = await trx
.insert({
name: user.name,
email: user.email,
joined: new Date()})
.into(config.DB_TABLE_USER)
.returning("*")
await trx
.insert({
email: user.email,
hash
}).into(config.DB_TABLE_LOGIN)
.returning("email")
await trx.commit();
return Promise.resolve(users[0]);
}
catch(error) {
await trx.rollback;
return Promise.reject("Error adding user: " + error)
}
}

Categories