Mongoose - Query exec() never resolves in model method - javascript

In an attempt to refactor some code that will be used in different places, I moved this method which was already working from a controller to a Mongoose model.
In the model, the trackQuery.exec() never reaches the callback, never resolves. If I resolve without the exec() or without waiting for the exec(), it works fine. The trackQuery is filled properly with a Mongoose Query.
What is the subtlety about Mongoose models that I don't get ?
ArtistSchema.methods.insertRelatedTracks = function (items) {
const newTracksToSave = items.map((item) => {
return new Promise((resolve, reject) => {
const TrackModel = mongoose.model('Track'),
trackQuery = TrackModel.findOne({ externalID: item.id })
;
return trackQuery.exec((err, track) => {
if (err) {
return reject(err);
}
if (!track) {
let track = new TrackModel({
externalID: item.id,
provider: item.provider,
title: item.title,
artist: item.artist,
artistID: this._id,
artistExternalID: this.externalID,
thumbnail: item.thumbnail,
publishedAt: item.publishedAt,
count: 0
});
return track.save((err, res) => {
if (err) {
return reject(err);
}
return resolve(res);
});
}
return resolve(track);
});
});
});
return Promise.all(newTracksToSave).then(() => {
return this;
}).catch((err) => {
console.error(err);
return this;
});
}

The solution for me was to manually import the TrackModel instead of relying on the usual runtime mongoose.model('Track') method. I have no explications why mongoose.model doesn't work in this case. Any hints are welcomed.

Related

Function returned undefined, expected Promise or value in Cloud Functions

I have a Cloud Function that is triggered when a document is created in Firestore, and I keep getting Function returned undefined, expected Promise or value. The function does what it is supposed to do, but it sometimes takes around 25-30 seconds, so I thought it may have something to do with this error. I would really appreciate if someone could help me understand what to return here. My function is below:
const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp();
const db = admin.firestore();
const Iyzipay = require('iyzipay');
const iyzipay = new Iyzipay({
apiKey: '...',
secretKey: '...',
uri: '...'
});
exports.pay = functions
.region('europe-west1')
.firestore
.document('requests/{docId}')
.onCreate((snap, context) => {
const newValue = snap.data();
const request = {
locale: Iyzipay.LOCALE.TR,
conversationId: newValue.uid,
price: newValue.price,
paidPrice: newValue.price,
currency: Iyzipay.CURRENCY.TRY,
installment: '1',
basketId: 'B67832',
paymentChannel: Iyzipay.PAYMENT_CHANNEL.MOBILE_IOS,
paymentGroup: Iyzipay.PAYMENT_GROUP.LISTING,
paymentCard: {
cardHolderName: newValue.cardHolderName,
cardNumber: newValue.cardNumber,
expireMonth: newValue.expireMonth,
expireYear: newValue.expireYear,
cvc: newValue.cvc
},
buyer: {
id: newValue.uid,
name: newValue.name,
surname: newValue.surname,
gsmNumber: newValue.gsmNumber,
email: newValue.email,
identityNumber: newValue.identityNumber,
registrationAddress: newValue.registrationAddress,
city: newValue.city,
country: newValue.country,
zipCode: newValue.zipCode
},
shippingAddress: {
contactName: newValue.name,
city: newValue.city,
country: newValue.country,
address: newValue.registrationAddress,
zipCode: newValue.zipCode
},
billingAddress: {
contactName: newValue.name,
city: newValue.city,
country: newValue.country,
address: newValue.registrationAddress,
zipCode: newValue.zipCode
},
basketItems: [
{
id: newValue.productid,
name: newValue.productname,
category1: newValue.category1,
itemType: Iyzipay.BASKET_ITEM_TYPE.PHYSICAL,
price: newValue.price
},
]
}
iyzipay.payment.create(request, function (err, result) {
console.log(err, result);
const docRef1 = db.collection('results').doc().set(result);
})
})
You need to terminate a Cloud Function when all the asynchronous work is completed, see the doc. In the case of a background triggered Cloud Function (e.g. Cloud Firestore function onCreate trigger, like your Cloud Function) you must return the chain of Promises returned by the asynchronous method calls.
I don't know the Iyzipay service nor the corresponding Node.js library, but it seems that there is no "promisified" version of the iyzipay.payment.create method. You should therefore wrap it in a Promise and chain this Promise with the Promise returned by the Firestore asynchronous set() method, as follows (untested).
exports.pay = functions
.region('europe-west1')
.firestore
.document('requests/{docId}')
.onCreate((snap, context) => {
const newValue = snap.data();
const request = { ... };
return new Promise(function (resolve, reject) {
iyzipay.payment.create(request, function (err, result) {
if (err) {
reject(err)
} else {
resolve(result)
}
})
})
.then(result => {
return db.collection('results').doc().set(result);
})
.catch(error => {
console.log(error);
return null;
});
});
If you want to write something to the log when the operation is completed, do as follows:
// ...
.onCreate((snap, context) => {
const newValue = snap.data();
const request = { ... };
return new Promise(function (resolve, reject) {
// ...
})
.then(result => {
return db.collection('results').doc().set(result);
})
.then(() => {
console.log("Operation completed: " + result);
return null;
})
.catch(error => {
console.log(error);
return null;
});
Update following your comment:
How can I add another Firestore query after return db.collection('results').doc().set(result);? For example, I want to
update a field in a document, so where can I add
db.collection('listings').doc(newValue.productid).update({sold : true})?
You have two possibilities:
Approach #1
Since the update() method is an asynchronous method which returns a Promise (like all Firebase asynchronous methods), you need to add it to the chain of promises, as follows:
return new Promise(function (resolve, reject) {
iyzipay.payment.create(request, function (err, result) {
if (err) {
reject(err)
} else {
resolve(result)
}
})
})
.then(result => {
return db.collection('results').doc().set(result);
})
.then(() => {
return db.collection('listings').doc(newValue.productid).update({sold: true});
})
.catch(error => {
console.log(error);
return null;
});
Approach #2
Since both the set() the update() methods write to a document, you could use a batched write, as follows:
return new Promise(function (resolve, reject) {
iyzipay.payment.create(request, function (err, result) {
if (err) {
reject(err)
} else {
resolve(result)
}
})
})
.then(result => {
const batch = db.batch();
var docRef1 = db.collection('results').doc();
batch.set(docRef1, result);
var docRef2 = db.collection('listings').doc(newValue.productid);
batch.update(docRef2, {sold: true});
return batch.commit();
})
.catch(error => {
console.log(error);
return null;
});
The difference with Approach #1 is that the two writes are done in one atomic action.
PS: Note that instead of doing db.collection('results').doc().set(result); you could do db.collection('results').add(result);

javascript promise after foreach loop with multiple mongoose find

I'm trying to have a loop with some db calls, and once their all done ill send the result. - Using a promise, but if i have my promise after the callback it dosent work.
let notuser = [];
let promise = new Promise((resolve, reject) => {
users.forEach((x) => {
User.find({
/* query here */
}, function(err, results) {
if(err) throw err
if(results.length) {
notuser.push(x);
/* resolve(notuser) works here - but were not done yet*/
}
})
});
resolve(notuser); /*not giving me the array */
}).then((notuser) => {
return res.json(notuser)
})
how can i handle this ?
Below is a function called findManyUsers which does what you're looking for. Mongo find will return a promise to you, so just collect those promises in a loop and run them together with Promise.all(). So you can see it in action, I've added a mock User class with a promise-returning find method...
// User class pretends to be the mongo user. The find() method
// returns a promise to 'find" a user with a given id
class User {
static find(id) {
return new Promise(r => {
setTimeout(() => r({ id: `user-${id}` }), 500);
});
}
}
// return a promise to find all of the users with the given ids
async function findManyUsers(ids) {
let promises = ids.map(id => User.find(id));
return Promise.all(promises);
}
findManyUsers(['A', 'B', 'C']).then(result => console.log(result));
I suggest you take a look at async it's a great library for this sort of things and more, I really think you should get used to implement it.
I would solve your problem using the following
const async = require('async')
let notuser = [];
async.forEach(users, (user, callback)=>{
User.find({}, (err, results) => {
if (err) callback(err)
if(results.length) {
notUser.push(x)
callback(null)
}
})
}, (err) => {
err ? throw err : return(notuser)
})
However, if you don't want to use a 3rd party library, you are better off using promise.all and await for it to finish.
EDIT: Remember to install async using npm or yarn something similar to yarn add async -- npm install async
I used #danh solution for the basis of fixing in my scenario (so credit goes there), but thought my code may be relevant to someone else, looking to use standard mongoose without async. I want to gets a summary of how many reports for a certain status and return the last 5 for each, combined into one response.
const { Report } = require('../../models/report');
const Workspace = require('../../models/workspace');
// GET request to return page of items from users report
module.exports = (req, res, next) => {
const workspaceId = req.params.workspaceId || req.workspaceId;
let summary = [];
// returns a mongoose like promise
function addStatusSummary(status) {
let totalItems;
let $regex = `^${status}$`;
let query = {
$and: [{ workspace: workspaceId }, { status: { $regex, $options: 'i' } }],
};
return Report.find(query)
.countDocuments()
.then((numberOfItems) => {
totalItems = numberOfItems;
return Report.find(query)
.sort({ updatedAt: -1 })
.skip(0)
.limit(5);
})
.then((reports) => {
const items = reports.map((r) => r.displayForMember());
summary.push({
status,
items,
totalItems,
});
})
.catch((err) => {
if (!err.statusCode) {
err.statusCode = 500;
}
next(err);
});
}
Workspace.findById(workspaceId)
.then((workspace) => {
let promises = workspace.custom.statusList.map((status) =>
addStatusSummary(status)
);
return Promise.all(promises);
})
.then(() => {
res.status(200).json({
summary,
});
})
.catch((err) => {
if (!err.statusCode) {
err.statusCode = 500;
}
next(err);
});
};

Sequelize Update Returning Before and After Results

I'm trying to create an Update API route using Sequelize that will:
Capture the record before the update (beforeRec)
Perform the update
Capture the updated record (updatedRec)
Return both the beforeRec and updatedRec
I'm having trouble with my promise chain, which is executing the before and after select queries before executing the update. I've tried several different ways of chaining and capturing results, but here's the latest code:
router.put('/:id', (req, res) => {
const pk = req.params.id;
const getBeforeRec = Master.findByPk(pk)
.then(rec => {return rec})
const updateRec = getBeforeRec
.then(
Master.update(
req.body,
{ where: {id: pk} }
)
)
const getUpdatedRec = updateRec
.then(
Master.findByPk(pk)
.then(rec => {return rec})
);
return Promise.all([getBeforeRec, updateRec, getUpdatedRec])
.then( ([beforeRec, updateRes, afterRec]) => {
return res.json({beforeRec, afterRec})
})
.catch(err => {
return res.status(400).json({'error': err});
});
});
Here's a sanitized example of how the results look:
{
"beforeRec": {
"id": 100,
"updated_col_name": false,
},
"afterRec": {
"id": 100,
"updated_col_name": false,
}
}
In the console, I can see that the update is executing last:
Executing (default): SELECT [id], [updated_col_name] FROM [master] WHERE [master].[id] = N'100';
Executing (default): SELECT [id], [updated_col_name] FROM [master] WHERE [master].[id] = N'100';
Executing (default): UPDATE [master] SET [updated_col_name]=1 WHERE [id] = N'106'
What's the best way to make the second select statement wait for the update?
Any help in clarifying how to chain promises while capturing results along the way will be greatly appreciated! Thanks.
After trying a number of ways, it finally works with nesting:
router.put('/:id', (req, res) => {
const pk = req.params.id;
let beforeRec;
Master.findByPk(pk)
.then(rec => { beforeRec = rec; })
.then(() => {
Master.update(
req.body,
{ where: {id: pk} }
)
.then(() => {
Master.findByPk(pk)
.then(rec => { return rec; })
.then((afterRec) => {
return res.json({beforeRec, afterRec})
})
})
})
.catch(err => {
return res.status(400).json({'error': err});
});
});
If I don't nest the second Master.findByPk, then Master.update() ends up executing last. Also, while I can set beforeRec outside of the promise chain, it didn't work for afterRec.
I don't love it, since I'm still confused by promises, but it's returning the desired results. However, with this nesting mess, I'm not sure where the catch() belongs. Will it catch errors within the nested then()s? Only further testing will tell.
You can do that with , previous method of the instance that returned by update query :
Master.update( req.body , { where: {id: pk} }).then(master => {
console.log(master.get()); // <---- Will give you latest values
console.log(master.previous()); // <---- returns the previous values for all values which have changed
})
For More Detail :
http://docs.sequelizejs.com/class/lib/model.js~Model.html#instance-method-previous
https://github.com/sequelize/sequelize/issues/1814
Give this a shot:
router.put('/:id', (req, res) => {
const pk = req.params.id;
let beforeRec, afterRec;
Master.findByPk(pk)
.then(rec => { beforeRec = rec; })
.then(() => {
Master.update(
req.body,
{ where: {id: pk} }
)
})
.then(() => {
Master.findByPk(pk)
.then(rec => { afterRec = rec; })
})
.then(() => {
res.json({beforeRec, afterRec})
})
.catch(errror => {
res.status(400).json({error});
});
});
Resurrecting an old question to help people in the future...
I've been using sequelize v6 with MySQL. I can't speak to other variances but assuming you just want the snapshot of the "previous" values, you can use the following method to create a copy the properties and their values before updating them
// then catch method
router.put('/:id', (req, res) => {
const pk = req.params.id;
let beforeRecord;
const updateRec = Master.findByPk(pk).then(rec => {
// .get() method is synchronous
beforeRecord = rec.get({ plain: true });
// calling .update on the model instance will also
// call .reload on the instance as well.
// Same thing happens when calling .save on the instance
return rec.update(req.body);
});
updateRec.then(rec => {
const afterRec = rec.get({ plain: true });
return res.json({beforeRec, afterRec})
}).catch(err => {
return res.status(400).json({'error': err});
});
});
// Async await method
router.put('/:id', async (req, res) => {
const pk = req.params.id;
try {
/** #type{import('sequelize').Model} */ // rec equals a sequelize model instance
const rec = await Master.findByPk(pk)
// .get() method is synchronous and returns an object (NOT a sequelize model instance)
const beforeRecord = rec.get({ plain: true });
// calling .update on the model instance will also
// call .reload on the instance as well.
// Same thing happens when calling .save on the instance
await rec.update(req.body); // after this call, rec contains the new updated values
const afterRec = rec.get({ plain: true });
return res.json({beforeRec, afterRec})
} catch (err) {
return res.status(400).json({'error': err});
}
});

Node.js - How to chain Promise in The right way

I recently moved from callback functions to promises in node.js. I want to preform async query to the DB (psql) in the most elegant way. I was wondering if the following code is the right way to do it or if I can chain for example the promises in a way of first().then(second).then(third).
function queryAll(req, res) {
searchQuery()
.then((rows) => {
console.log(rows);
totalQuery()
.then((total) => {
console.log(total);
});
});
res.json({"rows": rows, "total": total});
}
function searchQuery() {
return new Promise(function(resolve, reject) {
var rowData = { rows: {} };
pool.query('select age, sex from workers;', values, function(err, result) {
if(err) {
return console.error('error running query', err);
reject(err);
}
rowData.rows = result.rows;
resolve(rowData);
});
});
}
function totalQuery() {
return new Promise(function(reject, resolve) {
var totalData = { totals: {} };
pool.query('select sex, scores from workers group by sex;', values, function(err, result) {
if(err) {
return console.error('error running query', err);
reject(err);
}
totalData.totals = result.rows;
resolve(totalData);
});
});
}
var rowData = { rows: {} };
var totalData = { totals: {} };
First of all, these make no sense stored in variables since there's nothing else on the object. Just resolve with the rows directly instead.
return console.error('error running query', err);
Also, don't just console.log your errors. then accepts a second callback that executes on thrown errors or rejected promises. Throw this message in an error or reject with it instead. Also, I would leave logging to the consumer.
function queryAll(req, res) {
searchQuery()
.then((search) => {
console.log(rows);
totalQuery()
.then((total) => {
console.log(total);
});
});
res.json({"rows": rows, "total": total});
}
rows and total don't exist anywhere. Plus, by the time res.json executes, rows and total (assuming they come from inside the callbacks) won't exist yet since the whole sequence is async. You'll get undefined values as results.
I see little point in running searchQuery and totalQuery in sequence as they're not dependent on each other. It's better to run them parallel instead. Use Promise.all for that.
function queryAll(req, res) {
Promise.all([
searchQuery(),
totalQuery()
]).then(values => {
const rows = values[0];
const total = values[1];
res.json({"rows": rows, "total": total});
}, function(e){
// something went wrong
});
}
function searchQuery() {
return new Promise(function(resolve, reject) {
pool.query('select age, sex from workers;', values, function(err, result) {
if(err) reject(err);
else resolve(result.rows);
});
});
}
function totalQuery() {
return new Promise(function(reject, resolve) {
pool.query('select sex, scores from workers group by sex;', values, function(err, result) {
if(err) reject(err);
else resolve(result.rows);
});
});
}
You have a few issues in the code:
You return before executing reject()
There is an undefined rows variable (mismatch with search)
res.json is executed before the results are in.
The promises resolve to objects like { rows: rows }, but the main function seems to expect the plain numbers, not the objects. So let the promises just resolve to numeric values.
The second SQL is ambiguous since the second field is not aggregated and does not appear in the group by clause either. Assuming you want to sum the scores, use sum().
The second query is only launched after the first one has returned results, but this can be done in parallel
You have very similar code repeated. Try to reuse code and make the SQL statement an argument.
Here is how I would suggest to do it:
function queryAll(req, res) {
return Promise.all([searchQuery(), totalQuery()]).then(([rows, total]) => {
console.log('rows', rows);
console.log('total', total);
// Make sure to only access the promised values in the `then` callback
res.json({rows, total});
});
}
function searchQuery() {
return promiseQuery('select age, sex from workers;');
}
function totalQuery() {
// When you group, make sure to aggregate:
return promiseQuery('select sex, sum(scores) as scores from workers group by sex;');
}
function promiseQuery(sql) { // reusable for other SQL queries
return new Promise(function(resolve, reject) {
pool.query(sql, values, function(err, result) {
if(err) {
// Do not return before calling reject!
console.error('error running query', err);
reject(err);
return;
}
// No need for a variable or object, just resolve with the number of rows
resolve(result.rows);
});
});
}
The most elegant solution would be via pg-promise:
function queryAll(req, res) {
db.task(t => {
return t.batch([
t.any('SELECT age, sex FROM workers', values),
t.any('SELECT sex, scores FROM workers GROUP BY sex', values)
]);
})
.then(data => {
res.json({rows: data[0], total: data[1]});
})
.catch(error => {
// handle error
});
}
And that's everything. You don't have to reinvent promise patterns for working with the database, they are all part of the library already.
And if your queries have a dependency, see: How to get results from multiple queries at once.
Or if you prefer ES6 generators:
function queryAll(req, res) {
db.task(function* (t) {
let rows = yield t.any('SELECT age, sex FROM workers', values);
let total = yield t.any('SELECT sex, scores FROM workers GROUP BY sex', values);
return {rows, total};
})
.then(data => {
res.json(data);
})
.catch(error => {
// handle error
});
}
And with the ES7's await/async it would be almost the same.
First of all there are some errors in your code, you have to place the reject before the return, otherwise it will be never called, and create a dangling promise:
function searchQuery() {
return new Promise(function(resolve, reject) {
var rowData = {
rows: {}
};
pool.query('select age, sex from workers;', values, function(err, result) {
if (err) {
reject(err);
console.error('error running query', err);
} else {
rowData.rows = result.rows;
resolve(rowData);
}
});
});
}
Beside that you should not nest the Promises when ever possible.
So it should be:
function queryAll(req, res) {
var result = {};
searchQuery()
.then((search) => {
console.log(search);
result.rows = search;
return totalQuery();
})
.then((total) => {
result.total = total;
console.log(total);
});
}
The res.json has to be called in the then part of the Promise:
function queryAll(req, res) {
var result = {};
searchQuery()
.then((search) => {
console.log(search);
result.rows = search;
return totalQuery();
})
.then((total) => {
result.total = total;
console.log(total);
})
.then(() => {
res.json({
"rows": result.rows,
"total": result.total
});
});
}
If your queryAll is called as e.g. middleware of express, then you should handle the catch case within queryAll:
function queryAll(req, res) {
var result = {};
searchQuery()
.then((search) => {
console.log(search);
result.rows = search;
return totalQuery();
})
.then((total) => {
result.total = total;
console.log(total);
})
.then(() => {
res.json({
"rows": result.rows,
"total": result.total
});
})
.catch( err => {
res.status(500).json({error: 'some error'})
});
}
For postgress I would suggest to use pg-promise instead of using a callback style library and wrapping it into promises yourself.
You could simplify the code if you use a library like bluebird:
const bPromise = require('bluebird')
function queryAll(req, res) {
bPromise.all([
searchQuery(),
totalQuery()
])
.spread((rows, total) => {
res.json({
"rows": rows,
"total": total
});
})
.catch(err => {
res.status(500).json({
error: 'some error'
})
});
}
With nsynjs your logic may be coded as simple as this:
var resp = {
rows: dbQuery(nsynjsCtx, conn, 'select age, sex from workers', values1).data,
total: dbQuery(nsynjsCtx, conn, 'select sex, scores from workers group by sex', values2).data
};
Please see example of multiple sequential queries here: https://github.com/amaksr/nsynjs/tree/master/examples/node-mysql

Return mysql query for graphql resolver function in node-mysql

I'm new to graphql and node so sorry if this is really simple but I'm trying to perform a mysql query so that the query response is returned by graphql for the client to read from. The problem I'm running into is that because node-mysql does query asynchronously, I can't get the query response directly.
After some tinkering I figured it out, new code:
var root = {
login: function({username, password}) {
var s = `select password from users where username='${username}'`;
var result = sql.query(s);
return result.then(response => {
return password == response[0].password
});
}
Here is the function definition for sql.query
exports.query = function(s, callback) {
var promise = new Promise((resolve, reject) => {
con.query(s, function(err, response) {
if (err) throw err;
resolve(response);
});
});
return promise;
Graphql now return that response is not defined.
I personally use mysql, but shouldn't be too much of a difference.
So I do it like this:
exports.getUser = ({ id }) => {
return new Promise((resolve, reject) => {
let sql = `select * from users u where u.user_id = ?`;
sql = mysql.format(sql, [id]);
connection.query(sql, (err, results) => {
if (err) reject(err);
resolve(results);
});
});
};
Wrap the query around a Promise, which resolves when the query is done.
Then in the resolve of a field, you just call .then on the promise and do whatever you want.
const viewerType = new GraphQLObjectType({
name: 'Viewer',
fields: () => ({
user: {
type: userType,
args: {
id: {
type: GraphQLInt,
},
},
resolve: (rootValue, args) => {
return getUser({ id: args.id }).then(value => value[0]);
},
},
}),
});
I appreciate that this is an old post but I've found this topic somewhat confusing and have been investigating different approaches, both which are shown here. One is a bit less verbose then the other (getQuery). I include the code for a mutation which although largely irrelevant to this topic demonstrates how none of this promise complication applies to returning the result of a mutation, just saying.
The documentation describes another approach called promise wrappers https://github.com/sidorares/node-mysql2#using-promise-wrapper. In my case this would have needed a top level await which didn't really fit in with my needs although there are all kinds of workarounds, but I just wanted the simplest possible solution to a simple problem.
resolver.ts
import db from "db";
// this is one way of doing it
const getQuery = async (sql: string) => {
const [rows] = await db.promise().query(sql);
return rows;
};
// and this is another
const getQuery2 = (sql: string) => {
return new Promise((resolve, reject) => {
db.query(sql, (error, results) => {
if (error) return reject(error);
return resolve(results);
});
});
};
export const resolvers = {
Query: {
getUsers: () => getQuery("SELECT * FROM `users`"),
},
Mutation: {
userLogin: (_, { email, password }) => {
return {
__typename: "UserLoginResponse",
status: "success",
jwt: "abc123",
};
},
},
};
db.ts
import mysql2 from "mysql2";
console.log("connecting to RDS db:", process.env.DB_BOOK_IT_HOST);
export const db = mysql2.createConnection({
host: process.env.DB_BOOK_IT_HOST,
user: process.env.DB_BOOK_IT_USER,
password: process.env.DB_BOOK_IT_PASSWORD,
database: process.env.DB_BOOK_IT_DATABASE,
});
export default db;

Categories