I am trying to re-learn NodeJS after a couple years of putting it down, so I'm building a small banking website as a test. I decided to use Sequelize for my ORM, but I'm having a bit of trouble sending money between people in a way that I like.
Here was my first attempt:
// myUsername - who to take the money from
// sendUsername - who to send the money to
// money - amount of money to be sent from `myUsername`->`sendUsername`
// Transaction is created to keep a log of banking transactions for record-keeping.
module.exports = (myUsername, sendUsername, money, done) => {
// Create transaction so that errors will roll back
connection.transaction(t => {
return Promise.all([
User.increment('balance', {
by: money,
where: { username: myUsername },
transaction: t
}),
User.increment('balance', {
by: -money,
where: { username: sendUsername },
transaction: t
}),
Transaction.create({
fromUser: myUsername,
toUser: sendUsername,
value: money
}, { transaction: t })
]);
}).then(result => {
return done(null);
}).catch(err => {
return done(err);
});
};
This worked, but it didn't validate the model when it was incremented. I'd like for the transaction to fail when the model does not validate. My next attempt was to go to callbacks, shown here (same function header):
connection.transaction(t => {
// Find the user to take money from
return User
.findOne({ where: { username: myUsername } }, { transaction: t }) .then(myUser => {
// Decrement money
return myUser
.decrement('balance', { by: money, transaction: t })
.then(myUser => {
// Reload model to validate data
return myUser.reload(myUser => {
// Validate modified model
return myUser.validate(() => {
// Find user to give money to
return User
.findOne({ where: { username: sendUsername } }, { transaction: t })
.then(sendUser => {
// Increment balance
return sendUser
.increment('balance', { by: money, transaction: t })
.then(sendUser => {
// Reload model
return sendUser.reload(sendUser => {
// Validate model
return sendUser.validate(() => {
// Create a transaction for record-keeping
return Transaction
.create({
fromUser: myUser.id,
toUser: sendUser.id,
value: money
}, { transaction: t });
});
});
});
});
});
});
});
});
}).then(result => {
return done(null);
}).catch(err => {
return done(err);
});
This works, in that money is still transfered beetween people, but it still doesn't validate the models. I think the reason is that the .validate() and the .reload() methods do not have the ability to add the transaction: t parameter on it.
My question is if there's a way to do validation in a transaction, but I'd also like some help fixing this "callback hell." Again, I haven't done JS in a while, so there are probably better ways of doing this now that I'm just now aware of.
Thanks!
I believe you can't get validations to fire on the Model's increment and decrement and need to have instances. In some sequelize Model methods you can configure validations to run, but it doesn't look like it here
I'd do it like this
module.exports = async function(myUserId, sendUserId, money) {
const transaction = await connection.transaction();
try {
const [myUser, sendUser] = await Promise.all([
User.findById(myUserId, { transaction }),
User.findById(sendUserId, { transaction })
]);
await Promise.all([
myUser.increment('balance', {
by: money,
transaction
}),
myUser.increment('balance', {
by: -money,
transaction
})
]);
await Transaction.create({...}, { transaction })
await transaction.commit();
} catch(e) {
await transaction.rollback();
throw e;
}
}
Related
I am trying to check existing of data on firebase real-time database and then add new data.
When I add exist data, it works well.
When I add new data, it works two time and don't add new data.
registerStaff = async (model) => {
if (!firebase.apps.length) {
return false;
}
if (model) {
return new Promise((resolve, reject) => {
this.db.ref("tbl_phone_number").on("value", async (snapshot) => {
if (snapshot.exists()) {
const samePhone = await _.filter(snapshot.val(), (o) => {
`find same phone number`;
return o.phone.toString() === model.phone.toString();
});
console.log("checking...", snapshot.val());
if (samePhone.length > 0) {
`checking samephone Number`;
console.log("exist...");
`if exist, return error`;
resolve({
type: "phone",
message: "The phone number is already used.",
});
} else {
`If there is no, add new phone number`;
const newPostKey = this.db
.ref()
.child("tbl_phone_number")
.push().key;
this.db
.ref(`tbl_phone_number/${newPostKey}`)
.set({ phone: model.phone, type: model.type })
.then(() => {
console.log("making...==>");
resolve({
type: "success",
message: "Successfully registered.",
});
})
.catch((err) => {
resolve({
type: "phone",
message: "Sorry. Something went wrong",
});
});
}
}
});
});
}
};
//console log result checking... checking... exist... checking... exist... making...
I found solution to solve this problem in using function "on" and "once".
'on' function of firebase is used to keep data from firebase and add new data automatically when I add new data.
'once' function of firebase is used to change data only once.
Therefore in above question, if I use 'once' function instead of 'on', it will work well.
Good luck.
A fews weeks ago I have asked a first question about this topic Catch errors when using async/await
My register function was getting pretty long and some parts of it are reusable for other controller functions so I decided to decompose the code into various reusable functions.
const findOrCreateEntity = async (req, next) => {
let entity, entity_type;
if(req.body.enterprise_number) {
entity = await Company.findOne({ enterprise_number: req.body.enterprise_number });
entity_type = 'Company';
if(!entity) {
return next(new ErrorResponse('Company not found', `We were unable to find the company for the given enterprise number ${req.body.enterprise_number}`, 400, 'company_not_found'));
}
} else {
entity = await PrivatePerson.findOne({ email: req.body.email });
entity_type = 'PrivatePerson';
if(!entity) {
[entity] = await PrivatePerson.create([{
_id: mongoose.Types.ObjectId().toHexString(),
...req.body,
source: req.app
}], { session: req.session })
}
}
return {
entity, entity_type
};
};
const createEnergyMandate = async (req, entity_id, entity_type, next) => {
let energy_mandate;
energy_mandate = await EnergyMandate.findOne({ entity: entity_id });
if(energy_mandate) {
return next(new ErrorResponse('Email already in use', `An account with the provided email address already exists. Please login instead.`, 409, 'email_already_in_use'));
}
let preferences = {
push_notifications: true,
sms_notifications: true,
email_notifications: true
}
if(!req.app.notifications) {
preferences = {
push_notifications: false,
sms_notifications: false,
email_notifications: false
}
}
[energy_mandate] = await EnergyMandate.create([{
_id: mongoose.Types.ObjectId().toHexString(),
entity: entity_id,
start_date: Date.now(),
status: 'edit_customer',
legal: {
privacy_policy_version: '2.1',
privacy_policy_signed: true,
privacy_policy_signed_at: Date.now(),
privacy_policy_signed_from_ip: req.user_ip,
general_conditions_version: '2.1',
general_conditions_signed: true,
general_conditions_signed_at: Date.now(),
general_conditions_signed_from_ip: req.user_ip
},
preferences: preferences,
sales_channel: req.app.channel
}], { session: req.session });
return energy_mandate;
}
export const register = asyncHandler(async (req, res, next) => {
const session = await mongoose.startSession();
session.startTransaction();
req.session = session;
try {
// Find or create entity
const { entity, entity_type } = await findOrCreateEntity(req, next);
// Create energy mandate
const energy_mandate = await createEnergyMandate(req, entity._id, entity_type, next);
// Create auth0 user unless req.app.user is false
if(!req.app.user) {
auth0_user = await createAuth0User(req, next);
auth0_uid = auth0_user.user_id;
} else {
auth0_uid = req.user.uid;
}
return res.status(200).json({
message: 'User registered',
data: {
user_id: auth0_uid
}
});
} catch (error) {
await session.abortTransaction();
if(auth0_user) {
await auth0ManagementClient.deleteUser({
id: auth0_user.user_id
});
}
next(error);
} finally {
session.endSession();
}
});
As you can see there are some points where I use an ErrorResponse to write very specific errors like return next(new ErrorResponse('Company not found', `We were unable to find the company for the given enterprise number ${req.body.enterprise_number}, 400, 'company_not_found'));`
Based on the previous question I asked and the answer I received, I should wrap each function in the register controller function into it's own try/catch block so the register function stops executing.
I've got two question about that:
How can I return the specific errors like return next(new ErrorResponse('Company not found', `We were unable to find the company for the given enterprise number ${req.body.enterprise_number}`, 400, 'company_not_found')); to the client but also calling the catch block.
Will the 'global catch' where the transaction is being aborted still be hit if I have all these smaller try/catch blocks? In other words will the main catch block of a try/catch be executed if the try/catch block has one or multiple try/catch blocks?
I am a beginner with sequelize and cannot get the transactions to work. Documentation is unclear and makes the following example not able to adapt to my requirements.
return sequelize.transaction(t => {
// chain all your queries here. make sure you return them.
return User.create({
firstName: 'Abraham',
lastName: 'Lincoln'
}, {transaction: t}).then(user => {
return user.setShooter({
firstName: 'John',
lastName: 'Boothe'
}, {transaction: t});
});
}).then(result => {
// Transaction has been committed
// result is whatever the result of the promise chain returned to the transaction callback
}).catch(err => {
// Transaction has been rolled back
// err is whatever rejected the promise chain returned to the transaction callback
});
First I have to insert a tuple in 'Conto', then insert another tuple in 'Preferenze' and finally based on the 'tipo' attribute insert a tuple in 'ContoPersonale' or 'ContoAziendale'.
If only one of these queries fails, the transaction must make a total rollback, commit.
The queries are:
Conto.create({
id: nextId(),
mail: reg.email,
password: reg.password,
tipo: reg.tipo,
telefono: reg.telefono,
idTelegram: reg.telegram,
saldo: saldoIniziale,
iban: generaIBAN()
})
Preferenze.create({
refConto: 68541
})
if (tipo == 0) {
ContoPersonale.create({
nomeint: reg.nome,
cognomeint: reg.cognome,
dataN: reg.datan,
cf: reg.cf,
refConto: nextId()
})
}
else if (tipo == 1) {
ContoAziendale.create({
pIva: reg.piva,
ragioneSociale: reg.ragsoc,
refConto: nextId()
})
}
With a transaction you pass it to each query you want to be part of the transaction, and then call transaction.commit() when you finished, or transaction.rollback() to roll back all the changes. This can be done use thenables however it is clearer when using async/await.
Since none of your queries depend on each other you can also make them concurrently using Promise.all().
thenables (with auto commit)
sequelize.transaction((transaction) => {
// execute all queries, pass in transaction
return Promise.all([
Conto.create({
id: nextId(),
mail: reg.email,
password: reg.password,
tipo: reg.tipo,
telefono: reg.telefono,
idTelegram: reg.telegram,
saldo: saldoIniziale,
iban: generaIBAN()
}, { transaction }),
Preferenze.create({
refConto: 68541
}, { transaction }),
// this query is determined by "tipo"
tipo === 0
? ContoPersonale.create({
nomeint: reg.nome,
cognomeint: reg.cognome,
dataN: reg.datan,
cf: reg.cf,
refConto: nextId()
}, { transaction })
: ContoAziendale.create({
pIva: reg.piva,
ragioneSociale: reg.ragsoc,
refConto: nextId()
}, { transaction })
]);
// if we get here it will auto commit
// if there is an error it with automatically roll back.
})
.then(() => {
console.log('queries ran successfully');
})
.catch((err) => {
console.log('queries failed', err);
});
async/await
let transaction;
try {
// start a new transaction
transaction = await sequelize.transaction();
// run queries, pass in transaction
await Promise.all([
Conto.create({
id: nextId(),
mail: reg.email,
password: reg.password,
tipo: reg.tipo,
telefono: reg.telefono,
idTelegram: reg.telegram,
saldo: saldoIniziale,
iban: generaIBAN()
}, { transaction }),
Preferenze.create({
refConto: 68541
}, { transaction }),
// this query is determined by "tipo"
tipo === 0
? ContoPersonale.create({
nomeint: reg.nome,
cognomeint: reg.cognome,
dataN: reg.datan,
cf: reg.cf,
refConto: nextId()
}, { transaction })
: ContoAziendale.create({
pIva: reg.piva,
ragioneSociale: reg.ragsoc,
refConto: nextId()
}, { transaction })
]);
// if we get here they ran successfully, so...
await transaction.commit();
} catch (err) {
// if we got an error and we created the transaction, roll it back
if (transaction) {
await transaction.rollback();
}
console.log('Err', err);
}
I'm having severe doubts that the code I'm writing is an efficient/best way to achieve my goal.
I have a promise which makes an SQL query, after it's completed I loop through an array and it's sub arrays+objects. Even if any of the subloops fail for any specific reason I want the inner loops to continue executing until the entire array has been looped through. Right now I have a "try/catch" hell which I doubt is the correct way to do this. I should however say that it works as expected, but how bad code is it?
new Promise((resolve, reject) => {
sqlConnection.execute(
'INSERT INTO pms (userId, message, conversationId) VALUES (?, ?, ?)',
[userid, receivedMsg, receivedConvId],
function(err, results) {
if (err) throw err;
resolve("DEBUG: PM from "+username+" into conv "+receivedConvId+" was sucessfully inserted to DB");
}
);
}).then(() => {
users.forEach(function(userobj, i, arr) {
try {
if (userobj.memberof.includes(receivedConvId)) {
let rcptUsername = userobj.username;
let rcptUserid = userobj.userid;
debug(rcptUsername+" is member of the group "+receivedConvId);
Object.keys(userobj.sessions).forEach(function(session) {
try {
userobj.sessions[session].forEach(function(wsConn) {
try {
debug("DEBUG: Broadcasting message to "+rcptUsername+" for connections inside session "+session);
wsConn.send(JSON.stringify(msgToSend));
} catch(err) {
errorHandler(err);
}
});
} catch(err) {
errorHandler(err);
}
});
}
} catch(err) {
errorHandler(err);
}
});
}).catch((err) => {
debug(err);
}).then(() => {
debug("INFO: Message broadcast finished");
});
The array I'm looping through could look like this:
[
{ username: 'Root',
userid: '1',
memberof: [ 1, 2, 3 ],
sessions:
{
pvkjhkjhkj21kj1hes5: [Array],
'4duihy21hkk1jhhbbu52': [Array]
}
},
{
username: 'Admin',
userid: '2',
memberof: [ 1, 2, 4 ],
sessions:
{
cg2iouoiuiou111uuok7: [Array],
sl1l3k4ljkjlkmmmmkllkl: [Array]
}
}
]
Grateful for any advice.
Assuming wsConn is a https://github.com/websockets/ws websocket - then the code you are using will only ever "detect" immediate errors anyway - any socket write failures will not be caught
You'll also be outputting "INFO: Message broadcast finished" before any of the wsConn.send have finished - because it's asynchronous
Fortunately .send has a callback, which is called back on errors or success once the send has completed - this solves both issues
Using promises is a good idea, except you haven't used promises for anything but the initial SQL execute, which is why you've ended up in nesting hell
I'm fairly confident (without your full code I can't be sure) that the following code will not only run, it has far less nesting
new Promise((resolve, reject) => {
sqlConnection.execute(
'INSERT INTO pms (userId, message, conversationId) VALUES (?, ?, ?)',
[userid, receivedMsg, receivedConvId],
(err, results) => {
if (err) {
return reject(err);
}
resolve("DEBUG: PM from " + username + " into conv " + receivedConvId + " was sucessfully inserted to DB");
}
);
}).then(() => {
const allConnectionsArray = users
.filter(({memberof}) => memberof.includes(receivedConvId)) // filter out any userobj we aren't going to send to
.map(({rcptUsername, rcptUserid, sessions}) => {
debug(rcptUsername + " is member of the group " + receivedConvId);
const userSessionsArray = Object.entries(sessions).map(([session, value]) => {
return value.map((wsConn, index) => {
return { wsConn, rcptUserid, rcptUsername, session, index };
})
});
return [].concat(...userSessionsArray); // flatten the array
});
const promises = [].concat(...allConnectionsArray) // flatten the array
.map(({ wsConn, rcptUserid, rcptUsername, session, index }) => {
debug("DEBUG: Broadcasting message to " + rcptUsername + " for connections inside session " + session);
return new Promise((resolve) => {
wsConn.send(JSON.stringify(msgToSend), err => {
if (err) {
return resolve({ rcptUserid, rcptUsername, session, index, err });
}
resolve({ rcptUserid, rcptUsername, session, index, err: false });
});
});
});
return Promise.all(promises);
}).then(results => {
/* results is an array of {
rcptUserid
rcptUsername
session
index //(index is the ordinal position in user.sessions array
err //(===false if success)
}
*/
debug("INFO: Message broadcast finished");
}).catch(error => {
// the only error caught here would be in the `return reject(err);` in the sql execute,
// because any failure in wsConn.send is a resolved promise (with an error property)
// unless I'm not seeing something obvious, they are the only possible places an error could be thrown anyway
});
If I need to perform two or three different operations on a few collections, is there a better way than chaining together find/update operations? For example:
db.collection('contactinfos').findOneAndUpdate(
{ _id: ObjectID(contactID) },
{ $set: { sharedWith } }
).then(response => {
db.collection('users').update(
{ _id: { $in: sharedWith.map(id => ObjectID(id)) } },
{ $addToSet: { hasAccessTo: contactID } },
{ multi: true }
).then(response => {
db.collection('users').update(
{ _id: { $in: notSharedWith.map(id => ObjectID(id)) } },
{ $pull: { hasAccessTo: contactID } },
{ multi: true }
).then(response => {
return res.send({ success: true });
}).catch(err => {
logger.error(`in updating sharing permissions for ${contactID} by user ${_id}`, err);
return res.status(400).send({ reason: 'unknown' });
});
}).catch(err => {
logger.error(`in updating sharing permissions for ${contactID} by user ${_id}`, err);
return res.status(400).send({ reason: 'unknown' });
});
}).catch(err => {
logger.error(`in updating sharing permissions for ${contactID} by user ${_id}`, err);
return res.status(400).send({ reason: 'unknown' });
});
It just seems messy and there has to be some better way of doing it. Furthermore, if there is an error after the first findOneAndUpdate that prevents the other updates from running, then there will be inconsistent data across documents. The documents contain ID references to other documents for faster lookup.
Also, is there a way to catch all errors within a chain of promises?
From your callback hell I can see you do not use response argument of .then() method anywhere. If you do not need results of one query to perform another, consider using Promise.all() method:
const updateContactInfo = db.collection('contactinfos')
.findOneAndUpdate(
{ _id: ObjectID(contactID) },
{ $set: { sharedWith } }
);
const updateUsers = db.collection('users')
.update(
{ _id: { $in: sharedWith.map(id => ObjectID(id)) } }, //hint: use .map(ObjectId) instead.
{ $addToSet: { hasAccessTo: contactID } },
{ multi: true }
);
const updateUsers2 = db.collection('users')
.update(
{ _id: { $in: notSharedWith.map(id => ObjectID(id)) } }, //hint: use .map(ObjectId) instead.
{ $pull: { hasAccessTo: contactID } },
{ multi: true }
);
Promise
.all([updateContactInfo, updateUsers, updateUsers2])
.then((values) => {
const updateContactInfoResult = values[0];
const updateUsersResult = values[1];
const updateUsers2Result = values[2];
return res.send({ success: true });
})
.catch((reason) => {
logger.error(`msg`, reason);
return res.status(400).send({ reason: 'unknown' });
});
Promise.all() will continue executing following .then() only if all the promises do resolve, otherwise it'll fall into the .catch() method. As of error handling, you can easily chain multiple .catch() methods, which is nicely explained here.
If you cannot have any data inconsistency, either:
Get some SQL database with transactions (easier solution)
Look into MongoDB Two-Phase Commit
And if it is acceptable to happen, let's say once per 1kk times, do include checking it's consistency within your app's logic.