I want to utilize mongoose's withTransaction helper particularly for its ability to automatically retry transient transaction errors. However, it seems that the withTransaction helper is incapable of returning data, which is a problem for me.
I have code that looks like:
import { startSession } from 'mongoose';
async addItem(itemData) {
const session = await startSession();
session.startTransaction();
try {
const item = await new Item({ itemData }).save({ session });
// a bunch of other async operations...
await session.commitTransaction();
session.endSession();
return item;
} catch (error) {
await session.abortTransaction();
session.endSession();
throw error;
}
}
How can I either (1) use the withTransaction helper but still have this function returning the item as it currently does, or (2) make this function automatically retry on transient transaction errors through some way other than using withTransaction.
This appears to be a known issue in the node driver. Some workarounds are provided in that ticket.
I wrote a simple helper that internally uses withTransaction to solve the problem and make transactions less verbose with mongoose.
After installing mongoose-trx you can simply do:
const transaction = require('mongoose-trx');
const [customer] = await transaction(session => Customer.create([{ name: 'Test' }], { session }));
// do whatever you need to do with the customer then return it
It supports transaction options as well, see the documentation on how to do it.
Related
On the API(GraphQL) - Getting Started documentation here, it says to query your data using the following:
import { API } from 'aws-amplify';
import * as queries from './graphql/queries';
// Simple query
const allTodos = await API.graphql({ query: queries.listTodos });
console.log(allTodos); // result: { "data": { "listTodos": { "items": [/* ..... */] } } }
However, when I try to apply their code to my javascript code it says that it does not recognize the word await. Online it says I can only use the await keyword inside of a async function. When I take the await keyword out, the promise from the query function does not get settled so it returns the promise first before the data.
I tried setting up an async function before, and posted a stackoverflow post about it. The solution got a little messy, and did not quite work for me. So, I am wondering what is the best way to go about querying data using Graphql? And how do I implement that?
await can be used only with in an async context, so ideally what you would have to do is the following
const allTodos = async () => {
const todos = await API.graphql({ query: queries.listTodos });
return todos
}
I have a simple function which deletes a product entry from the database, and now I'm trying to delete the image file of that product as well. I checked the Node.js file system docs and found 2 functions which deal with that - fs.unlink(path, callback) and fs.unlinkSync(path). I understand that the first one is asynchronous and that the second one is synchronous, but I'm still not quite sure which one should I use and why.
module.exports.deleteProduct = async (req, res, next) => {
let productId = req.body.productId
try {
let product = await Product.destroy({
where: {
id: productId
}
})
res.status(200).json({
product: product
})
} catch (e) {
console.log(e)
res.status(500)
}
}
Some code and an idea for you:
As others have already said, async is better than sync, so you won't end up blocking, even though, unless your API volume is extremely high, it probably won't matter, as indicated in another answer.
You can use the fs promises API via
const fs = require('fs').promises; //es5 OR
import { promises as fs } from 'fs'; //es6
to use the async (non-blocking) API with a one-liner await.
Special note: you may not want your API request to fail if you failed to unlink the directory, as you did in fact delete the product from the database.
// make sure you are using the promise API from fs
const fs = require('fs').promises;
module.exports.deleteProduct = async (req, res, next) => {
let productId = req.body.productId
try {
let product = await Product.destroy({
where: {
id: productId
}
})
try {
await fs.unlink('the/path/to/the/product/image');
} catch {
// you may want to handle a failure to delete separately
}
res.status(200).json({product: product})
} catch (e) {
console.log(e)
res.status(500)
}
}
If your server OS is Linux or some other UNIX derivative with a local file system, both .unlinkSync() and .unlink() run quickly: the OS-level unlinking operation is designed to complete quickly and predictably. So, if you use the blocking .unlinkSync() version you won't do much harm, especially if your unlinking is infrequent.
That being said, if you can use the asynchronous version it's a good practice to do so.
It looks like you can; you can call res.status()... from within a callback or after an await.
Don't Block the event loop in Node Js
The synchronous methods blocks the event loop unnecessarily ,which affects your application performance .always use async methods ,wherever possible.
or if you want to use it with awaitoperation (pseudo sync) ,you can do something like below ,by wrapping it within promise
const fs=require("fs");
function unlinkPromise(file)
{
return new Promise((resolve,reject)=>{
fs.unlink(file,(err,data)=>{
if(err)
{
reject(err);
}
resolve(data);
})
})
}
async function data()
{
console.log(await unlinkPromise("file"));
}
I'm reading the docs on Transaction operations, and I figured the t.set() method would work similar to the docReference.set() documented in the the Add data page.
To my surprise, it doesn't:
const newCustomerRef = db.collection('customers').doc();
await db.runTransaction(t => {
const res = t.set(newCustomerRef, formData)
console.log(res)
});
The res object above (return value of t.set()) contains a bunch of props that looks obfuscated, and it doesn't look as if it's intended for you to work with them.
Is there any way to get the ID of the newly created document within a Transaction?
Update
What I'm trying to achieve is to have multiple data operations in 1 go, and have everything reverted back if it fails.
As per Doug answer, if newCustomerRef already contains the ID, it seems what I am missing is to delete it during the catch block in case the transaction fails:
try {
const newCustomerRef = db.collection('customers').doc();
await db.runTransaction(t => {
const res = t.set(newCustomerRef, formData)
console.log(res)
});
} catch (e) {
newCustomerRef.delete()
//...error handling...
}
This is sort of a manual thing to do, feels a little hacky. Is there a way to delete it automatically if the transaction fails?
newCustomerRef already contains the ID. It was generated randomly on the client as soon as doc() was called, before the transaction ever started.
const id = newCustomerRef.id
If a transaction fails for any reason, the database is unchanged.
The operation to add the document is performed in the set(..) call. This means by using set() on the transaction, everything is rolled back should the transaction fail.
This means in the following example
...
await db.runTransaction(t => {
t.set(newCustomerRef, formData)
... do something ...
if (someThingWentWrong) {
throw 'some error'
}
});
Should someThingWentWrong be true, no document will have been added.
here is what I am trying to do with firebase cloud function:
-Listen to any change in one of the documents under 'user' collection.
-Update carbon copies of the userinfo in the relevant documents in both 'comment' and 'post' collections.
Because I will need to query in relevant documents and update them at once, I am writing codes for transaction operations.
Here is the code that I wrote. It returns the error message, 'Function returned undefined, expected Promise or value'.
exports.useInfoUpdate = functions.firestore.document('user/{userid}').onUpdate((change,context) => {
const olduserinfo=change.before.data();
const newuserinfo=change.after.data();
db.runTransaction(t=>{
return t.get(db.collection('comment').where('userinfo','==',olduserinfo))
.then((querysnapshot)=>{
querysnapshot.forEach((doc)=>{
doc.ref.update({userinfo:newuserinfo})
})
})
})
.then(()=>{
db.runTransaction(t=>{
return t.get(db.collection('post').where('userinfo','==',olduserinfo))
.then((querysnapshot)=>{
querysnapshot.forEach((doc)=>{
doc.ref.update({userinfo:newuserinfo})
})
})
})
})
});
I am a bit confused because as far as I know, 'update' method returns a promise? I might be missing something big but I picked up programming only last November, so don't be too harsh. :)
Any advice on how to fix this issue? Thanks!
EDIT:
Building on Renaud's excellent answer, I created the below code in case someone may need it.
The complication with transaction is that the same data may be stored under different indices or in different formats. e.g. The same 'map' variable can be stored under an index in one collection, and as part of an array in another. In this case, each document returned by querying needs different update methods.
I resolved this issue using doc.ref.path, split, and switch methods. This enables application of different update methods based on the collection name. In a nutshell, something like this:
return db.runTransaction(t => {
return t.getAll(...refs)
.then(docs => {
docs.forEach(doc => {
switch (doc.ref.path.split('/')[0]) { //This returns the collection name and switch method assigns a relevant operation to be done.
case 'A':
t = t.update(doc.ref, **do whatever is needed for this collection**)
break;
case 'B':
t = t.update(doc.ref, **do whatever is needed for this collection**)
break;
default:
t = t.update(doc.ref, **do whatever is needed for this collection**)
}
})
})
})
Hope this helps!
Preamble: This is a very interesting use case!!
The problem identified by the error message comes from the fact that you don't return the Promise returned by the runTransaction() method. However there are several other problems in your code.
With the Node.js Server SDK you can indeed pass a query to the transaction's get() method (you cannot with the JavaScript SDK). However, in your case you want to update the documents returned by two queries. You cannot call twice db.runTransaction() because, then, it is not a unique transaction anymore.
So you need to use the getAll() method by passing an unpacked array of DocumentReferences. (Again, note that this getAll() method is only available in the Node.js Server SDK and not in the JavaScript SDK).
The following code will do the trick.
We run the two queries and transform the result in one array of DocumentReferences. Then we call the runTransaction() method and use the spread operator to unpack the array of DocumentReferences and pass it to the getAll() method.
Then we loop over the docs and we chain the calls to the transaction's update() method, since it returns the transaction.
However note that, with this approach, if the results of one of the two original queries change during the transaction, any new or removed documents will not be seen by the transaction.
exports.useInfoUpdate = functions.firestore.document('user/{userid}').onUpdate((change, context) => {
const olduserinfo = change.before.data();
const newuserinfo = change.after.data();
const db = admin.firestore();
const q1 = db.collection('comment').where('userinfo', '==', olduserinfo); // See the remark below: you probably need to use a document field here (e.g. olduserinfo.userinfo)
const q2 = db.collection('post').where('userinfo', '==', olduserinfo);
return Promise.all([q1.get(), q2.get()])
.then(results => {
refs = [];
results.forEach(querySnapshot => {
querySnapshot.forEach(documentSnapshot => {
refs.push(documentSnapshot.ref);
})
});
return db.runTransaction(t => {
return t.getAll(...refs)
.then(docs => {
docs.forEach(doc => {
t = t.update(doc.ref, { userinfo: newuserinfo })
})
})
})
})
});
Two last remarks:
I am not sure that db.collection('comment').where('userinfo', '==', olduserinfo); will be valid as olduserinfo is obtained through change.before.data(). You probably need to specify one field. This is probably the same for newuserinfo.
Note that you cannot do doc.ref.update() in a transaction, you need to call the transaction's update() method, not the one of a DocumentReference.
Have some Firestore .set() functions that are suddenly not working-- they are not doing anything in the then statement and no code after "await" is executing with them if formatted as async/await. No docs are being added to Firestore.
First had this from the Firestore docs:
export function addProfileToDB(advisor, uid, state) {
firebaseDB.collection("users").doc(uid).set(profile)
.then(function(docRef) {
console.log("User added with ID: ", docRef.id) // never fires
})
.catch(function(error) {
console.error(error) // never fires
})
}
Tried a la this answer:
export async function addProfileToDB(profile, uid, state) {
const docRef = firebaseDB.collection("users").doc(uid)
console.log(docRef) // returns correctly
await docRef.set(profile)
console.log('added') // never happens
}
Using add() and read functions in the same setup works fine, but I need to specify the doc id (uid) in this case. Thoughts?
You could wrap the async/await version with a try catch block.
Same for the code which calls addProfileToDB (and you should await it too).
Maybe you'll see something.
If it's a cloud function that trigger addProfileToDB, did you watch logs on the firebase console ?
This ended up being due to a sign out user function that was being called at the same time as the addProfileToDB. Although the user being signed in wasn't required for addProfileToDB, for some reason the interruption caused the set to not go through. I moved the sign out user function to fire later and all is now working as expected. Thank you all for helping!
If set() throw an error, console.log('added') can't be ritched.
My solution consist to catch if there is any error.
Solution
export async function addProfileToDB(profile, uid, state) {
const docRef = firebaseDB.collection("users").doc(uid)
console.log(docRef) // returns correctly
try {
await docRef.set(profile)
console.log('added')
} catch (e) {
console.log(e)
}
}