I'm trying to push data to a nested array in mongodb. I'm using mongoose as well.
This is just mock code to see if i can get it working:
User model:
import mongoose from "mongoose";
const CoinSchema = new mongoose.Schema({
coinID: { type: String },
});
const CoinsSchema = new mongoose.Schema({
coin: [CoinSchema],
});
const WatchlistSchema = new mongoose.Schema({
watchlistName: { type: String },
coins: [CoinsSchema],
});
const NameSchema = new mongoose.Schema({
firstName: { type: String },
lastName: { type: String },
username: { type: String },
});
const UserSchema = new mongoose.Schema({
name: [NameSchema],
watchlists: [WatchlistSchema],
test: String,
});
const User = mongoose.model("User", UserSchema);
export default User;
route:
fastify.put("/:id", async (request, reply) => {
try {
const { id } = request.params;
const newCoin = request.body;
const updatedUser = await User.findByIdAndUpdate(id, {
$push: { "watchlists[0].coins[0].coin": newCoin },
});
await updatedUser.save();
// console.dir(updatedUser, { depth: null });
reply.status(201).send(updatedUser);
} catch (error) {
reply.status(500).send("could not add to list");
}
});
request.body // "coinID": "test"
I've tried a lot of different ways to push this data but still no luck. I still get 201 status codes in my terminal which indicates something has been pushed to the DB, but when I check nothing new is there.
Whats the correct way to target nested arrays and push data to them?
It's not perfect but you could get the user document, update the user's watchlist, and then save the updated watchlist like so:
fastify.put("/:id", async (request, reply) => {
try {
const { id } = request.params;
const newCoin = request.body;
// get the user
let user = await User.findById(id);
// push the new coin to the User's watchlist
user.watchlists[0].coins[0].coin.push(newCoin);
//update the user document
const updatedUser = await User.findOneAndUpdate({ _id: id },
{
watchlists: user.watchlists,
},
{
new: true,
useFindAndModify: false
}
);
reply.status(201).send(updatedUser);
} catch (error) {
reply.status(500).send("could not add to list");
}
});
const updateTask = async (req, res) => {
const { id } = req.params;
let update = {};
if (req.body.taskTitle) update.taskTitle = req.body.taskTitle;
if (req.body.taskContent) update.taskContent = req.body.taskContent;
if (req.body.role) update.role = req.body.role;
let task = await Task.updateOne(
{ taskId: id },
{
$set: {
update,
},
},
{ runValidators: true }
);
};
This is my code to update my data in database
as I am trying to update single single data or key if I want to update single data but it's not updating any thing i don't know where its not working as i tried to console data data come perfectly
const updateTask = async (req, res) => {
const { id } = req.params;
let update = {};
if (req.body.taskTitle) update.taskTitle = req.body.taskTitle;
if (req.body.taskContent) update.taskContent = req.body.taskContent;
if (req.body.role) update.role = req.body.role;
let task = await Task.updateOne(
{ taskId: id },
{
$set: update,
},
{ runValidators: true }
);
};
all you have to do was remove curly brackets and it will work like a charm $set : update
what the variable Task is ? And I’m sorry, but I didn’t really understand your question, so could you rephrase it more clearly ?
If not, be careful, users might inject things into your database. You should use the mongo-sanitize script.
Like that :
And check if your id is in the right form. If in your database it is not in bjson with MongoId. If this is the case, do not hesitate to convert your id as realized below.
const sanitize = require('mongo-sanitize');
const mongo = require('mongodb');
const updateTask = async (req, res) => {
const { id } = req.params;
let update = {};
if (req.body.taskTitle) update.taskTitle = sanitize(req.body.taskTitle);
if (req.body.taskContent) update.taskContent = sanitize(req.body.taskContent);
if (req.body.role) update.role = sanitize(req.body.role);
let task = await Task.updateOne(
{ taskId: mongo.ObjectId(sanitize(id)) }, // You can remove the mongo.ObjectId if your id is not a objectid.
{
$set: {
update,
},
},
{ runValidators: true }
);
};
I have a cloud function receiving a json string in a pubsub topic.
The goal is to extracts some data into a new json string.
Next parse it as JSONL.
And finally stream it to Google Cloud Storage.
I notice that sometimes the files seem to contain data and sometimes they do not.
The pubsub is working fine and data is coming into this cloud function just fine.
I tried adding some async awaits where I seem it might fit but I am afraid it has do to with the bufferstream. Both topics on where I have trouble getting my head around.
What could be the issue?
const stream = require('stream');
const { Storage } = require('#google-cloud/storage');
// Initiate the source
const bufferStream = new stream.PassThrough();
// Creates a client
const storage = new Storage();
// save stream to bucket
const toBucket = (message, filename) => {
// Write your buffer
bufferStream.end(Buffer.from(message));
const myBucket = storage.bucket(process.env.BUCKET);
const file = myBucket.file(filename);
// Pipe the 'bufferStream' into a 'file.createWriteStream' method.
bufferStream.pipe(file.createWriteStream({
validation: 'md5',
}))
.on('error', (err) => { console.error(err); })
.on('finish', () => {
// The file upload is complete.
console.log(`${filename} is uploaded`);
});
};
// extract correct fields
const extract = (entry) => ({
id: entry.id,
status: entry.status,
date_created: entry.date_created,
discount_total: entry.discount_total,
discount_tax: entry.discount_tax,
shipping_total: entry.shipping_total,
shipping_tax: entry.shipping_tax,
total: entry.total,
total_tax: entry.total_tax,
customer_id: entry.customer_id,
payment_method: entry.payment_method,
payment_method_title: entry.payment_method_title,
transaction_id: entry.transaction_id,
date_completed: entry.date_completed,
billing_city: entry.billing.city,
billing_state: entry.billing.state,
billing_postcode: entry.billing.postcode,
coupon_lines_id: entry.coupon_lines.id,
coupon_lines_code: entry.coupon_lines.code,
coupon_lines_discount: entry.coupon_lines.discount,
coupon_lines_discount_tax: entry.coupon_lines.discount_tax,
});
// format json to jsonl
const format = async (message) => {
let jsonl;
try {
// extract only the necessary
const jsonMessage = await JSON.parse(message);
const rows = await jsonMessage.map((row) => {
const extractedRow = extract(row);
return `${JSON.stringify(extractedRow)}\n`;
});
// join all lines as one string with no join symbol
jsonl = rows.join('');
console.log(jsonl);
} catch (e) {
console.error('jsonl conversion failed');
}
return jsonl;
};
exports.jsonToBq = async (event, context) => {
const message = Buffer.from(event.data, 'base64').toString();
const { filename } = event.attributes;
console.log(filename);
const jsonl = await format(message, filename);
toBucket(jsonl, filename);
};
it's fixed by moving the bufferstream const into the tobucket function.
It seems im using async wrong, can anybody spot what I am doing wrong?
This is the function I am waiting on:
//~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
export async function firebaseAcceptTradeOffer(tradeOfferID, userData) {
var tradeInstanceID;
var senderID;
var receiverID;
var senderItemsTemp;
var receiverItemsTemp;
var response;
var tradeOffer = db.collection("tradeOffers").doc(tradeOfferID);
return tradeOffer
.get()
.then((doc) => {
senderItemsTemp = doc.data().sendersItems;
receiverItemsTemp = doc.data().receiversItems;
senderID = doc.data().senderID;
receiverID = doc.data().receiverID;
})
.then(() => {
var itemInTrade = false;
senderItemsTemp.forEach((item) => {
db.collection("listings")
.doc(item.itemID)
.get()
.then((doc) => {
if (doc.data().status !== "listed") {
itemInTrade = true;
}
})
.then(() => {
receiverItemsTemp.forEach((item) => {
db.collection("listings")
.doc(item.itemID)
.get()
.then((doc) => {
if (doc.data().status !== "listed") {
itemInTrade = true;
}
})
.then(() => {
if (itemInTrade) {
tradeOffer.update({
status: "declined",
});
return false;
} else {
db.collection("trades")
.add({
tradeOfferID: tradeOfferID,
senderTradeStatus: {
created: true,
sentToSeekio: "current",
inspection: false,
sentToPartner: false,
},
receiverTradeStatus: {
created: true,
sentToSeekio: "current",
inspection: false,
sentToPartner: false,
},
postagePhotos: [],
inspectionPhotos: [],
senderPaid: false,
receiverPaid: false,
senderUploadedProof: false,
receiverUploadedProof: false,
senderID: senderID,
receiverID: receiverID,
messages: [
{
message: `Trade created. A representative, will message this chat shortly with instructions and postage address. If you would like more information about the trading process, head to seekio.io/help. Thank you for using Seekio!`,
sender: "System",
timestamp: firebase.firestore.Timestamp.fromDate(
new Date()
),
},
],
})
.then((docRef) => {
tradeInstanceID = docRef.id;
tradeOffer
.set(
{
status: "accepted",
tradeInstanceID: docRef.id,
},
{ merge: true }
)
.then(() => {
var receiver = db.collection("users").doc(senderID);
var notification = {
from: auth.currentUser.uid,
fromUsername: userData.username,
type: "tradeOfferAccepted",
time: firebase.firestore.Timestamp.fromDate(
new Date()
),
seen: false,
};
receiver
.update({
notifications: firebase.firestore.FieldValue.arrayUnion(
notification
),
})
.then(() => {
response = {
sendersItems: senderItemsTemp,
receiversItems: receiverItemsTemp,
};
return response;
});
});
})
.catch((err) => console.log(err));
}
});
});
});
});
});
}
And here is where I am calling it:
//~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
async function acceptTradeOffer() {
var tradeOfferID = currentTradeFocus;
var senderID = "";
setLoading("loading");
if (userData !== null && tradeOfferID !== "") {
const response = await firebaseAcceptTradeOffer(
currentTradeFocus,
userData
);
console.log(
"RESPONSE FROM FIREBASE SERVICE>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>: ",
response
);
if (!response) {
setErrorMsg("One of the selected items is no longer available.");
} else if (
response.sendersItems !== null &&
response.receiversItems !== null
) {
setSenderItems(response.sendersItems);
setReceiverItems(response.receiversItems);
toggleConfirmScreen("cancel");
setLoading("idle");
setItemsSet(true);
}
fetch(
"https://europe-west2-seekio-86408.cloudfunctions.net/sendMail?type=tradeUpdate&userID=" +
senderID
).catch((err) => {
console.log(err);
setLoading("idle");
});
}
}
So basically I want to go and check if any of the items in this 'trade' are not equal to 'listed' (which means they are not available, I want to return false, if not, then I return the array of items so the trade can continue.
EDIT: I've tried to rejig it all and it's half working. A top level look at what I am trying to do:
User wants to accept a trade offer for some items >
Check through all items to make sure they are available and not sold >
If so, accept the trade >
Then once its accepted, go and cancel all remaining trade offers that include items from this accepted trade, cause they are not available anymore.
//~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
export async function firebaseAcceptTradeOffer(tradeOfferID, userData) {
console.log(
"----- starting firebaseAcceptTradeOffer--------- ",
unavailableItem
);
//==============
var tradeInstanceID;
var senderID;
var receiverID;
var senderItemsTemp;
var receiverItemsTemp;
var unavailableItem = false;
var response;
var itemsArray;
var notListed = false;
//==============
var tradeOffer = db.collection("tradeOffers").doc(tradeOfferID);
unavailableItem = tradeOffer
.get()
.then((doc) => {
senderID = doc.data().senderID;
receiverID = doc.data().receiverID;
itemsArray = doc.data().sendersItems.concat(doc.data().receiversItems);
})
.then(() => {
itemsArray.forEach((item) => {
db.collection("listings")
.doc(item.itemID)
.get()
.then((doc) => {
if (doc.data().status !== "listed") {
notListed = true;
}
});
});
})
.then(() => {
return notListed;
});
console.log(
"-----unavailableItem at the end of method --------- ",
unavailableItem
);
//^^^^^^^^^^ here i am getting a promise result of false (which is correct) but HOW CAN I ACCESS IT
if (unavailableItem) {
tradeOffer.update({
status: "declined",
});
return false;
} else {
response = await createTrade(
tradeOffer,
tradeOfferID,
senderID,
receiverID,
userData.username
);
console.log("response from createTrade", response);
return response;
}
}
I am getting a promise object back with the value false above. False is correct value I am expecting, but how can I access it? its in the form of a promise object?
I have some time on my hands, so let's break this down.
Notes on Variables
If you aren't using TypeScript (and even if you are), I highly recommend inserting the type into the name of your variables.
db # ✔ by convention, either firebase.database() or firebase.firestore()
tradeOffer # ❓ type unclear, could be a number, an object, a string, etc
tradeOfferDocRef # ✔ a DocumentReference
trades # ❓ type unclear, plural implies a collection of some sort
tradesColRef # ✔ a CollectionReference
You may also encounter these:
doc # ❓ by convention, a DocumentSnapshot, but with unknown data
tradeDoc # ✔ implies a DocumentSnapshot<TradeData> (DocumentSnapshot containing trade data)
When using just doc, you need to look around where its used for context on what this DocumentSnapshot contains.
db.collection('trades').doc(tradeOfferID).get()
.then((doc) => { // contents implied to be TradeData
const data = doc.data();
});
// or
tradeDocRef.get()
.then((doc) => { // contents implied to be TradeData
const data = doc.data();
});
You should rename doc as appropriate, especially when using async/await syntax, so you don't end up in situations like:
const doc = await db.collection('trades').doc(tradeOfferID).get();
/* ... many lines ... */
const senderID = doc.get("senderID"); // what was doc again?
As you've tagged reactjs in your question, this implies you are using modern JavaScript.
Ditch any use of var and replace it with the block-scoped versions: const (prevents reassigning the variable) or let (similar to var, but not quite). These are safer and prevents the chances of accidentally overwriting something you shouldn't.
You can also make use of Object destructuring to assign your variables.
const senderID = doc.data().senderID;
const receiverID = doc.data().receiverID;
const itemsArray = doc.data().sendersItems.concat(doc.data().receiversItems);
can become:
const { senderID, receiverID, sendersItems, receiversItems } = doc.data();
const itemsArray = sendersItems.concat(receiversItems);
If you ever need only a single property out of a document, you should use DocumentSnapshot#get() instead of DocumentSnapshot#data() so it will parse only the field you want instead of the whole document's data.
function getUserAddress(uid) {
return firebase.firestore()
.collection('users')
.doc(uid)
.get()
.then(userDoc => userDoc.get("address")); // skips username, email, phone, etc
}
Notes on Promises
var senderID;
var receiverID;
var itemsArray;
tradeOfferDocRef
.get()
.then((doc) => {
senderID = doc.data().senderID;
receiverID = doc.data().receiverID;
itemsArray = doc.data().sendersItems.concat(doc.data().receiversItems);
})
.then(() => {
/* use results from above */
});
While the above code block functions as intended, when you have many of these variables like this as you do, it becomes unclear when and where they are set.
It also leads to problems like this where you think the variable has a value:
var senderID;
var receiverID;
var itemsArray;
tradeOfferDocRef
.get()
.then((doc) => {
// this line runs after the line below
senderID = doc.data().senderID;
receiverID = doc.data().receiverID;
itemsArray = doc.data().sendersItems.concat(doc.data().receiversItems);
});
// this line before the line above
console.log(senderID); // will always log "undefined"
This can be avoided in one of three ways:
Returning data to pass through to the next handler (you wouldn't use this for this example, only if the next then() handler is elsewhere):
tradeOfferDocRef
.get()
.then((doc) => {
const { senderID, receiverID, sendersItems, receiversItems } = doc.data();
const itemsArray = sendersItems.concat(receiversItems);
return { senderID, receiverID, itemsArray }; // pass to next step
})
.then((neededData) =>
/* use neededData.senderID, neededData.receiverID, etc */
});
Using the data within the same handler:
tradeOfferDocRef
.get()
.then((doc) => {
const { senderID, receiverID, sendersItems, receiversItems } = doc.data();
const itemsArray = sendersItems.concat(receiversItems);
/* use results from above */
});
Using async-await syntax:
const tradeDoc = await tradeOfferDocRef.get();
const { senderID, receiverID, sendersItems, receiversItems } = tradeDoc.data();
const itemsArray = sendersItems.concat(receiversItems);
/* use results from above */
Writing to Firestore
Your current code consists of the following steps:
1. Get the trade offer document</li>
2. If successful, pull out the sender and receiver's IDs, along with any items in the trade
3. If successful, do the following for each item in the sender items array:
a) Check if any of the sender's items are unavailable</li>
b) If successful, do the following for each item in the receiver items array:
- If **any item** was unavailable prior to this, decline the trade & return `false`.
- If all items **so far** are available, do the following:
a) Create a document containing information about the trade with the needed data
b) If successful, edit the trade offer document to accept it
c) If successful, create a notification for the receiver
d) If successful, return the traded items
e) If any of a) to d) fail, log the error and return `undefined` instead
4. Return `undefined`
In the above steps, you can see some problems with your promise chaining. But aside from that, you can also see that you create and edit documents one-by-one instead of all-at-once ("atomically"). If any of these writes were to fail, your database ends up in an unknown state. As an example, you could have created and accepted a trade, but failed to create the notification.
To atomically write to your database, you need to use a batched write where you bundle a bunch of changes together and then send them off to Firestore. If any of them were to fail, no data is changed in the database.
Next, you store a user's notifications inside of their user document. For a small number of notifications this is fine, but do you need to download all of those notifications if you wanted to pull just an address or phone number like in the example in the above section? I recommend splitting them out into their own document (such as /users/{someUserId}/metadata/notifications), but ideally their own collection (such as /users/{someUserId}/notifications/{someNotificationID}). By placing them in their own collection, you can query them and use QuerySnapshot#docChanges to synchronize changes and use Cloud Firestore triggers to send push notifications.
Refactored Function
1. Get the trade offer document</li>
2. Once the retrieved, do the following depending on the result:
- If failed or empty, return an error
- If successful, do the following:
a) Pull out the sender and receiver's IDs, along with any items in the trade.
b) For each item in the trade, check if any are unavailable and once the check has completed, do the following depending on the result:
- If any item is unavailable, do the following:
a) Decline the trade
b) Return the list of unavailable items
- If all items are available, do the following:
a) Create a new write batch containing:
- Create a document about the trade
- Edit the trade offer document to accept it
- Create a notification for the receiver
b) Commit the write batch to Firestore
c) Once the commit has completed, do the following depending on the result:
- If failed, return an error
- If successful, return the traded items and the trade's ID
Because the steps here depend on each other, this is a good candidate to use async/await syntax.
To see this in action, closely study this:
import * as firebase from "firebase-admin";
// insert here: https://gist.github.com/samthecodingman/aea3bc9481bbab0a7fbc72069940e527
async function firebaseAcceptTradeOffer(tradeOfferID, userData) {
const tradeOfferDocRef = db.collection("tradeOffers").doc(tradeOfferID);
const tradeDoc = await tradeOfferDocRef.get();
const { senderID, receiverID, sendersItems, receiversItems } =
tradeDoc.data();
const itemsArray = sendersItems.concat(receiversItems);
// TODO: Check if this is an accurate assumption
if (sendersItems.length == 0 || receiversItems.length == 0) {
success: false,
message: "One-sided trades are not permitted",
detail: {
sendersItemsIDs: sendersItems.map(({ itemID }) => itemID),
receiversItemsIDs: receiversItems.map(({ itemID }) => itemID),
},
};
const listingsColQuery = db
.collection("listings")
.where("status", "==", "listed");
const uniqueItemIds = Array.from(
itemsArray.reduce(
(set, { itemID }) => set.add(itemID),
new Set()
)
);
const foundIds = {};
await fetchDocumentsWithId(
listingsColQuery,
uniqueItemIds,
(listingDoc) => {
// if here, listingDoc must exist because we used .where("status") above
foundIds[listingDoc.id] = true;
}
);
const unavailableItemIDs = uniqueItemIds
.filter(id => !foundIds[id]);
if (unavailableItems.length > 0) {
// one or more items are unavailable!
await tradeOfferDocRef.update({
status: "declined",
});
return {
success: false,
message: "Some items were unavailable",
detail: {
unavailableItemIDs,
},
};
}
const tradeDocRef = db.collection("trades").doc();
const tradeInstanceID = tradeDocRef.id;
const batch = db.batch();
batch.set(tradeDocRef, {
tradeOfferID,
senderTradeStatus: {
created: true,
sentToSeekio: "current",
inspection: false,
sentToPartner: false,
},
receiverTradeStatus: {
created: true,
sentToSeekio: "current",
inspection: false,
sentToPartner: false,
},
postagePhotos: [],
inspectionPhotos: [],
senderPaid: false,
receiverPaid: false,
senderUploadedProof: false,
receiverUploadedProof: false,
senderID,
receiverID,
messages: [
{
message: `Trade created. A representative, will message this chat shortly with instructions and postage address. If you would like more information about the trading process, head to seekio.io/help. Thank you for using Seekio!`,
sender: "System",
timestamp: firebase.firestore.Timestamp.fromDate(new Date()),
},
],
});
batch.set(
tradeOfferDocRef,
{
status: "accepted",
tradeInstanceID,
},
{ merge: true }
);
const receiverNotificationRef = db
.collection("users")
.doc(senderID)
.collection("notifications")
.doc();
batch.set(receiverNotificationRef, {
from: auth.currentUser.uid,
fromUsername: userData.username,
type: "tradeOfferAccepted",
time: firebase.firestore.Timestamp.fromDate(new Date()),
seen: false,
});
await batch.commit();
return {
success: true,
message: "Trade accepted",
detail: {
tradeID: tradeInstanceID,
senderItems,
receiversItems,
},
};
}
Usage:
try {
const tradeResult = await firebaseAcceptTradeOffer(someTradeId);
} catch (err) {
// if here, one of the following things happened:
// - syntax error
// - database read/write error
// - database rejected batch write
}
In general, when you are returning a promise where it can't be resolved you must await its result. Additionally, you must be returning a value from within a promise then chain, at minimal the last .then() needs to be returning a value, this can also be done within a .finally() method.
Using Get from any firebase resource, realtime, firestore, and storage are all Async processes and must be awaited. in your case, you are missing an await for the return:
var tradeOffer = db.collection("tradeOffers").doc(tradeOfferID);
return tradeOffer
and you don't appear to be returning anything inside your .then() statements, I would suggest a complete rewrite of what you are trying to so you are returning values as they are needed.