Creating MongoDB Document from Nested Object - javascript

I'm sure it's a simple fix, but I've been pulling my hair out trying to get the syntax correct for a nested object. I'm trying to use it to create a MongoDB document.
The Mongo documents store conversations between two users. Each message in the conversation is stored in separate MongoDB documents, and the conversation document will reference each message that belongs to it.
Here's the Conversation Schema (which I think is OK)
var ConversationSchema = new mongoose.Schema({
participants: [
{
user1: {
id: {
type: mongoose.Schema.Types.ObjectId,
ref: "User"
},
username: String
},
user2: {
id: {
type: mongoose.Schema.Types.ObjectId,
ref: "User"
},
username: String
},
},
],
started: Number,
messages: [
{
type: mongoose.Schema.Types.ObjectId,
ref: "Message"
}
]
});
And here's one of my many attempts at creating the object to pass into MongoDB.
var conv = {
participants : {
"participants.user1.id" : req.body.senderId,
"participants.user1.username" : req.body.senderName,
"participants.user2.id" : req.body.recipientId,
"participants.user2.username" : req.body.recipientName
},
created : Date.now(),
messages : [] // The message _id is pushed in later.
}
It's the 'participants' bit which is really tripping me up. This data is coming back from the client as it should, but I can't manage to get it into my var conv. What's the correct syntax to create the nested object I need here?
Any guidance would be awesome! Thanks peoples!!

Fixed it! Yep it was just a simple syntax error: here's correct form in case anyone else ends up here.
var conv = {
participants : {
"user1" : {
"id" : req.body.senderId,
"username" : req.body.senderName
},
"user2" : {
"id" : req.body.recipientId,
"username" : req.body.recipientName
}
},
created : Date.now(),
messages : [] // The message _id is pushed in later.
}
Also, pro tip: Go away and do the washing up. Things will be much clearer when you come back to them.

Related

What is the best way to keep track of changes of a document's property in MongoDB?

I would like to know how to keep track of the values of a document in MongoDB.
It's a MongoDB Database with a Node and Express backend.
Say I have a document, which is part of the Patients collection.
{
"_id": "4k2lK49938d82kL",
"firstName": "John",
"objective": "Burn fat"
}
Then I edit the "objective" property, so the document results like this:
{
"_id": "4k2lK49938d82kL",
"firstName": "John",
"objective": "Gain muscle"
}
What's the best/most efficient way to keep track of that change? In other words, I would like to know that the "objective" property had the value "Burn fat" in the past, and access it in the future.
Thanks a lot!
Maintaining/tracking history in the same document is not all recommended. As the document size will keep on increasing leading to
probably if there are too many updates, 16mb document size limit
Performance degrades
Instead, you should maintain a separate collection for history. You might have use hibernates' Javers or envers for auditing for your relational databases. if not you can check how they work. A separate table (xyz_AUD) is maintained for each table (xyz). For each row (with primary key abc) in xyz table, there exist multiple rows in xyz_AUD table, where each row is version of that row.
Moreover, Javers also support MongoDB auditing. If you are using java you can directly use it. No need to write your own logic.
Refer - https://nullbeans.com/auditing-using-spring-boot-mongodb-and-javers/
One more thing, Javers Envers Hibernate are java libraries. But I'm sure for other programming languages also, similar libraries will be present.
There is a mongoose plugin as well -
https://www.npmjs.com/package/mongoose-audit (quite oudated 4 years)
https://github.com/nassor/mongoose-history#readme (better)
Maybe you can change the type of "objective" to array and track the changes in it. the last one of the array is the latest value.
Maintain it as a sub-document like below
{
"_id": "4k2lK49938d82kL",
"firstName": "John",
"objective": {
obj1: "Gain muscle",
obj2: "Burn fat"
}
}
You can also maintain it as an array field but remember, mongodb doesn't allow you to maintain uniqueness in an array field and if you plan to index the "objective" field, you'll have to create a multi key index
I think the simplest solution would be to use and update an array:
const patientSchema = new Schema({
firstName: { type: String, required: true },
lastName: { type: String, required: true },
objective: { type: String, required: true }
notes: [{
date: { type: Date, default: Date.now() },
note: { type: String, required: true }
}],
});
Then when you want to update the objective...
const updatePatientObjective = async (req, res) => {
try {
// check if _id and new objective exist in req.body
const { _id, objective, date } = req.body;
if (!_id || !objective) throw "Unable to update patient's objective.";
// make sure provided _id is valid
const existingPatient = await Patient.findOne({ _id });
if (!existingPatient) throw "Unable to locate that patient.";
// pull out objective as previousObjective
const { objective: previousObjective } = existingPatient;
// update patient's objective while pushing
// the previous objective into the notes sub document
await existingPatient.updateOne({
// update current objective
$set { objective },
// push an object with a date and note (previouseObjective)
// into a notes array
$push: {
notes: {
date,
note: previousObjective
},
},
}),
);
// send back response
res
.status(201)
.json({ message: "Successfully updated your objective!" });
} catch (err) {
return res.status(400).json({ err: err.toString() });
}
};
Document will look like:
firstName: "John",
lastName: "Smith",
objective: "Lose body fat.",
notes: [
{
date: 2019-07-19T17:45:43-07:00,
note: "Gain muscle".
},
{
date: 2019-08-09T12:00:38-07:00,
note: "Work on cardio."
}
{
date: 2019-08-29T19:00:38-07:00,
note: "Become a fullstack web developer."
}
...etc
]
Alternatively, if you're worried about document size, then create a separate schema for patient history and reference the user's id (or just store the patient's _id as a string instead of referencing an ObjectId, whichever you prefer):
const patientHistorySchema = new Schema({
_id: { type: Schema.Types.ObjectId, ref: "Patient", required: true },
objective: { type: String, required: true }
});
Then create a new patient history document when the objective is updated...
PatientHistory.create({ _id, objective: previousObjective });
And if you need to access to the patient history documents...
PatientHistory.find({ _id });

Update an array of objects mongoose

I know that this question might be beginner level but I haven't find anything yet.
I would like to update an array of objects with mongoose. I am interested in updating one object from the users array according to the index.
Usually one user is getting changed at a time.
Here is my schema:
_id: Schema.Types.ObjectId,
name: { type: String, required: true },
gm: {
type: Schema.Types.ObjectId,
ref: 'User',
required: true
},
users: [],
I want to update an object in the users array which is like this:
{
id:"5bcb7c7ff9c5c01b9482d244",
gm:"5bcb7c7ff9c5c01b9482d246",
name:"room 1"
users: [
{
id:"5bcb7c7ff9c5c01b9482d243",
stats:{
power:10,
mobility: 5,
vitality: 20
},
bag:{itemSlot1: "Knife",itemSlot2:"Sword" }
},
{
id:"5bcb7c7ff9c5c01b9482d241",
stats:{
power:10,
mobility: 5,
vitality: 20
},
bag:{itemSlot1: "Knife",itemSlot2:"Sword" }
]
}
I want to perform a patch or a post request to update one user each time from the user array. i am getting the id of the user from req.body to match it with my db.
My request is like this:
I would like to update based on a request like this:
data = {
stats={
power:"10",
vitality:"20"
}
}
Thanks in advance,
Cheers
You can do an update like this:
YourSchema.update({
'users.id': '5bcb7c7ff9c5c01b9482d243'
}, {
$set: {
'users.$.stats': data.stats
}
})
Which would update the first user with id 5bcb7c7ff9c5c01b9482d243 power stats to 20
This is using the update with the $ positional operator to update the element in the array.
Just have it set up in your post/patch request.

Nested array query using mongoose

I'm using mongoose (with nodejs) to make my queries.
I got the following Schemas for my database model (minified ofc):
var HistorySchema = new Schema({
status : String,
time : Date
});
var TaskSchema = new Schema({
game_id : Schema.Types.ObjectId,
history : [HistorySchema]
}, {collection: 'task'});
Now I want to give an example (insertion) for this which I want to use to show my problem and wish:
(Insertion)
{
"_id" : ObjectId("5772ca87439632101510fa6b"),
"history" :
[
{
"status" : "open",
"time" : ISODate("2016-06-25T12:17:46.982Z")
},
{
"status" : "complete",
"time" : ISODate("2016-06-30T12:17:46.982Z")
}
]
}
so far so good... Now I have a given date, in this case:
ISODate("2016-06-28T12:17:46.982Z")
Now I want to get from my collection all TaskSchema objects including matching HistorySchmea objects from the array. So I want to exclude the not matching part in history array.
I tried many things like $pull operation like
db.task.find(
{
"game_id": ObjectId("57711397893a97aa170aa983"),
"history.time":{
$lte: ISODate("2016-06-28T12:17:46.982Z")
}
},{$pull: {
"history": {
time: {
$gte: ISODate("2016-06-28T12:17:46.982Z")
}
}
}
}
but then I get errors like
Unsupported projection option: $pull: { history: { time: { $gte: new Date(1467116266982) } } }
Does anyone know how I could realize this query? I'm working now for days on this and couldn't find any help.
Thanks in advance!
One solution is to use the aggregation framework :
db.task.aggregate([
{$match: {_id : ObjectId("5772ca87439632101510fa6b")}},
{$unwind : "$history"},
{$match :{"history.time" :
{
$lte: ISODate("2016-06-30T12:17:46.982Z"),
$gte: ISODate("2016-06-01T12:17:46.982Z")
}
}
},
{$group:
{
_id:"$_id",
history: { $push: { status: "$history.status", time: "$history.time" } }
}
}
]);

How to improve or avoid find / fetch cycle in meteor's publication?

TL;DR:
Chat is one collection. ChatMess another one that has messages refering to a Chat's _id. How do I get the last messages from a list of chats with the less computation possible ? Here, find / fetch cycle in a loop is way too heavy and long.
I have this publication that is used to return a set of cursor to the user :
The chats sessions he takes part in (from Chat collection)
The last message from each of the chat session referenced in the first cursor (from ChatMess collection)
Currently, the logic is to :
Get the list of chat sessions from the user profile
Find the Chat sessions and loop through it
In the loop, I findOne the last message from this chat session and store its _id in an array. In addition, I store all the other users _ids.
Then, I find the messages which _id match the ones in my array.
Here is my main problem :
Isn't there a way more faster way to get the last messages from each of my chat session ? With that algo, I easily reach the 8000ms of response time, which is a way too heavy computation time, as much of this time is spent to find / fetch the chat messages's _id (cf linked screen from Kadira).
Meteor.publish("publishNewChat", function() {
this.unblock();
// we get a list of chat _id
let chatIdList = _get_all_the_user_chats_ids(this.userId);
if (!chatList)
return ;
// get the chat sessions objects
let chats_cursor = Modules.both.queryGet({
type : 'chat',
method : 'find',
query : { _id: { $in: chatIdList } },
projection : { sort: { _id: 1 }, limit : 1000 }
});
let array_of_fetched_chats = chats_cursor.fetch();
let chat_ids = [];
// and here we loop through the chat documents in order to get the last message that's been attached to each of them
array_of_fetched_chats.forEach(function(e) {
let lastMess = Modules.both.queryGet({
type : 'chatMess',
method : 'findOne',
query : { chatId: e._id },
projection : { sort: { date: -1 } }
});
if (lastMess)
chat_ids.push(lastMess._id);
});
return ([
chats_cursor,
Modules.both.queryGet({
type : 'chatMess',
method : 'find',
query : { _id: { $in: chat_ids } },
projection : { sort: { date: -1 }, limit: 1000 }
})
]);
});
Finally, it also add latence to all my DDP request that follows. I currently use a this.unblock() to avoid that, but I'd prefer not to use it here.
FYI, I have another publish that is updated each time the client change his current active chat session : on the client, routing to a new chat add its _id in a reactive array that update my getChatMess subscription in order to get on the client the messages from every chats the user visited in this since he connected. The goal is obviously to spare the server the sending of every message from every chat session the user have visited in his life.
Unfortunately, I lack ideas to improve that algo without breaking all my chat logic :S. Have you any idea ? How would you do ?
Thanks you.
EDIT: here is a screen from kadira that clearly show the problem :
Have you considered using the reywood/publishComposite package?
With this package you can publish related data in the same method without having to do a bunch of logic to get the correct data published.
The below code should get you started:
Meteor.publishComposite("publishNewChat", function() {
return [{
find:function(){
return Users.find({ _id: this.userId },{fields:{"profile.chat":1}});
},
children:[{
find:function(user){ //this function is passed each user returned from the cursor above.
return UserChats.find({userId:user._id},{fields:{blah:1,blah:1}}); //find the user chats using whatever query
},
children:[
//if there are any children of user chats that you need to publish, do so here...
{
find:function(userchat){
return Chats.find({_id:userchat.chatId})
},
children:[
{
find:function(chat){
return ChatMess.find({chatId:chat._id},{ sort: { date: -1 } });
},
children:[
{
find:function(chatMess){
var uids = _.without(chatMess.participants, this.userId);
return Users.find({_id:{$in:uids}});
}
}
]
}
]
}
]
},
]
}]
This will publish the cursors for all of the documents related to each of the parent documents. It is pretty fast, I use this package on a production platform high traffic and large datasets with no problems. On the client you could then query the documents as normal to get the ones you need to display.
Something like:
Users.findOne({_id:Meteor.userId()});
UserChats.find({userId:Meteor.userId()});
etc...
Here is a solution I developped :
Meteor.publish("publishNewChat", function() {
this.unblock();
let user = Modules.both.queryGet({
type : 'users',
method : 'findOne',
query : { _id: this.userId },
projection : { fields: { "profile.chat": true } }
});
let thisUserschats = tryReach(user, "profile", "chat").value;
if (!thisUserschats)
return ;
thisUserschats = thisUserschats.map(function(e) { return (e.chatId); });
let chats = Modules.both.queryGet({
type : 'chat',
method : 'find',
query : { _id: { $in: thisUserschats } },
projection : { sort : { _id: 1 },
limit : 1000
}
});
let chatArray = chats.fetch(),
uids = cmid = [];
let messages_id_list = [],
i = chatArray.length;
let _parallelQuery = index => {
Meteor.setTimeout(function () {
let tmp = Modules.both.queryGet({
type : 'chatMess',
method : 'find',
query : { chatId: chatArray[index]._id },
projection: { limit: 1, sort: { date: -1 } }
});
tmp.forEach(doc => {
messages_id_list.push((doc && doc._id) ? doc._id : null);
});
}, 1);
}
while (--i >= 0)
_parallelQuery(i);
let cursors = {
chats : chats,
chatMessages : null
}
let interval = Meteor.setInterval(function () {
if (messages_id_list.length === chatArray.length)
{
Meteor.clearInterval(interval);
cursors.chatMessages = Modules.both.queryGet({
type : 'chatMess',
method : 'find',
query : { _id: { $in: messages_id_list } },
projection : { sort: { date: -1 }, limit: 1000 }
});
cursors.chats.observeChanges({
// ...
});
cursors.chatMessages.observeChanges({
// ...
});
self.ready();
self.onStop(() => subHandle.stop(); );
}
}, 10);
});
I used async function with Meteor.setTimeout to parallelize the queries and save an index refering to a chat _id to look for. Then, when a query is finished, I add the last message to an array. With a Meteor.setInterval, I check the array length to know when all the queries are done. Then, as I can't return cursors anymore, I use the Meteor publication low level API to handle the publishing of the documents.
FYI : in a first attempt, I was using 'findOne' in my _parallelQueries, which divided my computation time by 2/3. But then, thanks to a friend, I tried the cursor.foreach() function, which allowed me to divide the computation time by 2 again !
In production, the benchmarks allowed me to go from a 7/8 second response time to an average response time of 1.6 second :)
Hope this will be usefull to you people ! :)

How to update user.services for Jasmine testing in Meteor

I am trying to mock out a user for testing out my application, and I have gotten to the point where I can create a test user and log them into the mirror instance of my app.
I need to compare the gmail addresses for peoples accounts, and to test this functionality, I want to add a test email address under user.services.google.email within the Meteor users account database (which is where the accounts-google package stores it, I don't need to mock out an entire user account yet).
What I can't figure out is how to append this information, instead of just overwriting what is already there, this is what my code looks like:
if (Meteor.users.find().count() === 0) {
var testUserDetails = {
email: 'testEmail#gmail.com',
password: 'testPassword'
};
console.log("Creating the Test User");
var newUserId = Accounts.createUser(testUserDetails);
Meteor.users.update({
_id: newUserId
}, {
$set: {
services: {
google: {
email: "testEmail#gmail.com"
}
}
}
});
} else {
console.log("There are already users in the Test database");
}
console.log('***** Finished loading default fixtures *****');
},
And this is what a user looks like:
{
"_id" : "Dw2xQPDwKp58RozC4",
"createdAt" : ISODate("2015-07-30T04:02:03.261Z"),
"services" : {
"password" : {
"bcrypt" : "asdfasdfasdfdsafsadfasdsdsawf"
},
"resume" : {
"loginTokens" : [ ]
}
},
"emails" : [
{
"address" : "testEmail#gmail.com",
"verified" : false
}
]
}
Now $set just rewrites everything within services, and there is no $push operation for mongo or for js, so how should I go about doing this? Should I consume the object and parse it manually?
*Note I have also tried using Meteor's Accounts.onCreateUser(function(options, user) but facing the same issue.
[...] there is no $push operation for mongo [...]
Sure, there is a $push operator, which appends a specified value to an array.
However, I think what you are trying to do is to update a document and keep all values which are already set.
Here is how you can do that:
Query the document first to get the object you want to set.
Update the respective object.
Run the MongoDB update operation to set the new object.
For instance:
var user = Meteor.users.findOne({
_id: newUserId
});
var servicesUserData = user.services;
servicesUserData.google.email = "your_new_email#gmail.com";
Meteor.users.update({
_id: newUserId
}, {
$set: {
"services": {
servicesUserData
}
}
});

Categories