I'm trying to debug an issue with Node.JS and Mongoose not saving the data correctly. I have a large object with multiple nested objects and arrays of objects. I'm viewing it in Chrome's devtools and the Node devtools right before saving. It's only a couple of nested objects that have this issue. The member variables in the array of objects are enclosed with square brackets []'s. I discovered this while viewing the data in Node devtools right before saving.
0: Object
[attributeName]: "Grow raccoon tail"
[componentId]: "58918f2c6f92704b0868aa30"
[derivativeName]: ""
[entityId]: "58918f9d6f92704b0868aa3e"
[isStateVariable]: "false"
[name]: "Feather"
[parentId]: "0"
[startValue]: "0"
[variableName]: "tail"
Here is the schema of the problem object
var componentVariableSchema = mongoose.Schema({
componentId: { type: ObjectId },
entityId: { type: ObjectId },
name: String,
attributeName: String,
isStateVariable: { type: Boolean, default: false },
variableName: String,
derivativeName: String,
startValue: { type: Number, default: 0.0 }
},
{
timestamps: true
});
Here is the schema of the parent object
var simulationSchema = mongoose.Schema({
createdDate: { type: Date, default: Date.now },
modifiedDate: { type: Date, default: Date.now },
name: { type: String, default: Date.now },
description: { type: String },
parentId: { type: ObjectId, ref: 'Project' },
// Variables
componentVariables: [componentVariableSchema],
simulationVariables: [simulationVariableSchema],
// Integrator
integratorType: { type: String },
integratorParams: [Number],
// Conditions
startTime: 0,
stopTime: 0,
initialValues: [Number],
// Containers for the code portion of the simulation
initializationCode: { type: String },
preFireCommandCode: { type: String },
staveVariableDerivativesCode: { type: String },
postFireCommandCode: { type: String },
// A simulation can be run multiple times with different sets of results
resultsList: [simulationResultSchema],
createdByUserId: { type: ObjectId, ref: 'User' },
isViewableToOthers: Boolean,
deletedBy: { type: ObjectId },
deletedDate: { type: Date }
},
{
timestamps: true
});
The simulationVariables array has the same problem. The parent object simulationSchema is within an array in it's parent object, but all of the member variables look correct. I've never seen this before, and I can't figure out what's causing it.
Related
I need to get a nested object within a certain document (searched by user ID) that also has an object inside of it (there's no guarantee that this object will be the same object).
I have the User model to be:
const mongoose = require('mongoose');
const { bool } = require('#hapi/joi');
const monitoringSchema = new mongoose.Schema({
type: Object,
default: {}
})
const hubSchema = new mongoose.Schema({
hubID: {
type: String,
default: ""
},
isSetup: {
type: Boolean,
default: false
},
monitoring: {
type: monitoringSchema
}
}, {strict:false})
const finalUserSchema = new mongoose.Schema({
username: {
type: String,
required: true,
max: 255
},
email: {
type: String,
required: true,
max: 255,
},
password: {
type: String,
required: true,
min: 10,
max: 1024,
},
date: {
type: Date,
default: Date.now
},
isVerified: {
type: Boolean,
default: false
},
hub: {
type: hubSchema
}
}, {strict:false});
module.exports = mongoose.model('User', finalUserSchema);
OR It has the layout:
_id: "id"
isVerified: true
username: "nathan"
email: "email#email.com"
hub:
hubID: "id"
monitoring: // WHOLE OBJECT I NEED TO RETREIVE
exampleObject:
exampleValue: exampleKey
I have an array of user IDs I need to update and I tried the query:
for(i in usersToUpdate){
User.findOne({_id: usersToUpdate[i], "hub.monitoring": {}}, {}, callbackResponse);
function callbackResponse(err, data){
if(err) return console.log(err)
console.log(data)
}
}
But it returns null as the data so obviously the query is wrong. I know the error is:
{_id: usersToUpdate[i], "hub.monitoring": {}}
more specifically:
"hub.monitoring": {}
I'm using {} to reference an object within monitoring, what's the correct reference to reference an unknown object and get it's values back, like a wildcard? I've tried:
{_id: usersToUpdate[i], "hub.monitoring": Object}
and it still doesn't work. I've seen this answer, however they reference a value that they already know, like a name?
To retrieve only the monitoring object, aggregation pipeline can be used.
Using $match to filter and $project to output/ supress fields.
User.aggregate([
{
$match: {
_id: mongoose.Types.ObjectId(usersToUpdate[i]),
},
},
{
$project: {
monitoring: "$hub.monitoring",
_id: 0,
},
},
]).exec(callbackResponse);
Playground example
You can try using the 2 object form of findOne where the first object is the query and the second object is the projection of what you want to return.
User.findOne({_id: usersToUpdate[i]}, {"hub.monitoring": {$exists: true}}, callbackResponse);
function callbackResponse(err, data){
if(err) return console.log(err)
console.log(data)
}
This way, the object will be returned if the monitoring object exist.
I'm currently trying to figure out at mongodb what's the best way in terms of performance cost and redundancy the best way of building a big document data schema. The final JSON from my rest -> app will be likely how it is structured.
Now internally the data will not be used as many to many that's why i binded it into a single document. Only the id will be used as a reference in another collections.
What you guys think, is it better to spit as relational way, with multiple collection to store the content inside of deliverable and use reference or just embedded. (since NoSQL has no joins i though this way will speed up)
Current using mongoose at node app
The Schema:
projectSchema = new Schema({
name: {
type: String,
required: true,
minlength: 3,
maxlength: 50
},
companyId: {
type: mongoose.Types.ObjectId,
ref: 'companies',
required: true
},
deleted: {
type: Number,
enum: [0, 1],
default: 0
},
predictedStartDate: {
type: Date,
default: ""
},
predictedEndDate: {
type: Date,
default: ""
},
realStartDate: {
type: Date,
default: ""
},
realEndDate: {
type: Date,
default: ""
},
//not final version
riskRegister: [{
name: String,
wpId: {
type: mongoose.Types.ObjectId,
ref: 'projects.deliverables.workPackages.id',
required: true
},
probability: String,
impact: String,
riskOwner: String,
response: String,
duration: String,
trigger: String,
status: String,
plannedTimming: String
}],
deliverables: [{
body: String,
workPackages: [{
body: String,
activities: [{
body: String,
tasks: [{
content: String,
properties: [{
dependecies: Array,
risk: {
type: Number,
enum: [0,1],
required: true
},
estimatedTime: {
type: Number,
required: true
},
realTime: {
required: true,
default: 0,
type: Number
},
responsible: {
id: {
type: Number,
default: -1
},
type: {
type: String,
enum: [0, 1], //0 - user, 1 - team
default: -1
}
},
materialCosts: {
type: Number,
default: 0
},
status: {
type: Number,
default: 0
},
approval: {
type: Number,
default: 0
},
startDate: {
type: Date,
default: ""
},
finishDate: {
type: Date,
default: ""
},
endDate: {
type: Date,
default: ""
},
userStartDate: {
type: Date,
default: ""
},
endStartDate: {
type: Date,
default: ""
},
taskNum: {
type: Number,
required: true
},
lessonsLearn: {
insertedAt: {
type: Date,
default: Date.now
},
creatorId: {
type: mongoose.Types.ObjectId,
ref: 'users',
required: true
},
situation: {
type: String,
required: true
},
solution: {
type: String,
required: true
},
attachments: Array
}
}]
}]
}]
}]
}]
})
The only concern I would raise would be regarding deliverables. If in the future there is a use case to do some CRUD operation regarding activities or tasks on the workPackage, the mongodb position operator $ does not support inner arrays, so you would be forced to extract all the deliverables and in memory iterate over all and only after update the deliverables.
My sugestion would be to support only arrays in the first level on the object. The inner objects should be moduled in separate collection ( activities and tasks ). In latest versions of mongodb you now have support to transactions so you can implement ACID on your operations against database, so the manipulation of all this information can be done in an atomic way.
I have a NodeJS application where I use the mongoose library to communicate with my mongo database.
The application is about a game, where multiple rounds are played. And after each round, the results of the round are submitted! I want the values (a json) to be push to players.rounds. I have an _id and a players.id to determine where to push.
This is what I thought would be the right way (and I'm still a newbie in mongoose). It prints me no error, but the db document is not affected. Still zero items in players.rounds.
This is what I thought would be the right way (and I'm still a newbie in mongoose).
My mongoose schema:
const gameSchema = new mongoose.Schema(
{
categories: [
{ type: String }
],
countdown: Number,
players: [{
_id: false,
id: String,
rounds: [
{ type: Map, of: String }
],
score: { type: Number, default: 0 },
ready: { type: Boolean, default: false }
}]
}
);
The place where I'm executing:
Game.findOneAndUpdate(
{ _id: gameId, 'players.id': client.id },
{ $push: { 'players.$.rounds': values } }, function(err) {
if (err) {
console.log('ERROR when submitting round');
console.log(err);
}
});
It prints me no error, but the db document is not affected. Still zero items in players.rounds.
you need to change your schema Object. we need to specify {strict: false} for changing the inserted documents in mongoose.
const gameSchema = new mongoose.Schema(
{
categories: [
{ type: String }
],
countdown: Number,
players: [{
_id: false,
id: String,
rounds: [
{ type: Map, of: String }
],
score: { type: Number, default: 0 },
ready: { type: Boolean, default: false }
}]
}, {strict:false} );
I have message document with groupId and createdTS fields.
and for query i have array of objects with groupId and lastVisit.
I want to query all messages per groupId after lastVisit
I tried with $in with groupIds but it is not filtering createdTS with lastVisit
member schema
const GroupMemberSchema = new mongoose.Schema({
userId: { type: String, required: true },
groupId: { type: String, required: true },
addTS: { type: Date, default: Date.now },
lastVisit: { type: Date, default: Date.now }
});
Message Schema
const GroupMessageSchema = new mongoose.Schema({
id: { type: String, required: true },
groupId: { type: String, required: true },
content: { type: String, required: true },
createdTS: { type: Date, default: Date.now },
});
for query
GroupMessage.find({groupId: {$in: groupIds}})
If I understood the question correct then you need to fetch records that match each groupId and at the same time are greater than appropriate lastVisit. If to translate it to MongoDB query it would be something like this:
{
"$or": [
{
"$and": [
{ "groupId": _groupId[i] },
{ "createdTS": { "$gt": _lastVisit[i] } }
]
},
...
]
}
Where _groupId[i] and _lastVisit[i] are array elements for list of groups and lastVisit timestamps.
I have the following Schema -
const leadSchema = new Schema(
{
emails: [{ type: Email, default: null }],
name: { type: String },
country: { type: String },
city: { type: String, index: true },
source: {
type: Number,
min: 1,
max: leadConfig.sources.length,
required: true
},
course: { type: Schema.Types.ObjectId, ref: 'courses',required: true},
gender: { type: String, enum: leadConfig.gender },
status: {type: Schema.Types.ObjectId, ref: 'status' },
dob: Date,
parent_name: String,
counselor: { type: Schema.Types.ObjectId, ref: 'users', default: null },
consultant_amount: { type: Number, min: 0, default: 0 },
consultant_amount_paid: { type: Number, min: 0, default: 0 },
loan: { type: Boolean, default: false },
reported: { type: Boolean, default: false },
scholarship: { type: Number, default: 0 },
student_id: { type: Number, default: null },
next_interection_deadline: { type: Date, default: null },
session: { type: Schema.Types.ObjectId, ref: 'session' }
},
{ timestamps: true }
);
module.exports = mongoose.model('leads', leadSchema);
I want to store the update history of all the documents of this collection.
For Example -
If I change the name field of a lead from 'John' to 'Jane' then a record should be saved in a history table with the following schema -
{
_id:(ObjectId),
collectionName:"lead"
column_name:"name"
oldValue - 'John',
newValue - 'Jane'
updateAt - Date()
}
I googled some plugins like mongoose-diff-history and it serves the purpose well but the only drawback was that it only worked with .save() method and not with mongodb updates methods.
I have been working on this problem for so many days but couldn't find a correct and efficient solution. Any solutions to this problem will be very much appreciated.
Have you looked into the midldeware hooks? Usually what you want could be handled there. For example look into Mongoose hooks: http://mongoosejs.com/docs/middleware.html
You have basically "events" which allow you do intercept records just before "save" etc and do something (like in your case store/log somewhere).
Here is an example from their docs:
var schema = new Schema(..);
schema.pre('save', function(next) {
// do stuff
next();
})
Here is one for the 'update':
schema.pre('update', function() {
this.update({},{ $set: { updatedAt: new Date() } });
});