We Keep Coding

Increment ($inc) update not working with upsert (MongoDB, NodeJS)

I am currently working on a MongoDB Post-Hashtag system, I am using NodeJS and the library "Mongoose". Following the creation of a new "post", I wish to increment the quantity of the "posts" field of the given hashtag's document, else if it does not exist: create a new document for the hashtag. I attempted to do this by utilizing the upsert option and performing a single update (updateOne) for each hashtag. The upserting & new document creation process for a new hashtag appears to work fine, however, pre-existing hashtags do not have their "posts" field incremented.
My code:
await Hashtag.updateOne({hashtag: hashtag}, {
$set: {$inc: {posts: 1}},
$setOnInsert: {
uniqueID: hashtagID,
hashtag: hashtag,
timeOfCreation: now,
posts: 1
}
}, {upsert: true});
I have attempted both with & without the '$set' operator, as well as '$update' and '$inc' to no avail. Omitting it results in the following error:
{
"errorType": "MongoServerError",
"errorMessage": "Updating the path 'posts' would create a conflict at 'posts'",
"code": 40,
"index": 0,
"stack": [
"MongoServerError: Updating the path 'posts' would create a conflict at 'posts'",
...
}
My schema as defined in Mongoose:
const hashtagSchema = new mongoose.Schema({
uniqueID: {
type: String,
required: true
},
hashtag: {
type: String,
required: true
},
timeOfCreation: {
type: Number,
required: true
},
posts: {
type: Number,
required: true
},
});

Remove the $set which is used before $inc.
await Hashtag.updateOne({hashtag: hashtag}, {
$inc: {posts: 1},
$setOnInsert: {
uniqueID: hashtagID,
hashtag: hashtag,
timeOfCreation: now,
posts: 1enter code here
}
}, {upsert: true});

Mongoose - Model.updateOne isn't working for me when adding an embeded document to an existing document

When I run this code in node.js my embedded document id doesn't match with the related id in the other collection.
const fruitSchema = new mongoose.Schema({
name: {
type: String,
required: true
},
rating: {
type: Number,
min: 1,
max: 10,
},
review: String
});
const Fruit = mongoose.model("Fruit", fruitSchema);
const watermelon = new Fruit({
name: "Watermelon",
rating: 7,
review: "Meh."
});
// watermelon.save();
const personSchema = new mongoose.Schema({
name: String,
age: Number,
favouriteFruit: fruitSchema
});
const Person = mongoose.model("Person", personSchema);
const person = new Person({
name: "John",
age: 37,
favouriteFruit: watermelon
});
person.save();
As a result, in MongoDB I get, From
db.fruits.find()
{
"_id" : ObjectId("5e7444c37ce78dd238d0f994"),
"name" : "Watermelon",
"rating" : 7,
"review" : "Meh.",
"__v" : 0
}
From
db.people.find()
{
"_id" : ObjectId("5e7451d971df90096974be01"),
"name" : "John",
"age" : 37,
"favouriteFruit" :
{
"_id" : ObjectId("5e7451d971df90096974be00"),
"name" :"Watermelon",
"rating" : 7,
"review" : "Meh."
},
"__v" : 0
}
I think I'm missing something with the Model.updateOne method.
I'm just trying to add an embedded document into another document.
I'm just a beginner, so any links or help would be amazing!
Thanks!

Why such behavior?
The reason you are having different _ids in the two fruits object that are supposed to be the same is that you did not add the _id property to the fruit schema and because _id is a required property of all MongoDB document, mongoose would help you auto-generate a new _id when it creates the query to send to the database. The _id it generates when you run watermelon.save() is different from the _id it generates when you run person.save(), this is why you are seeing the two different _ids.
The fix:
What you need to do is add the _id property to the fruit schema(and maybe the person schema to avoid further surprises) and then explicitly generate an _id your self before saving the document into the database.
The fruit schema should be like this after adding the _id property:
const fruitSchema = new mongoose.Schema({
_id: mongoose.ObjectId,
name: {
type: String,
required: true
},
rating: {
type: Number,
min: 1,
max: 10,
},
review: String
});
And when instantiating the fruit model, add the _id value yourself:
const watermelon = new Fruit({
_id: mongoose.Types.ObjectId(), // this helps generate a unique objectId
name: "Watermelon",
rating: 7,
review: "Meh."
});

Mongoose deep populate results in [Object] at 2nd level

After spent 1 day trying to get things done, I think it's time to ask for help.
I want to populate 2 levels of a document.
I tried with populate() but it seems to work only for first level, not deep populate! I read a lot at SO and I know it should work but I think I'm missing something really stupid...
Please let me know where I'm making mistakes.
Here are the relevant code.
Schemas
var compositionSchema = new Schema({
name: {
type: String,
required: true
},
contributions: [{
type: Schema.Types.ObjectId,
ref: 'Contribution'
}]
});
mongoose.model('Composition', compositionSchema);
var contributionSchema = new Schema({
name: {
type: String,
required: true
},
media: {
type: Schema.Types.ObjectId,
ref: 'Media'
}
});
mongoose.model('Contribution', contributionSchema);
var mediaSchema = new Schema({
name: {
type: String,
required: true
}
});
mongoose.model('Media', mediaSchema);
Actual documents saved in MongoDB
compositions:
{
"_id" : ObjectId("59e5db4595fe650a71fb0e07"),
"name" : "Test Pop 7",
"contributions" : [
ObjectId("59e5db4595fe650a71fb0e05")
]
}
contributions:
{
"_id" : ObjectId("59e5db4595fe650a71fb0e05"),
"name" : "Honda",
"media" : ObjectId("59e4ac5dacacd709eac2c856")
}
media:
{
"_id" : ObjectId("59e4ac5dacacd709eac2c856"),
"name" : "Logo_big.png",
"width" : 662,
"height" : 540
}
My tries (= the wrong code?)
In Node JS, when I do this (as per documentation):
Composition.findOne({ name: "Test Pop 7" })
.populate({
path: 'contributions',
model: 'Contribution',
populate: {
path: 'media',
model: 'Media',
}
})
.exec((error, doc) => {
if (error) { console.log(error); }
else {
console.log(doc);
}
});
prints out this, without actually populate the media field:
{ _id: 59e5db4595fe650a71fb0e07,
name: 'Test Pop 7',
contributions:
[ { _id: 59e5db4595fe650a71fb0e05,
name: 'Honda',
media: [Object] } ]
}

It works, keeping in mind the key-word in your question: prints. Printed, with console.log(), it just shows you the type (checked with typeof) of the document included in an array for some (2nd) level of nesting. If you do:
console.log(doc.contributions[0].media[0])
you will see your populated media document.

How we remove null error encounter in referencing node js mongodb

I have to use populate method for referencing my second collection. I have two collections one is levels and second child.
I have referenced to my level collection through child collection. For this, I use the populated method but it returns null.
This is node js code
function(){
var query = {'_id': result._id };
var params = 'usercurrLevelId';
childQuizInfo.findOne(query, params).populate({
path : 'usercurrLevelId',
model : 'Level',
select : '_id level_num age'
}).exec(function(err,levelinfo){
if(err) return next(err);
res.send(levelinfo);
});
}
This level schema
var LevelSchema = new Schema({
_id: { type: String },
age: { type: Number },
level_num: { type: Number },
min_score: { type: Number },
max_questions: { type: Number }
});
var Level = mongoose.model('Level', LevelSchema);
This is child schema
usercurrLevelId: {
type: mongoose.Schema.Types.ObjectId,
ref: 'Level',
index: true
}
This is the level stored in mongoDB
{
"_id" : ObjectId("57cb1ef8b0b05a2ce9b0c24b"),
"age" : 5,
"level_num" : 1,
"min_score" : 12,
"max_questions" : 30
}
{
"_id" : ObjectId("57cb1ef9b0b05a2ce9b0c24c"),
"age" : 5,
"level_num" : 2,
"min_score" : 15,
"max_questions" : 33
}
This is the child stored in mongoDB
{
"_id" : ObjectId("57cc0410d51045452644251b"),
"usercurrLevelId" : ObjectId("57cb1ef8b0b05a2ce9b0c24b")
}
And the output i get
{
"_id": "57cc0410d51045452644251b",
"usercurrLevelId": null,
"__v": 0
}
I using this node js query so that I get level_num and _id from my level schema using this referencing.

Slow MongoDB queries with Sails.js

I wrote an app using Sails.js with mongoDb(sails-mongo).
Firstly, I decided to write all to a single document...
And database slowed on 5GB of data..
"Slowed" means that basic find query executed in 30-50s..
Than I rewrite all in an multiple collections and add indexing..
example of my models:
Markets.js
module.exports = {
attributes: {
name: {
type: 'string',
index: true
},
pairs: {
collection: 'Exchanges',
via: 'source',
}
}
};
and Exchanges.js
module.exports = {
attributes: {
s1: {
type: "string"
},
source:{
model: "Maklers",
index: true
},
s2: {
type: "string"
},
p: {
type: 'float'
},
v1: {
type: 'float'
},
v2: {
type: 'float'
},
vb: {
type: 'float'
}
}
};
and example of slow query
Markets.findOne({
name: info,
sort: 'createdAt DESC',
limit: 1,
createdAt: {
'<=': aft
}
}).populateAll().exec(function(err, items) {
callback(err, items);
});
result of db.stats
> db.stats()
{
"db" : "stats222",
"collections" : 8,
"objects" : 36620661,
"avgObjSize" : 238.26556139988844,
"dataSize" : 8725442352,
"storageSize" : 10033258480,
"numExtents" : 63,
"indexes" : 13,
"indexSize" : 2940024192,
"fileSize" : 14958985216,
"nsSizeMB" : 16,
"extentFreeList" : {
"num" : 0,
"totalSize" : 0
},
"dataFileVersion" : {
"major" : 4,
"minor" : 22
},
"ok" : 1
}
What you can advice me?
It`s about 2000 of records every minute..
How to increase perfomance?
Change db config? Change indexes? Change DB? Change models/collections config?
I using 2-core server with 2GB of Virtual Memory..
Sorry for bad english..

There is a drawback in the 0.12 version of Waterline when using mongodb. By default waterline is not case sensitive, and mongodb is!
Your queries are slow, because when searching strings, it is being used a REGEX to find any case, so your indexes are useless. But you can change it, by disabling the case sensitiveness with the wlnex attribute:
someMongodbServer: {
adapter: 'sails-mongo',
host: 'mongodb',
port: 27017,
user: 'username',
password: 'password',
database: 'databaseCoolName',
wlNext: {
caseSensitive: true
}
},
You can confirm this error by checking on the mongodb logs. And see what are the slow queries.