We Keep Coding

Unexpected MongoServerError 11000 (duplicate) for a field that is NOT even declared in schema

I'm developing an online store Node.js REST API with Mongoose (MongoDB), which I'm new to. I decided to test the orders service and saw that after I had made 1 successful order (so it worked once), it sent me a duplicate key error for the next one, for a key 'name' with value 'null', of the order.products collection that is an Array, and not a kvp object.
I should note that nowhere in my code is 'products.name' mentioned.
ERROR:
MongoServerError: E11000 duplicate key error collection: store.orders index: products.name_1 dup
at {...}{
key: { products.name: null }
index: 0,
code: 11000,
keyPattern: { 'products.name': 1 },
keyValue: { 'products.name': null },
[Symbol(errorLabels)]: Set(0) {}
}
when the error is handled, this message is received and it makes no sense:
{ "message": "Order with products.name "null" already exists" }
Order schema:
const schema = new Schema({
userId: {
type: Types.ObjectId,
ref: 'User'
},
address: {
type: addressSchema,
required: true
},
products: {
type: [orderProductSchema],
required: true,
validate: nonEmptyArray
},
status: {
type: Number,
validate: inCollection(Object.values(ORDER_STATUS))
},
price: { type: Number, required: true, min: 0 }
}, { timestamps: true });
don't bother with the validators or the address/status/user/price, it has nothing to do with them; what is more, nothing is specified as unique: true
As you can see, the 'products' field is just an array of products, no 'name' is declared
orderProductSchema:
const schema = new Schema({
product: {
_id: { type: Types.ObjectId, required: true },
name: {
type: String,
required: true,
maxLength: 250
},
displayImage: String,
price: { type: Number, required: true, min: 0 }
},
quantity: {
type: Number,
required: true,
validate: isInteger,
min: 1
},
}, { _id: false });
I have a 'name' field here, but it's just the name of a product. The error is thrown even when the names are unique.
Orders service:
// get data and format it to fit the Order model
console.dir(products); // --> all is how it's expected to be in the schema
return await Order.create({
userId,
address,
products,
status: ORDER_STATUS.AWAITING_CONFIRMATION,
price: totalOrderPrice
});
It seems to me that this is some weird MongoDB behaviour/specification that I missed. If someone has any idea what could cause the problem - please help.
I tried removing all parts such as other fields and validations to see if they might've caused the problem but that was not the case. I thought maybe I had formatted the object I send to the database wrong, but I console.log-ed it and it was fine ({products: Array})

Thanks to #user20042973 and #MTN I saw that my 'orders' database had index 'products.name' (no idea how it got there).. I just removed the index and the problem is solved.

How to skip the unique constraint validation on same document update

I have an update call as below in my NestJs project to update a mongoose model.
async updateRole(role_id: ObjectId, request: any): Promise<any> {
return this.roleModel.findByIdAndUpdate(role_id, {...request});
}
Here is the request I'm passing
{
"name":"Super-Admin",
"application": "62b2dbfd82045f40ea884334",
"active":true,
"privileges": ["62b2dbfd82045f40ea884334","62b2dbfd82045f40ea884334"],
"updated_by": "Abhilash.Shajan1#gmail.com"
}
Below is my role schema
import * as mongoose from 'mongoose';
const Schema = mongoose.Schema;
const RoleSchema = new Schema({
name: {
type: String,
required: true
},
application: {
type: Schema.Types.ObjectId,
ref: 'Application',
autopopulate: true,
required: true
},
active: {
type: Boolean,
required: true
},
privileges: [{
type: Schema.Types.ObjectId,
ref: 'Privilege',
autopopulate: true
}],
created_by: {
type: String
},
created_at: {
type: Date
},
updated_by: {
type: String
},
updated_at: {
type: Date
}
});
RoleSchema.index( { name: 1, application: 1 }, { unique: true } );
export { RoleSchema };
I already created a document (This is the only document now present in the roles collection) with the above request. Now I'm trying to update its active field to true.
Since i have unique compound index in the schema, it does not allow me to update the active field, I'm getting unique contraint error on both application and name field.
This error will be meaningful if i have another document with same name and application in the collection, but there is not.
Another way is to pass the active field alone in the request. But it will not help in my case because the UI is always passing the whole fields which include the unchanged values as well.
Any suggestions ?

How to push an element into an array inside an object inside another array in Mongoose?

I am developing a server using Expressjs, Mongodb and Mongoose. I need to push an element (a string) into the "tweets" array which is inside an object (a friend) which is in turn inside the "friends" array which is inside a "user" Object which document in the "users" collection. Here is an example of how my documents in the Mongodb collection looks like:
{
"loggedIn": true,
"_id": "5f91ef0ce75d3b1d40539da0",
"username": "username",
"email": "a#h.com",
"password": "$2a$10$9krWS9Kq5024lRTexqaweePrn8aughepqTkaj3oA48x0fJ2ajd79u",
"dateOfBirth": "2002-12-07",
"gender": "male",
"friends": [
{
"tweets": [],
"_id": "5f91effae75d3b1d40539da7",
"username": "Jonas"
},
],
"__v": 0
}
I need to pick the specified username from the "Users" arrary first and then access "friends" array within this user and then pick the right friend object and finally push the tweet on $position: 0 in this array. I I tried to achieve that as shown in this code and I could access the friend object with the given friendUsername
await Users.updateOne(
{ username: req.params.username },
{
$push: {
friends: {
$elemMatch: {
username: req.params.friendUsername,
},
},
},
}
);
And now the question is how to access the "tweets" array inside $elemMatch and push the req.body.tweet at $position: 0 into it?

Here is how I would solve your issue, I first would re-define the way I am defining schemas.
My User schema would look something like the following
User.js
const mongoose = require('mongoose')
const UserSchema = mongoose.Schema({
...
friends: {
type: [{
type: mongoose.Schema.Types.ObjectId,
ref: 'User'
}],
required: true,
default: []
},
tweets: {
type: [{
type: mongoose.Schema.Types.ObjectId,
ref: 'Tweet'
}],
required: true,
default: []
},
...
}, {timestamps: true})
module.exports = mongoose.model('User', UserSchema)
User.js
const mongoose = require('mongoose')
const TweetSchema = mongoose.Schema({
...
text: {
type: String,
required: true
},
tweeter: {
type: mongoose.Schema.Types.ObjectId,
required: true,
ref: 'User'
},
likes: {
type: [{
type: mongoose.Schema.Types.ObjectId,
ref: 'User'
}],
required: true,
default: []
},
...
}, {timestamps: true})
module.exports = mongoose.model('Tweet', TweetSchema)
This assumes that every user can have tweets and that a User can be friend with another User
And now if someone tweets something you can do something like
const Tweet = require('./Tweet.js')
const User = require('./User.js')
let tweet = new Tweet({
text: "My first tweet!",
tweeter: "ID Of user who is posting the tweet"
})
tweet.save()
// Now update the user who tweeted
User.findOneAndUpdate()
User.updateOne({ _id: "ID Of user who is posting the tweet" }, { $push: { tweets: tweet._id } })
and now whenever you request a user all of his friends will be referenced and all of their tweets will also be referenced! if you want to see the actual tweets then use something like .populate() here are the docs for .populate() https://mongoosejs.com/docs/populate.html
Keep in mind is really a good practice to only return the actual ids and your frontend takes care of requesting the appropriate objects from their perspective endpoints. And if you wish to reduce network calls then the frontend would cache the data.
If the above doesn't help and you still would like to achieve your goal with your schemas then something like this should work (assuming your schema is called User)
let tweetObj = {}
User.updateOne({_id: 'your userid'}, {$push: {"friends.$.tweets": tweetObj}})
NOTE: I have omitted callbacks as they are irrelevant to the question

Mongoose How to get a nested object within a document via findOne

I need to get a nested object within a certain document (searched by user ID) that also has an object inside of it (there's no guarantee that this object will be the same object).
I have the User model to be:
const mongoose = require('mongoose');
const { bool } = require('#hapi/joi');
const monitoringSchema = new mongoose.Schema({
type: Object,
default: {}
})
const hubSchema = new mongoose.Schema({
hubID: {
type: String,
default: ""
},
isSetup: {
type: Boolean,
default: false
},
monitoring: {
type: monitoringSchema
}
}, {strict:false})
const finalUserSchema = new mongoose.Schema({
username: {
type: String,
required: true,
max: 255
},
email: {
type: String,
required: true,
max: 255,
},
password: {
type: String,
required: true,
min: 10,
max: 1024,
},
date: {
type: Date,
default: Date.now
},
isVerified: {
type: Boolean,
default: false
},
hub: {
type: hubSchema
}
}, {strict:false});
module.exports = mongoose.model('User', finalUserSchema);
OR It has the layout:
_id: "id"
isVerified: true
username: "nathan"
email: "email#email.com"
hub:
hubID: "id"
monitoring: // WHOLE OBJECT I NEED TO RETREIVE
exampleObject:
exampleValue: exampleKey
I have an array of user IDs I need to update and I tried the query:
for(i in usersToUpdate){
User.findOne({_id: usersToUpdate[i], "hub.monitoring": {}}, {}, callbackResponse);
function callbackResponse(err, data){
if(err) return console.log(err)
console.log(data)
}
}
But it returns null as the data so obviously the query is wrong. I know the error is:
{_id: usersToUpdate[i], "hub.monitoring": {}}
more specifically:
"hub.monitoring": {}
I'm using {} to reference an object within monitoring, what's the correct reference to reference an unknown object and get it's values back, like a wildcard? I've tried:
{_id: usersToUpdate[i], "hub.monitoring": Object}
and it still doesn't work. I've seen this answer, however they reference a value that they already know, like a name?

To retrieve only the monitoring object, aggregation pipeline can be used.
Using $match to filter and $project to output/ supress fields.
User.aggregate([
{
$match: {
_id: mongoose.Types.ObjectId(usersToUpdate[i]),
},
},
{
$project: {
monitoring: "$hub.monitoring",
_id: 0,
},
},
]).exec(callbackResponse);
Playground example

You can try using the 2 object form of findOne where the first object is the query and the second object is the projection of what you want to return.
User.findOne({_id: usersToUpdate[i]}, {"hub.monitoring": {$exists: true}}, callbackResponse);
function callbackResponse(err, data){
if(err) return console.log(err)
console.log(data)
}
This way, the object will be returned if the monitoring object exist.

Sails.js waterline and mysql adapter, can't get populate() with one-to-many associations working

So I've been at this for awhile and can't see how my code is different from the documentation.
I've also checked out this question, this question, this question, and this unanswered different question.
For my admin panel I'm trying to query to get all the information associated with a user and display a 'master' user profile to the admin.
My User model looks like this:
module.exports = {
autoPK: true,
attributes : {
id: {
type: 'integer',
primaryKey: true,
unique: true
},
email : {
type : 'email',
unique : true,
required : true,
},
password : {
type : 'string',
minLength : 8,
required : true
},
admin:{
type: 'bool'
},
user_profile:{
collection: 'userprofile',
via: 'user_id',
},
properties: {
collection: 'subjectproperties',
via: 'user_id'
},
employment_info: {
collection: 'employmentinfo',
via: 'user_id'
},
file_uploads: {
collection: 'fileupload',
via: 'user_id'
},
nearest_living_relatives:{
collection: 'nearestlivingrelative',
via: 'user_id'
},
mortgage_info: {
collection: 'mortgageinfo',
via: 'user_id'
},
user_progression_state:{
collection: 'userprogressionstate',
via: 'user_id'
},
users_applied_loan_values:{
collection: 'usersappliedloanvalues',
via: 'user_id'
}
}
}
I don't want to list out all the belongs to user models cause there are a lot of them, but here is one of the simpler one's.
EmploymentInfo.js
module.exports = {
tableName: "employment_info",
attributes : {
employers_name:{
type: 'string',
required: true
},
employers_address:{
type: 'string',
required: true
},
employers_city:{
type: 'string',
required: true
},
employers_state:{
type: 'string',
required: true
},
employers_zip:{
type: 'string',
required: true
},
job_position:{
type: 'string',
required: true
},
years_in_position:{
type: 'string',
required: true
},
years_in_industry:{
type: 'integer',
required: true
},
user_id:{
model:'user'
}
}
};
And as for my controller:
create_admin_user_profile: function(req, res){
var user_id = req.query.userId;
User.find({'id': user_id}).populateAll().exec(function(err, user){
if(err || user.length === 0){
sails.log.verbose(err);
}else{
sails.log.verbose(user);
}
});
},
It doesn't error out but all I see in the terminal is this for the above:
[ { user_profile: [],
properties: [],
employment_info: [],
file_uploads: [],
nearest_living_relatives: [],
mortgage_info: [],
user_progression_state: [],
users_applied_loan_values: [],
id: 5,
email: 'test#test.com',
admin: 1 } ]
Even though there is an entry in all of those tables for that user.
If I change the line:
User.find({'id': user_id}).populateAll().exec(function(err, user){
To:
User.find({'id': user_id}).populate('employment_info').exec(function(err, user){
Same but shorter result:
[ { employment_info: [],
id: 5,
email: 'test#test.com',
admin: 1 } ]
I've tried changing the case, I've tried adding columnName to the user_id attribute, I've tried changing the column name across the entire breadth of the project to not have an under_score in it, though that never seemed to be issue in it picking up the names correctly, but nothing I've done seems to work. I've also tried uninstalling sails, and the sails-mysql adapter and clearing my npm cache.
At this point my just stuck, I really can't see a reason why it's not working.
As for project info:
Sails v: 0.12.11
npm v: 3.10.9
node v: 7.2.0
Additional info asked for in comments:
SQL row taken right from db for user 5
employers_name, employers_address, employers_city, employers_state, employers_zip, job_position, years_in_position, years_in_industry, user_id
'Company', 'Mill Steet', 'SLC', 'Utah', '88888', 'Developer', '2', '2', '5'
And json format returned by find method in EmploymentInfo.js controller
{
"employmentInfo": {
"employers_name": "Company",
"employers_address": "Mill Steet",
"employers_city": "SLC",
"employers_state": "Utah",
"employers_zip": "88888",
"job_position": "Developer",
"years_in_position": "2",
"years_in_industry": 2,
"user": 5
}
}
The reason the last param is user and not user_id is because I rename it in the find method to serve the front-end mvc which also has the ability to work with associations. It's also why the JSON has the format it does.
Code from the find method that replaces user_id:
EmploymentInfo.find({'user_id': user_id}).exec(function(err, profile){
if(err || !profile.length){
return res.json(err);
}else{
res.status(200);
profile[0].user = profile[0].user_id;
delete profile[0].user_id;
res.send({'employmentInfo': profile[0]});
}
});
However I've tried not renaming it; I've also tried getting rid of my find override and just relying on the blueprint find method, neither of those worked either.