I am having a strange issue querying a Mongo DB collection. I am using findById() to get a single item that works sometimes and not others.
I have checked the id being passed to the server route and in all cases, they match perfectly with the targeted document in the collection.
Here is the basic code:
router.get("/:postId", async (req, res) => {
console.log('id : ', req.params.postId)
console.log('type: ', typeof(req.params.postId)) // id is a string
try {
const post = await Post.findById(req.params.postId).exec();
console.log('post :', post) // sometimes null
res.json(post);
} catch (err) {
res.json({ message: err });
}
});
In the above route, only certain posts will be found while others come back null. This happens regardless of whether the id passed is correct and the document exists with the exact id.
If anyone has any ideas about what could be going wrong here I'd much appreciate the help!
EDIT
I have done some more debugging and think it is something to do with the Schema for the Post model.
For example, this object will be found:
{
"tags": ["foo"],
"_id": "8394839483fhg020834903",
"title": "bar",
"content": "baz",
"isPrivate": true,
}
But this one will not because of the missing isPrivate property.
{
"tags": [],
"_id": "5e0fdc631ef5c46b285a4734",
"title": "New post",
"content": "Some content here",
}
I have tested this across multiple queries and it appears to the root of the problem.
I have tried adding
isPrivate: {
required: false
}
To the Schema but it doesn't seem to solve the issue.
Here is the full Schema
const postSchema = mongoose.Schema({
title: {
type: String,
required: true
},
content: {
type: String,
required: true
},
tags: [{ type: String }],
date: {
type: Date,
default: Date.now
},
isPrivate: {
type: Boolean
required: false
}
});
I'm not a Mongo/Mongoose expert, so any guidance would be much appreciated.
If post id match with any record it return data, otherwise it will return null. You should handle the exception
router.get("/:postId", async (req, res) => {
try {
const post = await Post.findById(req.params.postId).exec();
if(post) {
return res.json(post);
}
res.json({ message:'No Post found' });
} catch (err) {
res.json({ message: err });
}
});
You can manually check is record exists against a post id. You can use MongoDB Compass for gui browse the record
I believe the issue might be with your _id as per mongo standard _id should be a String is of 12 bytes or a string of 24 hex characters.
We can check if the _id is valid using mongoose.isValidObjectId()
I did run this check on your objects that you posted and indeed 1 is invalid while other is valid
const mongoose = require('mongoose');
console.log(`is '8394839483fhg020834903' valid - ${mongoose.isValidObjectId('8394839483fhg020834903')}`);
console.log(`is '5e0fdc631ef5c46b285a4734' valid - ${mongoose.isValidObjectId('5e0fdc631ef5c46b285a4734')}`);
It gives me
You will have to check what is modifying your ID's in the code, you can upload your schema to get a better understanding as well.
Related
I'm trying to use Mongoose (MongoDB JS library) to create a basic database, but I can't figure out how to delete the documents / items, I'm not sure what the technical term for them is.
Everything seems to work fine, when I use Item.findById(result[i].id), it returns a valid id of the item, but when I use Item.findByIdAndDelete(result[i].id), the function doesn't seem to start at all.
This is a snippet the code that I have: (Sorry in advance for bad indentation)
const testSchema = new schema({
item: {
type: String,
required: true
},
detail: {
type: String,
required: true
},
quantity: {
type: String,
required: true
}
})
const Item = mongoose.model("testitems", testSchema)
Item.find()
.then((result) => {
for (i in result) {
Item.findByIdAndDelete(result[i].id), function(err, result) {
if (err) {
console.log(err)
}
else {
console.log("Deleted " + result)
}
}
}
mongoose.connection.close()
})
.catch((err) => {
console.log(err)
})
I'm not sure what I'm doing wrong, and I haven't been able to find anything on the internet.
Any help is appreciated, thanks.
_id is a special field on MongoDB documents that by default is the type ObjectId. Mongoose creates this field for you automatically. So a sample document in your testitems collection might look like:
{
_id: ObjectId("..."),
item: "xxx",
detail: "yyy",
quantity: "zzz"
}
However, you retrieve this value with id. The reason you get a value back even though the field is called _id is because Mongoose creates a virtual getter for id:
Mongoose assigns each of your schemas an id virtual getter by default which returns the document's _id field cast to a string, or in the case of ObjectIds, its hexString. If you don't want an id getter added to your schema, you may disable it by passing this option at schema construction time.
The key takeaway is that when you get this value with id it is a string, not an ObjectId. Because the types don't match, MongoDB will not delete anything.
To make sure the values and types match, you should use result[i]._id.
If I use await user.save() in the code below it seems to get stuck but if I use it without await it seems to move ahead and in the response the passwordResetToken is displayed.. However in the database, for some reason, the rest of the fields get saved but passwordResetToken just doesn't get saved.
I am completely lost about what the reason could be.. Any direction on what I should look for would help.
exports.forgotPassword = async function(req, res, next) {
// 1. Get user based on POST email
const user = await User.findOne({ email: req.body.email });
// 2. Generate random token
user.passwordResetToken = "123456";
await user.save({ validateBeforeSave: false });
// 3. Send it back as an email
//4. Send Response
res.status(200).json({
status: "success",
data: {
user: user,
},
});
};
this is the response
{
"status": "success",
"data": {
"user": {
"role": "user",
"_id": "6116aeb70aae0f7c7d8800bc",
"name": "Jane Doe",
"email": "jane#doe.com",
"__v": 0,
"passwordResetToken": "123456"
}
}
}
However the passwordResetToken still doesn't save it to the database although the rest of it does.. I do have the passwordResetToken as a field in the mongoose.Schema
I had not added this section below to my question earlier so I am adding this to help understand where the issue might lie..
This is the userModels.js
const userSchema = new mongoose.Schema({
name: {
type: String,
required: [true, "Name is required"],
},
email: {
type: String,
required: [true, "Email is required"],
validate: [validator.isEmail, "Please provide valid email"],
unique: true,
},
role: {
type: String,
enum: ["admin", "lead-guide", "guide", "user"],
default: "user",
},
photo: String,
password: {
type: String,
minlength: 8,
required: [true, "Password is required"],
select: false,
},
passwordConfirm: {
type: String,
validate: {
validator: function(el) {
return el === this.password;
},
message:
"Password confirmation does not match with Password. Please try again",
},
},
passwordChangedAt: Date,
passwordResetToken: String,
passwordResetExpires: Date,
});
and here's the line from userRouter.js for this route..
router.route("/forgotPassword").post(authController.forgotPassword);
Maybe you can try with
let user = await User.findOne({ email: req.body.email });
// 2. Generate random token
user.passwordResetToken = "123456";
user = await user.save();
or you can use
const user = await User.findOneAndUpdate({email:req.body.email},{$set:{passwordResetToken:'123456'},{new:true})
new: true will return updated data.
Make sure that the model file contains the passwordResetToken field. Otherwise, it will be ignored.
I wonder if the issue is with the HTTP request, you might want to have http.post not http.put request for your posted question.
I am not sure if you find your own bug, should you delete the question or update it with an answer. But I'll just answer it here just so that someone who does the same mistake as me, can know what could go wrong.
When it seems like something is not getting saved or when you await the save() method and it gets deadlocked, chances are that it's the pre-save middleware..
It might not be obvious to those like me and you might end up trying to fix and replace your "perfect" code while the bug lies elsewhere..
So 2 things to check if something is not getting saved and if await creates an unending response)
1. Check if the pre-save middleware is configured properly
2. Check if the middleware ends with a next()
In my case it was this.. In the code below I had missed out the ! before the this.isModified("password")
And so while testing the response would show me the right object but it would never get saved as the pre-save middleware moved to next().
The await deadlock issue was because I did not end the middleware with the next() method.
this is the faulty code
userSchema.pre("save", function(next) {
if (this.isModified("password") || this.isNew) return next();
this.passwordChangedAt = Date.now() - 1000;
});
this is the correct code
userSchema.pre("save", function(next) {
if (!this.isModified("password") || this.isNew) return next();
this.passwordChangedAt = Date.now() - 1000;
next();
});
Anyways I hope this saves someone out there some time during debugging their code.
I am having a problem with the user model that I'm using with Mongoose and MongoDB to create each profile in my database. It works fine to post one user, but throws the following error if I logout and try again:
{
"name": "MongoError",
"message": "E11000 duplicate key error collection: CourtAPIDev.users index: trackers.case_id_1 dup key: { : null }",
"driver": true,
"index": 0,
"code": 11000,
"errmsg": "E11000 duplicate key error collection: CourtAPIDev.users index: trackers.case_id_1 dup key: { : null }"
}
According to mongoose documentation: If there is more than one document (a second user) without a value for the indexed field or is missing the indexed field, the index build will fail with a duplicate key error. I don't know how to set this _id property for the trackers property –– I thought it generated automatically!
Here's the trackers part of my Schema. And the relevant case_id property, which seems to be throwing the "null" error.
The whole repository can be found on my Github here, but the likely problem spots are the ones I highlighted, I think. Here's the github link: https://github.com/KingOfCramers/node_login_with_trackers
user model:
const UserSchema = new mongoose.Schema({
email: {
type: String,
required: true,
trim: true,
minLength: 1,
unique: true,
validate: {
validator: (value) => {
return validator.isEmail(value);
},
message: '{VALUE} is not a valid email'
}
},
password: {
type: String,
required: true,
minlength: 6
},
tokens: [{
access: {
type: String,
required: true
},
token: {
type: String,
required: true
}
}],
trackers: {
tweets: [TwitterSchema],
legislation: [LegislationSchema],
court_cases: [CourtCaseSchema]
},
frequency: [EmailSchema]
});
Express route:
app.post("/users", (req,res) => {
var body = _.pick(req.body, ['email', 'password']);
body.frequency = {
alert_time: new Date(),
email: req.body.email
}
var user = new User(body);
user.save().then(() => {
return user.generateAuthToken();
}).then((token) => {
res.header("x-auth", token);
res.send(user);
}).catch((e) => {
res.status(400).send(e);
});
});
Test (mocha):
it("Should post a new user", (done) => {
var email = "uniqueemail#example.com"
var password = "9webipasd"
supertest(app)
.post("/users") // Post request to the /todos URL
.send({
email,
password
})
.expect(200)
.expect((res) => {
expect(res.headers).toIncludeKey('x-auth')
expect(res.body._id).toExist();
expect(res.body.email).toBe(email);
})
.end((err) => {
if(err){
return done(err);
}
User.findOne({email}).then((user) => {
expect(user).toExist();
expect(user.password).toNotBe(password);
done();
}).catch((e) => done(e));
});
});
My guess is that there is an index on CourtCaseSchema.case_id which does not allow duplicates.
I think you could check (in a mongo shell) that with CourtAPIDev.court_cases.getIndexes() (I think your db is named CourtAPIDev and the collection is named court_cases but I am not sure about that).
Also if you clean the test db after each run, that would explain why the tests are passing, since there is no more than one user.
Turns out, it was to do with my mongodb database, not any of my code. After searching around online, I found that if I logged into the mongo shell and then dropped all indexes from the users collection, it solved my problem. Could someone explain why this was causing my program to crash? I think it may have to do with an old user model, but I don't really understand. Thanks!
Even if you have all of your keys as unique=False, you may still get E11000 duplicate key error. So in that case, just follow these steps and check if your error is resolved.
Delete all documents from the collection (e.g. db.collection_name.deleteMany({}))
Drop the COLLECTION (NOT THE DATABASE) (e.g db.collection_name.drop())
Cheers !!
I have following schema
var Topic= new Schema({
text: String,
topicId: String,
comments: [{type: Schema.Types.ObjectId, ref:'Comment'}]
});
var Comment = new Schema({
text: String
});
I am writing RESTFul API that will give me the Comment details as per topic ID and Comment ID
/topics/{id}/comments/{id}
Following is the function that gets data from Mongo
getCommentsById: function(req, resp){
req.db.Topic.findOne({"topicId": req.params.topicId})
.populate({path:"Comments", match:{"_id": req.params.commentId}})
.exec(function(err, topic){
if(err) {
return resp.status(500).json({
message: 'Error when getting Topic.',
error: err
});
}
if (!topic) {
return resp.status(404).json({
message: 'No such Topic'
});
}
if (!topic.comments || topic.comments.length==0) {
return resp.status(404).json({
message: 'No such Comment'
});
}
resp.json(topic.comments[0]);
});
}
The code works fine if I specify the right comment ID, but if I specify non-existing comment ID in URL then I get following error
{
"message": "Error when getting Topic.",
"error": {
"message": "Cast to ObjectId failed for value \"57c738b66d790f0c1bdb179\" at path \"_id\"",
"name": "CastError",
"kind": "ObjectId",
"value": "57c738b66d790f0c1bdb179",
"path": "_id"
}
}
What is the issue here and how to fix it?? Is there better way to query the required object?
The issue isn't that your specifying a non-existing comment ID. It's that you're specifying a string that can't be converted into a valid ObjectId. Your test string, "57c738b66d790f0c1bdb179" is a 23 character hex string. It should be length 24.
If you want to validate before attempting your query, there are several different ways you could go about it. Here's one example: Can I determine if a string is a MongoDB ObjectID?
I have a MongoDb schema like this
var User = new Schema({
"UserName": { type: String, required: true },
"Email": { type: String, required: true, unique: true },
"UserType": { type: String },
"Password": { type: String }
});
I am trying to create a new user
This is done in NodeJs using mongoose ODM
And this is the code for creating:
controller.createUser = function (req, res) {
var user = new models.User({
"UserName": req.body.UserName.toLowerCase(),
"Email": req.body.Email.toLowerCase(),
"UserType": req.body.UserType.toLowerCase()
});
models.User.findOne({ 'Email': user.Email }, function (err, olduser) {
if (!err) {
if (olduser) {
res.send({ 'statusCode': 409, 'statusText': 'Email Already Exists' });
}
else if (!olduser) {
user.setPassword(req.body.Password);
user.save(function (err, done) {
if (!err) {
console.log(user);
res.send({ 'statusCode': 201, 'statusText': 'CREATED' });
}
else {
res.send({ 'Status code': 500, 'statusText': 'Internal Server Error' });
}
});
}
}
else {
res.send({ 'statusCode': 500, 'statusText': 'ERROR' });
}
});
};
The for creating new user,I am giving attributes and values as follows:
{
"UserName": "ann",
"Email": "ann#ann.com",
"UserType": "normaluser",
"Password":"123456"
}
And I am getting error like this:
{"Status code":500,"statusText":"Internal Server Error","Error":{"name":"MongoError","err":"E11000 duplicate key error index: medinfo.users.$UserName_1 dup key: { : \"ann\" }","code":11000,"n":0,"connectionId":54,"ok":1}}
I understand that this error is because UserName is duplicated ,but I haven't set UserName with unique constraint.Whenever I add a new row,I need only email to be unique,UserName can be repeated.How to achieve this??
#ManseUK Is probably right, that looks like UserName is a 'key' - in this case an index. The _id attribute is the "primary" index that is created by default, but mongodb allows you to have multiple of these.
Start a mongo console and run medinfo.users.getIndexes()? Something must have added an index on 'UserName'.
required: true wouldn't do that, but you might have played with other settings previously and the index hasn't been removed?
There should be an index that is blocking.
You can try the db.collection.dropIndex() method
medinfo.users.dropIndexes()
I got the similar issue on my project. I tried to clear out all the documents and the dup issue still keep popping up. Until I dropped this collection and re-start my node service, it just worked.
What I had realized is that my data-structures were changing -- this is where versioning comes in handy.
You may need to get a mongoose-version module, do a thing.remove({}, ...) or even drop the collection: drop database with mongoose
I use RoboMongo for an admin tool (and I highly recommend it!) so I just went in and right-clicked/dropped collection from the console.
If anyone knows how to easily version and/or drop a collection from within the code, feel free to post a comment below as it surely helps this thread ( and I :) ).