I have a Node.js app that is creating a 'unit' in the DB under a 'building' that is saving two 'units' inside the array included in 'building'. Both of these are identical in both data and created timestamp. Anyone have an idea why the below code would be causing this to happen? I am a bit confused at it. I don't see how it would be adding two objects to the array. The code is not being run twice, I checked that with console.log() and just looking at my API logs.
await Building.findOneAndUpdate(
{buildingID},
{"$push": {units: unitData}},
(err, doc) => {
if(err) {
logger.error(`POST unit/new save error for unit: ${unitID} - error message: `, err)
return res.json({success: false, err, message: 'Error saving new unit, please try again'})
}
logger.debug('POST unit/new save() doc: ', doc)
return res.json({success: true, message: `Successfully saved new unit with ID of: ${unitID}`, unitData})
}
)
I ended up figuring it out, when there is an await and a callback it will save the data twice. This is a documented issue on the MongoDB documentation.
Related
The picture above shows an example of a request getting sent to the following route:
and the following picture shows what have been inserted to the database in compass (notice how there are three entries):
As we know, Model.create() accepts an array of objects, or an object.
In this example, I am sending an array of objects, to insert them.
Model.create([]) will insert the documents one by one to the database, it doesn't skip the validation part, which is why I chose it.
and when it finds a document with a validation error, it skips it, and moves to the next one.
until it finishes, then it reports the errors it encounters.
That's what it should be, However it's not exactly working like that.
Note that I have two documents which holds validation errors:
However, mongoose is only reporting the first one, it's not reporting the second one, even though it passes by it, and it sees it.
Why this information is critical?
Because on the client side, I have to know which documents got inserted, and which did not.
In this case, (when I will know which are the ones got inserted and the ones that did not), I can show the client for example that the documents x, y, z has been inserted, while the documents f, g, h did not. So the user can correct his mistake and send the request again.
The current error report is useless, because it only tells "there was a validation error", but it doesn't tell you the "where"
The error report should include all the documents which refused to be written to the database in an array.
Update
I realized that
const data = await User.insertMany(req.body)
Has exactly the same behavior.
It doesn't only apply to Model.create().
Model.insertMany() has the same problem as well.
How to make mongoose report the full errors?
since we don't have a fold code option yet, I will include the code shown in the images, down here. and I hope this isn't going to polute the question.
mongoose.connect('mongodb://localhost:27017/temp', (err) => {
if (err) return log.error(log.label, 'an error occured while connecting to DB!')
log.success(log.label, 'Successfully connected to the database!')
})
app.use(express.json())
app.post('/users', async (req, res, next) => {
try {
const data = await User.collection.insertMany(req.body)
res.json({
message: 'success!',
data,
})
} catch (error) {
console.log(error)
res.json({
message: 'an error',
data: error,
})
}
})
I checked the source code of create() method. It indeed only saves the first error and not all the errors.
However, since the create() method will send one request for each item anyway, you can implement your own logic where you will wrap all the items with Promise.allSettled() and use the create() method for each item. That way, you will know exactly which item was successfully added, and which threw an error:
app.post('/users', async (req, res, next) => {
try {
const items = req.body;
const results = await Promise.allSettled(
items.map((item) => User.create(item))
);
constole.log(results.map((result) => result.status);
// Items that were successfully created will have the status "fulfilled",
// and items that were not successfully created will have the status "rejected".
return res.status(200).json({ message: 'success', results })
} catch (error) {
console.log(error)
return res.status(400).json({ message: 'an error', data: error })
}
})
Just want to draw your attention to insertMany issue-5337 which is similar to your question, they resolved it differently, like below:
Comment on issue: https://github.com/Automattic/mongoose/issues/5783#issuecomment-341590245
In hind sight, going to have to punt on this one until a later release because we need to return a different structure if rawResult is false. We can't just return a ValidationError like in #5698 because that would cause a promise rejection, which isn't correct with insertMany() with ordered: false because that's very inconsistent with how the driver handles it. Using rawResult: true is the way to go right now.
ordered: false should give you multiple validation errors, but if ordered is not set (true by default) we should fall back to the current behavior.
User.collection.insertMany(req.body, { rawResult: true, ordered: false })
.then(users => {
console.log(users)
})
.catch(err => {
console.log(`Error: ${err}`);
});
Console Print:
{
acknowledged: true,
insertedCount: 3,
insertedIds: {
'0': new ObjectId("63a09dcaf4f03d04b07ec1dc"),
'1': new ObjectId("63a09dcaf4f03d04b07ec1de")
'2': new ObjectId("63a0a0bdfd94d1d4433e77da")
},
mongoose: {
validationErrors: [
[
Error: WASetting validation failed: ....
at ValidationError.inspect (...)
...
errors: { itemId: [ValidatorError] },
_message: '.....'
],
[
Error: WASetting validation failed: ....
at ValidationError.inspect (...)
...
errors: { itemId: [ValidatorError] },
_message: '....'
]
]
}
}
You can see the above response, this is not giving which object failed.
Currently, I would suggest #NeNaD's solution.
I am trying to update MongoDB database with mongoose
Works but I can't get a call back with new data
Templates.updateOne((req.query, req.body))
does not work
Templates.updateOne(req.query, req.body, { new: true }, (err, data) => {
console.log("just call back", data);
})
the data that is coming from req.query
{title: "Hello World"}
the data that is coming from req.body
{_ref: "your ref}
I am hoping to be able to update with callback and new data to console.log
According to the doc, Document.prototype.updateOne() does not accept {new:true} (it is a native MongoDB command, I believe), but Model.findOneAndUpdate() does.
I am new to nodejs and mongodb.
I am trying to create simple to-do app with nodejs and mongodb.
I have added the task in database.
Now in post method, I am using insertOne method of mongodb and in res.json I am having the following error.
res.json(info.ops[0].data)
TypeError: Cannot read property '0' of undefined
Code :
app.post('/create-item', function(req, res){
db.collection('items').insertOne({ text:req.body.text }, function(err, info){
res.json(info.ops[0])
})
})
Below is the screenshot of Error.
In current versions, there is no property returned named ops when insertOne is successful.
Hence the error TypeError: Cannot read property '0' of undefined
insertOne returns two properties:
acknowledged
Indicates whether this write result was acknowledged. If not, then all other members of this result will be undefined
insertedId
The identifier that was inserted. If the server generated the identifier, this value will be null as the driver does not have access to that data
See:
InsertOne
InsertOneResult
app.post('/create-item', function(req, res){
db.collection('items').insertOne({ text:req.body.text }, function(err, info){
console.log(info.acknowledged)
console.log(info.acknowledged)
res.json(info.acknowledged)
})
})
In previous versions, for example 3.2, different properties were returned for insertOne:
insertOne #3.2
insertOneWriteOpCallback #3.2
Similarly, different properties were returned for updateOne:
updateOne #3.2
updateOneWriteOpCallback #3.2
For more information about migrating to version 4 from earlier versions, see the article:
Changes in 4.x (and how to migrate!)
I encountered the same problem with the to-do app tutorial today. For those interested try changing the line - res.json(info.ops[0])
to - res.json({ _id: info.insertedId.toString(), text: req.body.text })
This gets the inserted id from the database for the newly added item. As "info" doesn't seem to return the inserted text in the current version of mongodb - i added that from the request parameter.
I got my information from here - Get the _id of inserted document in Mongo database in NodeJS
you are doing it wrong way firstly check the err then send the res.json because the error is because your info might be null if data not inserted successfully so you need to do it like
app.post('/create-item', function(req, res){
db.collection('items').insertOne({ text:req.body.text }, function(err, info){
if (err) {
res.json({message: "not inserted successFully"});
return;
}
res.json(info.ops[0])
})
})
now in above code what will happen if data is not inserted successfully it will send the error as response and return from function.
Currently developing an API with nodejs that communicates with a MongoDB database, I noticed a special behavior after deleting a document.
Indeed, my API has several endpoints that allow to retrieve all the animals in the database, to retrieve a specific one using the corresponding id or to delete a specific one, again using the id of the document to delete.
The results I don't understand happen once a document is deleted. Indeed, as you can see in the picture below, when I delete the document of the animal called "Johnny" the queries to find it via its id or to delete it via the same ID continue to work, even if the get returns nothing and the deletion indicates that no operation has been performed.
Personally I expected the same behavior as if I passed a wrong id for a deletion (visible below), but if the id has already been assigned in the database the queries work even after a deletion.
Does MongoDB have a "cache" of deleted documents in order to perform a rollback in case of unwanted deletion?
You will find below the different endpoints that use find, deleteOne & findById
exports.getAllAnimal = (req, res, next) => {
Animal.find().sort({'customer' : 1})
.then(animals => res.status(200).send(animals))
.catch(error => res.status(400).send({ error: error.message }));
};
exports.getOneAnimal = (req, res, next) => {
Animal.findOne({ _id: req.params.id })
.then(animal => res.status(200).send(animal))
.catch(error => res.status(400).send({ error: error.message }));
};
exports.deleteAnimal = (req, res, next) => {
Animal.deleteOne({ _id: req.params.id })
.then(thing => res.status(200).send({ message : 'Animal successfully deleted'}))
.catch(error => res.status(400).send({ error: error.message }));
};
MongoDB does not cache deleted id anywhere.
The thing is that when you said I passed a wrong id for a deletion ... you are passing an id with the same length but not in the required format. That's why Mongoose is throwing you an error.
However, if you follow the id structure of MongoDB to create an id that does not exist in the database and run an operation against it, MongoDB will still return you with success and an empty result.
Try using 5ea08034385a46666b05020f and run the .findById() query function against it. It's going to return you with a success an empty result.
The success only means that the operation is successful, but it doesn't necessarily mean that it actually finds something in the database.
I don't have access to your database, so the id is generated randomly but following the MongoDB ObjectId rules below:
The 12-byte ObjectId value consists of:
a 4-byte timestamp value, representing the ObjectId’s creation, measured in
seconds since the Unix epoch
a 5-byte random value
a 3-byte incrementing counter, initialized to a random value
Generate arbitrary MongoDB ObjectId:
https://observablehq.com/#hugodf/mongodb-objectid-generator
I'm getting this error:
Unhandled rejection SequelizeUniqueConstraintError: Validation error
How can I fix this?
This is my models/user.js
"use strict";
module.exports = function(sequelize, DataTypes) {
var User = sequelize.define("User", {
id: { type: DataTypes.INTEGER, autoIncrement: true, primaryKey: true},
name: DataTypes.STRING,
environment_hash: DataTypes.STRING
}, {
tableName: 'users',
underscored: false,
timestamps: false
}
);
return User;
};
And this is my routes.js:
app.post('/signup', function(request, response){
console.log(request.body.email);
console.log(request.body.password);
User
.find({ where: { name: request.body.email } })
.then(function(err, user) {
if (!user) {
console.log('No user has been found.');
User.create({ name: request.body.email }).then(function(user) {
// you can now access the newly created task via the variable task
console.log('success');
});
}
});
});
The call to User.create() is returning a Promise.reject(), but there is no .catch(err) to handle it. Without catching the error and knowing the input values it's hard to say what the validation error is - the request.body.email could be too long, etc.
Catch the Promise reject to see the error/validation details
User.create({ name: request.body.email })
.then(function(user) {
// you can now access the newly created user
console.log('success', user.toJSON());
})
.catch(function(err) {
// print the error details
console.log(err, request.body.email);
});
Update, since it's 2019 and you can use async/await
try {
const user = await User.create({ name: request.body.email });
// you can now access the newly created user
console.log('success', user.toJSON());
} catch (err) {
// print the error details
console.log(err, request.body.email);
}
Check in your database if you have an Unique Constraint created, my guess is that you put some value to unique: true and changed it, but sequelize wasn't able to delete it's constraint from your database.
I had this issue with my QA database. Sometimes a new record would save to the database, and sometimes it would fail. When performing the same process on my dev workstation it would succeed every time.
When I caught the error (per #doublesharp's good advice) and printed the full results to the console, it confirmed that a unique constraint as being violated - specifically, the primary key id column, which was set to default to an autoincremented value.
I had seeded my database with records, and even though the ids of those records were also set to autoincrement, the ids of the 200-some records were scattered between 1 and 2000, but the database's autoincrement sequence was set to start at 1. Usually the next id in sequence was unused, but occasionally it was already occupied, and the database would return this error.
I used the answer here to reset the sequence to start after the last of my seeded records, and now it works every time.
When using SQLite as a database, Sequelize (currently 5.21.5) throws SequelizeUniqueConstraintError on every constraint error, even if it has nothing to do with unique indexes. One examle would be inserting NULL into a non-nullable column. So be sure to also check for other types of errors when debugging this exception.
Building on #Ricardo Machado's answer, if you add unique:true to a model and already have values in an existing table that wouldn't be allowed under this new constraint you will get this error. To fix you can manually delete the rows or delete the table and so Sequelize builds it again.
In case you stumble upon this Validation error from Sequelize: check that the query populating (creating) the table is performed once.
My Nest.js app was not properly set up and executed the same command twice thus violating the unique constraint.