MongoDB insert without duplicates - javascript

Right now I am running mongodb and I just realized, I am inserting into collections and I am not sure if I am preventing duplicates. Here is how I am inserting:
function insertCompanies(companyID, companyURL, companyAppID) {
MongoClient.connect(url, function(err, db) {
if (err) {
console.log(err);
} else {
console.log("We are connected");
}
var collection = db.collection('Companies');
var company = {
"companyProfileID": companyID,
"url": companyURL,
"appID": companyAppID
};
collection.insert(company, function(err, result) {
if (err) {
console.log(err);
} else {
console.log(result);
}
});
db.close();
});
}
I am using NodeJS. When a user calls the insertCompanies method, if the company already exists (via companyID or URL), it seems like the collection allows duplicates.
Is there a different insert function that prevents duplicates? I looked around and could not find a straight forward solution tailored to me.

Instead of db.insert() you should use db.update() and specify $upsert: true.
https://docs.mongodb.com/v3.2/reference/method/db.collection.update/

The answer is not to insert, instead use update. The update() method either modifies the fields in existing documents or replaces an existing document entirely.

Related

Only one document in dbs - how to update it?

I have a node js program where I only want to allow one document in a specific collection.
Mongo Schema:
var DefaultPollSchema = new mongoose.Schema({
milliseconds: Number
});
So the collection always looks like this:
> db.defaultpolls.find()
{ "_id" : ObjectId("58500051a4b71d0c4d5c6ab3"), "milliseconds" : 15000, "__v" : 0 }
I want to create an API for updating this value, however I do not want to use the objectId in order to do the update. Since I will only have one document in this collection - is there a way to do it?
I tried using this:
router.put('/update', function(req, res, next) {
var milliseconds = req.body.milliseconds;
defaultPoll.updateMany({}, {$set: {milliseconds: milliseconds}}, function (err, interval){
if (err) {
res.json(err);
} else {
res.json(interval);
}
})
});
However this gives me the following error:
TypeError: Object function model(doc, fields, skipId) {
if (!(this instanceof model)) {
return new model(doc, fields, skipId);
}
Model.call(this, doc, fields, skipId);
} has no method 'updateMany'
Is there any other way to write the API?
Thanks
The method for Mongoose updates is called update, so the approach is fine, but it should be:
defaultPoll.update({}, {$set: {milliseconds: milliseconds}}, function (err, raw) { ...
JohnnyHK's solution will not work. Mongoose documentation mentions the following:
Passing an empty object {} as the doc will result in a no-op unless
the overwrite option is passed. Without the overwrite option set, the
update operation will be ignored and the callback executed without
sending the command to MongoDB so as to prevent accidently
overwritting documents in the collection.
So the following code should work in your case
(Note the { overwrite: true } as options):
router.put('/update', function(req, res, next) {
var milliseconds = req.body.milliseconds;
defaultPoll.update({}, {$set: {milliseconds: milliseconds}}, { overwrite: true }, function (err, interval){
if (err) {
res.json(err);
} else {
res.json(interval);
}
})
});

Deleting / removing with Mongoose with multiple parameters

This is my first attempt at deleting data in a MongoDB database. I'm loosely following this tutorial (just the delete part) to no avail, https://www.airpair.com/javascript/complete-expressjs-nodejs-mongodb-crud-skeleton. I just want to delete all the requested people who are in the requested country. All of my other requests work so I will just post the code that I know is not working, everything else is fine.
EDIT
The error I get in the log is "404 Not Found". When testing w/ Postman the response I get is, "Cannot DELETE /deletepeople/USA/John"
app.delete('deletepeople/:country/:name', function(req, res) {
var countryReq = req.params.country;
var nameReq = req.params.name;
peopleModel
.find({"country":countryReq}, function(err, country) {
country.find({"name": nameReq}, function (err, person) {
person.remove(function (err, person) {
if (err) {
console.log(err);
res.status(500).send();
}
return res.status(200).send();
})
})
})
});
});
country.find({"name": nameReq}, function (err, person) {
The above line is causing you an error, what are you searching in a returned document? Its just an document and not a collection.
You can use the id() method in embedded docs:
Look at the subdocuments [http://mongoosejs.com/docs/subdocs.html]

How to call delete from other class MEAN

I can't figure out how I delete data in another MongoDB scheme when I create.
I'm running on mean.js stack.
exports.create = function(req, res) {
var sign = new Sign(req.body);
sign.save(function(err) {
if (err) {
return res.status(400).send({
message: errorHandler.getErrorMessage(err)
});
} else {
sign.timesheets.forEach(function(entry) {
console.log(entry._id);
});
res.jsonp(sign);
}
});
};
Here I make a call to create a sign. Sign includes some strings and an array 'timesheets' with timesheet objects.
I also got a scheme for timesheets, I want to delete all timesheets objects that are included in my sign from timesheets scheme.
Here is my timesheets delete controller:
exports.delete = function(req, res) {
var timesheet = req.timesheet;
timesheet.remove(function(err) {
if (err) {
return res.status(400).send({
message: errorHandler.getErrorMessage(err)
});
} else {
res.jsonp(timesheet);
}
});
};
How do I call this from min sign controller, for each timesheet in sign?
Edit:
Route:
module.exports = function(app) {
var sign = require('../../app/controllers/sign.server.controller');
var timesheets = require('../../app/controllers/timesheets.server.controller');
app.route('/sign')
.post(sign.create, timesheets.deleteAll);
delete all
exports.deleteAll = function(req, res) {
var timesheet = req.timesheet;
timesheet.timesheets._id.forEach(function(entry) {
entry.remove(function(err) {
console.log(entry);
if (err) {
return res.status(400).send({
message: errorHandler.getErrorMessage(err)
});
} else {
res.jsonp(entry);
}
});
});
};
It runs and I can see id in the console, but it doesn't delete anything.
If you've implemented this as an array of TimesheetSchema documents inside each SignSchema document, then deleting the parent document would take everything that's part of it with it.
If you stored it as an array of ids referencing a document in another collection, then you'd have to go through those one by one and remove them as well. But I think it's better to go with the first approach if you don't need to do anything fancy. This was you can handle the removal easily, and it makes better sense semantically and performance-wise to retrieve everything you need to handle a "sign" in one go.

Handling timeouts with Node.js and mongodb

I am currently testing how some code stands up against the following scenario:
Node.js application is started and successfully establishes a connection to mongodb
After it has successfully setup a connection, the mongodb server dies and all subsequent requests fail
To do this I have got the following code which makes use of the official driver (found here: https://github.com/mongodb/node-mongodb-native) :
MongoClient.connect('mongodb://localhost:27017/testdb', function(err, db) {
app.get('/test', function(req, res) {
db.collection('users', function (err, collection) {
console.log(err);
if (err) {
// ## POINT 1 ##
// Handle the error
}
else {
collection.find({ 'username': username }, { timeout: true }).toArray(function(err, items) {
console.log(err);
if (err) {
// ## POINT 2 ##
// Handle the error
}
else {
if (items.length > 0) {
// Do some stuff with the document that was found
}
else {
// Handle not finding the document
}
}
});
}
});
});
});
As the mongodb server is no longer running when the request is being handled, I'd made the assumption that at either the points which I have labelled ## POINT 1 ## or ## POINT 2 ##, it would return an error indicating a timeout; this however, isn't the case.
I have tried a number of different settings (including one you can see here that explicitly allows the cursor to timeout), however I cannot seem to enable it in any way. In every configuration I've tried Node.js will simply keep waiting for the find() operation to callback and it never does.
If I start the Node.js application before running mongodb, it catches the error in the connect callback fine, but if the connection dies after that it doesn't seem to handle it in any way.
Is there a setting I am missing or is there no way to detect connections being terminated after they've been established?
Edit: just to be clear, the username variable used in the find method is actually declared in my full code, the code I've put in this post is a cut down version to illustrate the structure and error checking.
UPD:
Based on this post, looks like they've deployed fix that will do the same as what we do here. Not sure if this is already within npm (15.10.13). https://github.com/mongodb/node-mongodb-native/issues/1092#ref-commit-2667d13
After some investigation I've managed to understand what is going on there:
Every time you call any method to deal with database (find, update, insert, etc.) it creates cursor, that has own ID and registers itself to EventEmitter of Db for being called back later. In meantime it registers itself to _notReplied object within same CallBackStore.
But once connection is closed, I couldn't locate anything that would iterate through _notReplied cursors and would trigger them with errors or any logic with timers (it still might be somewhere there). So I've managed to write small work around, that does force triggers cursors with error when DB emits close event:
new mongodb.Db('testdb', new mongodb.Server('localhost', 27017, { }), { safe: true }).open(function (err, db) {
if (!err) {
db.on('close', function() {
if (this._callBackStore) {
for(var key in this._callBackStore._notReplied) {
this._callHandler(key, null, 'Connection Closed!');
}
}
});
// ...
} else {
console.log(err)
}
});
I recommend using first approach instead of MongoClient. Reasons are few: for example when you close connection and then call .find it will properly trigger error in callback, while with MongoClient it won't.
If you are using MongoClient:
MongoClient.connect('mongodb://localhost:27017/testdb', function(err, db) {
if (!err) {
db.on('close', function() {
if (this._callBackStore) {
for(var key in this._callBackStore._notReplied) {
this._callHandler(key, null, 'Connection Closed!');
}
}
});
// ...
} else {
console.log(err);
}
});
What this will do? Once connection is closed, it will iterate through All _notReplied cursors and trigger events for them with error Connection Closed!.
Test case:
items.find({ }).toArray(function(err, data) {
if (!err) {
console.log('Items found successfully');
} else {
console.log(err);
}
});
db.close();
That will force close database connection and trigger close event that you handle earlier and will make sure that cursor will be closed.
UPD:
I've added Issue on GitHub: https://github.com/mongodb/node-mongodb-native/issues/1092 we'll see what they say regarding this.
I had the same problem, and found this page from google.
But your choosed answer didn't resolve the problem and it is as same as you, this._callBackStore can't use
but i tried to wrap the Mongo, and it seems work fine
var MongoClient = require('mongodb').MongoClient;
var mongo = {};
mongo.init = function() {
MongoClient.connect('mongodb://localhost:27017/testdb', function(err, db) {
if (err) {
mongo.DB = '';
} else {
mongo.DB = db;
}
db.on('close', function() {
mongo.DB = '';
});
db.on('reconnect', function() {
mongo.DB = db;
});
}
}
mongo.getdb = function(callback) {
if (mongo.DB) {
callback(null, mongo.DB);
} else {
callback('can not connect to db', null);
}
}
module.exports = mongo;
firstly start server and init() it
and then you can require it and use
mongo.getdb(function(err, db) {
if (err) {
console.log(err);
} else {
db.collection('user').find({'xxx':'xxx'}).toArray(function(err, items) {
console.log(items);
});
}
});
After some further investigation, it seems that you can't specify "offline" timeouts such as in the scenario outlined above. The only timeout that can be specified is one which informs the server to timeout the cursor after 10 minutes of inactivity, however as in the scenario above the connection to the server is down this does not work.
For reference, I found the information here: https://github.com/mongodb/node-mongodb-native/issues/987#issuecomment-18915263 by who I believed to be one of the main contributors to the project.
I'm making api with Hapi and Mongodb (w/o mongoose). Features:
Start responding to API request only if mongo db is available
Stop responding if mongo dies during cycle
Re-start when mongo available again
Keep single connection for all requests
Combining some ideas from other answers and this post https://productbuilder.wordpress.com/2013/09/06/using-a-single-global-db-connection-in-node-js/ my approach is this:
server.js
Utilities.initializeDb(() => {
server.start((err) => {
if (err) throw err;
console.log('Server running at:', server.info.uri);
});
}, () => {
server.stop((err) => {
if (err) throw err;
console.log('Server stopped');
});
});
Utilities.js
"use strict";
const MongoClient = require('mongodb').MongoClient;
const MongoUrl = 'mongodb://localhost:27017/db';
export const Utilities = {
initializeDb: (next, onCrash) => {
const ConnectToDatabase = (params) => {
MongoClient.connect(MongoUrl, (err, db) => {
if (err !== null) {
console.log('#t4y4542te Can not connect to mongo db service. Retry in 2 seconds. Try #' + params.retry);
console.error(err);
setTimeout(() => {
ConnectToDatabase({retry: params.retry + 1});
}, 2000);
} else {
db.on('close', () => {
onCrash();
console.log('#21df24sf db crashed!');
ConnectToDatabase({retry: 0});
});
global.db = global.db || db;
next();
}
});
};
ConnectToDatabase({retry: 0});
}
};
I'm exporting db connection to global space. It feels like not best solution, but I had projects where db connection was passed as param to all modules and that sucked more. Maybe there should be some modular approach where you import db connection where you need it, but in my situation i need it almost everywhere, I would have to write that include statement in most files. This API is pointless w/o connection to db, so I think it might be best solution even if I'm against having something flying magically in global space..

Sails.js - issues when trying to display model data

I have no problem retrieving all my models from the database and displaying them on page using this code:
index: function(req, res) {
Applicant.find(function(err, applicants) {
if (err) {
return console.log(err);
}else{
res.view({
apps: applicants
});
}
});
}
But, if I try to pull just one model and display it, my browser gets stuck on loading. This is the code that I use for pulling just one model:
display: function(req, res) {
Applicant.find().where({id: 2}).done(function(err, appl) {
if (err) {
return console.log('HAI');
}else{
res.view({
applicant: appl
});
}
});
}
Likely, your browser is stuck because an error happens when you're trying to find an Applicant, and your code doesn't return any response in this case. So browser waits for response forever.
Please try something like this
if (err) {
console.log('HAI');
return res.send(err, 500);
}
P.S. By the way, as of Sails v0.9, find() method will alwals return an array, even if only one record is found. If you want to find just one record by id, and expect single object in your view, you can use findOne() method.
.find() returns an array. You may be expecting a single applicant object.
Using appl[0] would solve this. Please note that Sails' Waterline ORM provides .findOne() for situations such as these. Here's more info on .findOne()
display: function(req, res) {
Applicant.find().where({id: 2}).done(function(err, appl) {
if (err) {
return console.log('HAI');
}else{
res.view({
applicant: appl[0]
});
}
});
}
Or better yet...
display: function(req, res) {
Applicant.findOne({id: 2}, function(err, appl) {
if (err) {
return console.log('HAI');
}else{
res.view({
applicant: appl
});
}
});
}

Categories