Mongodb - Handling insertMany unique constraint violation - javascript

I'm working with node and mongodb 5. I have a unique index added to the Parcel property of my collection. As I run the program while testing, I get:
MongoBulkWriteError: E11000 duplicate key error collection: mydata.pima2 index: Parcel_1 dup key: { Parcel: "AARON" }
My code:
for (let j = 0; j < lastNameRecords.length; j++) {
const ln = lastNameRecords[j].name;
const recordsObj = { 'Parcel': ln, 'recordNum': 'TBD' };
recordsArr.push(recordsObj);
}
console.log('number of records: ', recordsArr.length);
try {
--> const response = await collection.insertMany(recordsArr, { ordered: false });
const updated = await collection.updateOne(result, { recordNum: 'ERD' });
} catch (error) {
console.log(error);
}
Th error is occurring at the line with the arrow above. Obviously as I test, I am inserting multiple records , all of which have been previously inserted into the 'pima2' collection. How do I avoid causing an error or handle it so that I can move on to the next statement

Your error is related to your constraint violation, so it's prevent the scenario of repeated Parcel data.
{ Parcel: "AARON" }
There's two options to you,
Basically you can remove this constraint if you really don't need it and the error may disappear
Validate if this value already exists in your collection, after that create a find query passing this Parcel value and check if this value already exists. So you can advice to the user that this value already exists.

Related

how to use a for loop after a mongoDB update?

I have a route like this :
router.put("/:id", upload.single('doc_upload'), async(req,res)=>{
try{
const updatedUser = await User.findOneAndUpdate({ "client._id": req.params._id,"clients.documents._id":req.body.doc_id},
{"$set":{"clients.$[clientFilter].documents.$[documentFilter].status":"uploaded"}},
{"arrayFilters":[
{"clientFilter._id":req.params.id},
{"documentFilter._id":req.body.doc_id}
]} ,
);
for (i=0; i < updatedUser.clients.length; i++) {
if (updatedUser.clients[i]._id == req.params.id) {
for (j=0; j < updatedUser.clients[i].documents.length; j++) {
console.log("this is the status of all my docs" + updatedUser.clients[i].documents[j].status)
}
}
}
res.send(req.file)
} catch(err){
res.status(500).json(err);
}
It first searches for a user then upload the « status » (which is by default « pending »for « uploaded »)
My problem is the loop, I’m trying to console log the status of the doc, but even if the update is made in the db it still prints « pending » for all my docs. I guess what I’m trying to do is to find a way to first update the user then console.log everything.
Any idea on how to do that please ?
As you said you saw db and your required document was updated successfully. Mongoose findOneAndUpdate updates the document but it does not return updated document. To return updated document you should use { new: true } object as additional argument as :
let options = { new: true }; DBModel.findOneAndUpdate( findCondition, updateData, options, (error, updatedDoc) => {});
So now since your query returned a updated document, your problem will be solved.

How to update array inside object, inside array mongoDB

So the document contains an array of objects, each object containing it's own array. So how would I go about updating one of the elements in the array that's inside the object which is inside another array. I've read some things with $. But I don't understand completely how to use it to call a position. I know the position of the element. But I can't just say $[] because the position is defined in a variable and not a string...
I've tried doing a simple
db.collection.findOne({...}, (err, data) => {...});
and then changing the arrays in the objects in the array in there with a simple:
data.arr[x].type[y] = z; data.save().catch(err => {console.log(err)});
But it doesn't save the new values I set for for the element of the array.
Sample structure after proposed solution from #Tom Slabbaert:
Data.findOne({
userID: 'CMA'
}, (err, doc) => {
if(err) {console.log(err)}
if(doc) {
for(var i = 0; i<CMA.stockMarket.length; i++) {
if(CMA.stockMarket[i].name == data.userID) {
for(var z = 0; z<CMA.stockMarket[i].userStock.length; z++) {
if(z == company) {
var updateAmount = CMA.stockMarket[i].userStock[z]+args[1]
var updateKey = `stockMarket.${i}.userStock.${z}`
Data.updateOne({userID: 'CMA'}, {'$set': {[updateKey]: updateAmount}})
}
}
}
}
}
});
-------------------------EDIT-------------------------
So I tried changing some things around in the data base to see if that would fix the problem I was having. I modified the updated code that was provided by #Tom Slabbaert. But nothing seems to work for some reason :/ Here's what I have so far, at this point I hope it's just a syntax error somewhere. Cause this is really frustrating at this point. Note that I'm still using the for loops here to find if the info exists. And if not, push that info into the database. This might only be temporary until I find a better way / if there is a better way.
for(var i = 0; i<CMA.userStocks.length; i++) {
if(CMA.userStocks[i].name == data.userID) {
for(var z = 0; z<CMA.userStocks[i].shares.length; z++) {
//console.log(CMA.userStocks[i].shares[z].companyName)
if(CMA.userStocks[i].shares[z].companyName == args[0]) {
var updateKey = `CMA.userStocks.$[elem1].shares.$[elem2].amount`
Data.updateOne(
{userID: 'CMA'},
{
"$inc": {
[updateKey]: args[1]
}
},
{
arrayFilters: [
{
"elem1.name": data.userID,
"elem2.companyName": args[0]
}
]
}
)
purchaseComplete(); return;
}
}
CMA.userStocks[i].shares.push({companyName: args[0], amount: parseInt(args[1])})
CMA.save().catch(err => {console.log(err)});
purchaseComplete(); return;
}
}
CMA.userStocks.push({name: data.userID, shares: [{companyName: args[0], amount: parseInt(args[1])}]});
CMA.save().catch(err => {console.log(err)});
purchaseComplete(); return;
The data I'm trying to find and change is structured like the following:
And what I'm trying to change in the end is the 'amount' (which is an integer)
_id: (Not relavent in this question)
userID: 'CMA'
stockMarket: [...] (Not relavent in this question)
userStocks: [
Object: (position 0 in userStocks array)
name: 'string' (equal to data.userID in the code)
shares: [
Object: (position 0 in shares array)
companyName: 'string' (this is args[0] in the code)
amount: integer
]
]
You can just prepare the "key" ahead of time. like so:
const updateKey = `arr.${x}.type.${y}`
db.collection.updateOne(
{...},
{
"$set": {
[updateKey]: z
}
})
Mongo Playground
Using Mongo's positional operators ($ and $[]) are usually required when you don't know the position in the array and want to use a condition to update the element.
------ EDIT-----
After given your sample code you just have a minor syntax error:
var updateKey = `stockMarket.${i}.userStock.${z}`
Should just be:
var updateKey = `CMA.stockMarket.${i}.userStock.${z}`
However After seeing your code I recommend you execute the following solution which uses a single update with arrayFilters, it just cleans up the code quite a bit:
const updateKey = `CMA.stockMarket.$[elem1].userStock.${company}`;
db.collection.update(
{userID: 'CMA'},
{
"$inc": {
[updateKey]: args[1]
}
},
{
arrayFilters: [
{
"elem1.name": data.userID
}
]
})
Mongo Playground
Well I found something that worked. Apparently it didn't save the db.collection.updateMany unless I made a .then() function on the end? I have no idea why, but it's the same with an aggregate I made. (It basically does the same as a Data.findOne and save it too, but it isn't limited by the parallel save error)
Solution I found with aggregation:
<collection field> = <new data for collection field>
Data.aggregate([
{
$match: { //This is used to create a filter
['<insert field>']: <insert filter>
}
}, {
$addFields: { //This is used to update existing data, or create a new field containing the data if the field isn't found
['<collection field>']: <new data for collection field>
}
}, {
$merge: { //This is used to merge the new data / document with the rest of the collection. Thus having the same effect as a standard save
into: {
db: '<insert database name>',
coll: '<insert collection name>'
}
}
}
]).then(() => {
//After it's done, do something here. Or do nothing at all it doesn't matter as long as the .then() statement remains. I found that not having this part will break the code and make it not save / work for some reason.
}); return;
Solution I found with db.collection.updateMany
db.collection.updateMany(
{<insert field>: filter}, {$set: {'<insert field>': <new data>}}
).then(() => {
//This .then() statment in my case was needed for the updateMany function to work correctly. It wouldn't save data without it for some reason, it does not need to contain any actual info in this part. As long as it's here.
});
With this new info I could simply access and change the data that I was trying to before using the previous instructions provided by #Tom Slabbaert and my new method of actually making it save the changes made into the document.

Getting object store already exists inside onupgradeneeded

My code is as follows (usually naming convention for the well-known objects):
var DBOpenRequest = window.indexedDB.open("messages", 6);
//...
DBOpenRequest.onupgradeneeded = function(event) {
console.log("Need to upgrade.");
var db = event.target.result;
console.log(db);
db.onerror = function(event) {
console.log("Error upgrading.");
};
// Create an objectStore for this database
var objectStore = db.createObjectStore("messages", { keyPath: "id", autoIncrement: true });
};
This ran fine for versions 3 and 4. When it came to version 5, I get the error:
Failed to execute 'createObjectStore' on 'IDBDatabase': An object store with the specified name already exists.
at IDBOpenDBRequest.DBOpenRequest.onupgradeneeded
Isn't the createObjectStore operating on a new version of the database which is empty? How do I fix the error?
I happened to log the db object and the details are below:
I am curious why the version number is different in the summary line and when expanded.
Isn't the createObjectStore operating on a new version of the database which is empty?
When you get upgradeneeded the database is in whatever state you left it in before. Since you don't know what versions of your code a user will have visited, you need to look at the event's oldVersion to find out what that was. The typical pattern is something like this:
var rq = indexedDB.open('db', 5);
rq.onupgradeneeded = function(e) {
var db = rq.result;
if (e.oldVersion < 1) {
// do initial schema creation
db.createObjectStore('users');
}
if (e.oldVersion < 2) {
// do 1->2 upgrade
var s = db.createObjectStore('better_users');
s.createIndex('some_index', ...);
db.deleteObjectStore('users'); // migrating data would be better
}
if (e.oldVersion < 3) {
// do 2->3 upgrade
rq.transaction.objectStore('better_users').createIndex('index2', ...);
}
if (e.oldVersion < 4) {
// do 3->4 upgrade
db.createObjectStore('messages', ...);
}
if (e.oldVersion < 5) {
// do 4->5 upgrade
// ...
}
}
I am curious why the version number is different in the summary line and when expanded.
That one is subtle... I believe at the point where the 5 was logged the database had started the upgrade. But because an exception was thrown in the upgradeneeded handler the upgrade was aborted, and the version number was rolled back to 4 before the details were logged.
The best way to upgrade the DB is checking if the store name is already there. In this example I'm using https://npmjs.com/idb
openDB('db-name', version, {
upgrade(db, oldVersion, newVersion, transaction) {
if(!db.objectStoreNames.contains('messages')) {
db.createObjectStore('messages', { keyPath: "id", autoIncrement: true })
}
}
})
If you need to check if an indexName already exist, you can get the objectStore and check for the indexNames property if it contains the indexName you need.
openDB('db-name', version, {
upgrade(db, oldVersion, newVersion, transaction) {
const storeName = transaction.objectStore('storeName')
if(!storeName.indexNames.contains('indexName')) {
storeName.createIndex('indexName', 'propertyName', { unique: false });
}
}
})
Using indexDB API with indexNames and objectStoreNames to check if something is either there or not makes my code way more reliable and easy to maintain, it is also briefly mentioned on Working with IndexDB Using database versioning

Why is hashtagseen[] empty after I call the addposthashtags function?

I am trying to add hashtags in the post's hashtag[] array as a object with a num:1 variable to the users hashtagseen[] array if it is not already in it else add 1 the num if the hashtag is already in the hashtagseen[] array. How do I fix my code? Here is the code, thanks in advanced.
edit: I think I am not finding post.hashtag with this.hashtag and that is why it will not go to else. Just a guess.
The user object
Accounts.createUser({
username: username,
password: password,
email: email,
profile: {
hashtagsl:[],
}
});
collections/post.js
var post = _.extend(_.pick(postAttributes, 'title', 'posttext','hashtags'), {
userId: user._id,
username: user.username,
submitted: new Date().getTime(),
commentsCount: 0,
upvoters: [], votes: 0,
});
calling it
Meteor.call('addposthashtags',this.hashtags,Meteor.user().profile.hashtagsl);
lib/usershash
Meteor.methods({
addposthashtags: function (hashtags,hashtagsl) {
//supposed to make hashtagseen a array with the names from the hashtagsl object in it
var hashtagseen = _.pluck(hashtagsl, 'name');
//supposed to run once per each hashtag in the posts array.
for (var a = 0; a < hashtags.length; a++) {
//supposed set hashtagnumber to the number indexOf spits out.
var hashnumber=hashtagseen.indexOf(hashtags[a]);
//supposed to check if the current hashtag[a] === a idem in the hashtagseen.
if(hashnumber===-1){
var newhashtag = this.hashtags[a];
//supposed to make the object with a name = to the current hashtags
Meteor.users.update({"_id": this.userId},{"$push":{"profile.hashtagsl": {name: newhashtag, num: 1}}})
} else {
var hashi = hashtagseen[hashnumber];
//supposed to ad one to the num variable within the current object in hashtagsl
Meteor.users.update({"_id": this.userId, "profile.hashtagsl.name":hashi},{"$inc":{"profile.hashtagsl.num":1}});
}
}
}
});
Your addposthashtags function is full of issues. You also haven't provided a "schema" for hashtag objects.
addposthashtags: function () {
for (a = 0; a < this.hashtag.length; a++) {
// Issue1: You're querying out the user for every iteration of the loop!?
for (i = 0; i < Meteor.user().profile.hashtagseen.length; i++) {
// Issue2: You're comparing two _objects_ with ===
// Issue3: Even if you use EJSON.equals - the `num` property wont match
// Issue4: You're querying out the user again?
if (this.hashtag[a] === Meteor.user().profile.hashtagseen[i]) {
// Issue5 no `var` statement for hashtagseeni?
// Issue6 You're querying out the user again??
hashtagseeni = Meteor.user().profile.hashtagseen[i];
//Issue7 undefined hashtagsli?
//Issue8 Calling multiple methods for the one action (eg in a loop) is a waste of resources.
Meteor.call('addseen', hashtagsli);
} else {
//Issue9 no `var` statement for newhashtag?
newhashtag = this.hashtag[a];
newhashtag.num = 1;
//Issue8b Calling multiple methods for the one action (eg in a loop) is a waste of resources.
Meteor.call('updateUser', newhashtag, function (err, result) {
if (err)
console.log(err);
});
}
}
}
}
Also, the method has similiar issues:
addseen: function (hashtagseeni) {
// Issue10: var `profile` is undefined
// Issue11: should use `this.userId`
// Issue12: hashtagseeni wouldn't match profile.hashtagseen due to "num" field.
Meteor.users.update({"_id": Meteor.userId, "profile.hashtagseen": profile.hashtagseeni}, {"$inc":{"profile.hashtagseen.$.num":1}});
}
New issues with your new set of code:
Meteor.methods({
addposthashtags: function (hashtags,hashtagsl) {
//Issue1 `hashtag` is undefined, guessing you mean `hashtags`
//Issue2 no `var` for a
for (a = 0; a < hashtag.length; a++) {
//Issue3 no `var` for i
//Issue4 Why are you looping through both?
// don't you just want to check if hashtag[a] is in hashtagsl?
for (i = 0; i < hashtagsl.length; i++) {
if (hashtags[a] === hashtagsl[i].name) {
var hashi = hashtagsl[i].name;
//supposed to ad one to the num variable within the current object in hashtagsl.
// Issue5: This query wont do what you think. Test until you've got it right.
Meteor.users.update({"_id": Meteor.userId, 'profile.hashtagsl':hashi}, {"$inc":{"num":1}});
} else {
// Issue6 `this.hashtag` isn't defined. guessing you mean `hashtags[a]`
var newhashtag = this.hashtag[a];
// Issue7 superfluous statement
var newhashtagnum = num = 1;
// Issue8 Obvious syntax errors
// Perhaps try Meteor.users.update({"_id": this.userId},{"$push":{"profile.hashtagsl": {name: newhashtag, num: 1}}})
Meteor.users.update({"_id": Meteor.userId, 'profile'},{"$addToSet":{"hashtagsl"[newhashtag]=newhashtagnum}})
};
};
};
};
});
I'd suggest you trying the following
1) Assuming that after newhashtag=hashtag[a] you get a JSON object in newhashtag variable, try replacing newhashtag:{num:1}; with newhashtag.num = 1 - this will add the num variable to the object and set the value.
1.a) For debugging purposes try adding some console.log(JSON.stringify(newhashtag)); after each of the two lines where you're setting and changing the newhashtag variable - this way you'll know exactly what you're trying to add to the mongoDB document.
2) The update to increment the views also doesn't seem to me that will work. Couple of things to note here - $set:{'profile.hashtagseen[i]':num++} - MongoDB won't be able to identify the 'i' in 'profile.hashtagseen[i]' and 'num++' is not how increments are done in Mongo.
I'd suggest you look into the $inc and to the positional update documentation of MongoDB.
Your final increment update statement will look something like
Meteor.users.update({"_id": Meteor.userId, "profile.hashtagseen": profile.hashtagseen[i]}, {"$inc":{"profile.hashtagseen.$.num":1}});
I see that executing addposthashtags is in the client, and you must to pay attention because this function will execute in minimongo and doesn't work all operations. First you try execute this operation under mongo if it's work you must to create one function inside the folder server.
Add text of the documentation of Minimongo
In this release, Minimongo has some limitations:
$pull in modifiers can only accept certain kinds of selectors.
findAndModify, aggregate functions, and map/reduce aren't supported.
All of these will be addressed in a future release. For full Minimongo
release notes, see packages/minimongo/NOTES in the repository.
Minimongo doesn't currently have indexes. It's rare for this to be an
issue, since it's unusual for a client to have enough data that an
index is worthwhile.
You try create one method on the server, with the same operation.
Server:
Meteor.methods({
updateUser: function (newhashtag) {
Meteor.users.update(this.userId,
{
$addToSet: {'profile.$.hashtagseen': newhashtag}
});
}
});
Client:
Meteor.call('updateUser',newhashtag,function(err,result){
if (err)
console.log(err);// there you can print the erro if there are
});
Minimongo doesn't support alls operation, for test you can to execute in the console for testing the method if supported. After that you can to execute the operation under mongo directly, that clears your doubts.

Check for duplicate record in Chrome Storage extension before saving

I'm developing a small Chrome extension that would allow me to save some records to chrome.storage and then display them.
I've managed to make the set and get process work as I wanted (kinda), but now I'd like to add a duplicate check before saving any record, and I'm quite stuck trying to find a nice and clean solution.
That's what I came up for now:
var storage = chrome.storage.sync;
function saveRecord(record) {
var duplicate = false;
var recordName = record.name;
storage.get('records', function(data) {
var records = data.records;
console.log('im here');
for (var i = 0; i < records.length; i++) {
var Record = records[i];
if (Record.name === recordName) {
duplicate = true;
break;
} else {
console.log(record);
}
}
if (duplicate) {
console.log('this record is already there!');
} else {
arrayWithRecords.push(record);
storage.set({ bands: arrayWithRecords }, function() {
console.log('saved ' + record.name);
});
}
});
}
I'm basically iterating on the array containing the records and checking if the name property already exists. The problem is it breaks basic set and get functionality -- in fact, when saving it correctly logs 'im here' and the relative record object, but it doesn't set the value. Plus, after a while (generally after trying to list the bands with a basic storage.get function) it returns this error:
Error in response to storage.get: TypeError: Cannot read property
'name' of null
I'm guessing this is due to the async nature of the set and get and my incompetence working with it, but I can't get my head around it in order to find a better alternative. Ideas?
Thanks in advance.

Categories