Collection insert multiple - ensure uniqueness - javascript

The main goal is: Insert multiple documents at a time but ensure only unique ones are stored in the collection Explanation below
I'm inserting many documents into a mongodb collection at once like so:
db.users.insert([{name: 'Joe'}{'name': 'Bob'}]);
Some of the values going into the insert may be duplicate and i would like MongoDB to ignore these. This is to ensure uniqueness within the collection.
db.users.insert([{name: 'Joe'}{'name': 'Jack'}]);
Using an index the whole .insert() operation will fail if the whole insert is not unique instead of just the non unique items being ignored.
db.users.createIndex({ 'name': 1 }, { unique: true });
Log from mongoDB:
BulkWriteResult({
"writeErrors" : [
{
"index" : 0,
"code" : 11000,
"errmsg" : "E11000 duplicate key error collection: test.users index: name_1 dup key: { : \"Joe\" }",
"op" : {
"_id" : ObjectId("57f924ee7d864acfefaca700"),
"name" : "joe"
}
}
],
"writeConcernErrors" : [ ],
"nInserted" : 0,
"nUpserted" : 0,
"nMatched" : 0,
"nModified" : 0,
"nRemoved" : 0,
"upserted" : [ ]
})
Is there another way of doing this?

Forget the comment. Maybe you're looking fro something like this. UnorderedBulkOp
The magic cames from that Unordered Bulk Operations occur in parallel. if one ( or some) of them fai, MongoDB continue with the rest and shouts the errors at the end.

For people looking for an answer to this. I eventually use a bulk update.upsert() operation. This meant that I could keep my capped collection, make sure it was ordered and for the whole operation to still be in bulk.
This removed the need for the unique index, as upsert always updates/inserts the data ensuring we don't have duplicates.
The find operation should include the unique keys.
const bulk = db.collection.initializeOrderedBulkOp();
bulk.find({name: 'Joe'}).upsert().updateOne({name: 'Joe'});
bulk.find({name: 'Bob'}).upsert().updateOne({name: 'Bob'});
bulk.find({name: 'Jack'}).upsert().updateOne({name: 'Jack'});
bulk.execute();

For inserting multiple documents with some unique key, you may create an index, where you can declare that key to be unique.
Like in your case, you can create index using this command,
db.users.createIndex({"name":1},{unique:true})

Related

Mongodb E11000 duplicate key error on array field

I have a quick question about mongoose schema real quick. Here is the code: https://i.ibb.co/Db8xPMw/5555.png
I tried to create a document without the property "work". It works in the first time, but it didn't start to work on the second time that I do the same thing again.
Do you have any idea?
Basically I create two documents without an "work" property, which causes a duplicate key error. However, I didn't set up unqiue: true though.
Error :
"errmsg" : "E11000 duplicate key error collection: test.user index work_1 dup key: { : null }
From the message it says your collection has an index with name work_1 probably on field work, Since you've created a document without work field then basically you cannot create another document without work field what so ever in the same collection, cause two documents with no work field or even work field with value as null or same cannot exist as it violates unique constraint policies (it says dup key : { : null}) !! Uniques indexes can be created via mongoose schemas or can also be created by manually running queries on database.
Ref : Search for Unique Index and Missing Field in index-unique
So you need to drop the existing index using dropIndex & then if needed recreate it using createIndex. MongoDB would automatically convert a created index to index-multikey (multi-key index - indexes on array fields) if at least one existing document has array value for that indexed field by the time you create index or even if an array value gets inserted for that field in future.
Through code - Drop index : yourSchema.dropIndex({yourFieldName: 1}) && Create index : yourSchema.index({yourFieldName : 1})
NOTE : Just in case if you want to have certain criteria in unique indexes like situation from this question where indexed field can be missing in some documents but it shouldn't be considered as duplicate insertion, then you can take use of partial-indexes (Search for Partial Index with Unique Constraint) which would only index documents where work field exists.
Ex of partial-indexes :-
db.yourCollectionName.createIndex(
{ work: 1 },
{ unique: true, partialFilterExpression: { work: { $exists: true } } }
)
Ref : mongoose-Indexes

How to update an element in a redux/vuex store in O(1)?

How do you usually update an element(hash) in a list in the store (redux, vuex) in O(1)?
I need the order of the elements, it's important (I can add/remove elements from the list). I will update elements on each millisecond, so a lot of updates will be made.
My question is, it's very bad to save the index of the element as a property of the element? Or if the list has 500-1000 elements, it's ok to use .find by Id for example?
Normalize your store shape with two properties
Map of id to entity
List of ids in the required order.
{
byId : {
"post1" : {
id : "post1",
author : "user1",
body : "......",
comments : ["comment1", "comment2"]
},
"post2" : {
id : "post2",
author : "user2",
body : "......",
comments : ["comment3", "comment4", "comment5"]
}
},
allIds : ["post1", "post2"]
}
With this structure, you can update a particular entity in O(1) complexity, and also retrieve the entities in the order of insertion by iterating allIds.
More information can be found here

Index on array keypath doesn't find any values

I want to get familiar with indexedDB to built my Firefox WebExtension.
My sample data is structured like this:
const sampleDataRaw = [
{
"ent_seq" : 1413190,
"att1" : [ {
"sub11" : "content1",
"sub12" : [ "word" ]
}, {
"sub11" : "content2"
} ],
"att2" : [ {
"sub21" : "other content",
"sub22" : [ "term" ]
} ]
}, {
"ent_seq" : 1000010,
"att2" : [ {
"sub21" : "more content"
}, {
"sub22" : "more words"
} ]
}
] // end sampleRawData
I got as far as opening/creating my database, adding this sample data and querying it by the ent_seq key using objectStore.get() and objectStore.openCursor().
The problem arises when I want to search the sub11 or sub21 fields using indexes I should have created for these like this:
objectStore.createIndex("sub11Elements", "att1.sub11", { unique: false });
objectStore.createIndex("sub21Elements", "att2.sub21", { unique: false });
When I want to search, say, fields sub11 as here:
var index = objectStore.index("sub11Elements");
index.get("content1").onsuccess = function(event) {
// I should have the first object of my data now, alas the result is undefined instead
};
It certainly does succeed, but the returned value is undefined since the get() didn't actually find anything.
I want to know why it doesn't find the entry and how to make it find it. I figured it might be because the keypath is wrong, but as stated, if I instead search by the key (ent_seq) I can successfully get the result.att1[i].sub11 values.
On mozilla's websites it's stated that keys can be of type string and array (or array within array etc) amongst others, and keypath parts are supposed to be concatenated using dots.
From searching on stackexchange I've so far found that it's not possible to have variable keys inside the keypath, but that shouldn't be the case here anyway.
Therefore, I really don't see what might be causing the search to not find the object inside the database.
It looks like the second level of objects are arrays, not properties of the first level of objects. The . accessor accesses sub properties, not indices of an array.
IDBObjectStore.prototype.get always yields success when there is no error, and is not indicative of whether a match was found.
A bit more on point 1. Look at "att1":[{"sub11" : "content1","sub12" : [ "word" ]}.... Pretend this was was an actual basic JavaScript object. Would you be able to use att1.sub11? No. Because the value of att1 is an array, not an object.

Node Js - $addToSet nor $push don't work in update with Mongoose (MongoDB)

I have a problem with Mongoose and I want to update a document in MongoDB using the module mongoose, this is my schema:
var User = new mongoose.Schema({
name: String,
email: String,
list_houses: [{
id_house: String,
price: Double
}],
...
});
When I want to add a value 'id_house' without repeating using the code below, this it does not work
User.update({'name': Name},
{$addToSet: {'list_houses': { 'id_house': new_houses , 'price': new_prices } } } ,
if(err)console.log(err);
console.log(result);
});
When I execute this, the result in BD is:
>db.users.find().pretty()
{
"__v" : 0,
"_id" : qweqweqweqweqweqwe,
"email" : adadasdweqwe#gmail.com,
"name" : Richard Smith,
"list_houses" : [ ],
.....
}
I used $push too, but neither it works, I don't know what is the problem with my code now.
Sorry to tell you but $addToSet doesn`t work with Object values in the array.
You ask me why?
Logic is simple, In JS when you compare strings/number etc with same values you do get a true but not with Object
So {a:1} is not equal to {a:1}. Of course because Objects are compared with reference.
Now the solution to your problem is you'll have to do it manually
get the list using find. Check in the list if your object exists. If not add to the list.
You can also specify your object details within find query and just find may be the documents not with this data(to be inserted) in it

Querying Nested List Existence in Mongo

I have a document in Mongo that is structured in the following way:
{
"_id" : ObjectId("4eea7237d0ba3a04f20008fb"),
"code" : "b2677c2809c844cc9d7e3e4ff8d95b46",
"city_id" : 4,
"datetime" : ISODate("2011-12-13T18:41:44.062Z"),
"plays" : [
{
"play_id" : 717224,
"clicks" : [ ],
"order" : 1,
"mysql_id" : 145
}
I want to query for docs whose plays.clicks attribute is a non-empty list. I've tried exists with no luck. I thought that something like this might work:
db.collection.find({plays.clicks.0: {$exists:true}})
But I believe that this would return only docs whose first element in the plays array contains a non-empty clicks list.
Any thought on how I might get this done?
Thanks
db.collection.find({plays.clicks.0: {$exists:true}})
is the right syntax, however as plays is a list the query will match any document that has clicks in plays. There is no way to retrieve a subset of an Array for subelements in this way[1]. There is a ticket for sub / virtual collections[2]
[1] http://www.mongodb.org/display/DOCS/Retrieving+a+Subset+of+Fields#RetrievingaSubsetofFields-RetrievingaSubrangeofArrayElements
[2] https://jira.mongodb.org/browse/SERVER-828
Save the size of the list as a separate attribute (e.g. num_plays). Then you can query for documents where num_plays is greater than 0:
Haven't tested, but I think the query you want is
{ "plays.clicks" : { "$size" : 0 } }
http://www.mongodb.org/display/DOCS/Advanced+Queries#AdvancedQueries-%24size

Categories