In my MongoDB/Node backend, I have a function that updates a date on a field titled subscriptionEnd based on subscription _ids that are passed in. It looks like this:
try {
db.collection('clients').updateMany(
{ "subscription._id": { $in: mongoArrSubscription } },
{ $set : {"subscription.$.subscriptionEnd": lastDayOfMonth } },
function (err, res) {
if (err) throw err;
});
res.sendStatus(200);
} catch (e) {
console.log(e);
}
}
This works as is. However, as is, this will only target the first element in the "subscription" array. I realize now we have instances where there are two elements within the same array that may be passed in. So, my thought is to use $unwind within this function, to first $unwind the "subscription" array. I'm a little unclear on the syntax however, and if this is doable.
My unwind aggregation would look like this:
Client.aggregate( [ { $unwind : "$subscription" } ] );
Is there a way I can chain together this $unwind aggregation, so it happens just prior to the updateMany() within my try/catch block? What does that syntax look like? Do I chain the operations together, or do I pass the $unwind in as an argument to the updateMany() ?
I tried this, passing the $unwind operation in as the first parameter:
try {
db.collection('clients').updateMany( { $unwind : "$subscription" },
{ "subscription._id": { $in: mongoArrSubscription } },
{ $set : {"subscription.$.subscriptionEnd": lastDayOfMonth } },
function (err, res) {
if (err) throw err;
});
res.sendStatus(200);
} catch (e) {
console.log(e);
}
}
... but it errors out:
MongoError: Unknown modifier: $unwind
So maybe I need to do the aggregation $unwind first, as a separate operation, and then run the updateMany() ?
UPDATE: Someone pointed out that $unwind is not useable with updateMany(), so perhaps I need to do the $unwind operation first and separately, and then run my updateMany() operation?
You can mimic the 3.6 multi update using aggregation and bulk write for lower versions.
Something like
var bulk = db.getCollection('clients').initializeUnorderedBulkOp();
var count = 0;
var batch = 1;
db.getCollection('clients').aggregate([
{$match:{"subscription._id":{$in:mongoArrSubscription}}},
{$unwind:"$subscription"},
{$match:{"subscription._id":{$in:mongoArrSubscription}}},
{$project: {_id:0, subscription_id:"$subscription._id"}}
]).forEach(function(doc){
var subscription_id = doc.subscription_id;
bulk.find({"subscription._id":subscription_id}).updateOne(
{$set:{"subscription.$.subscriptionEnd": lastDayOfMonth}}
);
count++;
if (count == batch) {
bulk.execute();
bulk = db.getCollection('clients').initializeUnorderedBulkOp();
count = 0;
}
});
if (count > 0) {
bulk.execute();
}
Related
This is my Operator Models:
const operatorSchema = new Schema({
operatorName: {
type: String
},
users:[{
email:String,
payment:Number,
paymentsData: Date,
product: String,
}],
});
I need to filter by operatorName and email in users block. But when I try with this I get all users in related OperatorName how can I query correctly ?
Operators.find( { $and: [{operatorName: operatorName}, {'users.email': 'super#m.com'}]}, function (err, docs) {
) {
if (err) console.log(err)
else {
docs.forEach(function(data){
console.log(data)
})
// res.render('total_earns_operator_tables', { operators: docs });
}
});
EDIT
I also try with aggregate method like this but again, I get same result and I gel all bunch of user data, but I want only demouser#mail.com
Operators.aggregate([
{ $match: {$and: [{ operatorName: operatorName},{'users.email':
'demouser#mail.com' }]}},
]
,function (err, docs) {
// console.log(Operators)
// Operators.find( { $and: [{operatorName: operatorName}, {users: {$elemMatch: {email:['super#m.com']}}}]}, function (err, docs) {
// Operators.find( {operatorName: operatorName, "users.email": "demouser#mail.com"}, function (err, docs) {
if (err) console.log(err)
else {
docs.forEach(function(data){
console.log(data)
})
// res.render('total_earns_operator_tables', { operators: docs });
}
});
It is very basic but I couldnt find solution.
Your query is doing exactly what it is supposed to do. It is returning all documents that satisfy you two criteria: 1. having a specified operatorName and 2. users array having at least one user matching the specified email.
If you want to reashape your documents by filtering the user array to only include the user matching your condition, you'll have to use an aggregation.
EDIT
As per your edit: Your aggregation only have a $match stage, which is identical to your query above. To change the shape of a document, the aggregation framework provides you with the $project stage, see the example below:
Operators.aggregate([
{
$match: {
operatorName: operatorName,
"users.email": "demouser#mail.com"
}
},
{
$project: {
operatorName: '$operatorName',
users: {
$filter: {
input: "$users",
as: "user",
cond: {
$eq: [
"$$user.email",
"demouser#mail.com"
]
}
}
}
}
}
]
Here, we first filter the collection to get only the documents you want, using the $match stage, then we use the $filteroperator in $project stage, to return only the matching users within the array.
See the working playground
I have an array that looks like this :
"attributes": [{"id": "5dad5242e7038210842ec59c","value": 3},{"id": "5dbade6a824f3e2244b2870b","value": 6},{"id": "5d7c943a5759f91e187520cc","value": 17}]
Each value in the array corresponds to a schema from where I will fetch that data.
Ex : if(value == 1){ fetch data from schemaA}
Based on the data fetched from each schema I will repopulate the array with additional information, so at the end the array would look like this:
"attributes": [{"id": "5dad5242e7038210842ec59c","value": 3, "name": "attributeA"},{"id": "5dbade6a824f3e2244b2870b","value": 6, "name": "attributeF"},{"id": "5d7c943a5759f91e187520cc","value": 17, "name": "attributeQ"}]
So far I have written a function :
exports.fetchAttributes = (attr, callback) => {
try {
switch (attr.value) {
case 3:
this.fetchAttributeC(attr.id, (err, attributeB) => {
callback(err, attributeB);
});
break;
case 6:
this.fetchAttributeF(attr.id, (err, attributeF) => {
callback(err, attributeF);
});
break;
case 7:
this.fetchAttributeQ(attr.id, (err, attributeQ) => {
callback(err, attributeQ);
});
break;
default : console.log("Invalid value");
}
} catch(e){
console.log("Catch in fetchAttributes "+e);
return callback(e, null);
}
}
The above function is being called within another function:
exports.functionAttributes = (attributes, callback) => {
let newAttributes = [];
attributes.forEach((attr) => {
this.fetchAttributes(attr, (err, attributeFull) => {
newAttributes.push(attributeFull);
})
}
//need to pass in callback here, but obviously there is a scope issue
}
I need this array with newAttributes as I have to pass it to the final callback in async waterfall where this function is. Due to the scope issue newAttributes is always empty. I need help finding a better way to achieve this. So I can finally end up with result array as I have shown above. Any help is greatly appreciated.
P.S : I have tried conditional callbacks, promises but I couldn't make it work as I need to pass parameters in each step. Feel free to point out where I am wrong, or if there is a better method of achieving this, I would be happy to learn.
I think a promise list would perfectly solve your problem as long as you structured it correctly.
First: Make fetchAttributes return a promise:
return new Promise(function(resolve, reject) {
...Your try catch using resolve() and reject()
})
Second: Create the Array of your promises
let fetchList= [];
attributes.forEach((attr) => {
fetchList.push(...your function)
}
Third: If you don't care whether one fails, make sure to add in a reflection so the promise.all() always completes -> Check this stack post
Finally Once your promise.all() resolves, you can loop through all of the returned values and push them to the newAttributes variable!!!
If you have trouble with it I can provide more info but that should get you started!
In MongoDB, is it possible to update the value of a field using the value from another field? The equivalent SQL would be something like:
UPDATE Person SET Name = FirstName + ' ' + LastName
And the MongoDB pseudo-code would be:
db.person.update( {}, { $set : { name : firstName + ' ' + lastName } );
The best way to do this is in version 4.2+ which allows using the aggregation pipeline in the update document and the updateOne, updateMany, or update(deprecated in most if not all languages drivers) collection methods.
MongoDB 4.2+
Version 4.2 also introduced the $set pipeline stage operator, which is an alias for $addFields. I will use $set here as it maps with what we are trying to achieve.
db.collection.<update method>(
{},
[
{"$set": {"name": { "$concat": ["$firstName", " ", "$lastName"]}}}
]
)
Note that square brackets in the second argument to the method specify an aggregation pipeline instead of a plain update document because using a simple document will not work correctly.
MongoDB 3.4+
In 3.4+, you can use $addFields and the $out aggregation pipeline operators.
db.collection.aggregate(
[
{ "$addFields": {
"name": { "$concat": [ "$firstName", " ", "$lastName" ] }
}},
{ "$out": <output collection name> }
]
)
Note that this does not update your collection but instead replaces the existing collection or creates a new one. Also, for update operations that require "typecasting", you will need client-side processing, and depending on the operation, you may need to use the find() method instead of the .aggreate() method.
MongoDB 3.2 and 3.0
The way we do this is by $projecting our documents and using the $concat string aggregation operator to return the concatenated string.
You then iterate the cursor and use the $set update operator to add the new field to your documents using bulk operations for maximum efficiency.
Aggregation query:
var cursor = db.collection.aggregate([
{ "$project": {
"name": { "$concat": [ "$firstName", " ", "$lastName" ] }
}}
])
MongoDB 3.2 or newer
You need to use the bulkWrite method.
var requests = [];
cursor.forEach(document => {
requests.push( {
'updateOne': {
'filter': { '_id': document._id },
'update': { '$set': { 'name': document.name } }
}
});
if (requests.length === 500) {
//Execute per 500 operations and re-init
db.collection.bulkWrite(requests);
requests = [];
}
});
if(requests.length > 0) {
db.collection.bulkWrite(requests);
}
MongoDB 2.6 and 3.0
From this version, you need to use the now deprecated Bulk API and its associated methods.
var bulk = db.collection.initializeUnorderedBulkOp();
var count = 0;
cursor.snapshot().forEach(function(document) {
bulk.find({ '_id': document._id }).updateOne( {
'$set': { 'name': document.name }
});
count++;
if(count%500 === 0) {
// Excecute per 500 operations and re-init
bulk.execute();
bulk = db.collection.initializeUnorderedBulkOp();
}
})
// clean up queues
if(count > 0) {
bulk.execute();
}
MongoDB 2.4
cursor["result"].forEach(function(document) {
db.collection.update(
{ "_id": document._id },
{ "$set": { "name": document.name } }
);
})
You should iterate through. For your specific case:
db.person.find().snapshot().forEach(
function (elem) {
db.person.update(
{
_id: elem._id
},
{
$set: {
name: elem.firstname + ' ' + elem.lastname
}
}
);
}
);
Apparently there is a way to do this efficiently since MongoDB 3.4, see styvane's answer.
Obsolete answer below
You cannot refer to the document itself in an update (yet). You'll need to iterate through the documents and update each document using a function. See this answer for an example, or this one for server-side eval().
For a database with high activity, you may run into issues where your updates affect actively changing records and for this reason I recommend using snapshot()
db.person.find().snapshot().forEach( function (hombre) {
hombre.name = hombre.firstName + ' ' + hombre.lastName;
db.person.save(hombre);
});
http://docs.mongodb.org/manual/reference/method/cursor.snapshot/
Starting Mongo 4.2, db.collection.update() can accept an aggregation pipeline, finally allowing the update/creation of a field based on another field:
// { firstName: "Hello", lastName: "World" }
db.collection.updateMany(
{},
[{ $set: { name: { $concat: [ "$firstName", " ", "$lastName" ] } } }]
)
// { "firstName" : "Hello", "lastName" : "World", "name" : "Hello World" }
The first part {} is the match query, filtering which documents to update (in our case all documents).
The second part [{ $set: { name: { ... } }] is the update aggregation pipeline (note the squared brackets signifying the use of an aggregation pipeline). $set is a new aggregation operator and an alias of $addFields.
Regarding this answer, the snapshot function is deprecated in version 3.6, according to this update. So, on version 3.6 and above, it is possible to perform the operation this way:
db.person.find().forEach(
function (elem) {
db.person.update(
{
_id: elem._id
},
{
$set: {
name: elem.firstname + ' ' + elem.lastname
}
}
);
}
);
I tried the above solution but I found it unsuitable for large amounts of data. I then discovered the stream feature:
MongoClient.connect("...", function(err, db){
var c = db.collection('yourCollection');
var s = c.find({/* your query */}).stream();
s.on('data', function(doc){
c.update({_id: doc._id}, {$set: {name : doc.firstName + ' ' + doc.lastName}}, function(err, result) { /* result == true? */} }
});
s.on('end', function(){
// stream can end before all your updates do if you have a lot
})
})
update() method takes aggregation pipeline as parameter like
db.collection_name.update(
{
// Query
},
[
// Aggregation pipeline
{ "$set": { "id": "$_id" } }
],
{
// Options
"multi": true // false when a single doc has to be updated
}
)
The field can be set or unset with existing values using the aggregation pipeline.
Note: use $ with field name to specify the field which has to be read.
Here's what we came up with for copying one field to another for ~150_000 records. It took about 6 minutes, but is still significantly less resource intensive than it would have been to instantiate and iterate over the same number of ruby objects.
js_query = %({
$or : [
{
'settings.mobile_notifications' : { $exists : false },
'settings.mobile_admin_notifications' : { $exists : false }
}
]
})
js_for_each = %(function(user) {
if (!user.settings.hasOwnProperty('mobile_notifications')) {
user.settings.mobile_notifications = user.settings.email_notifications;
}
if (!user.settings.hasOwnProperty('mobile_admin_notifications')) {
user.settings.mobile_admin_notifications = user.settings.email_admin_notifications;
}
db.users.save(user);
})
js = "db.users.find(#{js_query}).forEach(#{js_for_each});"
Mongoid::Sessions.default.command('$eval' => js)
With MongoDB version 4.2+, updates are more flexible as it allows the use of aggregation pipeline in its update, updateOne and updateMany. You can now transform your documents using the aggregation operators then update without the need to explicity state the $set command (instead we use $replaceRoot: {newRoot: "$$ROOT"})
Here we use the aggregate query to extract the timestamp from MongoDB's ObjectID "_id" field and update the documents (I am not an expert in SQL but I think SQL does not provide any auto generated ObjectID that has timestamp to it, you would have to automatically create that date)
var collection = "person"
agg_query = [
{
"$addFields" : {
"_last_updated" : {
"$toDate" : "$_id"
}
}
},
{
$replaceRoot: {
newRoot: "$$ROOT"
}
}
]
db.getCollection(collection).updateMany({}, agg_query, {upsert: true})
(I would have posted this as a comment, but couldn't)
For anyone who lands here trying to update one field using another in the document with the c# driver...
I could not figure out how to use any of the UpdateXXX methods and their associated overloads since they take an UpdateDefinition as an argument.
// we want to set Prop1 to Prop2
class Foo { public string Prop1 { get; set; } public string Prop2 { get; set;} }
void Test()
{
var update = new UpdateDefinitionBuilder<Foo>();
update.Set(x => x.Prop1, <new value; no way to get a hold of the object that I can find>)
}
As a workaround, I found that you can use the RunCommand method on an IMongoDatabase (https://docs.mongodb.com/manual/reference/command/update/#dbcmd.update).
var command = new BsonDocument
{
{ "update", "CollectionToUpdate" },
{ "updates", new BsonArray
{
new BsonDocument
{
// Any filter; here the check is if Prop1 does not exist
{ "q", new BsonDocument{ ["Prop1"] = new BsonDocument("$exists", false) }},
// set it to the value of Prop2
{ "u", new BsonArray { new BsonDocument { ["$set"] = new BsonDocument("Prop1", "$Prop2") }}},
{ "multi", true }
}
}
}
};
database.RunCommand<BsonDocument>(command);
MongoDB 4.2+ Golang
result, err := collection.UpdateMany(ctx, bson.M{},
mongo.Pipeline{
bson.D{{"$set",
bson.M{"name": bson.M{"$concat": []string{"$lastName", " ", "$firstName"}}}
}},
)
i am trying to toggle a boolean value inside a object, which is a subdocument, i am having hard time to update a particular object within an array.
Document:
"_id" : ObjectId("54afaabd88694dc019d3b628")
"Invitation" : [
{
"__v" : 0,
"ID" : ObjectId("54af6ce091324fd00f97a15f"),
"__t" : "USER",
"_id" : ObjectId("54b4ceb50fc380001bea1752"),
"Accepted" : false
},
{
"__v" : 0,
"ID" : ObjectId("54afac5412f5fdcc007a5c4d"),
"__t" : "USER",
"_id" : ObjectId("54b4cebe0fc380001bea1753"),
"Accepted" : false
}
],
Controller:
User.aggregate([{$match: {_id: ObjectId(54afaabd88694dc019d3b628)}},{$unwind: '$Invitation'},{$project: {_id: '$_id',Invitation: '$Invitation'}}],function(err,results){
function updateInvitation(_id){
var query = {'_id': _id, 'Invitation.ID': ObjectId("54af6ce091324fd00f97a15f")};
var operator = {$inc: {'Invitation.Accepted': 1}};
User.update(query,operator,{multi:true},function(err,updated){
if(err){
console.log(err);
}
console.log('updating'+updated);
});
}
res.jsonp(results);
updateInvitation(results[0]._id);
});
i tried using $set but it didnt worked out as whole Invitation array got replaced with 'Accepted = 1'
How can i toggle 'Accepted' field of the document with particular 'ID'.
Invitation.$.Accepted
Positional operator doesnot apply to field containing array so can't iterate to Accepted field
EDIT:
User.find({_id: req.user._id},'Invitation',function(err,docs){
if(err){
console.log(err);
}
console.log(docs);
var results = [];
async.each(docs,function(doc,err) {
if(err){
console.log('error'+ err);
}
async.each(docs.Invitation,function(invite,callback) {
console.log('second async');
User.update(
{ '_id': doc._id, 'Invitation._id': invite._id },
{ '$set': {'Invitation.$.Accepted': !invite.Accepted}},
function(err,doc) {
results.push(doc);
console.log('updated'+doc);
callback(err);
}
);
});
},function(err) {
if (err)
console.log(err);
console.log(results);
});
});
The control is not getting inside second async.each, error is thrown on first async here is the error:
error-function () {
if (called) throw new Error("Callback was already called.");
called = true;
fn.apply(root, arguments);
}
I really don't think that even as a feeder query the aggregation framework is the right operation to use here. All you are doing is "denormalizing" the array as individual documents. There really should be no need. Just fetch the document instead:
var query = {}; // whatever criteria
Users.find(query,"Invitation",function(err,docs) {
if (err) {
console.log(err);
var results = [];
async.each(docs,function(doc,callback) {
async.each(docs.Invitation,function(invite,callback) {
Users.findOneAndUpdate(
{ "_id": doc._id, "Invitation._id": invite._id },
{ "$set": { "Invitation.$.Accepted": !invite.Accepted } },
function(err,doc) {
results.push( doc );
callback(err);
}
);
},callback);
},function(err) {
if (err)
console.log(err);
console.log(results);
});
});
So there is no problem iterating the list of documents in a response for what you are doing, it's just that you also want to iterate the array members as well. The catch is when issuing any kind of .update() that you need to be aware then the asynchronous call is complete.
So I'm using async.each but you probably want async.eachLimit to control the looping. The matching of the element comes from the positional $ operator, corresponding to the matched array element in the query.
It's just JavaScript code, so simply "toggle" the value with !invite.accepted which will inverse it. For additional fun, return the "results" array by pushing the modified document from .findOneAndUpdate().
Use the positional update operator for this:
http://docs.mongodb.org/manual/reference/operator/update/positional/
User.update(
{
"_id": ObjectId("54afaabd88694dc019d3b628"),
"Invitation": {"$elemMatch": {"ID" : ObjectId("54af6ce091324fd00f97a15f"), "Accepted":false}}
},
{
"$set" : {
"Invitation.$.Accepted" : true
}
},{multi:false, upsert:false, safe:true}, function (err, numAffectedDocuments){
// TODO
}
);
Note : you don't need the aggregation step since it actually does nothing.
I need to replace a string in certain documents. I have googled this code, but it unfortunately does not change anything. I am not sure about the syntax on the line bellow:
pulpdb = db.getSisterDB("pulp_database");
var cursor = pulpdb.repos.find();
while (cursor.hasNext()) {
var x = cursor.next();
x['source']['url'].replace('aaa', 'bbb'); // is this correct?
db.foo.update({_id : x._id}, x);
}
I would like to add some debug prints to see what the value is, but I have no experience with MongoDB Shell. I just need to replace this:
{ "source": { "url": "http://aaa/xxx/yyy" } }
with
{ "source": { "url": "http://bbb/xxx/yyy" } }
It doesn't correct generally: if you have string http://aaa/xxx/aaa (yyy equals to aaa) you'll end up with http://bbb/xxx/bbb.
But if you ok with this, code will work.
To add debug info use print function:
var cursor = db.test.find();
while (cursor.hasNext()) {
var x = cursor.next();
print("Before: "+x['source']['url']);
x['source']['url'] = x['source']['url'].replace('aaa', 'bbb');
print("After: "+x['source']['url']);
db.test.update({_id : x._id}, x);
}
(And by the way, if you want to print out objects, there is also printjson function)
The best way to do this if you are on MongoDB 2.6 or newer is looping over the cursor object using the .forEach method and update each document usin "bulk" operations for maximum efficiency.
var bulk = db.collection.initializeOrderedBulkOp();
var count = 0;
db.collection.find().forEach(function(doc) {
print("Before: "+doc.source.url);
bulk.find({ '_id': doc._id }).update({
'$set': { 'source.url': doc.source.url.replace('aaa', 'bbb') }
})
count++;
if(count % 200 === 0) {
bulk.execute();
bulk = db.collection.initializeOrderedBulkOp();
}
// Clean up queues
if (count > 0)
bulk.execute();
From MongoDB 3.2 the Bulk() API and its associated methods are deprecated you will need to use the db.collection.bulkWrite() method.
You will need loop over the cursor, build your query dynamically and $push each operation to an array.
var operations = [];
db.collection.find().forEach(function(doc) {
print("Before: "+doc.source.url);
var operation = {
updateOne: {
filter: { '_id': doc._id },
update: {
'$set': { 'source.url': doc.source.url.replace('aaa', 'bbb') }
}
}
};
operations.push(operation);
})
operations.push({
ordered: true,
writeConcern: { w: "majority", wtimeout: 5000 }
})
db.collection.bulkWrite(operations);
Nowadays,
starting Mongo 4.2, db.collection.updateMany (alias of db.collection.update) can accept an aggregation pipeline, finally allowing the update of a field based on its own value.
starting Mongo 4.4, the new aggregation operator $replaceOne makes it very easy to replace part of a string.
// { "source" : { "url" : "http://aaa/xxx/yyy" } }
// { "source" : { "url" : "http://eee/xxx/yyy" } }
db.collection.updateMany(
{ "source.url": { $regex: /aaa/ } },
[{
$set: { "source.url": {
$replaceOne: { input: "$source.url", find: "aaa", replacement: "bbb" }
}}
}]
)
// { "source" : { "url" : "http://bbb/xxx/yyy" } }
// { "source" : { "url" : "http://eee/xxx/yyy" } }
The first part ({ "source.url": { $regex: /aaa/ } }) is the match query, filtering which documents to update (the ones containing "aaa")
The second part ($set: { "source.url": {...) is the update aggregation pipeline (note the squared brackets signifying the use of an aggregation pipeline):
$set is a new aggregation operator (Mongo 4.2) which in this case replaces the value of a field.
The new value is computed with the new $replaceOne operator. Note how source.url is modified directly based on the its own value ($source.url).
Note that this is fully handled server side which won't allow you to perform the debug printing part of your question.
MongoDB can do string search/replace via mapreduce. Yes, you need to have a very special data structure for it -- you can't have anything in the top keys but you need to store everything under a subdocument under value. Like this:
{
"_id" : ObjectId("549dafb0a0d0ca4ed723e37f"),
"value" : {
"title" : "Top 'access denied' errors",
"parent" : "system.admin_reports",
"p" : "\u0001\u001a%"
}
}
Once you have this neatly set up you can do:
$map = new \MongoCode("function () {
this.value['p'] = this.value['p'].replace('$from', '$to');
emit(this._id, this.value);
}");
$collection = $this->mongoCollection();
// This won't be called.
$reduce = new \MongoCode("function () { }");
$collection_name = $collection->getName();
$collection->db->command([
'mapreduce' => $collection_name,
'map' => $map,
'reduce' => $reduce,
'out' => ['merge' => $collection_name],
'query' => $query,
'sort' => ['_id' => 1],
]);