We Keep Coding

how to replace first 5 occurrences in string using mongodb query

How to replace the first n occurance of string in mongodb database.
sample JSON :
[{
"NAME":"Peter Parker",
"NUMBER":"6882859392"
},
{
"NAME":"Jhon Ripper",
"NUMBER":"9285346361"
},
{
"NAME":"James",
"NUMBER":"8754859390"
}]
i want to update 1st 5 occurence of NUMBER field of whole JSON
Expected output:
[{
"NAME":"Peter Parker",
"NUMBER":"*****59392"
},
{
"NAME":"Jhon Ripper",
"NUMBER":"*****46361"
},
{
"NAME":"James",
"NUMBER":"*****59390"
}]

How about this? without regex
Mongodb playground
db.collection.aggregate([
{
$set: {
NUMBER: {
$concat: [
"*****",
{
$substrCP: [
"$NUMBER",
5,
{
$strLenCP: "$NUMBER"
}
]
},
]
}
}
}
])

In MongoDB, you can use the updateMany() method to update multiple documents that match a specific condition. In your case, you want to update the first 5 occurrences of the "NUMBER" field.
Here is an example of how you can use the updateMany() method to update the first 5 occurrences of the "NUMBER" field in your JSON:
// Define a filter to find the first 5 occurrences of the "NUMBER" field
var filter = { NUMBER: { $regex: /^[0-9]{5}/ } };
// Define the update operation
var update = { $set: { NUMBER: "*****" + "$" + "NUMBER" } };
// Use the updateMany() method to update the documents that match the filter
db.yourCollectionName.updateMany(filter, update);
The filter { NUMBER: { $regex: /^[0-9]{5}/ } } uses a regular expression to find the documents where the value of the "NUMBER" field starts with 5 digits.
The update { $set: { NUMBER: "*****" + "$" + "NUMBER" } } uses the $set operator to update the "NUMBER" field to a string that starts with "*****" and concatenates the rest of the original number using the $ operator.
The updateMany() method updates all the documents that match the filter, in your case the first 5 occurrences of NUMBER field.
Please keep in mind that you need to be connected to a MongoDB server and a specific database before running this code.

1st i'm storing all numbers with respective _id to the variable a, then iterate the collection with forEach function and updating the document.
This code help me out in this problem statement
var a = db.coll.find({},{"NUMBER":1})
a.forEach(function( row ) {
var dict = '*****'+row.NUMBER.substring(5,row.NUMBER.length);
db.coll.updateOne({"_id" : row._id},{$set : {"NUMBER" : dict}});
});

how do I increment field in array of objects after finding it (using findOne()) and before saving it?

I want to update an object inside an array of schemas without having to do two requests to the database. I currently am incrementing the field using findOneAndUpdate() if the object already exists and it works fine. but in case the object does not exist then I am having to make another request using update() to push the new object and make it available for later increments.
I want to be able to do only one request (e.g. findOne()) to get the user and then increment the field only if object exists in the array and if not I would like to push the new object instead. then save the document. this way I am only making one read/request from the database instead of two.
this is the function now:
async addItemToCart(body, userId) {
const itemInDb = await Model.findOneAndUpdate(
{
_id: userId,
'cart.productId': body.productId,
},
{ $inc: { 'cart.$.count': 1 } }
);
if (itemInDb) return true;
const updated = await Model.update(
{ _id: userId },
{ $push: { cart: body } }
);
if (updated.ok !== 1)
return createError(500, 'something went wrong in userService');
return true;
}
what I would like to do is:
async addItemToCart(body, userId) {
const itemInDb = await Model.findOne(
{
_id: userId,
'cart.productId': body.productId,
}
);
if (itemInDb) {
/**
*
* increment cart in itemInDb then do itemInDb.save() <<------------
*/
} else {
/**
* push product to itemInDb then save
*/
}
Thank you!

You can try findOneAndUpdate with upsert.
upsert: true then create data if not exists in DB.
Model.findOneAndUpdate(
{
_id: userId,
'cart.productId': body.productId,
},
{ $inc: { 'cart.$.count': 1 } },
{
upsert: true,
}
)
Use $set and $inc in one query.
try {
db.scores.findOneAndUpdate(
{
_id: userId,
'cart.productId': body.productId,
},
{ $set: { "cart.$.productName" : "A.B.C", "cart.$.productPrice" : 5}, $inc : { "cart.$.count" : 1 } },
{ upsert:true, returnNewDocument : true }
);
}
catch (e){
//error
}
reference Link : here

You can use upsert.
upsert is defined as an operation that creates a new document when no document matches the query criteria and if matches then it updates the document. It is an option for the update command. If you execute a command like below it works as an update, if there is a document matching query, or as an insert with a document described by the update as an argument.
Example: I am just giving a simple example. You have to change it according to your requirement.
db.people.update(
{ name: "Andy" },
{
name: "Andy",
rating: 1,
score: 1
},
{ upsert: true }
)
So in the above example, if the people with name Andy is found then the update operation will be performed. If not then it will create a new document.

If the specific field is double, convert it to Int [duplicate]

I am trying to change the type of a field from within the mongo shell.
I am doing this...
db.meta.update(
{'fields.properties.default': { $type : 1 }},
{'fields.properties.default': { $type : 2 }}
)
But it's not working!

The only way to change the $type of the data is to perform an update on the data where the data has the correct type.
In this case, it looks like you're trying to change the $type from 1 (double) to 2 (string).
So simply load the document from the DB, perform the cast (new String(x)) and then save the document again.
If you need to do this programmatically and entirely from the shell, you can use the find(...).forEach(function(x) {}) syntax.
In response to the second comment below. Change the field bad from a number to a string in collection foo.
db.foo.find( { 'bad' : { $type : 1 } } ).forEach( function (x) {
x.bad = new String(x.bad); // convert field to string
db.foo.save(x);
});

Convert String field to Integer:
db.db-name.find({field-name: {$exists: true}}).forEach(function(obj) {
obj.field-name = new NumberInt(obj.field-name);
db.db-name.save(obj);
});
Convert Integer field to String:
db.db-name.find({field-name: {$exists: true}}).forEach(function(obj) {
obj.field-name = "" + obj.field-name;
db.db-name.save(obj);
});

Starting Mongo 4.2, db.collection.update() can accept an aggregation pipeline, finally allowing the update of a field based on its own value:
// { a: "45", b: "x" }
// { a: 53, b: "y" }
db.collection.updateMany(
{ a : { $type: 1 } },
[{ $set: { a: { $toString: "$a" } } }]
)
// { a: "45", b: "x" }
// { a: "53", b: "y" }
The first part { a : { $type: 1 } } is the match query:
It filters which documents to update.
In this case, since we want to convert "a" to string when its value is a double, this matches elements for which "a" is of type 1 (double)).
This table provides the code representing the different possible types.
The second part [{ $set: { a: { $toString: "$a" } } }] is the update aggregation pipeline:
Note the squared brackets signifying that this update query uses an aggregation pipeline.
$set is a new aggregation operator (Mongo 4.2) which in this case modifies a field.
This can be simply read as "$set" the value of "a" to "$a" converted "$toString".
What's really new here, is being able in Mongo 4.2 to reference the document itself when updating it: the new value for "a" is based on the existing value of "$a".
Also note "$toString" which is a new aggregation operator introduced in Mongo 4.0.
In case your cast isn't from double to string, you have the choice between different conversion operators introduced in Mongo 4.0 such as $toBool, $toInt, ...
And if there isn't a dedicated converter for your targeted type, you can replace { $toString: "$a" } with a $convert operation: { $convert: { input: "$a", to: 2 } } where the value for to can be found in this table:
db.collection.updateMany(
{ a : { $type: 1 } },
[{ $set: { a: { $convert: { input: "$a", to: 2 } } } }]
)

For string to int conversion.
db.my_collection.find().forEach( function(obj) {
obj.my_value= new NumberInt(obj.my_value);
db.my_collection.save(obj);
});
For string to double conversion.
obj.my_value= parseInt(obj.my_value, 10);
For float:
obj.my_value= parseFloat(obj.my_value);

db.coll.find().forEach(function(data) {
db.coll.update({_id:data._id},{$set:{myfield:parseInt(data.myfield)}});
})

all answers so far use some version of forEach, iterating over all collection elements client-side.
However, you could use MongoDB's server-side processing by using aggregate pipeline and $out stage as :
the $out stage atomically replaces the existing collection with the
new results collection.
example:
db.documents.aggregate([
{
$project: {
_id: 1,
numberField: { $substr: ['$numberField', 0, -1] },
otherField: 1,
differentField: 1,
anotherfield: 1,
needolistAllFieldsHere: 1
},
},
{
$out: 'documents',
},
]);

To convert a field of string type to date field, you would need to iterate the cursor returned by the find() method using the forEach() method, within the loop convert the field to a Date object and then update the field using the $set operator.
Take advantage of using the Bulk API for bulk updates which offer better performance as you will be sending the operations to the server in batches of say 1000 which gives you a better performance as you are not sending every request to the server, just once in every 1000 requests.
The following demonstrates this approach, the first example uses the Bulk API available in MongoDB versions >= 2.6 and < 3.2. It updates all
the documents in the collection by changing all the created_at fields to date fields:
var bulk = db.collection.initializeUnorderedBulkOp(),
counter = 0;
db.collection.find({"created_at": {"$exists": true, "$type": 2 }}).forEach(function (doc) {
var newDate = new Date(doc.created_at);
bulk.find({ "_id": doc._id }).updateOne({
"$set": { "created_at": newDate}
});
counter++;
if (counter % 1000 == 0) {
bulk.execute(); // Execute per 1000 operations and re-initialize every 1000 update statements
bulk = db.collection.initializeUnorderedBulkOp();
}
})
// Clean up remaining operations in queue
if (counter % 1000 != 0) { bulk.execute(); }
The next example applies to the new MongoDB version 3.2 which has since deprecated the Bulk API and provided a newer set of apis using bulkWrite():
var bulkOps = [];
db.collection.find({"created_at": {"$exists": true, "$type": 2 }}).forEach(function (doc) {
var newDate = new Date(doc.created_at);
bulkOps.push(
{
"updateOne": {
"filter": { "_id": doc._id } ,
"update": { "$set": { "created_at": newDate } }
}
}
);
})
db.collection.bulkWrite(bulkOps, { "ordered": true });

To convert int32 to string in mongo without creating an array just add "" to your number :-)
db.foo.find( { 'mynum' : { $type : 16 } } ).forEach( function (x) {
x.mynum = x.mynum + ""; // convert int32 to string
db.foo.save(x);
});

What really helped me to change the type of the object in MondoDB was just this simple line, perhaps mentioned before here...:
db.Users.find({age: {$exists: true}}).forEach(function(obj) {
obj.age = new NumberInt(obj.age);
db.Users.save(obj);
});
Users are my collection and age is the object which had a string instead of an integer (int32).

You can easily convert the string data type to numerical data type.
Don't forget to change collectionName & FieldName.
for ex : CollectionNmae : Users & FieldName : Contactno.
Try this query..
db.collectionName.find().forEach( function (x) {
x.FieldName = parseInt(x.FieldName);
db.collectionName.save(x);
});

I need to change datatype of multiple fields in the collection, so I used the following to make multiple data type changes in the collection of documents. Answer to an old question but may be helpful for others.
db.mycoll.find().forEach(function(obj) {
if (obj.hasOwnProperty('phone')) {
obj.phone = "" + obj.phone; // int or longint to string
}
if (obj.hasOwnProperty('field-name')) {
obj.field-name = new NumberInt(obj.field-name); //string to integer
}
if (obj.hasOwnProperty('cdate')) {
obj.cdate = new ISODate(obj.cdate); //string to Date
}
db.mycoll.save(obj);
});

demo change type of field mid from string to mongo objectId using mongoose
Post.find({}, {mid: 1,_id:1}).exec(function (err, doc) {
doc.map((item, key) => {
Post.findByIdAndUpdate({_id:item._id},{$set:{mid: mongoose.Types.ObjectId(item.mid)}}).exec((err,res)=>{
if(err) throw err;
reply(res);
});
});
});
Mongo ObjectId is just another example of such styles as
Number, string, boolean that hope the answer will help someone else.

I use this script in mongodb console for string to float conversions...
db.documents.find({ 'fwtweaeeba' : {$exists : true}}).forEach( function(obj) {
obj.fwtweaeeba = parseFloat( obj.fwtweaeeba );
db.documents.save(obj); } );
db.documents.find({ 'versions.0.content.fwtweaeeba' : {$exists : true}}).forEach( function(obj) {
obj.versions[0].content.fwtweaeeba = parseFloat( obj.versions[0].content.fwtweaeeba );
db.documents.save(obj); } );
db.documents.find({ 'versions.1.content.fwtweaeeba' : {$exists : true}}).forEach( function(obj) {
obj.versions[1].content.fwtweaeeba = parseFloat( obj.versions[1].content.fwtweaeeba );
db.documents.save(obj); } );
db.documents.find({ 'versions.2.content.fwtweaeeba' : {$exists : true}}).forEach( function(obj) {
obj.versions[2].content.fwtweaeeba = parseFloat( obj.versions[2].content.fwtweaeeba );
db.documents.save(obj); } );
And this one in php)))
foreach($db->documents->find(array("type" => "chair")) as $document){
$db->documents->update(
array('_id' => $document[_id]),
array(
'$set' => array(
'versions.0.content.axdducvoxb' => (float)$document['versions'][0]['content']['axdducvoxb'],
'versions.1.content.axdducvoxb' => (float)$document['versions'][1]['content']['axdducvoxb'],
'versions.2.content.axdducvoxb' => (float)$document['versions'][2]['content']['axdducvoxb'],
'axdducvoxb' => (float)$document['axdducvoxb']
)
),
array('$multi' => true)
);
}

The above answers almost worked but had a few challenges-
Problem 1: db.collection.save no longer works in MongoDB 5.x
For this, I used replaceOne().
Problem 2: new String(x.bad) was giving exponential number
I used "" + x.bad as suggested above.
My version:
let count = 0;
db.user
.find({
custID: {$type: 1},
})
.forEach(function (record) {
count++;
const actualValue = record.custID;
record.custID = "" + record.custID;
console.log(`${count}. Updating User(id:${record._id}) from old id [${actualValue}](${typeof actualValue}) to [${record.custID}](${typeof record.custID})`)
db.user.replaceOne({_id: record._id}, record);
});
And for millions of records, here are the output (for future investigation/reference)-

Mongodb: How to update $min and $max if value passed is null

I have an upsert query which relies and a data object containing the values. Now, it may so happen that certain fields such as data.sumPacketSize.value
could be null - in which case mongo throws an error stating that $min / $max requires a number value.
My original query is:
profiler.upsert({
name: target,
}, {
$inc: {
flowCount: data.doc_count
},
$inc: {
sumPacketSize: data.sumPacketSize.value
},
$inc: {
sumFlowSize: data.sumFlowSize.value
},
$max: {
maxPacketSize: data.maxPacketSize.value,
maxFlowSize: data.maxFlowSize.value
},
$min: {
minPacketSize: data.minPacketSize.value,
minFlowSize: data.minFlowSize.value
},
});
One approach to solving this is to figure which of the keys of data object have null values in them and form a specific mongo query depending on that. However, it isn't elegant and is more so a crude approach.
Another approach would be to first query the db for existing values. If the current value passed is null, then take in the existing values instead:
...
...
let existingProfiler = profiler.findOne({name: target});
data.maxPacketSize.value === null && existingProfiler && (data.maxPacketSize.value = existingProfiler.maxPacketSize);
data.minFlowSize.value === null && existingProfiler && (data.minFlowSize.value = existingProfiler.minFlowSize);
... no change with the mongo query...
$max: {
maxPacketSize: data.maxPacketSize.value,
maxFlowSize: data.maxFlowSize.value
},
...
...
My questions is how can I NOT include the relevant fields in the query if they are null, without having multiple queries for each scenario.
Thanks.

As $max and $min compares current value with specified value, one solution is to use a dummy value which will force mongo to use current value, i.e.
profiler.upsert({
name: target,
}, {
$inc: {
flowCount: data.doc_count
},
$inc: {
sumPacketSize: data.sumPacketSize.value
},
$inc: {
sumFlowSize: data.sumFlowSize.value
},
$max: {
maxPacketSize: data.maxPacketSize.value || SOME_LOWER_BOUND,
maxFlowSize: data.maxFlowSize.value || SOME_LOWER_BOUND
},
$min: {
minPacketSize: data.minPacketSize.value || SOME_UPPER_BOUND,
minFlowSize: data.minFlowSize.value || SOME_UPPER_BOUND
},
});

How to get the latest and oldest record in mongoose.js (or just the timespan between them)

Basic problem
I have a bunch of records and I need to get latest (most recent) and the oldest (least recent).
When googling I found this topic where I saw a couple of queries:
// option 1
Tweet.findOne({}, [], { $orderby : { 'created_at' : -1 } }, function(err, post) {
console.log( post );
});
// option 2
Tweet.find({}, [], {sort:[['arrival',-1]]}, function(err, post) {
console.log( post );
});
Unfortunatly they both error:
TypeError: Invalid select() argument. Must be a string or object.
The link also has this one:
Tweet.find().sort('_id','descending').limit(15).find(function(err, post) {
console.log( post );
});
and that one errors:
TypeError: Invalid sort() argument. Must be a string or object.
So how can I get those records?
Timespan
Even more ideally I just want the difference in time (seconds?) between the oldest and the newest record, but I have no clue on how to start making a query like that.
This is the schema:
var Tweet = new Schema({
body: String
, fid: { type: String, index: { unique: true } }
, username: { type: String, index: true }
, userid: Number
, created_at: Date
, source: String
});
I'm pretty sure I have the most recent version of mongoDB and mongoose.
EDIT
This is how I calc the timespan based on the answer provided by JohnnyHK:
var calcDays = function( cb ) {
var getOldest = function( cb ) {
Tweet.findOne({}, {}, { sort: { 'created_at' : 1 } }, function(err, post) {
cb( null, post.created_at.getTime() );
});
}
, getNewest = function( cb ) {
Tweet.findOne({}, {}, { sort: { 'created_at' : -1 } }, function(err, post) {
cb( null, post.created_at.getTime() );
});
}
async.parallel({
oldest: getOldest
, newest: getNewest
}
, function( err, results ) {
var days = ( results.newest - results.oldest ) / 1000 / 60 / 60 / 24;
// days = Math.round( days );
cb( null, days );
}
);
}

Mongoose 3.x is complaining about the [] parameter in your findOne calls as the array format is no longer supported for the parameter that selects the fields to include.
Try this instead to find the newest:
Tweet.findOne({}, {}, { sort: { 'created_at' : -1 } }, function(err, post) {
console.log( post );
});
Change the -1 to a 1 to find the oldest.
But because you're not using any field selection, it's somewhat cleaner to chain a couple calls together:
Tweet.findOne().sort({created_at: -1}).exec(function(err, post) { ... });
Or even pass a string to sort:
Tweet.findOne().sort('-created_at').exec(function(err, post) { ... });

Fast and Simple - One Line Solution
Get 10 latest documents
MySchema.find().sort({ _id: -1 }).limit(10)
Get 10 oldest documents
MySchema.find().sort({ _id: 1 }).limit(10)
In case you want sorting based on some other property i.e. createdAt and get the oldest or latest. It is similar to the above query.
MySchema.find().sort({ createdAt: -1 }).limit(10) // 10 latest docs
MySchema.find().sort({ createdAt: 1 }).limit(10) // 10 oldest docs

for version ~3.8 mongoose
to find the last entry
model.findOne().sort({ field: 'asc', _id: -1 }).limit(1)
or using
model.findOne().sort({ field: -_id }).limit(1)

collectionName.findOne().sort({$natural: -1}).limit(1).exec(function(err, res){
if(err){
console.log(err);
}
else{
console.log(res);
}
}
This will give you the last document recorded on the database. Just follow the same concept.

await Model.find().sort({$natural:-1}).limit(1); //for the latest record
await Model.find().sort({$natural:1}).limit(1); //for the oldest record
This one works for me. using mongodb natural order https://docs.mongodb.com/manual/reference/operator/meta/natural/

We have method called sort using that we can able to get first element(old document) which means 1 for sort field or last element(new document) which means -1 for sort field of collection.

The best way is to have an async function like that:
async function findLastElement () {
return await Mymodel.findOne().sort('-_id');
}
this way you get the last element and you ensure reusability.

Here is the answer with async - await
const olderDoc: any = await Model.findOne().sort({ createdAt: 1 }).lean().exec()
console.log('olderDoc', olderDoc)
const newerDoc: any = await Model.findOne().sort({ createdAt: -1 }).lean().exec()
console.log('newerDoc', newerDoc)