I'm trying to find documents in mongoDB that are older than a specific amount of time.
Here is an example of a document in my database that should be returned,
{
"_id" : ObjectId("5cc07ea943eeba155840b62f"),
"name" : "test3",
"value" : 3,
"created_at" : {
"$date" : 1556119209477
},
"updatedAt" : {
"$date" : 1556119209477
},
"__v" : 0
}
Here is the query I have so far....
Item.find({"created_at":{$lt:new Date(Date.now() - keys.anAmountOfMilliseconds)}}, (err, foundItems) => {
When run like this, the following error is returned: { CastError: Cast to date failed for value "Invalid Date" at path "created_at" for model "items"
When searching for documents using, createdAt or "createdAt", the query returns with nothing....
Related
I created a TTL index on my collection's created_at field to autoremove document after 30s but after sometime passed (5min later) the document didn't get removed. I read some related questions and I'm sure there was no typo in my collection.
Here is the JSON after running db[collectionName].getIndexes()
[
{
"v" : 2,
"key" : {
"_id" : 1
},
"name" : "_id_",
"ns" : "record_db.collectionName"
},
{
"v" : 2,
"key" : {
"created_at" : 1.0
},
"name" : "created_at_1",
"ns" : "record_db.collectionName",
"expireAfterSeconds" : 30.0
}
]
And here is an example document:
{
"_id" : ObjectId("5fc3823af29f9d9eb1518857"),
"record_path" : path/to/file,
"timestamp" : 1606648392,
"humantime" : "29/11/2020 18:12:58",
"created_at" : ISODate("2020-11-29T18:12:58.859Z")
}
ISODate("2020-11-29T18:12:58.859Z") not happened yet.
const iso = "2020-11-29T18:12:58.859Z"
const date = new Date(iso);
console.log(date.toString());
In MongoDB, I have models of User, Token, and Boost.
A user can have one or more tokens and one or more boosts.
Token has a 2dsphere location field.
And Boost has startTime and stopTime Date fields.
A user is said to have an active boost if Date.now() is greater than boost.startTime() and less than boost.stopTime().
What Mongo aggregation can I write to fetch me all the tokens near a particular location that belong to users with at least one active boost?
Based on your question, I have created a mock data
token collection:
{
"_id" : ObjectId("5b97541c6af22cc65216ffd8"),
"userid" : "5b9753726af22cc65216ffd6",
"location" : {
"longitude" : 80.250875,
"latitude" : 13.052519
}
},
{
"_id" : ObjectId("5b97543a6af22cc65216ffd9"),
"userid" : "5b97537e6af22cc65216ffd7",
"location" : {
"longitude" : 80.249995,
"latitude" : 13.051819
}
}
boost collection :
{
"_id" : ObjectId("5b9754796af22cc65216ffda"),
"startTime" : ISODate("2018-09-11T05:36:57.149Z"),
"stopTime" : ISODate("2018-09-11T05:36:57.149Z"),
"userid" : "5b9753726af22cc65216ffd6"
},
{
"_id" : ObjectId("5b9754b46af22cc65216ffdb"),
"startTime" : ISODate("2018-10-08T18:30:00.000Z"),
"stopTime" : ISODate("2018-10-08T18:30:00.000Z"),
"userid" : "5b97537e6af22cc65216ffd7"
}
Users collection :
{
"_id" : ObjectId("5b9753726af22cc65216ffd6"),
"userName" : "user111"
},
{
"_id" : ObjectId("5b97537e6af22cc65216ffd7"),
"userName" : "user222"
}
The aggregate query to fetch all the tokens near a particular location that belong to users with at least one active boost is:
db.token.aggregate([
{
"$geoNear": {
"near": { type: "Point", coordinates: [80.248797,13.050599] },
"distanceField": "location",
"maxDistance": 1000,
"includeLocs": "location",
"spherical": true
}
},
{"$lookup" : {"from":"boost",
"localField" : "userid",
"foreignField" : "userid",
"as" : "boostDocs"
}},
{"$unwind" : "$boostDocs"},
{"$match" : {"$and":[{"boostDocs.startTime":{"$lte":new Date("11/09/2018")}},{"boostDocs.stopTime":{"$gte":new Date("10/09/2018")}}]}}
])
Notice that query to match the location is at the top of the query as $geoNear will only work if its the first stage of the aggregation pipeline.
The Date that I've used for comparison is just to check if my query works. You can specify your date or Date.now() as per your requirement.
I've to push a given array of Number values into a selected Document inside my MongoDB database.
The Document that I'm going to update as the following structure:
{
"_id" : {
"id" : 17,
"type" : "f"
},
"__v" : 0,
"created_at" : ISODate("2017-03-22T11:16:21.403Z"),
"token" : {
"expDate" : ISODate("2017-12-31T00:00:00Z"),
"token" : "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJlbWFpbCI6ImFsYWRpbkBjb25zb3J6aW9jZXIuaXQiLCJleHAiOjE1MTQ2Nzg0MDB9.QvbT146bA_KH5XA7MH8ASXm9cr3sPZChJ3prYyDireI"
},
"updated_at" : ISODate("2017-07-24T09:42:33.741Z"),
"plots" : {
"idPlot" : [
23570,
23475
]
},
"machines" : [
{
"idPlotBind" : 1,
"ip" : "",
"mac" : "18-5F-00-4A-FE-F4",
"irrId" : 31,
"_id" : ObjectId("59084f527d634d301338aac6"),
"addr" : "pialadin.ddns.net"
},
{
"idPlotBind" : null,
"ip" : "",
"mac" : "12-01-02-FE-AB-B2",
"irrId" : 35,
"_id" : ObjectId("59084f7d7d634d301338aac7")
}
]
}
I'm using the Mongoose library for JS, and the accused query is this one:
userSchema.findOneAndUpdate({$and:[{ '_id.id': resData.PlotRows.IdUser}, {'_id.type': 'f'}]},{$addToSet:{'plots.$.idPlot': {$each: plotData}}}, {upsert: false}, function(err, usr){
if(err){
console.log(err);
return;
}
});
But when I try to execute it, gives me back:
Error: Can't use $each with Number
I am trying to extract aggregate data from Mongodb for each filename each version where platform is win
My data has following format
[
{ "_id" : { "platform" : "mac", "filename" : "f1.json", "version" : "1.4.14" }, "avgSecs" : 15 },
{ "_id" : { "platform" : "win", "filename" : "f2.json", "version" : "3000.0.0_developer" }, "avgSecs" : 1217 },
{ "_id" : { "platform" : "win", "filename" : "f3.json", "version" : "1.4.14" }, "avgSecs" : 1711 },
{ "_id" : { "platform" : "win", "filename" : "f3.json", "version" : "3000.0.0_developer" }, "avgSecs" : 1191 }
]
My code is as follows
[ {$sort:{filename:1}},{$match:{platform:"win"}},{ $group:{ _id: {filename:"$filename", version:"$version"}, avgSecs: {$avg:"$int_secs"} } }]
which is send to mongodb using JS apis as follows
collection.aggregate(aggQuery).toArray(function(err,docs){
assert.equal(null,err)
resolve(docs);
})
but I dont get parsed and aggregate data but complete data as is from the db.
I have checked the same aggregate query works fine with mongodb shell. I am not sure what is the problem with JS api. Am I using it the right way
Ok found it.
AggQuery was sent in as string while it was expecting an Array Object.
Made the object Array and now results are as expected.
How do I take the difference between adjacent records in mongoDB using javascript? For example, if I have the following three documents in a collection:
{
"_id" : ObjectId("50ed90a55502684f440001ac"),
"time" : ISODate("2013-02-13T15:45:41.148Z")
}
{
"_id" : ObjectId("50ed90a55502684f440001ad"),
"time" : ISODate("2013-02-13T15:45:42.148Z")
}
{
"_id" : ObjectId("50ed90a55502684f440001ae"),
"time" : ISODate("2013-02-13T15:45:45.148Z")
}
I want to take the difference in the "time" field between adjacent values to get:
{
"_id" : ObjectId("50ed90a55502684f440001ac"),
"time" : ISODate("2013-02-13T15:45:41.148Z"),
"time_difference" : null
}
{
"_id" : ObjectId("50ed90a55502684f440001ad"),
"time" : ISODate("2013-02-13T15:45:42.148Z"),
"time_difference" : 1
}
{
"_id" : ObjectId("50ed90a55502684f440001ae"),
"time" : ISODate("2013-02-13T15:45:45.148Z"),
"time_difference" : 3
}
Any ideas on how to do this efficiently in javascript/mongoDB? Thanks.
I don't know whether this was true when the question was asked seven years ago, but this can be solved completely within the aggregation framework. Assuming the collection name is AdjacentDocument, the following aggregation will get the results you're looking for:
db.AdjacentDocument.aggregate(
{$sort: {time: 1}},
{$group: {_id: 0, document: {$push: '$$ROOT'}}},
{$project: {documentAndPrevTime: {$zip: {inputs: ['$document', {$concatArrays: [[null], '$document.time']}]}}}},
{$unwind: {path: '$documentAndPrevTime'}},
{$replaceWith: {$mergeObjects: [{$arrayElemAt: ['$documentAndPrevTime', 0]}, {prevTime: {$arrayElemAt: ['$documentAndPrevTime', 1]}}]}},
{$set: {time_difference: {$trunc: [{$divide: [{$subtract: ['$time', '$prevTime']}, 1000]}]}}},
{$unset: 'prevTime'}
);
Aggregation pipeline walkthrough
First, the documents are sorted from oldest to newest. They are grouped into a single document with the documents stored in an ordered array field:
{$sort: {time: 1}},
{$group: {_id: 0, document: {$push: '$$ROOT'}}}
/*
{
"_id" : 0,
"document" : [
{
"_id" : ObjectId("50ed90a55502684f440001ac"),
"time" : ISODate("2013-02-13T15:45:41.148Z")
},
{
"_id" : ObjectId("50ed90a55502684f440001ad"),
"time" : ISODate("2013-02-13T15:45:42.148Z")
},
{
"_id" : ObjectId("50ed90a55502684f440001ae"),
"time" : ISODate("2013-02-13T15:45:45.148Z")
}
]
}
*/
Next, the previous times are zipped into the document array, creating an array of [document, previousTime]:
{$project: {documentAndPrevTime: {$zip: {inputs: ['$document', {$concatArrays: [[null], '$document.time']}]}}}}
/*
{
"_id" : 0,
"documentAndPrevTime" : [
[
{
"_id" : ObjectId("50ed90a55502684f440001ac"),
"time" : ISODate("2013-02-13T15:45:41.148Z")
},
null
],
[
{
"_id" : ObjectId("50ed90a55502684f440001ad"),
"time" : ISODate("2013-02-13T15:45:42.148Z")
},
ISODate("2013-02-13T15:45:41.148Z")
],
[
{
"_id" : ObjectId("50ed90a55502684f440001ae"),
"time" : ISODate("2013-02-13T15:45:45.148Z")
},
ISODate("2013-02-13T15:45:42.148Z")
]
]
}
*/
Next, the document & time array is unwound, creating a document for each of the initial documents:
{$unwind: {path: '$documentAndPrevTime'}}
/*
{
"_id" : 0,
"documentAndPrevTime" : [
{
"_id" : ObjectId("50ed90a55502684f440001ac"),
"time" : ISODate("2013-02-13T15:45:41.148Z")
},
null
]
}
{
"_id" : 0,
"documentAndPrevTime" : [
{
"_id" : ObjectId("50ed90a55502684f440001ad"),
"time" : ISODate("2013-02-13T15:45:42.148Z")
},
ISODate("2013-02-13T15:45:41.148Z")
]
}
{
"_id" : 0,
"documentAndPrevTime" : [
{
"_id" : ObjectId("50ed90a55502684f440001ae"),
"time" : ISODate("2013-02-13T15:45:45.148Z")
},
ISODate("2013-02-13T15:45:42.148Z")
]
}
*/
Next, we replace the document with the value of the document array element, merged with previous time element (using null if it's the initial index):
{$replaceWith: {$mergeObjects: [{$arrayElemAt: ['$documentAndPrevTime', 0]}, {prevTime: {$arrayElemAt: ['$documentAndPrevTime', 1]}}]}}
/*
{
"_id" : ObjectId("50ed90a55502684f440001ac"),
"time" : ISODate("2013-02-13T15:45:41.148Z"),
"prevTime" : null
}
{
"_id" : ObjectId("50ed90a55502684f440001ad"),
"time" : ISODate("2013-02-13T15:45:42.148Z"),
"prevTime" : ISODate("2013-02-13T15:45:41.148Z")
}
{
"_id" : ObjectId("50ed90a55502684f440001ae"),
"time" : ISODate("2013-02-13T15:45:45.148Z"),
"prevTime" : ISODate("2013-02-13T15:45:42.148Z")
}
*/
Finally, we update the document by setting the time_difference to the difference of the two time fields, and removing the temporary prevTime field. Since the difference between two dates is in milliseconds and your example uses seconds, we calculate the seconds by dividing by 1000 and truncating.
{$set: {time_difference: {$trunc: [{$divide: [{$subtract: ['$time', '$prevTime']}, 1000]}]}}},
{$unset: 'prevTime'}
/*
{
"_id" : ObjectId("50ed90a55502684f440001ac"),
"time" : ISODate("2013-02-13T15:45:41.148Z"),
"time_difference" : null
}
{
"_id" : ObjectId("50ed90a55502684f440001ad"),
"time" : ISODate("2013-02-13T15:45:42.148Z"),
"time_difference" : 1
}
{
"_id" : ObjectId("50ed90a55502684f440001ae"),
"time" : ISODate("2013-02-13T15:45:45.148Z"),
"time_difference" : 3
}
*/
The one thing you will want to make sure of here is that you have a sort on the query you wish to use to garnish your records. If no sort is used it will actually use find order, which is not $natural order.
Find order can differ between queries so if you run the query twice within the period of 2 minutes you might find that they don't return the same order. It does seem however that your query would be logically sorted on tiem_difference.
It should also by noted that this is not possible through normal querying. I also do not see an easy way doing this through the aggregation framework.
So already it seems the next plausible method is either using multiple queries or client side processing. Client side processing is probably the better here using a function like the one defined by #Marlon above.
One thing, I want to clear you. Unlike MYSQL, MongoDB is not give gurantee to the position. I mean, MongoDB will give you different sort at different time. So compare adjacent document may give different result, on every reading.
If you are fine with that and you want to compare then try with MongoDB's MapReduce http://docs.mongodb.org/manual/applications/map-reduce/
Assuming those 3 objects are coming through in an array, you could do something like the below:
var prevTime;
var currentTime;
for(var i = 0; i < records.length; i++)
{
currentTime = new Date(records[i].time).getTime();
records[i].time_difference = currentTime - prevTime;
prevTime = currentTime;
}
Of course you'll need to swap bits out to make it use the records from mongo.
If you need to do any more complex date calculations, I highly suggest checking out datejs (which you can get a node wrapper for if you want).