I have a problem,
I do a group on my database and the problem is the $$ROOT can exceed the memory limit, so I would like to have the X first elements.
this is my group:
$group: {
_id: {
DOG_RACE: '$DOG_RACE',
FOOD: '$FOOD'
},
total: {
$count: {}
},
_raw: {
$push: '$$ROOT'
}
}
the _raw contain all dogs grouped and there can be millions.
I tried to replace _raw: {} with an array with a slice but I got this error:
The field '_raw' must be an accumulator object
Can someone please have the syntaxe to do it ?
Thanks
EDIT
No this is what I'm trying:
$group: {
_id: {
DOG_RACE: '$DOG_RACE',
FOOD: '$FOOD'
},
total: {
$count: {}
},
_raw: {
$firstN: {
input: '$$ROOT',
n: 10
}
}
}
And I'm still getting an error with MongoServerError: unknown group operator '$function'
same with $function
also my mongodb database after running mongod is v6,0,1
Related
I have a collection of documents, which I need to first narrow down by set criteria, then sort alphabetically by string value inside those documents — let's say that's a "search result". I then need to find document that matches a given _id and then pick a document next to it (before or after) from the above "search result".
Background:
I use mongoose to query my database via Node.js.
I have a set of "special sections" in my blog that are comprised of all the articles that must have three particular conditions associated within the keys in the document. I can get the list of articles belonging to said section like so:
const specialSectionListQuery = Article.find({
tag: { $ne: "excluded" },
[`collections.cameras`]: { $exists: true },
status: "published",
})
To finish creating the "special section," I must sort the documents alphabetically via their title attribute:
.sort({ [`collections.cameras.as.title`]: "asc" })
Now I want to add a link to "next article within the same special section" at the bottom of such articles. I know _id and any other value needed from the current article. The above query gives me an ordered list of documents within the section so I can easily find it within that list specialSectionListQuery.findOne({ _id: "xxx" }).exec().
However, I need to find the next article within the above list. How do I do that?
My attempts thus far:
I tried to create article list via aggregation, which led me nowhere (I simply made my app do exactly the same thing — make a list for a "special sectin"):
Article.aggregate([
{
$match: {
tag: { $ne: "excluded" },
[`collections.cameras`]: { $exists: true },
status: "published",
},
},
{
$sort: {
[`collections.cameras.as.title`]: 1,
},
}
]).exec()
But I can't for the life of me figure out how to iterate to the next document in the list properly.
I have thought of saving the list in Node.js memory (variable) and then finding what I need via JavaScript but that can't be scalable.
I have considered creating a new collection and saving the above list there but that would require me to either 1) do it every time a document is altered/added/deleted via Node.js — which is a lot of code and it may break if I interact with database another way 2) rebuild the colleciton every time I run the query, but that feels like it'll lack in performance.
Please help and thank you!
P.S.:
Example collection which should cover most of the cases I'm looking to solve for:
[
{
_id: 1,
name: "Canon",
collections: { cameras: { as: { title: "Half-Frame" } } },
tag: "included",
status: "published"
},
{
_id: 2,
name: "Pentax",
collections: { cameras: { as: { title: "Full-Frame" } } },
tag: "included",
status: "published"
},
{
_id: 3,
name: "Kodak",
collections: { film: { as: { title: "35mm Film" } } },
tag: "included",
status: "published"
},
{
_id: 4,
name: "Ricoh",
collections: { cameras: { as: { title: "Full-Frame" } } },
tag: "included",
status: "published"
},
{
_id: 5,
name: "Minolta",
collections: { cameras: { as: { title: "Half-Frame Review" } } },
tag: "excluded",
status: "published"
},
{
_id: 4,
name: "FED",
collections: { cameras: { as: { title: "Full-Frame" } } },
tag: "included",
status: "draft"
}
]
One thing you can try is to extend your $sort by adding _id so that it always returns documents in deterministic order:
{
$sort: {
"collections.cameras.as.title": 1,
_id: 1
}
},
{
$limit: 1
}
Once your first query returns the document with _id: 2 and collections.cameras.as.title: Full-Frame, you can use below query to get subsequent document:
{
$match: {
$and: [
{
tag: { $ne: "excluded" },
"collections.cameras": { $exists: true },
status: "published",
},
{
$or: [
{
$and: [
{ "collections.cameras.as.title": { $eq: "Full-Frame" } },
{ "_id": { $gt: 2 } }
]
},
{ "collections.cameras.as.title": { $gt: "Full-Frame" } }
]
}
]
}
},
{
$sort: {
"collections.cameras.as.title": 1,
_id: 1
}
},
{
$limit: 1
}
In this case due to deterministic $sort you can exclude previously found document by adding additional filtering criteria and the order should be preserved.
Mongo Playground
I have the following aggregation code which returns all the students for each school in my school ID list. For each student there is an academic year ID that looks like this: ObjectId("5ede4682341e8426f1cf6285")
return await this.studentSchoolModel.aggregate([
{
$match: {
school: { $in: schoolIDs },
},
},
{
$group: {
_id: '$school',
academic_years: '$academic_year',
total_students: { $sum: 1 },
},
},
]);
If I try to group by the school, it works, but if I try to group by the school and academic year I get the following error:
MongoError: The field 'academic_year' must be an accumulator object
Does anyone know what I'm doing wrong? I want the output for each school to look like this:
_id: 12345678 //schoolID
academic_years: {
_id: 1111111 //academic year ID
total_students: 5 //number of students in that year
},
{
_id: 2222222 //academic year ID
total_students: 8 //number of students in that year
}
You can use $group
first group is to group by school and academic year. The second group is to group by school
Here is the code
db.collection.aggregate([
{
"$group": {
"_id": { sc: "$school", ay: "$academic_year" },
total: { $sum: 1 }
}
},
{
$group: {
_id: "$_id.sc",
total_students: {
$push: {
_id: "$_id.ay",
total_students: "$total"
}
}
}
}
])
Working Mongo playground
I've an analytics API written using MongoDB.
This is my sessions model
const sessionSchema = new Schema(
{
user: { id: Number, name: String, email: String },
},
{ timestamps: true },
);
I want to get the unique users count by the date.
for the date 2019-10-24 there maybe 10 sessions from two users (id 1, id 2)
and
for the date 2019-10-25 there maybe 20 sessions from two users (id 3, id 8)
So my expected results is
2019-10-24 2 users
2019-10-25 2 users
I tried this
db.Session.aggregate([
{
$group: {
_id: { user: '$user.id', day: { $dayOfYear: '$createdAt' } },
count: { $sum: 1 },
},
},
])
and this doesn't seem to work.
My createdAt field's type is Date (eg:- createdAt: 2019-10-16T13:11:17.935Z) That is why I used $dayOfYear: '$createdAt'
db.Session.aggregate([
{
$project: {
date: { $dateToString: { format: "%Y-%m-%d", date: "$createdAt" } }
}
},
{
$group: {
_id: "$date" ,
count: { $sum: 1 }
}
}
]);
At first group according to createdAt, as we want final result according to createdAt.
In this $group stage just push the userIds into the array, use $addToSet to keep it unique.
Then get the count of userIds with $size operator in $project stage.
You get the result.
Here is the query.
(convert the date as you want, its just a format, and you have done this, so I am skipping this task).
db.sessions.aggregate({$match:{}},
{$group:{_id: "$createdAt", userId : {$addToSet: "$user.id"}}},
{$project:{_id: 1, noOfUsers:{$size:"$userId"}}}).pretty()
Hope this helps!
data=[{
locId: '332wn',
locadetails: [
{ loc: 'ny',
status: true
},
{ loc: 'ca',
status: null
},
{ loc: 'tx',
status: null
}
]
}]
I have following query that is trying to find all the locdetails that have open value as null or false
Loc.find({'locId': id}, {'locadetails' : {$elemMatch: {'status': {$ne: true}}}}, (err, locs)=>{
if(err) {
retrun callback(err);
}
callback(null, locs)
});
Problem I have is this query will only return one value o locadetails with null while it should return two as seen in the data.
Please let me know what to do so I can get whole array of items that have status field as null or false ...Thanks
$elemMatch will return first matching element from an array based on a condition. Use Aggregation instead.
Both the $ operator and the $elemMatch operator project the first
matching element from an array based on a condition. Reference
Loc.aggregate([
{ $match: { "locId": "332wn" } },
{ $unwind: "$locadetails" },
{ $match: { "locadetails.status": { $ne: true } } },
{ $group: { _id: "$_id", locId: { $first: "$locId" }, locadetails: { $push: "$locadetails" }, } }
])
If I want to sum up numerical values in MongoDB I do it like this:
totalOpenBalance: {
$sum: "$openBalance"
} // sum up all "openBalance" values
But what I'm wondering is, what operator do I use when I want to sum up instances of something? Say if I have a property such as customer_id, and data that looks like this:
{
"customer_id" : 445,
"other_prop" : value
},
{
"customer_id" : 446,
"other_prop" : value
},
Note that I don't want to sum up the values assigned to "customer_id", but rather tally up how many instances of "customer_id" are in the collection of data. In other words, according to the data above, I should get "2" as my output. What operator do I use to do that?
To clarify, this is a step I need to add to an aggregation pipeline I'm using to generate a mongo view.
Any of the below should get you going:
Simple find:
db.collection.find({
"customer_id": { $exists: true }
}).count()
Aggregate with $count:
db.collection.aggregate({
$match: {
"customer_id": { $exists: true }
}
}, {
$count: "numberOfInstances"
})
Aggregate with $group:
db.collection.aggregate({
$match: {
"customer_id": { $exists: true }
}
}, {
$group: {
_id: null,
"numberOfInstances": { $sum: 1 } // count instances
}
})
You can simply use find and $exists and then count the returned rows
db.collection.find( { customer_id: { $exists: true } } ).count()
Or if you want to use aggregate (which I don't think you should do for such simple task) this is how you can do it.
db.collection.aggregate({
$match: {
"customer_id": {
$exists: true
}
}
}, {
$group: {
_id: null,
"total": {
$sum: 1
}
}
})
Here total property will give you the number of instances containing customer_id in them