Here is my MongoDB collection schema:
company: String
model: String
tags: [String]
I need to aggregate it so I get the following output:
[{
"_id": {
"company": "Lenovo",
"model": "T400"
},
"tags": {
tag: "SomeTag"
count: 124 // number of times, this tag was found in `Lenovo T400`
}
}...]
I tried to do the following:
var aggParams = {};
aggParams.push({ $unwind: '$tags' });
aggParams.push({ $group: {
_id: { company: '$company', model: '$model' },
tags: { $push: { tag: '$tags', count: { $sum: 1 } } },
}});
But I got the following error:
invalid operator '$sum'
What is the right way to do this with aggregation?
You need to process $unwind on an array in order to deal with it meaninfully in aggregation. Also you adding to an array and "counts" stage a separate. As well an aggregation pipeline in an "array" of arguments itself, and not an object as you have defined:
Model.aggregate([
{ "$unwind": "$tags" },
{ "$group": {
"_id": {
"company": "$company",
"model": "$model",
"tag": "$tags"
},
"count": { "$sum": 1 }
}},
{ "$group": {
"_id": {
"company": "$_id.company",
"model": "$_id.model",
},
"tags": { "$push": { "tag": "$_id.tag", "count": "$count" }
}}
], function(err,result) {
})
So two $group stages does the job here. One to sum up the tags within company and model and the other to group on just the "company" and "model" and add the distinct tags and counts to an array.
Related
currently, I am struggling with how the MongoDB document system works. I want to fetch array elements with an auto-generated id but how to fetch that specific data that I don't know.
my current schema is
const ItemPricesSchema = new mongoose.Schema({
_id : {
type: String
},
ItemsPrices: {
type: [{
barcode : {
type: String
},
itemName : {
type: String
},
price : {
type: String
}
}]
}
});
current data is stored in this way
{
"_id": "sha#c.c",
"ItemsPrices": [
{
"barcode": "345345",
"itemName": "maggie",
"price": "45",
"_id": "620a971e11120abbde5f4c3a"
},
{
"barcode": "356345",
"itemName": "monster",
"price": "70",
"_id": "620a971e11120abbde5f4c3b"
}
],
"__v": 0
}
what I want to achieve is that I want to find array elements through ids
if I want a specific array element with id "620a971e11120abbde5f4c3b" what should I do??
I have tried $unwind , $in, $match...
the result should be like
{
"_id": "sha#c.c",
"ItemsPrices": [
{
"barcode": "356345",
"itemName": "monster",
"price": "70",
"_id": "620a971e11120abbde5f4c3b"
}
],
"__v": 0
}
what I tried is like this from the answer
router.get('/filter/:id', async (req, res) => {
try {
const item = await ItemPricesSchema.aggregate([
{$project: {
"ItemsPrices": {
$filter: {
input: "$ItemsPrices",
as: "item",
cond: {
$eq: [
"$$item._id",
"620a8dd1c88ae3eb88a8107a"
]
}
}
}
}
}
])
res.json(item);
console.log(item);
} catch (error) {
res.status(500).json({message: error.message});
}
})
and returns something like this (Empty arrays)
[
{
"_id": "xvz#zyx.z",
"ItemsPrices": []
},
{
"_id": "zxc#xc.czx",
"ItemsPrices: []
},
{
"_id": "asd#asd.asd",
"ItemsPrices": []
},
{
"_id": "qwe#qwe.qwe",
"ItemsPrices": []
}
]
but If I search for price $$item.price
cond: {
$eq: [
"$$item.price",
"30"
]
}
it returns the perfect output
[
{
"_id": "xvz#zyx.z",
"ItemsPrices": []
},
{
"_id": "zxc#xc.czx",
"ItemsPrices: []
},
{
"_id": "asd#asd.asd",
"ItemsPrices": []
},
{
"_id": "qwe#qwe.qwe",
"ItemsPrices": [
{
"barcode":"234456345",
"price":"30",
"itemName":"monster",
"_id":"620a8dd1c88ae3eb88a8107a"
}
]
}
]
You can do an aggregation with $project and apply $filter on the array part. In mongoose you can apply the aggregation query in a more or less similar way https://mongoosejs.com/docs/api/aggregate.html
db.collection.aggregate([
{
$project: {
"ItemsPrices": {
$filter: {
input: "$ItemsPrices",
as: "item",
cond: {
$eq: [
"$$item._id",
mongoose.Types.ObjectId("620a971e11120abbde5f4c3b")
]
}
}
},
"__v": 1 //when projecting 1 means in the final result this field appears
}
}
])
more examples
demo
Option 1:
Use $filter in an aggregation query as explained by cmgchess
Option 2:
If you only want one object from array you can use $elemMatch like this:
db.collection.find({
"ItemsPrices._id": "620a971e11120abbde5f4c3b"
},
{
"ItemsPrices": {
"$elemMatch": {
"_id": "620a971e11120abbde5f4c3b"
}
}
})
Example here
But take care, using $elemMatch only the first element is returned. Check this other example where there are two objects with the desired _id but only returns one.
As said before, if you only one (or only exists one) maybe you can use find and $elemMatch to avoid a filter by the entire array. But if can be multiple values use $filter.
I have a query result from a mongoose aggregation query that I need to further process, or at best do it in the aggregation itself.
The aggregation looks like this
result = await TokenBalance.aggregate([
{
$match: {
$and: [
{ ethervalue: { $gte: minBalance } },
{
ethervalue: { $lte: maxBalance }
}
]
}
},
{ $limit:limit }
])
This returns an array of Objects of this format
{
"_id": "61013d6dda7d7c0015af5ccf",
"balances": [
{
"address": "0x1fc3ddeb035310930a444c0fa59c01618d5902af",
"symbol": "HBTC",
"balance": 5.21419339e-10,
"usdvalue": 0.000020969961637162402
},
{
"address": "0x1fc3ddeb035310930a444c0fa59c01618d5902af",
"symbol": "NSBT",
"balance": 1.258566,
"usdvalue": 27.427343477595258
},
{
"address": "0x1fc3ddeb035310930a444c0fa59c01618d5902af",
"symbol": "CRV",
"balance": 517.985955847106,
"usdvalue": 806.7017064052314
},
{
"address": "0x1fc3ddeb035310930a444c0fa59c01618d5902af",
"symbol": "USDT",
"balance": 0.003469,
"usdvalue": 0.003470159747979122
}
],
"address": "0x1fc3ddeb035310930a444c0fa59c01618d5902af",
"ethervalue": 0.7604598621232733,
"createdAt": "2021-07-28T11:20:13.927Z",
"updatedAt": "2021-07-28T11:20:13.927Z",
"__v": 0
},
What I need, is the "balances" property to be processed as grouped by symbol and for each of these symbols sum the balance and usdvalue fields.
I would prefer this do be done in the aggregation if possible, but I can not seem to get it right, even not in pure nodejs.
I want the result to be like this:
[
{
symbol: USDC, balance: xxx, usdvalue: yyy
},
{
symbol: USDT, balance: zzz, usdvalue: jjj
}
]
You can use the below approach,
$unwind to deconstruct the balances array
$group by symbol and sum balance and usdvalue
$addFields to rename _id field to symbol and and remove _id field
result = await TokenBalance.aggregate([
{
$match: {
$and: [
{ ethervalue: { $gte: minBalance } },
{ ethervalue: { $lte: maxBalance } }
]
}
},
{ $unwind: "$balances" },
{
$group: {
_id: "$balances.symbol",
balance: { $sum: "$balances.balance" },
usdvalue: { $sum: "$balances.usdvalue" }
}
},
{
$addFields: {
symbol: "$_id",
_id: "$$REMOVE"
}
},
{ $limit:limit }
])
Playground
I'm using nodejs with mongoose (mongodb) and I want to filter inside a subdocument array the language selected.
User schema:
var localUserSchema = new mongoose.Schema({
firstName: {
type: String
},
moreInformation: {
experience: {
type: Number
},
specializations: [{
...
sports:[{
type: Schema.Types.ObjectId,
ref: 'sport'
}]
}]
});
User data:
[{
"_id": {
"$oid": "5fc6a379b1d5ff2c42a9a536"
},
"moreInformation": {
"experience" : 2,
"specializations": [{
"sports": [{
"$oid": "5fc6aa91b1db6cd15702241c"
}, {
"$oid": "5fcb741e786f0703646befe2"
}]
}]
}
}]
Sport schema:
var sportSchema = new Schema({
name: {
en: {
type: String
},
it: {
type: String
}
},
icon: {
type: Schema.Types.ObjectId, ref: 'file'
}
});
Sport data:
[{
"_id": {
"$oid": "5fc6aa91b1db6cd15702241c"
},
"name": {
"en": "Football",
"it": "Calcio"
},
"icon": {
"$oid": "5fc9598a0955177dee8a3bc4"
}
},{
"_id": {
"$oid": "5fcb741e786f0703646befe2"
},
"name": {
"en": "Swimming",
"it": "Nuoto"
},
"icon": {
"$oid": "5fc9598a0955177dee8a3bc5"
}
}
I want all users joined with sports by sport id, but filtered by language key like below and replace the sports name languages with the name selected without the language key.
So, if I wanted to select and filter the english language 'en', i would like to get this result:
[
{
"_id": "5fc6a379b1d5ff2c42a9a536",
"moreInformation": {
"specializations": [{
...
"sports": [
{
"_id": "5fc6aa91b1db6cd15702241c",
"name": "Football",
"icon": "5fc9598a0955177dee8a3bc4"
},
{
"_id": "5fcb741e786f0703646befe2",
"name": "Swimming",
"icon": "5fc9598a0955177dee8a3bc5"
}]
}]
}
}
}
How I can do it?
You will need to use aggregation methods like $unwind, $lookup, $group last but not least $project.
They define a sequence of steps to help you retrieve your data as you expect
The get the response that you want take a look into the following code and the running example in here
Ps: I suggest you to look into the example in the mongo playground ( link above ) and separate the aggregation pipelines to understand better the proccess
db.users.aggregate([
{
// move specializations array to separated objects
"$unwind": "$moreInformation.specializations"
},
{
// move specialization.sports to separated objects
"$unwind": "$moreInformation.specializations.sports"
},
{
// search into sports collection based on the temporary "sports" attribute
$lookup: {
from: "sports",
localField: "moreInformation.specializations.sports",
foreignField: "_id",
as: "fullSport"
}
},
{
// as the lookup resolves an array of a single result we move it to be an object
"$unwind": "$fullSport"
},
{
// here we select only the attributes that we need and the selected language
"$project": {
"moreInformation.experience": 1,
"moreInformation.specializations.sports._id": "$fullSport._id",
"moreInformation.specializations.sports.icon": "$fullSport.icon",
"moreInformation.specializations.sports.name": "$fullSport.name.en"
}
},
{
// then we group it to make the separated objects an array again
"$group": {
"_id": "$_id",
// we group by the $_id and move it to temporary "sports" attribute
"sports": {
$push: "$moreInformation.specializations.sports"
},
"moreInformation": {
$first: "$moreInformation"
}
}
},
{
$project: {
"moreInformation.experience": 1,
// move back the temporary "sports" attribute to its previous path
"moreInformation.specializations.sports": "$sports"
}
}
])
I need to output a list of unique values in a collection. I've used the distinct method when the values are either a string or a number. But in this situation, the values are an array of objects.
The simplified model looks like this:
const mongoose = require('mongoose');
const ItemModel = mongoose.Schema({
category: [{
lang: String,
text: String
}],
discount: {
type: String,
required: true
},
date: {
type: Date,
default: Date.now
}
});
module.exports = mongoose.model('Items', ItemModel);
A typical find() query without parameters produces this result:
[
{
"_id": "5fd8712b374a9a1410f786bf",
"category": [
{
"_id": "5fd8712b374a9a1410f786c2",
"lang": "RU",
"text": "Домашняя одежда"
},
{
"_id": "5fd8712b374a9a1410f786c3",
"lang": "EN",
"text": "Homewear"
}
],
"discount": "45%",
"date": "2020-12-12T11:12:37.811Z",
"__v": 0
},
{
"_id": "5fd4a5b95e1a251ac96b2e08",
"category": [
{
"_id": "5fd4a5b95e1a251ac96b2e0b",
"lang": "RU",
"text": "Домашняя одежда"
},
{
"_id": "5fd4a5b95e1a251ac96b2e0c",
"lang": "EN",
"text": "Homewear"
}
],
"discount": "35%",
"date": "2020-12-12T11:12:37.811Z",
"__v": 0
},
{
"_id": "5fd49e415e1a251ac96b2dfc",
"category": [
{
"_id": "5fd49e415e1a251ac96b2dff",
"lang": "RU",
"text": "Активный отдых"
},
{
"_id": "5fd49e415e1a251ac96b2e00",
"lang": "EN",
"text": "Active"
}
],
"discount": "50%",
"date": "2020-12-12T10:06:53.120Z",
"__v": 0
}
]
I need to output a list of unique "category.text" values where "lang" equals "EN". At the output, we should get an array like this:
[ "Active", "Homewear" ]
How to do it the right way, with good performance?
Regards.
You can use aggregation pipeline,
$unwind to deconstruct category array
$match category.lang is EN
$group by null, and get unique text using $addToSet
StockModel.aggregate([
{ $unwind: "$category" },
{ $match: { "category.lang": "EN" } },
{
$group: {
_id: null,
test: { $addToSet: "$category.text" }
}
}
])
Playground
use aggregation :
take a look at the example below :
StockModel.aggregate([
{
$group: {
_id: 0,
title: { $addToSet: '$title' },
stock_id: { $addToSet: '$stock_id' },
},
},
]);
with aggregation $group you can distinct your collections via filter.
I have created an aggregate function and I feel it's pretty long and non-DRY. I'm wondering what ways I can improve it.
My Thread model has a sub-document called revisions. The function tries to get the most recent revision that has the status of APPROVED.
Here is the full model.
{
"_id": ObjectId("56dc750769faa2393a8eb656"),
"slug": "my-thread",
"title": "my-thread",
"created": 1457249482555.0,
"user": ObjectId("56d70a491128bb612c6c9220"),
"revisions": [
{
"body": "This is the body!",
"status": "APPROVED",
"_id": ObjectId("56dc750769faa2393a8eb657"),
"comments": [
],
"title": "my-thread"
}
]
}
And here is the aggregate function I want to improve.
Thread.aggregate([
{ $match: {
slug: thread
} },
{ $project: {
user: '$user',
created: '$created',
slug: '$slug',
revisions: {
$filter: {
input: '$revisions',
as: 'revision',
cond: { $eq: [ '$$revision.status', 'APPROVED' ] }
}
}
} },
{ $sort: { 'revisions.created': -1 } },
{ $project: {
user: '$user',
created: '$created',
slug: '$slug',
revisions: { $slice: ["$revisions", 0, 1] }
} },
{ $unwind: '$revisions'},
{ $project: {
body: '$revisions.body',
title: '$revisions.title',
user: '$user',
slug: '$slug',
created: '$created'
}}
])
Well you cannot really since there are $sort and $unwind stages in between on purpose. It's also basically "wrong", since the $sort cannot re-order the array until you $unwind it first.
Then it is better to use $group and $first instead, to just get the first element from the sort in each document:
Thread.aggregate([
{ "$match": {
"slug": thread
} },
{ "$project": {
"user": 1,
"created": 1,
"slug": 1,
"revisions": {
"$filter": {
"input": "$revisions",
"as": "revision",
"cond": { "$eq": [ "$$revision.status", "APPROVED" ] }
}
}
} },
// Cannot sort until you $unwind
{ "$unwind": "$revisions" },
// Now that will sort the elements
{ "$sort": { "_id": 1, "revisions.created": -1 } },
// And just grab the $first boundary for everything
{ "$group": {
"_id": "$_id",
"body": { "$first": "$revisions.body" },
"title": { "$first": "$revisions.title" },
"user": { "$first": "$user" },
"slug": { "$first": "$slug" },
"created": { "$first": "$created" }
}}
])
You could always reform the array with $push and then apply $arrayElemAt instead of the $slice to yield just a single element, but it's kind of superflous considering it would need another $project after the $group in the first place.
So even though there are "some" operations you can do without using $unwind, unfortunately "sorting" the arrays generated out of functions like $filter is not something that can be presently done, until you $unwind the array first.
If you didn't "need" the $sort on the "revisions.created" ( notably missing from your sample document ) then you can instead just use normal projection instead:
Thread.find(
{ "slug": slug, "revisions.status": "APPROVED" },
{ "revisions.$": 1 },
)
Only when sorting array elements would you need anything else, since the $ positional operator will just return the first matched element anyway.