I have a collection named Vote that looks like the following:
{
postId: "1",
comment:{
text_sentiment: "positive",
topic: "A"
}
}, // DOC-1
{
postId: "2",
comment:{
text_sentiment: "negative",
topic: "A"
}
}, // DOC-2
{
postId: "3",
comment:{
text_sentiment: "positive",
topic: "B"
}
},..//DOC-3 ..
I want to do an aggregation on this collection such that it returns the following structure.
[
{
_id: "hash",
topic: "A",
topicOccurance: 2,
sentiment: {
positive: 1,
negative: 1,
neutral: 0
},
postIds: [1,2]
},
..
]
I created the following aggregation:
db.Vote.aggregate([
{
$match: {
surveyId: "e6d38e1ecd",
"comment.topic": {
$exists: 1
},
}
},
{
$group: {
_id: {
topic: "$comment.topic",
text_sentiment: "$comment.text_sentiment"
},
total: {
$sum: 1
},
}
},
{
$group: {
_id: "$_id.topic",
total: {
$sum: "$total"
},
text_sentiments: {
$push: {
k: "$_id.text_sentiment",
v: "$total"
}
}
}
},
{
$project: {
topic: "$_id",
topicOccurance: "$total",
sentiment: {
"$arrayToObject": "$text_sentiments"
}
}
},
{
$sort: {
"topicOccurance": -1
}
}
])
This works fine but I do not know how can I also get an array in the response holding the key postIds. Each document inside the collection vote has postId and I want to group the posts having the same topic and push to an array. How can I do this?
2nd stage ($group) - Add postId into postIds array via $push.
3rd stage ($group) - Add postIds array into postIds array via $push. This will leads postIds become nested array.
[[1,2], ...]
4th stage ($project) - For postIds field, use $reduce operator to flatten the postIds array by $concat. Update: with $setUnion to distinct items in array.
db.collection.aggregate([
// match stage
{
$group: {
_id: {
topic: "$comment.topic",
text_sentiment: "$comment.text_sentiment"
},
total: {
$sum: 1
},
postIds: {
$push: "$postId"
}
}
},
{
$group: {
_id: "$_id.topic",
total: {
$sum: "$total"
},
text_sentiments: {
$push: {
k: "$_id.text_sentiment",
v: "$total"
}
},
postIds: {
"$push": "$postIds"
}
}
},
{
$project: {
topic: "$_id",
topicOccurance: "$total",
sentiment: {
"$arrayToObject": "$text_sentiments"
},
postIds: {
$setUnion: [
{
$reduce: {
input: "$postIds",
initialValue: [],
in: {
$concatArrays: [
"$$value",
"$$this"
]
}
}
}
]
}
}
},
// sort stage
])
Sample Mongo Playground
What I want to achieve is finding a specific document on that current month based on the provided date. The date is stored as a string, in order for me to compare the date I need to convert the date first. However I have trouble on converting the datestring in a nested array of objects.
My collections:
{
sections: [{
fields: [{
name: 'Date',
value: '2020-11-30T15:59:59.999Z' // this is string
},
{
name: 'Title',
value: 'My book'
},
{
name: 'Author',
value: 'Henry'
}
]
]
}
}
What I have tried:
1)
const existingReport = await Report.find({
$expr: {
$gte: [
{
$dateFromString: {
dateString: "$sections.field[0].value",
},
},
moment(payload.forPeriod).startOf("month").toDate(),
],
$lt: [
{
$dateFromString: {
dateString: "$sections.field[0].value",
},
},
moment(payload.forPeriod).endOf("month").toDate(),
],
},
});
const existingReport1 = await Report.aggregate([
{
$addFields: {
formattedData: {
$cond: {
if: {
$eq: ["$sections.field.value", "Date"],
},
then: {
$dateFromString: {
dateString: "$sections.field.value",
},
},
else: "$sections.field.value",
},
},
},
},
]);
You can simply do a $toDate with the help of 2 $reduce to iterate the sections and fields array.
db.collection.aggregate([
{
"$match": {
$expr: {
$eq: [
true,
{
"$reduce": {
"input": "$sections",
"initialValue": false,
"in": {
"$reduce": {
"input": "$$this.fields",
"initialValue": false,
"in": {
$or: [
"$$value",
{
$and: [
{
$gte: [
{
"$toDate": "$$this.value"
},
new Date("2020-11-01")
]
},
{
$lte: [
{
"$toDate": "$$this.value"
},
new Date("2020-11-30")
]
}
]
}
]
}
}
}
}
}
]
}
}
}
])
Here is the Mongo playground for your reference.
I would like to return this only if there are 2 by in the data array. The number of _id can be unlimited.
However, the code { $size: { data: 2 }, } does not work because I get $size is not allowed in this atlas tier error.
Expected return:
[
{
"_id": "Something1?",
"data": [
{
"by": "user1",
},
{
"by": "user2",
}
]
},
]
I want to include something like $size in the code, otherwise it will return the data even if there is only 1 by, or 3 by, or 0 by. I only want to return the data if there are 2 by.
What should I do? Full code without $size:
let x = await Answer.aggregate([
{
$match: {
$and: [
{
by: {
$in: [user.email, user2[0].email],
},
},
],
},
},
{
$group: {
_id: "$question",
data: {
$push: "$$ROOT",
},
},
},
{
$project: {
"data._id": 0,
"data.question": 0,
"data.__v": 0,
},
},
{ $sort: { "data.date": -1 } },
]);
Looks like your atlas tier doesn't support $size.
But you can have a field like count that increments by 1 when grouping:
db.collection.aggregate([
{
$group: {
_id: "$question",
data: {
$push: "$$ROOT",
},
count: {
$sum: 1
}
}
},
{
$match: {
count: 2
}
}
])
Try this in playground
Update
Finally, your aggregation should look like this:
[
{
$match: {
$and: [
{
by: {
$in: [user.email, user2[0].email],
},
},
],
},
},
{
$group: {
_id: "$question",
data: {
$push: "$$ROOT",
},
count: {
$sum: 1
}
},
},
{
$match: {
count: 2
}
},
{
$project: {
"data._id": 0,
"data.question": 0,
"data.__v": 0,
"count": 0
},
},
{ $sort: { "data.date": -1 } },
]
You can learn more about $sum here.
Presuming your model is called Employee:
Employee.find({ { "social_account.2": { "$exists": false }} },function(err,docs) {
})
As $exists asks for the 2 index of an array which means it has something in it.
The same applies to a maximum number:
Employee.find({ { "social_account.9": { "$exists": true}} },function(err,docs) {
})
For your perspective I think this should be your answer:
Employee.find({ { "data.2": { "$exists": false }} },function(err,docs) {
})
I have this endpoint, it's the initial endpoint when a customer is visiting the eshop:
export const getAllProductsByCategory = async (req, res, next) => {
const pageSize = parseInt(req.query.pageSize);
const sort = parseInt(req.query.sort);
const skip = parseInt(req.query.skip);
const { order, filters } = req.query;
const { brands, tags, pricesRange } = JSON.parse(filters);
try {
const aggregate = Product.aggregate();
aggregate.lookup({
from: 'categories',
localField: 'categories',
foreignField: '_id',
as: 'categories'
});
aggregate.match({
productType: 'product',
available: true,
categories: {
$elemMatch: {
url: req.params
}
}
});
aggregate.lookup({
from: 'tags',
let: { tags: '$tags' },
pipeline: [
{
$match: {
$expr: { $in: ['$_id', '$$tags'] }
}
},
{
$project: {
_id: 1,
name: 1,
slug: 1
}
}
],
as: 'tags'
});
aggregate.lookup({
from: 'brands',
let: { brand: '$brand' },
pipeline: [
{
$match: {
$expr: { $eq: ['$_id', '$$brand'] }
}
},
{
$project: {
_id: 1,
name: 1,
slug: 1
}
}
],
as: 'brand'
});
if (brands.length > 0) {
const filterBrands = brands.map((_id) => utils.toObjectId(_id));
aggregate.match({
$and: [{ brand: { $elemMatch: { _id: { $in: filterBrands } } } }]
});
}
if (tags.length > 0) {
const filterTags = tags.map((_id) => utils.toObjectId(_id));
aggregate.match({ tags: { $elemMatch: { _id: { $in: filterTags } } } });
}
if (pricesRange.length > 0 && pricesRange !== 'all') {
const filterPriceRange = pricesRange.map((_id) => utils.toObjectId(_id));
aggregate.match({
_id: { $in: filterPriceRange }
});
}
aggregate.facet({
tags: [
{ $unwind: { path: '$tags' } },
{ $group: { _id: '$tags', tag: { $first: '$tags' }, total: { $sum: 1 } } },
{
$group: {
_id: '$tag._id',
name: { $addToSet: '$tag.name' },
total: { $addToSet: '$total' }
}
},
{
$project: {
name: { $arrayElemAt: ['$name', 0] },
total: { $arrayElemAt: ['$total', 0] },
_id: 1
}
},
{ $sort: { total: -1 } }
],
brands: [
{ $unwind: { path: '$brand' } },
{
$group: {
_id: '$brand._id',
name: { $first: '$brand.name' },
slug: { $first: '$brand.slug' },
total: {
$sum: 1
}
}
},
{ $sort: { name: 1 } }
],
pricesRange: [
{
$bucket: {
groupBy: {
$cond: {
if: { $ne: ['$onSale.value', true] },
then: '$price',
else: '$sale.salePrice'
}
},
boundaries: [0, 20.01, 50.01],
default: 'other',
output: {
count: { $sum: 1 },
products: { $push: '$_id' }
}
}
}
],
products: [
{ $skip: (skip - 1) * pageSize },
{ $limit: pageSize },
{
$project: {
_id: 1,
images: 1,
onSale: 1,
price: 1,
quantity: 1,
slug: 1,
sale: 1,
sku: 1,
status: 1,
title: 1,
brand: 1,
tags: 1,
description: 1
}
},
{ $sort: { [order]: sort } }
],
total: [
{
$group: {
_id: null,
count: { $sum: 1 }
}
},
{
$project: {
count: 1,
_id: 0
}
}
]
});
aggregate.addFields({
total: {
$arrayElemAt: ['$total', 0]
}
});
const [response] = await aggregate.exec();
if (!response.total) {
response.total = 0;
}
res.status(httpStatus.OK);
return res.json(response);
} catch (error) {
console.log(error);
return next(error);
}
};
If no filters are applied all products matches the category requested with no problem.
My issue is when a customer selects a brand or tag, then the facet returns the products, but returns only one brand/tag (as it should be since the products filtered have only this brand).
What I must do in order to retain all brands/tags and let the user select more than one brand/tag? If customer selects a brand, then the tags should match the returned products tags and vice versa.
Is there a better way to implement tags stage in $facet since tags is an array and the desired output is: [{_id: 123, name: {label: 'test', value: 123]}]
The request is like:(1,2,3,4 represents _id)
http://locahost:3000/get-products/?filters={brands: [1, 2], tags: [3,4], pricesRange:[]}
Update
This is the products schema with tags and brands:
brand: {
ref: 'Brand',
type: Schema.Types.ObjectId
},
tags: [
{
ref: 'Tags',
type: Schema.Types.ObjectId
}
]
tags schema:
{
metaDescription: {
type: String
},
metaTitle: {
type: String
},
name: {
label: {
type: String,
index: true
},
value: {
type: Schema.Types.ObjectId
},
},
slug: {
type: String,
index: true
},
status: {
label: {
type: String
},
value: {
default: true,
type: Boolean
}
}
}
brands schema:
description: {
default: '',
type: String
},
name: {
required: true,
type: String,
unique: true
},
slug: {
type: String,
index: true
},
status: {
label: {
default: 'Active',
type: String
},
value: {
default: true,
type: Boolean
}
}
Scenario:
User visits store, selects a category and all matching products should return with matched brands, tags, priceRange & pagination.
Case 1:
User clicks a brand from checkbox, then the request returns matching products,tags & priceRanges and all brands of the selected category, not of matched products
Case 2:
User selects a brand like Case 1, but then decides to check a tag too, then the request should return all brands and tags again, but products matched against them.
Case 3:
User do not select brand but selects a tag only, the request should return all matching products that have that tag/tags and return the brands that matched the products returned.
Case 4:
Same as case 3, but user selects a brand after selecting a tag/tags, the request should return matching products, brands & tags.
In all cases pagination should return proper total, also priceRanges should match the returned results.
I hope it's clear now, I think I've not missed any other case. I could probably grey out/disable the tags/brands that do not match the response in the front end but I don't know if this is user friendly.
This is what I ended up with:
export const getAllProductsByCategory = async (req, res, next) => {
const pageSize = parseInt(req.query.pageSize);
const sort = parseInt(req.query.sort);
const skip = parseInt(req.query.skip);
const { order, filters } = req.query;
const { brands, tags, pricesRange } = JSON.parse(filters);
try {
const aggregate = Product.aggregate();
aggregate.lookup({
from: 'categories',
localField: 'categories',
foreignField: '_id',
as: 'categories'
});
aggregate.match({
productType: 'product',
available: true,
categories: {
$elemMatch: {
url: `/${JSON.stringify(req.params['0']).replace(/"/g, '')}`
}
}
});
aggregate.lookup({
from: 'tags',
let: { tags: '$tags' },
pipeline: [
{
$match: {
$expr: { $in: ['$_id', '$$tags'] }
}
},
{
$project: {
_id: 1,
name: 1,
slug: 1
}
}
],
as: 'tags'
});
aggregate.lookup({
from: 'brands',
let: { brand: '$brand' },
pipeline: [
{
$match: {
$expr: { $eq: ['$_id', '$$brand'] }
}
},
{
$project: {
_id: 1,
name: 1,
slug: 1
}
}
],
as: 'brand'
});
const filterBrands = brands.map((_id) => utils.toObjectId(_id));
const filterTags = tags.map((_id) => utils.toObjectId(_id));
const priceRanges = pricesRange ? pricesRange.match(/\d+/g).map(Number) : '';
aggregate.facet({
tags: [
{ $unwind: { path: '$brand' } },
{ $unwind: { path: '$tags' } },
{
$match: {
$expr: {
$and: [
filterBrands.length ? { $in: ['$brand._id', filterBrands] } : true
]
}
}
},
{ $group: { _id: '$tags', tag: { $first: '$tags' }, total: { $sum: 1 } } },
{
$group: {
_id: '$tag._id',
name: { $addToSet: '$tag.name' },
total: { $addToSet: '$total' }
}
},
{
$project: {
name: { $arrayElemAt: ['$name', 0] },
total: { $arrayElemAt: ['$total', 0] },
_id: 1
}
},
{ $sort: { name: 1 } }
],
brands: [
{ $unwind: { path: '$brand' } },
{ $unwind: { path: '$tags' } },
{
$match: {
$expr: {
$and: [
filterTags.length ? { $in: ['$tags._id', filterTags] } : true
]
}
}
},
{
$group: {
_id: '$brand._id',
name: { $first: '$brand.name' },
slug: { $first: '$brand.slug' },
total: {
$sum: 1
}
}
},
{ $sort: { name: 1 } }
],
products: [
{ $unwind: { path: '$brand', preserveNullAndEmptyArrays: true } },
{ $unwind: { path: '$tags', preserveNullAndEmptyArrays: true } },
{
$match: {
$expr: {
$and: [
filterBrands.length ? { $in: ['$brand._id', filterBrands] } : true,
filterTags.length ? { $in: ['$tags._id', filterTags] } : true,
pricesRange.length
? {
$and: [
{
$gte: [
{
$cond: {
if: { $ne: ['$onSale.value', true] },
then: '$price',
else: '$sale.salePrice'
}
},
priceRanges[0]
]
},
{
$lte: [
{
$cond: {
if: { $ne: ['$onSale.value', true] },
then: '$price',
else: '$sale.salePrice'
}
},
priceRanges[1]
]
}
]
}
: true
]
}
}
},
{ $skip: (skip - 1) * pageSize },
{ $limit: pageSize },
{
$project: {
_id: 1,
brand: 1,
description: 1,
images: 1,
onSale: 1,
price: 1,
quantity: 1,
sale: 1,
shipping: 1,
sku: 1,
skuThreshold: 1,
slug: 1,
status: 1,
stock: 1,
tags: 1,
title: 1
}
},
{ $sort: { [order]: sort } }
],
pricesRange: [
{ $unwind: { path: '$brand', preserveNullAndEmptyArrays: true } },
{ $unwind: { path: '$tags', preserveNullAndEmptyArrays: true } },
{
$match: {
$expr: {
$and: [
filterBrands.length ? { $in: ['$brand._id', filterBrands] } : true,
filterTags.length ? { $in: ['$tags._id', filterTags] } : true
]
}
}
},
{
$project: {
price: 1,
onSale: 1,
sale: 1,
range: {
$cond: [
{
$and: [
{
$gte: [
{
$cond: {
if: { $ne: ['$onSale.value', true] },
then: '$price',
else: '$sale.salePrice'
}
},
0
]
},
{
$lte: [
{
$cond: {
if: { $ne: ['$onSale.value', true] },
then: '$price',
else: '$sale.salePrice'
}
},
20
]
}
]
},
'0-20',
{
$cond: [
{
$and: [
{
$gte: [
{
$cond: {
if: { $ne: ['$onSale.value', true] },
then: '$price',
else: '$sale.salePrice'
}
},
20
]
},
{
$lte: [
{
$cond: {
if: { $ne: ['$onSale.value', true] },
then: '$price',
else: '$sale.salePrice'
}
},
50
]
}
]
},
'20-50',
'50+'
]
}
]
}
}
},
{
$group: {
_id: '$range',
count: { $sum: 1 }
}
},
{
$project: {
_id: 0,
range: '$_id',
count: 1
}
},
{ $unwind: { path: '$range', preserveNullAndEmptyArrays: true } },
{
$sort: {
range: 1
}
}
],
total: [
{ $unwind: { path: '$brand', preserveNullAndEmptyArrays: true } },
{ $unwind: { path: '$tags', preserveNullAndEmptyArrays: true } },
{
$match: {
$expr: {
$and: [
filterBrands.length ? { $in: ['$brand._id', filterBrands] } : true,
filterTags.length ? { $in: ['$tags._id', filterTags] } : true,
pricesRange.length
? {
$and: [
{
$gte: [
{
$cond: {
if: { $ne: ['$onSale.value', true] },
then: '$price',
else: '$sale.salePrice'
}
},
priceRanges[0]
]
},
{
$lte: [
{
$cond: {
if: { $ne: ['$onSale.value', true] },
then: '$price',
else: '$sale.salePrice'
}
},
priceRanges[1]
]
}
]
}
: true
]
}
}
},
{
$group: {
_id: null,
count: { $sum: 1 }
}
},
{
$project: {
count: 1,
_id: 0
}
}
]
});
aggregate.addFields({
total: {
$arrayElemAt: ['$total', 0]
}
});
const [response] = await aggregate.exec();
if (!response.total) {
response.total = 0;
}
res.status(httpStatus.OK);
return res.json(response);
} catch (error) {
console.log(error);
return next(error);
}
};
I have an aggregation query that returns the sum / total number of reviews submitted for a given location ( not the average star rating ). Reviews are scored 1 - 5 stars. This particular query groups these reviews into two categories, "internal" and "google".
I have a query that returns results that are almost what I'm looking for. However, I need to add an additional condition for internal reviews. I want to ensure that the internal reviews "stars" value exists / is not null and contains a value of at least 1. So, I was thinking adding something similar to this would work:
{ "stars": {$gte: 1} }
This is the current aggregation query:
[
{
$match: { createdAt: { $gte: fromDate, $lte: toDate } }
},
{
$lookup: {
from: 'branches',
localField: 'branch',
foreignField: '_id',
as: 'branch'
}
},
{ $unwind: '$branch' },
{
$match: { 'branch.org_id': branchId }
},
{
$group: {
_id: '$branch.name',
google: {
$sum: {
$cond: [{ $eq: ['$source', 'Google'] }, 1, 0]
}
},
internal: {
$sum: {
$cond: [ { $eq: ['$internal', true]}, 1, 0 ],
},
}
}
}
]
Truncated Schema:
{
branchId: { type: String, required: true },
branch: { type: Schema.Types.ObjectId, ref: 'branches' },
wouldRecommend: { type: String, default: '' }, // RECOMMENDATION ONLY
stars: { type: Number, default: 0 }, // IF 1 - 5 DOCUMENT IS A REVIEW
comment: { type: String, default: '' },
internal: { type: Boolean, default: true },
source: { type: String, required: true },
},
{ timestamps: true }
I need to make sure that I'm not counting "wouldRecommend" recommendations in the sum of the internal reviews. Do determine if something is a review it will have a star rating of 1 or more stars. Recommendations will have a star value of 0.
How can I add the condition that ensures the internal "$stars" value is >= 1 ( greater than or equal to 1 ) ?
Using Ashh's answer I was able to form this query:
[
{
$lookup: {
from: 'branches',
localField: 'branch',
foreignField: '_id',
as: 'branch'
}
},
{ $unwind: '$branch' },
{
$match: {
'branch.org_id': branchId
}
},
{
$group: {
_id: '$branch.name',
google: {
$sum: {
$cond: [{ $eq: ['$source', 'Google'] }, 1, 0]
}
},
internal: {
$sum: {
$cond: [
{
$and: [{ $gte: ['$stars', 1] }, { $eq: ['$internal', true] }]
},
1,
0
]
}
}
}
}
];
You can use $and with the $cond operator
{ "$group": {
"_id": "$branch.name",
"google": { "$sum": { "$cond": [{ "$eq": ["$source", "Google"] }, 1, 0] }},
"internal": { "$sum": { "$cond": [{ "$eq": ["$internal", true] }, 1, 0 ] }},
"rating": {
"$sum": {
"$cond": [
{
"$and": [
{ "$gte": ["$stars", 1] },
{ "$eq": ["$internal", true] }
]
},
1,
0
],
}
}
}}