I have a schema, Comment, like the one below. It's a system of "comments" and "replies", but each comment and reply has multiple versions. When a user wants to view a comment, I want to return just the most recent version with the status of APPROVED.
const Version = new mongoose.Schema({
user: {
type: mongoose.Schema.Types.ObjectId,
ref: 'User'
},
body: String,
created: Date,
title: String,
status: {
type: String,
enum: [ 'APPROVED', 'OPEN', 'CLOSED' ]
}
})
const Reply = new mongoose.Schema({
user: {
type: mongoose.Schema.Types.ObjectId,
ref: 'User'
},
created: Date,
versions: [ Version ]
})
const Comment = new mongoose.Schema({
user: {
type: mongoose.Schema.Types.ObjectId,
ref: 'User'
},
created: Date,
versions: [ Version ],
replies: [ Reply ]
})
I've gotten the parent Comment to display how I want with the code below. However, I've had trouble applying that to the sub-document, Reply.
const requestedComment = yield Comment.aggregate([
{ $match: {
query
} },
{ $project: {
user: 1,
replies: 1,
versions: {
$filter: {
input: '$versions',
as: 'version',
cond: { $eq: [ '$$version.status', 'APPROVED' ] }
}
},
}},
{ "$unwind": "$versions" },
{ $sort: { 'versions.created': -1 } },
{ $group: {
_id: '$_id',
body: { $first: '$versions.body' },
title: { $first: '$versions.title' },
replies: { $first: '$replies' }
}}
])
.exec()
Any help achieving the same result with the replies subdocuments would be appreciated. I would like to return the most recent APPROVED version of each reply in a form like this:
comment: {
body: "The comment's body.",
user: ObjectId(...),
replies: [
{
body: "The reply's body."
user: ObjectId(...)
}
]
}
Basically you just need to continue the same process on from the existing pipeline. But this time to $unwind out the "versions" per each "replies" entry and $sort them there.
So these are "additional" stages to your pipeline.
// Unwind replies
{ "$unwind": "$replies" },
// Unwind inner versions
{ "$unwind": "$replies.versions" },
// Filter for only approved
{ "$match": { "replies.versions.status": "APPROVED" } },
// Sort on all "keys" and then the "version" date
{ "$sort": {
"_id": 1,
"replies._id": 1,
"replies.versions.created": -1
}},
// Group replies to get the latest version of each
{ "$group": {
"_id": {
"_id": "$_id",
"body": "$body",
"title": "$title",
"replyId": "$replies._id",
"replyUser": "$replies.user",
"replyCreated": "$replies.created"
},
"version": { "$first": "$replies.version" }
}},
// Push replies back into an array in the main document
{ "$group": {
"_id": "$_id._id",
"body": { "$first": "$_id.body" },
"title": { "$first": "$_id.title" },
"replies": {
"$push": {
"_id": "$_id.replyId",
"user": "$_id.replyUser" },
"created": "$_id.replyCreated", // <-- Value from Reply
"body": "$version.body", // <-- Value from specific Version
"title": "$version.title"
}
}
}}
All depending of course on which fields you want, being either from ther Reply or from the Version.
Whichever fields, since you "un-wound" two arrays, you $group back "twice".
Once to get the $first items after sorting per Reply
Once more to re-construct the "replies" array using $push
That's all there is too it.
If you were still looking at ways to "sort" the array "in-place" without using $unwind, well MongoDB just does not do that yet.
Bit of advice on your design
As a note, I see where you are going with this and this is the wrong model for the type of usage that you want.
It makes little sense to store "revision history" within the embdedded structure. You are rarely going to use it in general update and query operations, and as this demonstrates, most of the time you just want the "latest".
So just do that instead, and store a "flag" indicating "revisions" if really necessary. That data can then be stored external to the main structure, and you won't have to jump through these hoops just to get the "latest accepted version" on every request.
Related
I have a collection of documents, which I need to first narrow down by set criteria, then sort alphabetically by string value inside those documents — let's say that's a "search result". I then need to find document that matches a given _id and then pick a document next to it (before or after) from the above "search result".
Background:
I use mongoose to query my database via Node.js.
I have a set of "special sections" in my blog that are comprised of all the articles that must have three particular conditions associated within the keys in the document. I can get the list of articles belonging to said section like so:
const specialSectionListQuery = Article.find({
tag: { $ne: "excluded" },
[`collections.cameras`]: { $exists: true },
status: "published",
})
To finish creating the "special section," I must sort the documents alphabetically via their title attribute:
.sort({ [`collections.cameras.as.title`]: "asc" })
Now I want to add a link to "next article within the same special section" at the bottom of such articles. I know _id and any other value needed from the current article. The above query gives me an ordered list of documents within the section so I can easily find it within that list specialSectionListQuery.findOne({ _id: "xxx" }).exec().
However, I need to find the next article within the above list. How do I do that?
My attempts thus far:
I tried to create article list via aggregation, which led me nowhere (I simply made my app do exactly the same thing — make a list for a "special sectin"):
Article.aggregate([
{
$match: {
tag: { $ne: "excluded" },
[`collections.cameras`]: { $exists: true },
status: "published",
},
},
{
$sort: {
[`collections.cameras.as.title`]: 1,
},
}
]).exec()
But I can't for the life of me figure out how to iterate to the next document in the list properly.
I have thought of saving the list in Node.js memory (variable) and then finding what I need via JavaScript but that can't be scalable.
I have considered creating a new collection and saving the above list there but that would require me to either 1) do it every time a document is altered/added/deleted via Node.js — which is a lot of code and it may break if I interact with database another way 2) rebuild the colleciton every time I run the query, but that feels like it'll lack in performance.
Please help and thank you!
P.S.:
Example collection which should cover most of the cases I'm looking to solve for:
[
{
_id: 1,
name: "Canon",
collections: { cameras: { as: { title: "Half-Frame" } } },
tag: "included",
status: "published"
},
{
_id: 2,
name: "Pentax",
collections: { cameras: { as: { title: "Full-Frame" } } },
tag: "included",
status: "published"
},
{
_id: 3,
name: "Kodak",
collections: { film: { as: { title: "35mm Film" } } },
tag: "included",
status: "published"
},
{
_id: 4,
name: "Ricoh",
collections: { cameras: { as: { title: "Full-Frame" } } },
tag: "included",
status: "published"
},
{
_id: 5,
name: "Minolta",
collections: { cameras: { as: { title: "Half-Frame Review" } } },
tag: "excluded",
status: "published"
},
{
_id: 4,
name: "FED",
collections: { cameras: { as: { title: "Full-Frame" } } },
tag: "included",
status: "draft"
}
]
One thing you can try is to extend your $sort by adding _id so that it always returns documents in deterministic order:
{
$sort: {
"collections.cameras.as.title": 1,
_id: 1
}
},
{
$limit: 1
}
Once your first query returns the document with _id: 2 and collections.cameras.as.title: Full-Frame, you can use below query to get subsequent document:
{
$match: {
$and: [
{
tag: { $ne: "excluded" },
"collections.cameras": { $exists: true },
status: "published",
},
{
$or: [
{
$and: [
{ "collections.cameras.as.title": { $eq: "Full-Frame" } },
{ "_id": { $gt: 2 } }
]
},
{ "collections.cameras.as.title": { $gt: "Full-Frame" } }
]
}
]
}
},
{
$sort: {
"collections.cameras.as.title": 1,
_id: 1
}
},
{
$limit: 1
}
In this case due to deterministic $sort you can exclude previously found document by adding additional filtering criteria and the order should be preserved.
Mongo Playground
I have an array of objects call "extra" with different properties: some objects have "plus" and some haven't.
I want to create inside this "extra" array, 2 different arrays one called "cheap" with all the object that don't have the "plus" property and one called "exp" with only the objects with the "plus" property.
I think I can use the $reduce method in mongodb aggregate with $concatArrays and check with $cond if the property plus exists or not.
Something like that:
Data example:
{
extra: [
{
description: "laces",
type: "exterior",
plus: '200'
},
{
description: "sole",
type: "interior"
},
{
description: "logo",
type: "exterior"
},
{
description: "stud",
type: "exterior",
plus: '450'
}
],
}
{
$project: {
extra: {
$reduce: {
input: ['$extra'],
initialValue: {cheap: [], exp: []},
$cond: {
if: {$eq: ['$$this.plus', null]},
then: {
in: {
cheap: {
$concatArrays: ['$$value.cheap', '$$this'],
},
},
},
else: {
in: {
exp: {
$concatArrays: ['$$value.exp', '$$this'],
},
},
},
},
},
},
},
}
It doesn't work...I tried many ways or writing the $cond part without luck.
I can't figure it out.
Thank you all.
K.
Apart from some minor syntax issues you've had another problem is your understand of the $ne operator.
In this case you expect a missing value to be equal to null, this is not how Mongo works. so for a document:
{ name: "my name" }
The aggregation query:
{ $cond: { $eq: ["$missingField", null] } }
Will not give true as you expect as missing is not equal to null. I took the liberty to fix the syntax issues you've had, this working pipeline is the way to go:
db.collection.aggregate([
{
$project: {
extra: {
$reduce: {
input: "$extra",
initialValue: {
cheap: [],
exp: []
},
in: {
cheap: {
"$concatArrays": [
"$$value.cheap",
{
$cond: [
"$$this.plus",
[],
[
"$$this"
],
]
}
]
},
exp: {
"$concatArrays": [
"$$value.exp",
{
$cond: [
"$$this.plus",
[
"$$this"
],
[]
]
}
]
}
}
},
},
},
}
])
Mongo Playground
One thing to note is that $cond evaluates the plus field, meaning if the field does exist with a null value or a 0 value then it will consider this document matched for the cheap array. This is something to consider and change in case these are possible.
I'm struggling trying to find all the docs with a specific property in the child. For example, I want to find all the users with their child active.
These are my models
const userSchema = new mongoose.Schema({
child: {
type: mongoose.Schema.Types.ObjectId,
ref: 'child'
}
});
const childSchema = new mongoose.Schema({
active: {
type: Boolean,
default: true
}
});
I tried with populate and match ( .populate({path:'child', match: {active: true}})) but I'm getting all the users with the property child as null if not active. I need only the users with an active child. All my researches head to use the dot syntax, but for any reason I get an empty array. See below:
let usersWithActiveChild = await User.find({'child.active': true}));
console.log(usersWithActiveChild) // --> displays '[]'
Thanks for your help!
This can be accomplished easily by using aggregation framework.
First we join two collections with $lookup.
Lookup result is array, but our relation with User and Child is one to one, so we get the first item by using $arrayElemAt: ["$child", 0].
And lastly, we apply our filter "child.active": true, using $match.
Playground
let usersWithActiveChild = await User.aggregate([
{
$lookup: {
from: "childs", //must be PHYSICAL collection name
localField: "child",
foreignField: "_id",
as: "child",
},
},
{
$addFields: {
child: {
$arrayElemAt: ["$child", 0],
},
},
},
{
$match: {
"child.active": true,
},
},
]);
Sample docs:
db={
"users": [
{
"_id": ObjectId("5a834e000102030405000000"),
"child": ObjectId("5a934e000102030405000000")
},
{
"_id": ObjectId("5a834e000102030405000001"),
"child": ObjectId("5a934e000102030405000001")
},
{
"_id": ObjectId("5a834e000102030405000002"),
"child": ObjectId("5a934e000102030405000002")
},
],
"childs": [
{
"_id": ObjectId("5a934e000102030405000000"),
"active": true
},
{
"_id": ObjectId("5a934e000102030405000001"),
"active": false
},
{
"_id": ObjectId("5a934e000102030405000002"),
"active": true
}
]
}
Output:
[
{
"_id": ObjectId("5a834e000102030405000000"),
"child": {
"_id": ObjectId("5a934e000102030405000000"),
"active": true
}
},
{
"_id": ObjectId("5a834e000102030405000002"),
"child": {
"_id": ObjectId("5a934e000102030405000002"),
"active": true
}
}
]
Or as a better approach would be first getting activ childs, and then lookup with users like this:
db.childs.aggregate([
{
$match: {
"active": true
}
},
{
$lookup: {
from: "users",
localField: "_id",
foreignField: "child",
as: "user"
}
}
])
Playground
When you use a ref to refer to another schema, Mongoose stores the documents in separate collections in MongoDB.
The actual value stored in the child field of the user document is a DBRef.
If you were to look at the data directly in MongoDB you would find something similar to this:
User collection
{
_id: ObjectId("5a934e000102030405000000")
child: DBRef("child",ObjectId("5a934e000102030405000001"),"DatabaseName")
}
Child collection:
{
_id: ObjectId("5a934e000102030405000001"),
active: true
}
When you populate the user object, Mongoose fetches the user document, and then fetches the child. Since the user documents have been retrieved already, the match in the populate call filters the children, as you noted.
The dotted notation 'child.active' can only be used if the child is stored in MongoDB as a subdocument, like
{
_id: ObjectId("5a934e000102030405000000")
child:{
_id: ObjectId("5a934e000102030405000001"),
active: true
}
}
But your child is defined as a ref, so this will not be the case.
In order to filter the list of user documents based on the content of the referenced child, you will need to either
- populate with match as you have done and then filter the result set, or
- aggregate the user collection, lookup the child documents, and then match the child field.
This is a sample JSON object, among 1000 like them, stored in my MongoDB collection.
{
"_id": ObjectId("5b1bb74ffc7ee601c6915939"),
"groupId": "-abcde",
"applicationId": "avcvcvc",
"integration": "web",
"Category": "message",
"Action": "message",
"Type": "newMessage",
"Id": "activity",
"data": {
"test": "good morning"
},
"timestamp": 1528543055858.0,
"createdAt": ISODate("2018-06-09T11:17:35.868+0000"),
"updatedAt": ISODate("2018-06-09T11:17:35.868+0000"),
"__v": NumberInt(0)
}
This object is an example of data with date as 2018-06-09. There are objects with successive dates in db. I have to fetch data with a specific date and perform this operation on all objects with that date
db.collection.aggregate([{
$match: {
$or: [{
Type: "on mouse hover click"
}, {
Type: "on mouse out"
},
{
Type: "on chat start"
}, {
Type: "Load Event"
}
]
}
},
{
$group: {
_id: null,
count: {
$sum: 1
}
}
}
]);
The api, when called, should show a specific date and the calculation performed in the upper query alongwith it. this should happen on all objects with diff. dates. For this, i have made a sample schema
let { Schema } = require("mongoose");
let mongoose = require("mongoose");
let dataSchema = Schema({
date: { type: Date },
calculation: { type: Number }
});
let NewData = mongoose.model('dataSchema', dataSchema);
module.exports = { NewData };
How can i achieve this? I am new to MongoDB so i cant quite figure out how to do this.
You cannot group by a date cause are differents minutes seconds millis etc etc
you need make a project for date instead datetime, those can have same date but different time
Then you can perform grouping and what you want to do
db.getCollection("collection")
.aggregate([
{
$project : {
"groupId":1,
"applicationId":1,
"integration":1,
"Category":1,
"Action":1,
"Type":1,
"Id":1,
"data":1,
"timestamp":1,
"createdAt":1,
"updatedAt":1,
"__v":1
createdDate: { $dateToString: {format: "%G-%m-%d", date: "$createdAt"}},
}
},
{
$group: { _id: "$createdDate", count: { $sum: 1 } },
}
]);
I have document called question
var QuestionSchema = new Schema({
title: {
type: String,
default: '',
trim: true
},
body: {
type: String,
default: '',
trim: true
},
user: {
type: Schema.ObjectId,
ref: 'User'
},
category: [],
comments: [{
body: {
type: String,
default: ''
},
root: {
type: String,
default: ''
},
user: {
type: Schema.Types.ObjectId,
ref: 'User'
},
createdAt: {
type: Date,
default: Date.now
}
}],
tags: {
type: [],
get: getTags,
set: setTags
},
image: {
cdnUri: String,
files: []
},
createdAt: {
type: Date,
default: Date.now
}
});
As a result, I need to sort comments by root field, like this
I tried to sort the array of comments manually at backend and tried to use aggregation, but I was not able to sort this. Help please.
Presuming that Question is a model object in your code and that of course you want to sort your "comments by "date" from createdAt then using .aggregate() you would use this:
Question.aggregate([
// Ideally match the document you want
{ "$match": { "_id": docId } },
// Unwind the array contents
{ "$unwind": "comments" },
// Then sort on the array contents per document
{ "$sort": { "_id": 1, "comments.createdAt": 1 } },
// Then group back the structure
{ "$group": {
"_id": "$_id",
"title": { "$first": "$title" },
"body": { "$first": "$body" },
"user": { "$first": "$user" },
"comments": { "$push": "$comments" },
"tags": { "$first": "$tags" },
"image": { "$first": "$image" },
"createdAt": { "$first": "$createdAt" }
}}
],
function(err,results) {
// do something with sorted results
});
But that is really overkill since you are not "aggregating" between documents. Just use the JavaScript methods instead. Such as .sort():
Quesion.findOneById(docId,function(err,doc) {
if (err) throw (err);
var mydoc = doc.toObject();
mydoc.questions = mydoc.questions.sort(function(a,b) {
return a.createdAt > b.createdAt;
});
console.log( JSON.stringify( doc, undefined, 2 ) ); // Intented nicely
});
So whilst MongoDB does have the "tools" to do this on the server, it makes the most sense to do this in client code when you retrieve the data unless you actually need to "aggregate" accross documents.
But both example usages have been given now.