I'm struggling trying to find all the docs with a specific property in the child. For example, I want to find all the users with their child active.
These are my models
const userSchema = new mongoose.Schema({
child: {
type: mongoose.Schema.Types.ObjectId,
ref: 'child'
}
});
const childSchema = new mongoose.Schema({
active: {
type: Boolean,
default: true
}
});
I tried with populate and match ( .populate({path:'child', match: {active: true}})) but I'm getting all the users with the property child as null if not active. I need only the users with an active child. All my researches head to use the dot syntax, but for any reason I get an empty array. See below:
let usersWithActiveChild = await User.find({'child.active': true}));
console.log(usersWithActiveChild) // --> displays '[]'
Thanks for your help!
This can be accomplished easily by using aggregation framework.
First we join two collections with $lookup.
Lookup result is array, but our relation with User and Child is one to one, so we get the first item by using $arrayElemAt: ["$child", 0].
And lastly, we apply our filter "child.active": true, using $match.
Playground
let usersWithActiveChild = await User.aggregate([
{
$lookup: {
from: "childs", //must be PHYSICAL collection name
localField: "child",
foreignField: "_id",
as: "child",
},
},
{
$addFields: {
child: {
$arrayElemAt: ["$child", 0],
},
},
},
{
$match: {
"child.active": true,
},
},
]);
Sample docs:
db={
"users": [
{
"_id": ObjectId("5a834e000102030405000000"),
"child": ObjectId("5a934e000102030405000000")
},
{
"_id": ObjectId("5a834e000102030405000001"),
"child": ObjectId("5a934e000102030405000001")
},
{
"_id": ObjectId("5a834e000102030405000002"),
"child": ObjectId("5a934e000102030405000002")
},
],
"childs": [
{
"_id": ObjectId("5a934e000102030405000000"),
"active": true
},
{
"_id": ObjectId("5a934e000102030405000001"),
"active": false
},
{
"_id": ObjectId("5a934e000102030405000002"),
"active": true
}
]
}
Output:
[
{
"_id": ObjectId("5a834e000102030405000000"),
"child": {
"_id": ObjectId("5a934e000102030405000000"),
"active": true
}
},
{
"_id": ObjectId("5a834e000102030405000002"),
"child": {
"_id": ObjectId("5a934e000102030405000002"),
"active": true
}
}
]
Or as a better approach would be first getting activ childs, and then lookup with users like this:
db.childs.aggregate([
{
$match: {
"active": true
}
},
{
$lookup: {
from: "users",
localField: "_id",
foreignField: "child",
as: "user"
}
}
])
Playground
When you use a ref to refer to another schema, Mongoose stores the documents in separate collections in MongoDB.
The actual value stored in the child field of the user document is a DBRef.
If you were to look at the data directly in MongoDB you would find something similar to this:
User collection
{
_id: ObjectId("5a934e000102030405000000")
child: DBRef("child",ObjectId("5a934e000102030405000001"),"DatabaseName")
}
Child collection:
{
_id: ObjectId("5a934e000102030405000001"),
active: true
}
When you populate the user object, Mongoose fetches the user document, and then fetches the child. Since the user documents have been retrieved already, the match in the populate call filters the children, as you noted.
The dotted notation 'child.active' can only be used if the child is stored in MongoDB as a subdocument, like
{
_id: ObjectId("5a934e000102030405000000")
child:{
_id: ObjectId("5a934e000102030405000001"),
active: true
}
}
But your child is defined as a ref, so this will not be the case.
In order to filter the list of user documents based on the content of the referenced child, you will need to either
- populate with match as you have done and then filter the result set, or
- aggregate the user collection, lookup the child documents, and then match the child field.
Related
I have a collection of documents, which I need to first narrow down by set criteria, then sort alphabetically by string value inside those documents — let's say that's a "search result". I then need to find document that matches a given _id and then pick a document next to it (before or after) from the above "search result".
Background:
I use mongoose to query my database via Node.js.
I have a set of "special sections" in my blog that are comprised of all the articles that must have three particular conditions associated within the keys in the document. I can get the list of articles belonging to said section like so:
const specialSectionListQuery = Article.find({
tag: { $ne: "excluded" },
[`collections.cameras`]: { $exists: true },
status: "published",
})
To finish creating the "special section," I must sort the documents alphabetically via their title attribute:
.sort({ [`collections.cameras.as.title`]: "asc" })
Now I want to add a link to "next article within the same special section" at the bottom of such articles. I know _id and any other value needed from the current article. The above query gives me an ordered list of documents within the section so I can easily find it within that list specialSectionListQuery.findOne({ _id: "xxx" }).exec().
However, I need to find the next article within the above list. How do I do that?
My attempts thus far:
I tried to create article list via aggregation, which led me nowhere (I simply made my app do exactly the same thing — make a list for a "special sectin"):
Article.aggregate([
{
$match: {
tag: { $ne: "excluded" },
[`collections.cameras`]: { $exists: true },
status: "published",
},
},
{
$sort: {
[`collections.cameras.as.title`]: 1,
},
}
]).exec()
But I can't for the life of me figure out how to iterate to the next document in the list properly.
I have thought of saving the list in Node.js memory (variable) and then finding what I need via JavaScript but that can't be scalable.
I have considered creating a new collection and saving the above list there but that would require me to either 1) do it every time a document is altered/added/deleted via Node.js — which is a lot of code and it may break if I interact with database another way 2) rebuild the colleciton every time I run the query, but that feels like it'll lack in performance.
Please help and thank you!
P.S.:
Example collection which should cover most of the cases I'm looking to solve for:
[
{
_id: 1,
name: "Canon",
collections: { cameras: { as: { title: "Half-Frame" } } },
tag: "included",
status: "published"
},
{
_id: 2,
name: "Pentax",
collections: { cameras: { as: { title: "Full-Frame" } } },
tag: "included",
status: "published"
},
{
_id: 3,
name: "Kodak",
collections: { film: { as: { title: "35mm Film" } } },
tag: "included",
status: "published"
},
{
_id: 4,
name: "Ricoh",
collections: { cameras: { as: { title: "Full-Frame" } } },
tag: "included",
status: "published"
},
{
_id: 5,
name: "Minolta",
collections: { cameras: { as: { title: "Half-Frame Review" } } },
tag: "excluded",
status: "published"
},
{
_id: 4,
name: "FED",
collections: { cameras: { as: { title: "Full-Frame" } } },
tag: "included",
status: "draft"
}
]
One thing you can try is to extend your $sort by adding _id so that it always returns documents in deterministic order:
{
$sort: {
"collections.cameras.as.title": 1,
_id: 1
}
},
{
$limit: 1
}
Once your first query returns the document with _id: 2 and collections.cameras.as.title: Full-Frame, you can use below query to get subsequent document:
{
$match: {
$and: [
{
tag: { $ne: "excluded" },
"collections.cameras": { $exists: true },
status: "published",
},
{
$or: [
{
$and: [
{ "collections.cameras.as.title": { $eq: "Full-Frame" } },
{ "_id": { $gt: 2 } }
]
},
{ "collections.cameras.as.title": { $gt: "Full-Frame" } }
]
}
]
}
},
{
$sort: {
"collections.cameras.as.title": 1,
_id: 1
}
},
{
$limit: 1
}
In this case due to deterministic $sort you can exclude previously found document by adding additional filtering criteria and the order should be preserved.
Mongo Playground
I have an User like so the id is NOT the _id)
{
id: string;
}
Which can create files like so
{
name: string;
author: User;
}
I would like to get all Files where the author is a given User, but I do not know how to use the "filter" function to do that.
So currently I do
const author = await this.userModel.find({ id });
return this.filesModel.find({ author });
Is there a more efficient way to do it ?
(I use NestJS with the Mongoose integration, the syntax used is the same as the Mongoose library)
EDIT
Given the User document
{
_id: 'OVZVIbovibiugb44'
id: 10
}
And the Files documents
[
{ name: "1.docx", author: ObjectId('OVZVIbovibiugb44') },
{ name: "2.docx", author: ObjectId('voisbvOVISBEIVBv') },
]
I would like to use the function
findOwned(authorId = 10) {
const author = await this.userModel.find({ id });
return this.filesModel.find({ author });
// But do it only with "filesModel"
}
And get, as a result,
[
{ name: '1.docx', author: 'ObjectId('OVZVIbovibiugb44') },
]
You can use $lookup into an aggregation query to merge collections.
Also, as your id is an String and your author is an ObjectId you will need one previous stage using $toObjectId
So the query is similar to this:
$match stage (optional) to query only with documents you want. Like a filter
$project to convert id String field to ObjectId. You can user $set also.
$lookup to merge collection and the ouput is in a field called files.
$project to output only files array from the merge.
db.User.aggregate([
{ "$match": { "id": "5a934e000102030405000001" } },
{ "$project": { "id": { "$toObjectId": "$id" } } },
{ "$lookup": {
"from": "Files",
"localField": "id",
"foreignField": "author",
"as": "files" }
},
{ "$project": { "files": 1 } }
])
Example here
I want generate an ObjectID for each Object present inside my array. The thing is I'm getting the products with a .forEach statement from another server and push them inside my array without a Schema that generates an ObjectID....
Product Schema:
const productsSchema = new mongoose.Schema({
apiKey: String,
domain: String,
totalcount: Number,
totaldone: Number,
allSKUS: Array,
allProducts: Array,
created_at: { type: Date },
updated_at: { type: Date },
}, { collection: 'products', timestamps: true });
productsSchema.plugin(uniqueValidator);
const Products = mongoose.model('Products', productsSchema);
module.exports = Products;
My Code:
const newProduct = {
apiKey: userApiProducts.apiKey,
domain: userApiProducts.domain,
totalcount: userApiProducts.totalcount,
totaldone: userApiProducts.totaldone,
allSKUS: userApiProducts.allSKUS,
allProducts: userApiProducts.allProducts // generate ObjectID for each object that gets pushed inside the Array
};
Products.findOneAndUpdate( userApiProducts.domain, newProduct, {upsert:true} , (err, existingProducts) => {
if (err) { return next(err); }
});
Output:
// Please Check ADD OBJECT ID HERE comment. This is where i want to generate an unique ObjectID before I push the data. I tried with var id = mongoose.Types.ObjectId(); but i'm afraid it will not be Unique...
{
"_id" : ObjectId("58780a2c8d94cf6a32cd7530"),
"domain" : "http://example.com",
"updatedAt" : ISODate("2017-01-12T23:27:15.465Z"),
"apiKey" : "nf4fh3attn5ygkq1t",
"totalcount" : 11,
"totaldone" : 11,
"allSKUS" : [
"Primul",
"Al doilea",
"Al treilea"
],
"allProducts" : [
{
// ADD OBJECT ID HERE
"id": 1,
"sku": "Primul",
"name": "Primul",
"status": 1,
"total_images": 2,
"media_gallery_entries": [
{
"id": 1,
"media_type": "image",
"label": null,
"position": 1,
"disabled": false,
"types": [
"image",
"small_image",
"thumbnail",
"swatch_image"
],
"file": "/g/r/grafolio_angel_and_devil.png"
},
{
"id": 2,
"media_type": "image",
"label": null,
"position": 2,
"disabled": false,
"types": [],
"file": "/g/r/grafolio_angel_and_devil_thumbnail.jpg"
}
]
},
{
// ADD OBJECT ID HERE
"id": 3,
"sku": "Al doilea",
"name": "Al doilea",
"status": 1,
"total_images": 2,
"media_gallery_entries": [
{
"id": 4,
"media_type": "image",
"label": null,
"position": 2,
"disabled": false,
"types": [],
"file": "/g/r/grafolio_angel_and_devil_thumbnail_1.jpg"
},
{
"id": 5,
"media_type": "image",
"label": null,
"position": 3,
"disabled": false,
"types": [],
"file": "/b/e/before.png"
}
]
}, etc ......
],
"__v" : 0,
"createdAt" : ISODate("2017-01-12T22:58:52.524Z")
}
Is there any way of doing this without having to make a ton of DB Calls? I can't imagine saving like this
array.forEach((x)=> {
Products.save({})
})
Hope someone has already worked on something similar and found the perfect solution for this !
If you want to add ObjectId automatically, you need to define a separate schema for it and set the _id options for the schema as true.
Do the following:
Change your productsSchema as CatalogueSchema (for ease of
understanding).
Define a new ProductSchema for Product (element of allProducts)
In CatalogueSchema define allProducts type as [Product.schema]. This will automatically add _id (ObjectId).
Also, you don't need to add created_at and updated_at as part of schema when you set timestamps option as true.
Catalogue Schema
const Product = require('Product_Schema_Module_Path'); // Edit
const CatalogueSchema = new mongoose.Schema({
apiKey: String,
domain: String,
totalcount: Number,
totaldone: Number,
allSKUS: Array,
allProducts: [Product.schema]
// Note the change here (Array -> [Product.schema]
// Creating a separate schema ensures automatic id (ObjectId)
}, { collection: 'catalogue', timestamps: true });
CatalogueSchema.plugin(uniqueValidator);
const Catalogue = mongoose.model('Catalogue', CatalogueSchema);
module.exports = Catalogue;
Product Schema
(New schema to ensure adding of ObjectId)
const ProductSchema = new mongoose.Schema({
id: Number,
sku: String,
name: String,
status: Number,
total_images: Number,
media_gallery_entries: Array
}, { _id: true, timestamps: true });
// _id option is true by default. You can ommit it.
// If _id is set to false, it will not add ObjectId
ProductSchema.plugin(uniqueValidator);
const Product = mongoose.model('Product', ProductSchema);
module.exports = Product;
EDIT (Save Products in Catalogue)
(Also, note that you have to require the ProductSchema module in your CatalogueSchema module)
// Map userApiProducts.allProducts to array of Product documents
const products = userApiProducts.allProducts.map(product => {
return new Product(product);
})
const newProduct = {
apiKey: userApiProducts.apiKey,
domain: userApiProducts.domain,
totalcount: userApiProducts.totalcount,
totaldone: userApiProducts.totaldone,
allSKUS: userApiProducts.allSKUS,
allProducts: products
};
Catalogue
.findOneAndUpdate({ domain: userApiProducts.domain }, newProduct, { upsert:true } , (err, products) => {
// Handle error
});
To add multiple documents into Mongo you can use db.collection.insert():
https://docs.mongodb.com/manual/reference/method/db.collection.insert/
In Mongoose you can use Model.insertMany():
http://mongoosejs.com/docs/api.html#model_Model.insertMany
But keep in mind that when you have one document inside of other document in Mongoose they are not actually stored like that in Mongo. Mongo only stores the IDs of the child documents and not their contents in the parent document - and not even any info on which collection those IDs belong to.
When you use population then Mongoose actually retrieves the relevant documents from the DB in separate requests to Mongo. So, population is a concept of Mongoose. Mongo just stores the IDs, so you need to create the documents first before you can insert the IDs.
The thing that you are trying to do would be easy without using Mongoose. You can store multiple documents in one request in Mongo using your own IDs if you want and you can store another document with an array of those IDs in another request.
Of course however you do it you will get inconsistent state during the operation because Mongo doesn't support transactions.
I am trying to use $match to find items with a specific _id in a double embedded document.
I have a document called users which contains information such as name, email, and it also contains an embedded document which has the business this user is with.
I also have a document called businesses, which contains an embedded document which has the building that this business is in.
I also have a document called building.
I am trying to have a mongo query which returns all of the users with a business at a certain building ID.
I have an aggregate function which uses $lookup to match the users to the building they are in. and this does work. However now I am trying to use $match to only return the documents with a specific building id.
Here is an example of my user, business and building documents:
_id: 5ca487c0eeedbe8ab59d7a7a
name: "John Smith"
email: "jsmith9#gmail.com"
business: Object
_id: 5ca48481eeedbe8ab59d7a38
name: "Visitors"
_id: 5ca48481eeedbe8ab59d7a38
name: "Visitors"
building: Object
_id: 5ca48481eeedbe8ab59d7a36
name: "Building1"
_id: 5ca48481eeedbe8ab59d7a36
name: "Building1"
When I return the aggregated query it returns documents in the following format:
{
"_id": "5ca487c0eeedbe8ab59d7a7a",
"name": "John Smith",
"email": "jsmith9#gmail.com",
"business": {
"_id": "5ca48481eeedbe8ab59d7a38",
"name": "Visitors"
},
"__v": 0,
"user_building": {
"_id": "5ca48481eeedbe8ab59d7a38",
"name": "Visitors",
"building": {
"_id": "5ca48481eeedbe8ab59d7a36",
"name": "Building1"
},
"__v": 0
}
},
However when I add the match in, it returns []. What am i doing wrong here?
router.get("/:id", async (req, res) => {
const users_buildings = await User.aggregate([
{
$lookup: {
from: "businesses",
localField: "business._id",
foreignField: "_id",
as: "user_building"
}
},
{ $unwind: "$user_building" },
{
$match: {
"user_building.building": { _id: req.params.id }
}
}
]);
You need to match _id inside the building object. Try with this
{
$match: {
"user_building.building._id": req.params.id
}
}
if not working
{
$match: {
"user_building.building._id": ObjectId(req.params.id)
}
}
op edit: I imported ObjectId with:
var ObjectId = require('mongodb').ObjectID;
and used the second solution and it worked correctly.
I have a schema, Comment, like the one below. It's a system of "comments" and "replies", but each comment and reply has multiple versions. When a user wants to view a comment, I want to return just the most recent version with the status of APPROVED.
const Version = new mongoose.Schema({
user: {
type: mongoose.Schema.Types.ObjectId,
ref: 'User'
},
body: String,
created: Date,
title: String,
status: {
type: String,
enum: [ 'APPROVED', 'OPEN', 'CLOSED' ]
}
})
const Reply = new mongoose.Schema({
user: {
type: mongoose.Schema.Types.ObjectId,
ref: 'User'
},
created: Date,
versions: [ Version ]
})
const Comment = new mongoose.Schema({
user: {
type: mongoose.Schema.Types.ObjectId,
ref: 'User'
},
created: Date,
versions: [ Version ],
replies: [ Reply ]
})
I've gotten the parent Comment to display how I want with the code below. However, I've had trouble applying that to the sub-document, Reply.
const requestedComment = yield Comment.aggregate([
{ $match: {
query
} },
{ $project: {
user: 1,
replies: 1,
versions: {
$filter: {
input: '$versions',
as: 'version',
cond: { $eq: [ '$$version.status', 'APPROVED' ] }
}
},
}},
{ "$unwind": "$versions" },
{ $sort: { 'versions.created': -1 } },
{ $group: {
_id: '$_id',
body: { $first: '$versions.body' },
title: { $first: '$versions.title' },
replies: { $first: '$replies' }
}}
])
.exec()
Any help achieving the same result with the replies subdocuments would be appreciated. I would like to return the most recent APPROVED version of each reply in a form like this:
comment: {
body: "The comment's body.",
user: ObjectId(...),
replies: [
{
body: "The reply's body."
user: ObjectId(...)
}
]
}
Basically you just need to continue the same process on from the existing pipeline. But this time to $unwind out the "versions" per each "replies" entry and $sort them there.
So these are "additional" stages to your pipeline.
// Unwind replies
{ "$unwind": "$replies" },
// Unwind inner versions
{ "$unwind": "$replies.versions" },
// Filter for only approved
{ "$match": { "replies.versions.status": "APPROVED" } },
// Sort on all "keys" and then the "version" date
{ "$sort": {
"_id": 1,
"replies._id": 1,
"replies.versions.created": -1
}},
// Group replies to get the latest version of each
{ "$group": {
"_id": {
"_id": "$_id",
"body": "$body",
"title": "$title",
"replyId": "$replies._id",
"replyUser": "$replies.user",
"replyCreated": "$replies.created"
},
"version": { "$first": "$replies.version" }
}},
// Push replies back into an array in the main document
{ "$group": {
"_id": "$_id._id",
"body": { "$first": "$_id.body" },
"title": { "$first": "$_id.title" },
"replies": {
"$push": {
"_id": "$_id.replyId",
"user": "$_id.replyUser" },
"created": "$_id.replyCreated", // <-- Value from Reply
"body": "$version.body", // <-- Value from specific Version
"title": "$version.title"
}
}
}}
All depending of course on which fields you want, being either from ther Reply or from the Version.
Whichever fields, since you "un-wound" two arrays, you $group back "twice".
Once to get the $first items after sorting per Reply
Once more to re-construct the "replies" array using $push
That's all there is too it.
If you were still looking at ways to "sort" the array "in-place" without using $unwind, well MongoDB just does not do that yet.
Bit of advice on your design
As a note, I see where you are going with this and this is the wrong model for the type of usage that you want.
It makes little sense to store "revision history" within the embdedded structure. You are rarely going to use it in general update and query operations, and as this demonstrates, most of the time you just want the "latest".
So just do that instead, and store a "flag" indicating "revisions" if really necessary. That data can then be stored external to the main structure, and you won't have to jump through these hoops just to get the "latest accepted version" on every request.