Related
I want to search for my users in DB using their first name and last name and then returning the full object of it but what I've found in another post is returning the _id and name which is concatenated firstName and lastName
This is the code I am using.
results = await StageOne.aggregate([
{ $project: { "name": { $concat: ["$firstName", " ", "$lastName"] } } },
{ $match: { "name": { $regex: searchInput, $options: 'i' } } }
]).collation(
{ locale: 'en', strength: 2 }
).limit(limit).skip(offset);
And the response looks something like this
{ _id: 5f064921a8900b73174f76a1, name: 'John Doe' }
What I want to be returned is something like this
{ _id: 5f08fc3b8f2719096146f767, firstName: 'John', lastName: 'Doe', email: 'johndoe#email.com' ... createdAt: 2020-07-10T23:39:39.310Z, updatedAt: 2020-07-10T23:39:39.310Z, __v: 0 }
Which I can do it by running it like this separately for firstName or lastName
results = await StageOne.find({ firstName: { $regex: searchInput, $options: 'i' } }).collation(
{ locale: 'en', strength: 2 }
).limit(limit).skip(offset);
I would suggest you to add $project at the end of your pipeline.
{$project: {$firstName:1, $lastName:1, $email:1, $name:1}}
As In
db.collection.aggregate([
{
$project: {
"name": {
$concat: [
"$firstName",
" ",
"$lastName"
]
},
firstName: 1,
lastName: 1,
data: 1
}
},
{
$match: {
"name": {
"$regex": "ohn",
"$options": "i"
}
}
},
{
$project: {
firstName: 1,
lastName: 1,
data: 1
}
}
])
play
You need to add part of your first project pipeline what are all the fields needs to be projected.
Another way to achieve this.
I'm trying to do an inner join that matches, but for some reason I get a left join.
I have 2 relations tables, and I want to get the movie with the genre names.
Consider the following models:
// Movie
const MovieSchema = new mongoose.Schema({
id: {
type: Number,
default: null,
required: true,
unique: true
},
title: {
type: String,
default: null,
required: true,
unique: true,
trim: true
},
});
const Movie = mongoose.model('Movie', MovieSchema);
module.exports = Movie;
// Genre
const GenreSchema = new mongoose.Schema({
id: {
type: Number,
default: null,
required: true,
unique: true
},
name: {
type: String,
default: null,
required: false,
trim: true,
unique: true
}
});
const Genre = mongoose.model('Genre', GenreSchema);
module.exports = Genre;
// MovieGenre
const MovieGenreSchema = new mongoose.Schema({
genreId: {
type: Number,
default: null,
required: true
},
movieId: {
type: Number,
default: null,
required: true
}
});
const MovieGenre = mongoose.model('MovieGenre', MovieGenreSchema);
module.exports = MovieGenre;
I try to do the following query:
{
$lookup:
{
from: MovieGenre.collection.name,
localField: 'id',
foreignField: 'movieId',
as: 'movieGenres'
}
},
{
$lookup: {
from: Genre.collection.name,
localField: 'g.id',
foreignField: 'genreId',
as: 'genreNames'
}
},
{
$match: {
'genreNames.name': 'Action'
}
}
and I get the results:
{
_id: 5ee9b51609f44c0f38262c94,
id: 26583,
title: 'The Keeper',
__v: 0,
movieGenres: [
{
_id: 5ee8b8cf0d186c20b4bf3ccd,
genreId: 28,
movieId: 26583,
__v: 0
},
{
_id: 5ee8b8cf0d186c20b4bf3cce,
genreId: 53,
movieId: 26583,
__v: 0
}
],
genreNames: [
{ _id: 5ee8b68f0d186c20b4b03a3d, id: 35, name: 'Comedy', __v: 0 },
{ _id: 5ee8b68f0d186c20b4b03a3e, id: 80, name: 'Crime', __v: 0 },
{ _id: 5ee8b68f0d186c20b4b03a40, id: 18, name: 'Drama', __v: 0 },
{ _id: 5ee8b68f0d186c20b4b03a42, id: 53, name: 'Thriller', __v: 0 },
{ _id: 5ee8b68f0d186c20b4b03a43, id: 28, name: 'Action', __v: 0 },
{ _id: 5ee8b68f0d186c20b4b03a45, id: 14, name: 'Fantasy', __v: 0 },
{ _id: 5ee8b68f0d186c20b4b03a46, id: 27, name: 'Horror', __v: 0 },
{ _id: 5ee8b68f0d186c20b4b03a4a, id: 10752, name: 'War', __v: 0 },
{ _id: 5ee8b68f0d186c20b4b03a4b, id: 10402, name: 'Music', __v: 0 },
{ _id: 5ee8b68f0d186c20b4b03a4c, id: 37, name: 'Western', __v: 0 },
{ _id: 5ee8b68f0d186c20b4b03a4d, id: 36, name: 'History', __v: 0 }
]
}
but what i expected to get is:
{
_id: 5ee9b51609f44c0f38262c94,
id: 26583,
title: 'The Keeper',
__v: 0,
movieGenres: [
{
_id: 5ee8b8cf0d186c20b4bf3ccd,
genreId: 28,
movieId: 26583,
__v: 0
},
{
_id: 5ee8b8cf0d186c20b4bf3cce,
genreId: 53,
movieId: 26583,
__v: 0
}
],
genreNames: [
{ _id: 5ee8b68f0d186c20b4b03a43, id: 28, name: 'Action', __v: 0 },
{ _id: 5ee8b68f0d186c20b4b03a42, id: 53, name: 'Thriller', __v: 0 },
]
}
Can you please tell me,
What am I doing wrong?
Thanks.
You just need to correct your second lookup with genre collection, i have added 2 approaches you can use anyone,
1) Using your approach:
localField pass previous lookup result's movieGenres.genreId
foreignField pass id of genre collection
{
$lookup: {
from: Genre.collection.name,
localField: "movieGenres.genreId",
foreignField: "id",
as: "genreNames"
}
}
if you want to filter the genreNames names from above lookup by name,
$filter to iterate loop of genreNames array and filter by name: Action
{
$addFields: {
genreNames: {
$filter: {
input: "$genreNames",
cond: { $eq: ["$$this.name", "Action"] }
}
}
}
}
Your final query would be,
{
$lookup: {
from: MovieGenre.collection.name,
localField: "id",
foreignField: "movieId",
as: "movieGenres"
}
},
{
$lookup: {
from: Genre.collection.name,
localField: "movieGenres.genreId",
foreignField: "id",
as: "genreNames"
}
},
{
$match: {
"genreNames.name": "Action"
}
},
{
$addFields: {
genreNames: {
$filter: {
input: "$genreNames",
cond: { $eq: ["$$this.name", "Action"] }
}
}
}
}
Playground
2) Using lookup with pipeline approach:
The alternate way to do this using lookup with pipeline,
let to pass movieGenres.genreId from above lookup
$match to match genreId using $expr expression match and name field and combine conditions using $and operations
{
$lookup: {
from: MovieGenre.collection.name,
localField: "id",
foreignField: "movieId",
as: "movieGenres"
}
},
{
$lookup: {
from: Genre.collection.name,
let: { genreIds: "$movieGenres.genreId" },
pipeline: [
{
$match: {
$and: [
{ $expr: { $in: ["$id", "$$genreIds"] } },
{ name: "Action" }
]
}
}
],
as: "genreNames"
}
}
Playground
Pick specific field from $arrayElemAt inside $map.
I want to pick only the name field returned from the object at $arrayElemAt
const data = await this.aggregate([
{
$match: { provider_id: providerId },
},
{
$lookup: {
from: 'users',
localField: 'staff.user_id',
foreignField: '_id',
as: 'staffUsers',
},
},
{
$project: {
staff: {
$map: {
input: '$staff',
in: {
_id: '$$this._id',
email_login: '$$this.email_login',
full_calendar_view: '$$this.full_calendar_view',
verified: '$$this.verified',
user_id: '$$this.user_id',
description: '$$this.description',
name: {
$arrayElemAt: [
'$staffUsers',
{
$indexOfArray: ['$staffUsers._id', '$$this.user_id'],
},
],
},
},
},
},
},
},
]);
Simply use .dot with the $staffUsers
{ "name": {
"$arrayElemAt": [
"$staffUsers.name",
{ "$indexOfArray": ["$staffUsers._id", "$$this.user_id"] }
]
}}
my collection:
groups : [{_id: 001, name: ABC, type: a}, {_id: 002, name: DEF, type: b}]
I'm doing the below coding to get result in mongodb:
.aggregate([
{
$lookup: {
from: 'groups',
localField: 'groupId',
foreignField: '_id',
as: 'groupName'
}
},
{
$unwind: {
path: '$groupName',
preserveNullAndEmptyArrays: true
}
},
{
$project:
groupName: {
name: 1
}
}
The result I get from the above coding is :
groupName: Object { name: "ABC" }
But I don't want the ABC to be as an object.
I want my result to be single element:
groupName: "ABC"
Any idea how to do it?
Using $project you can achieve your expected result.
.aggregate([
{
$lookup: {
from: 'groups',
localField: 'groupId',
foreignField: '_id',
as: 'groupName'
}
},
{
$unwind: {
path: '$groupName',
preserveNullAndEmptyArrays: true
}
},
{
$project:
{groupName: "$groupName.name"}
}
Here is a code I have:
const _ = require('lodash')
const Box = require('./models/Box')
const boxesToBePicked = await Box.find({ status: 'ready', client: 27 })
const boxesOriginalIds = _(boxesToBePicked).map('original').compact().uniq().value()
const boxesOriginal = boxesOriginalIds.length ? await Box.find({ _id: { $in: boxesOriginalIds } }) : []
const attributes = ['name']
const boxes = [
...boxesOriginal,
...boxesToBePicked.filter(box => !box.original)
].map(box => _.pick(box, attributes))
Let's say, we have following data in "boxes" collection:
[
{ _id: 1, name: 'Original Box #1', status: 'pending' },
{ _id: 2, name: 'Nested box', status: 'ready', original: 1 },
{ _id: 3, name: 'Nested box', status: 'ready', original: 1 },
{ _id: 4, name: 'Nested box', status: 'pending', original: 1 },
{ _id: 5, name: 'Original Box #2', status: 'ready' },
{ _id: 6, name: 'Original Box #3', status: 'pending' },
{ _id: 7, name: 'Nested box', status: 'ready', original: 6 },
{ _id: 8, name: 'Original Box #4', status: 'pending' }
]
Workflow
Find all boxes, which are ready to be picked:
const boxesToBePicked = await Box.find({ status: 'ready' })
// Returns:
[
{ _id: 2, name: 'Nested box', status: 'ready', original: 1 },
{ _id: 3, name: 'Nested box', status: 'ready', original: 1 },
{ _id: 5, name: 'Original Box #2', status: 'ready' },
{ _id: 7, name: 'Nested box', status: 'ready', original: 6 }
]
Get all the IDs of original (parent) boxes of those:
const boxesOriginalIds = _(boxesToBePicked).map('original').compact().uniq().value()
// Returns:
[1, 6]
Get those boxes by their IDs:
const boxesOriginal = boxesOriginalIds.length ? await Box.find({ _id: { $in: boxesOriginalIds } }) : []
// Returns
[
{ _id: 1, name: 'Original Box #1', status: 'pending' },
{ _id: 6, name: 'Original Box #3', status: 'pending' }
]
Join those boxes with not nested boxes to be picked:
const boxes = [
...boxesOriginal,
...boxesToBePicked.filter(box => !box.original)
].map(box => _.pick(box, attributes))
// Returns
[
{ name: 'Original Box #1' },
{ name: 'Original Box #3' },
{ name: 'Original Box #2' }
]
So basically what we are doing here is getting all the original boxes if they have at least one nested box with status "ready", and all not nested boxes with status "ready".
I think it can be simplified by using aggregation pipeline and projection. But how?
You can try something like below. Uses $lookUp to self join to collection and $match stage with $or in combination with $and for second condition and the next part of $or for first condition and $group stage to remove duplicates and $project stage to format the response.
db.boxes.aggregate([{
$lookup: {
from: "boxes",
localField: "original",
foreignField: "_id",
as: "nested_orders"
}
}, {
$unwind: {
path: "$nested_orders",
preserveNullAndEmptyArrays: true
}
}, {
$match: {
$or: [{
$and: [{
"status": "ready"
}, {
"nested_orders": {
$exists: false,
}
}]
}, {
"nested_orders.status": "pending"
}]
}
}, {
$group: {
"_id": null,
"names": {
$addToSet: {
name: "$name",
nested_name: "$nested_orders.name"
}
}
}
}, {
$unwind: "$names"
}, {
$project: {
"_id": 0,
"name": {
$ifNull: ['$names.nested_name', '$names.name']
}
}
}]).pretty();
Sample Response
{ "name" : "Original Box #1" }
{ "name" : "Original Box #2" }
{ "name" : "Original Box #3" }
To decompose the aggregation :
a $group which creates
an array ids which match ready status for which it will add the *original value
an array box_ready which match ready status and keep the other fields as is (it will be used later)
an array document which contain the whole original document ($$ROOT)
{
$group: {
_id: null,
ids: {
$addToSet: {
$cond: [
{ $eq: ["$status", "ready"] },
"$original", null
]
}
},
box_ready: {
$addToSet: {
$cond: [
{ $eq: ["$status", "ready"] },
{ _id: "$_id", name: "$name", original: "$original", status: "$status" },
null
]
}
},
document: { $push: "$$ROOT" }
}
}
$unwind document field to remove the array
{
$unwind: "$document"
}
use a $redact aggregation to keep or remove records based on matching of $document._id in the array ids previously created (that contain the matching original and status)
{
$redact: {
"$cond": {
"if": {
"$setIsSubset": [{
"$map": {
"input": { "$literal": ["A"] },
"as": "a",
"in": "$document._id"
}
},
"$ids"
]
},
"then": "$$KEEP",
"else": "$$PRUNE"
}
}
}
$group to push all documents that matched the previous $redact to another array named filtered (we have now 2 array which can be united)
{
$group: {
_id: null,
box_ready: { $first: "$box_ready" },
filtered: { $push: "$document" }
}
}
use a $project with a setUnion to union the arrays box_ready and filtered
{
$project: {
union: {
$setUnion: ["$box_ready", "$filtered"]
},
_id: 0
}
}
$unwind the array you have obtained to get distinct records
{
$unwind: "$union"
}
$match only those which have original missing and that are not null (as initially a the status:ready condition has obliged to get a null value on the first $group
{
$match: {
"union.original": {
"$exists": false
},
"union": { $nin: [null] }
}
}
The whole aggregation query is :
db.collection.aggregate(
[{
$group: {
_id: null,
ids: {
$addToSet: {
$cond: [
{ $eq: ["$status", "ready"] },
"$original", null
]
}
},
box_ready: {
$addToSet: {
$cond: [
{ $eq: ["$status", "ready"] },
{ _id: "$_id", name: "$name", original: "$original", status: "$status" },
null
]
}
},
document: { $push: "$$ROOT" }
}
}, {
$unwind: "$document"
}, {
$redact: {
"$cond": {
"if": {
"$setIsSubset": [{
"$map": {
"input": { "$literal": ["A"] },
"as": "a",
"in": "$document._id"
}
},
"$ids"
]
},
"then": "$$KEEP",
"else": "$$PRUNE"
}
}
}, {
$group: {
_id: null,
box_ready: { $first: "$box_ready" },
filtered: { $push: "$document" }
}
}, {
$project: {
union: {
$setUnion: ["$box_ready", "$filtered"]
},
_id: 0
}
}, {
$unwind: "$union"
}, {
$match: {
"union.original": {
"$exists": false
},
"union": { $nin: [null] }
}
}]
)
It gives you :
{ "union" : { "_id" : 1, "name" : "Original Box #1", "status" : "pending" } }
{ "union" : { "_id" : 5, "name" : "Original Box #2", "status" : "ready" } }
{ "union" : { "_id" : 6, "name" : "Original Box #3", "status" : "pending" } }
Use an additional $project if you want to select specific fields
For mongoose, you should be able to do like this to perform aggregation :
Box.aggregate([
//the whole aggregation here
], function(err, result) {
});
Several of the answers are close but here's the most efficient way. It accumulates the "_id" values of boxes to be picked up and then uses $lookup to "rehydrate" the full details of each (top-level) box.
db.boxes.aggregate(
{$group: {
_id:null,
boxes:{$addToSet:{$cond:{
if:{$eq:["$status","ready"]},
then:{$ifNull:["$original","$_id"]},
else:null
}}}
}},
{$lookup: {
from:"boxes",
localField:"boxes",
foreignField:"_id",
as:"boxes"
}}
)
Your result based on sample data:
{
"_id" : null,
"boxIdsToPickUp" : [
{
"_id" : 1,
"name" : "Original Box #1",
"status" : "pending"
},
{
"_id" : 5,
"name" : "Original Box #2",
"status" : "ready"
},
{
"_id" : 6,
"name" : "Original Box #3",
"status" : "pending"
}
] }
Note that the $lookup is done only for the _id values of boxes to be picked up which is far more efficient than doing it for all boxes.
If you wanted the pipeline to be more efficient you would need to store more details about original box in the nested box documents (like its name).
To achieve your goal you can follow bellow steps:
First of all select record for status is ready (because you want to get parent who has no nested box but status is ready and who
has nested box at least one with stats is ready )
Find parent box using $lookup
then $group to get unique parent box
then $project box name
So can try this query:
db.getCollection('boxes').aggregate(
{$match:{"status":'ready'}},
{$lookup: {from: "boxes", localField: "original", foreignField: "_id", as: "parent"}},
{$unwind: {path: "$parent",preserveNullAndEmptyArrays: true}},
{$group:{
_id:null,
list:{$addToSet:{"$cond": [ { "$ifNull": ["$parent.name", false] }, {name:"$parent.name"}, {name:"$name"} ]}}
}
},
{$project:{name:"$list.name", _id:0}},
{$unwind: "$name"}
)
OR
get record for status is ready
get desired recordID
get name according to recordID
db.getCollection('boxes').aggregate(
{$match:{"status":'ready'}},
{$group:{
_id:null,
parent:{$addToSet:{"$cond": [ { "$ifNull": ["$original", false] }, "$original", "$_id" ]}}
}
},
{$unwind:"$parent"},
{$lookup: {from: "boxes", localField: "parent", foreignField: "_id", as: "parent"}},
{$project: {"name" : { $arrayElemAt: [ "$parent.name", 0 ] }, _id:0}}
)
Using mongoose (4.x)
Schema:
const schema = mongoose.Schema({
_id: Number,
....
status: String,
original: { type: Number, ref: 'Box'}
});
const Box = mongoose.model('Box', schema);
Actual Query:
Box
.find({ status: 'ready' })
.populate('original')
.exec((err, boxes) => {
if (err) return;
boxes = boxes.map((b) => b.original ? b.original : b);
boxes = _.uniqBy(boxes, '_id');
console.log(boxes);
});
Docs on Mongoose#populate: http://mongoosejs.com/docs/populate.html