We Keep Coding

Node.js: Mongoose filter data from array of ObjectIDs stored in collection [duplicate]

This question already has answers here:
mongodb/mongoose findMany - find all documents with IDs listed in array
(9 answers)
Closed 3 months ago.
I am trying to search using node.js, ejs and mongoose. All the filter parameters are working perfectly but only categoryIds is not (stored as a collection of ObjectIDs in the mongodb document, referring to the respective document in categories collection), always giving me the empty record set.
For example:
If I need to find the a movie called Cosmos (see the attached screenshot) then I can easily find it with all or any filter except categories. Once I select any category, the record-set will go blank even if the I have selected the one which it belongs to.
model.js
const Model = mongoose.model('Movie', new Schema({
...
categoryIds: [{
type: Schema.Types.ObjectId,
trim: true,
default: null,
ref: 'Category',
}],
copyrightId: {
type: Schema.Types.ObjectId,
trim: true,
default: null,
ref: 'Copyright',
},
...
}, {
timestamps: true
});
Controller.js
Router.get('/', (req, res) => {
const search = req.query;
const conditions = (() => {
let object = {};
['releaseYear', 'languageId', 'copyrightId'].forEach(filter => {
if (search[filter] != '') {
object[filter] = search[filter];
}
});
if (typeof search.categoryIds !== 'undefined') {
object.categoryIds = [];
search.categoryIds.forEach(item => object.categoryIds.push(item));
}
if (search.keywords != '') {
object.title = {
$regex: search.keywords,
$options: 'i'
};
}
return object;
})();
const count = await Model.count(conditions);
const items = await Model.find(conditions, {
__v: false,
imdb: false,
trailer: false,
createdAt: false,
updatedAt: false,
}).sort({
status: -1,
releaseYear: -1,
title: 1
})
.populate('languageId', ['title'])
.populate('copyrightId', ['title'])
.populate('categoryIds', ['title'])
.skip(serialNumber)
.limit(perPage);
...
});
All the fields in the search form
{
categoryIds: [
'6332a8a2a336e8dd78e3fe30',
'6332a899a336e8dd78e3fe2e',
'6332a87ba336e8dd78e3fe2c',
'634574ab339b1a6b09c1e144'
],
languageId: '',
copyrightId: '',
releaseYear: '',
rating: '',
seen: '',
status: '',
keywords: '',
submit: 'search' // button
}
filtered search parameters
{
categoryIds: [
'6332a8a2a336e8dd78e3fe30',
'6332a899a336e8dd78e3fe2e',
'6332a87ba336e8dd78e3fe2c',
'634574ab339b1a6b09c1e144'
]
}
Here is the screenshot of mongodb document.

...
if (typeof search.categoryIds !== 'undefined') {
object.categoryIds = {
$in: []
};
search.categoryIds.forEach(item => object.categoryIds.$in.push(
mongoose.Types.ObjectId(item))
);
}
console.log(object);
return object;
The is the final filter object
{
categoryIds: {
'$in': [
new ObjectId("6332a87ba336e8dd78e3fe2c"),
new ObjectId("634669f4a2725131e80d99f1")
]
}
}
Now, all the filters are working perfectly.
Thank you everyone.

The filter should contain all categoryIds and in the same order to match the document. It's not quite clear from the question if it is the intended functionality. If not, most popular usecases are documented at https://www.mongodb.com/docs/manual/tutorial/query-arrays/
I don't recall how mongoose handles types when you query with array function like $all, so you may need to convert string IDs to ObjectIDs manually, e.g.:
search.categoryIds.forEach(item => object.categoryIds.push(
mongoose.Types.ObjectId(item))
);

JOI validation for a string with json as a value

i am trying to validate the string using the JOI package available in npm, i checked this documentation which has many useful string formats e.g. date, IP, base64 but i need to validate the following JSON which contains a stringified JSON as a value, and there is no example in the documentation for such case
{
"id": 232,
"name": "Trojan Horse",
"file": "download.exe",
"infected": true,
"engines": "['Norton', 'AVG', 'NOD32']"
}
So for example what if i want to check engines have valid JSON value and have at-least one engine defined if infected key is set to true
The following schema works only if the engines value is written as parsed JSON
Joi.object().keys({
id: Joi.number().required(),
name: Joi.string().min(5).required(),
file: Joi.string().min(3).required(),
infected: Joi.boolean().required(),
engines: Joi.array().when('infected', {
is: Joi.exists().valid(true),
then: Joi.min(1).required()
})
});

What you need to do is to create a custom JOI validator by extending the array validator of the JOI package and using that custom validator for the engines property.
const custom = Joi.extend({
type: 'array',
base: Joi.array(),
coerce: {
from: 'string',
method(value, helpers) {
if (typeof value !== 'string' ||
value[0] !== '[' && !/^\s*\[/.test(value)) {
return;
}
try {
return { value: JSON.parse(value) };
}
catch (ignoreErr) { }
}
}
});
const schema = Joi.object({
id: Joi.number().required(),
name: Joi.string().min(5).required(),
file: Joi.string().min(3).required(),
infected: Joi.boolean().required(),
engines: custom.array().when('infected', {
is: true,
then: custom.array().min(1).required()
})
})
const validateTest = async (joiSchema, testObject) => {
try {
const value = await joiSchema.validateAsync(testObject);
console.log(value);
}
catch (err) {
console.error(err)
}
};
validateTest(schema, {
"id": 232,
"name": "Trojan Horse",
"file": "download.exe",
"infected": true,
"engines": `["Norton", "AVG", "NOD32"]`
})
You can see more examples like that here

Mongodb, Delete one object from an array of objects using javascript

I am trying to remove one object from an array of my collection, which looks like this. It s a collection in Mongodb
Before deleting a specific object based on chartId, I need to check the userId and the name of the array. Then I need to delete the object.
I have written this code, but its not working. someone will tell me what exactly I am missing in this code.
delChartObj.updateOne(
{ 'userId': userId },
{ $pull: { "Color": { "chartId": req_chart_id } } },
{ safe: true, multi: true}, function (err, obj) {
if (err) { res.send.err }
res.status(200).send({ msg: "Deleted Sucessfully" });
});
In my case, userId = ADAM, array = "Color" and chartID = time
I am using mongoose for performing action
delChartObj is an object of model
const UserSchema = mongoose.Schema({
userId: { type: String, required: true, unique: true },
charts: { type: Object },
});

You should do findOneAndUpdate, the syntax will be something like:
Model.findOneAndUpdate(
< condition>,
{ $pull: { "Color.$.chartId": req_chart_id } } }, // The actual Query
{ new: true }
)

try this in pull
{ $pull: { "Chart.Color.$.chartId": req_chart_id } } },

mongoose check if id exists but that id is nested inside an array

When i fetch new alerts, i want to check if the ID of the new alert was already recorded. The issue is that that ID is nested inside an array. There's the alertsDetails array, which contains objects and those objects have an _ID filed which is what i want to check. I am not sure how to achieve that. I got the code below but then i have to iterate over the result to check the exists value. Im sure there must be a better way.
const mongoose = require('mongoose');
const { Schema } = mongoose;
const G2AlertsSchema = new Schema(
{
status: { type: String, required: true },
openDate: { type: Date, required: true },
alertType: { type: Array, required: true },
severity: { type: Array, required: true },
locationName: { type: Array, required: true },
history: { type: Array, required: true },
alertDetails: { type: Array, required: false },
assignedTo: { type: Schema.Types.ObjectId, ref: 'user' },
},
{
timestamps: true,
},
);
const G2Alerts = mongoose.model('G2Alert', G2AlertsSchema);
module.exports = G2Alerts;
This is the code i found on mongodb's website. I just want to see if the ID exists only. Basically when i fetch the new alerts i get an array and i iterate over it, i want to check each item's ID against what's inside the Database. If it's there, skip and go to the next. If it's new, then create a new alert and save it.
const exists = await G2Alerts.aggregate([
{
$project: {
exists: {
$in: ['5f0b4f508bda3805754ab343', '$alertDetails._id'],
},
},
},
]);
EDIT: Another thing. I am getting a eslint warning saying i should use array iteration instead of a for loop. The issue is, i need to use await when looking up the Alert ID. If i use, reduce or filter, i can't use await. If i use async inside the reduce or filter function, then it will return promises in or just an empty array.
This below works, based on the answer provided by Tom Slabbaert
const newAlertsData = [];
for (let item of alertData.data.items) {
const exists = await G2Alerts.find({ 'alertDetails._id': `${item._id}` });
if (exists.length === 0) {
newAlertsData.push(item);
}
}
if (newAlertsData.length !== 0) {......
But this does not
const filteredAlerts = alertData.data.items.reduce((filtered, item) => {
const exists = await G2Alerts.find({ 'alertDetails._id': `${item._id}` });
if (exists.length === 0) {
filtered.push(item);
}
return filtered;
}, []);

You're not far off, here is an example using the correct syntax:
const exists = await G2Alerts.findOne({"alertDetails._id": '5f0b4f508bda3805754ab343'}});
if (!exists) {
... do something
}
This can also be achieve using aggregate with a $match stage instead of a $project stage or even better countDocuments which just returns the count instead of the entire object if you do not require it.
One more thing I'd like to add is that make sure alertDetails._id is string type as you're using string in you're $in. otherwise you'll need to cast them to ObjectId type in mongoose like so:
new mongoose.Types.ObjectId('5f0b4f508bda3805754ab343')
And for Mongo:
import {ObjectId} from "mongodb"
...
new ObjectId('5f0b4f508bda3805754ab343')
EDIT
Try something like this?
let ids = alertData.data.items.map(item => item._id.toString());
let existing = await G2Alerts.distinct("alertsDetails._id", {"alertsDetails._id": {$in: ids}});
const filteredAlerts = alertData.data.items.reduce((filtered, item) => {
if (!existing.includes(item._id.toString())) {
return [item].concat(filtered)
}
return filtered;
}, []);
This way you only need to call the db once and not multiple times.
Final code based on the provided answer.
const ids = alertData.data.items.map(item => item._id);
const existing = await G2Alerts.find({ 'alertDetails._id': { $in: ids } }).distinct(
'alertDetails._id',
(err, alerts) => {
if (err) {
res.send(err);
}
return alerts;
},
);
const filteredAlerts = alertData.data.items.reduce((filtered, item) => {
if (!existing.includes(item._id.toString()) && item.openDate > dateLimit) {
return [item].concat(filtered);
}
return filtered;
}, []);

mongoose finding records with multiple conditions and checking if the request parameter not empty

I'm new to this technology and working with Node and Express server that uses Mongoose and MongoDB 3.3 . I have following schema for a document collection.
exports.schemaLivre = new mongoose.Schema({
Id : Number,
Nom: { type: String, maxlength: 50 },
Auteur: String,
Description: String,
Categorie: String,
}
exports.Livre = mongoose.model('Livre', schemas.schemaLivre);
I'm trying to run a web service having 3 parameters .
app.get('/Livres/search=:search&category:=category&auteur=:auteur', controller.RecupererLivresMultiple);
each parameter can be empty and foreach parameter I have a different kind of search logic ( for the param search , I will use regex expression to check through "Nom" and "Description" and "Auteur" fields in the same time, for category and auteur , the value submitted must be equal to the content of the field).
I will need something like that but I didn't found the right query .
exports.RecupererLivresMultiple = function (req, res) {
var re = new RegExp(req.params.search, 'i');
models.Livre.find($and: [
{
$or: [
{ 'Auteur': req.params.auteur },
]
},
{
$or: [
{ 'Category': req.params.Category },
]
}
]).or([{'Nom': {$regex: re}}, {'Auteur': {$regex: re}},{'Description':{$regex: re}}]).exec(function (err, livres) {
if (err) {
return console.error(err);
}
res.json(livres);
});
};
The question is How to check the request parameter sent from the web service is empty and How I disable it in my query when searching for records .
My second question is how to combine those logic of search together ?

I think the basic structure you want for your query is:
{
'Auteur': req.params.auteur,
'Categorie': req.params.category,
$or: [
{ 'Auteur': re},
{ 'Categorie': re},
{ 'Nom': re},
]
}
To make each parameter optional, you can do something like this to build the query:
// create empty query
var findJSON = {
};
// append criteria if parameters present
if (req.params.auteur){
findJSON.Auter = req.params.auter
}
if (req.params.category){
findJSON.Categorie = req.params.category
}
if (req.params.search){
var re = new RegExp(req.params.search, 'i');
findJSON.$or = [
{ 'Auteur': re},
{ 'Categorie': re},
{ 'Nom': re},
];
}
models.Livre.find(findJSON).exec(function (err, livres) {
if (err) {
return console.error(err);
}
res.json(livres);
});
Because each parameter is optional, you should handle the case where none is provided. You could do:
if(req.params.auteur || req.params.category || req.params.search){
models.Livre.find(findJSON).exec(function (err, livres) {
if (err) {
return console.error(err);
}
res.json(livres);
});
} else {
res.json([]);
}
You may also want to handle the error condition differently. As you have it now, you aren't actually writing to the response if your find() returns an error.