We Keep Coding

Sequelize filtering one to many relation

I have a User model and Timeoffs model with one to many relation. I want get all the users that doesn't have timeoffs in the specified date. Right now in the code I'm doing the exact opposite of this - getting only the users with timeoffs in the given date. How can I do this in right way? Is it possible to just invert include in that part or should I write a more complex query?
getQuestionHourUsersByDate (date, timeId) {
return UsersModel.findAll({
where: {
[USERS.COLUMNS.QH_STATUS]: true,
},
include: [
{
model: TimeoffsModel,
where: {
[Operator.and]: {
[TIMEOFFS.COLUMNS.PROCEED_STATUS]: true,
[TIMEOFFS.COLUMNS.STATUS]: TimeOffEnums.TIMEOFF_REQUEST_STATUS.APPROVED,
},
[Operator.and]: {
[TIMEOFFS.COLUMNS.START_DATE]: {
[Operator.lte]: date,
},
[TIMEOFFS.COLUMNS.END_DATE]: {
[Operator.gte]: date,
},
},
},
},
{
model: RolesModel,
where: {
[ROLES.COLUMNS.ROLE_GROUP_ID]: 1,
},
through: {
attributes: [],
where: {
[USER_ROLES.COLUMNS.STATE]: true,
},
},
},
{
model: ResponsibilitiesModel,
attributes: [
RESPONSIBILITIES.COLUMNS.NAME,
RESPONSIBILITIES.COLUMNS.RESPONSIBILITY_GROUP_ID,
RESPONSIBILITIES.COLUMNS.COLOR_CODE,
],
where: {
[RESPONSIBILITIES.COLUMNS.RESPONSIBILITY_GROUP_ID]: 2,
},
required: false,
through: {
attributes: [],
where: {
[USER_RESPONSIBILITIES.COLUMNS.ACTIVE]: true,
},
},
},
{
model: LanguagesModel,
attributes: [
LANGUAGES.COLUMNS.CODE,
LANGUAGES.COLUMNS.NAME,
],
},
{
model: UserCalendarsModel,
include: [
{
model: CalendarsModel,
where: {
[CALENDARS.COLUMNS.START_DATE]: date,
[CALENDARS.COLUMNS.ACTIVE]: true,
},
include: [
{
model: QuestionHourSlotsModel,
where: {
[QUESTION_HOUR_SLOTS.COLUMNS.TIME_ID]: timeId,
},
},
],
},
],
},
],
});
}

Unfortunately, I should use Sequelize.literal in where option to add a subquery condition with NOT EXISTS like this:
where: Sequelize.where(Sequelize.literal('NOT EXISTS (SELECT 1 FROM ... WHERE startDate <= #date and endDate >= #date)'), '=', true)
and also to pass a specified date you add to indicate bind option along with main query options like this:
where: {
[USERS.COLUMNS.QH_STATUS]: true,
},
bind: {
date
}

Request with 2 many to many with sequelize (merge result)

I would like to know how can I "merge" the result to get result from 2 tables.
Currently I have 3 tables :
posts [id, title...]
feeds [id, fk_people_id, fk_post_id]
posts_peoples [id, fk_people_id, fk_post_id]
I would like to return the posts where people is present in feeds table and posts_peoples table.
When I run this request, I have only the post where people is present in feeds table :
// Request
const resultRequest = await db.Post.findAll({
include: [
{
model: db.Feed,
as: "Feed",
where: {
fk_people_id: 2,
},
},
],
})
When I run this request, I have only the post where people is present in posts_peoples table :
// Request
const resultRequest = await db.Post.findAll({
include: [
{
model: db.PostPeople,
as: "PostPeople",
where: {
fk_people_id: 2,
},
},
],
})
When I add feeds and posts_peoples, it doesn't work.
// Request
const resultRequest = await db.Post.findAll({
include: [
{
model: db.Feed,
as: "Feed",
where: {
fk_people_id: 2,
},
},
{
model: db.PostPeople,
as: "PostPeople",
where: {
fk_people_id: 2,
},
},
],
})
The result is an empty array.

Add required: false to your includes to generate SQL with a LEFT JOIN to include results from both tables.
// Request
const resultRequest = await db.Post.findAll({
include: [{
model: db.Feed,
as: "Feed",
where: {
fk_people_id: 2,
},
required: false,
},
{
model: db.PostPeople,
as: "PostPeople",
where: {
fk_people_id: 2,
},
required: false,
}],
})

Virtual populate mongoose

I've tried to use virtual populate between two models I created:
in this case to get all the reviews with the tour id and show them with the tour.
(when using query findById() to show only this tour)
my virtual is set to true in the Schema (I've tried to set them to true after using the virtual populate but it doesn't work - by this soultion)
after checking the mongoose documentation its seems to be right but it doesn't work.
my tourSchema:
const tourSchema = new mongoose.Schema({
name: {
type: String,
required: [true, 'A tour must have a name'], //validator
unique: true,
trim: true,
maxlength: [40, 'A tour name must have less or equal then 40 characters'],
minlength: [10, 'A tour name must have at least 10 character']
//validate: [validator.isAlpha, 'A tour name must have only alphabetic characters']
},
etc...
etc...
etc...
//guides: Array --- array of user id's || embedding
guides: [
//Reference to the user data model without saving the guides in the tour data model
//Child referencing
{
type: mongoose.Schema.ObjectId,
ref: 'User'
}
]
},
{
//passing options, getting the virual properties to the document/object
toJSON: { virtuals: true },
toObject: { virtuals: true }
}
);
//Define virtual properties
tourSchema.virtual('durationWeeks').get(function () {
console.log('Virtual 1');
//using function declaration => using this keyword
return this.duration / 7;
});
//Virtual populate
tourSchema.virtual('reviews', {
ref: 'review',
localField: '_id', // Find tour where `localField`
foreignField: 'tour', // is equal to `foreignField`
//look for the _id of the tour in the tour field in review
});
my reviewSchema:
** I used in the review schema in for the tour and user populate for the tour id **
const reviewSchema = new mongoose.Schema({
review: {
type: String,
required: [true, 'Review can not be empty!']
},
rating: {
type: Number,
min: 1,
max: 5
},
createdAt: {
type: Date,
default: Date.now(),
},
tour: [
{
type: mongoose.Schema.ObjectId,
ref: 'Tour',
required: [true, 'Review must be belong to a tour.']
}
],
user: [
{
type: mongoose.Schema.ObjectId,
ref: 'User',
required: [true, 'Review must be belong to a user.']
}
]
},
{
//passing options, getting the virual properties to the document/object
toJSON: { virtuals: true },
toObject: { virtuals: true },
}
);
//Query middleware
reviewSchema.pre(/^find/, function (next) {
this.populate({
path: 'tour',
select: 'name'
})
.populate({
path: 'user',
select: 'name'
});
next();
});
My output:
get all reviews (review model data):
{
"status": "success",
"data": {
"review": [
{
"_id": "5f6ba5b45624454efca7e0b1",
"review": "What an amzing tour",
"tour": {
"guides": [],
"_id": "5c88fa8cf4afda39709c2955",
"name": "The Sea Explorer",
"durationWeeks": null,
"id": "5c88fa8cf4afda39709c2955"
},
"user": {
"_id": "5f69f736e6eb324decbc3a52",
"name": "Liav"
},
"createdAt": "2020-09-23T19:44:52.519Z",
"id": "5f6ba5b45624454efca7e0b1"
}
]
}
}
and the get tour by id:
{
"status": "success",
"data": {
"tour": {
"startLocation": {
"type": "Point",
"coordinates": [
-80.185942,
25.774772
],
"description": "Miami, USA",
"address": "301 Biscayne Blvd, Miami, FL 33132, USA"
},
"ratingsAverage": 4.8,
"ratingsQuantaity": 0,
"images": [
"tour-2-1.jpg",
"tour-2-2.jpg",
"tour-2-3.jpg"
],
"startDates": [
"2021-06-19T09:00:00.000Z",
"2021-07-20T09:00:00.000Z",
"2021-08-18T09:00:00.000Z"
],
"secretTour": false,
"guides": [],
"_id": "5c88fa8cf4afda39709c2955",
.
.
.
.
"slug": "the-sea-explorer",
"__v": 0,
"durationWeeks": 1,
"id": "5c88fa8cf4afda39709c2955"
}
}
}
as you can see the review has the tour as an arr and the id is inside the arr of the tour is there an option that the populate is not targeting the right field?

You need an option virtuals: true passed into the schema creation:
const tourSchema = new mongoose.Schema({
...
}, {
virtuals: true
}
In addition, we use the mongoose-lean-virtuals module to help with .lean and virtuals. e.g.
const mongooseLeanVirtuals = require('mongoose-lean-virtuals');
...
tourSchema.plugin(mongooseLeanVirtuals);
tourSchema.set('toJSON', { virtuals: true });
tourSchema.set('toObject', { virtuals: true });
though I'm guessing that's not strictly necessary.

So I figure it out.
First i asked in github - mongoose repo and got answerd:
reviewSchema.pre(/^find/, function (next) {
this.populate({
path: 'tour',
options: { select: 'name' } // <-- wrap `select` in `options` here...
}).populate({
path: 'user',
options: { select: 'name photo' } // <-- and here
});
next();
});
We should improve this: nested options are very confusing and it's
hard to remember whether something should be options.select or select
The second issue was to add populate after using the FindById method in the tour controller, using the populate without using the wrap 'select' didn't work for me.
exports.getTour = catchAsync(async (req, res, next) => { //parameter => :id || optinal parameter => :id?
//populate reference to the guides in the user data model
const tour = await Tour.findById(req.params.id).populate('reviews');
if (!tour) {
return next(new AppError('No tour found with that id', 404));
}
res.status(200).json({
status: 'success',
data: {
tour
}
});
})
and in the tour model, I changed the foreign key from "tour_id" (as I saw in other questions to "tour").
//Virtual populate
tourSchema.virtual('reviews', {
ref: 'Review',
localField: '_id', // Find tour where `localField`
foreignField: 'tour' // is equal to `foreignField`
//look for the _id of the tour in the tour field in review
});
Now i do have reviews in my tour data and it does virtual populate to the tour by id

How to search through mongoose ObjectId in feathers?

I have two feathers services, one for profiles and the other one for labels.
A profile can have array of ObjectId labels from other collections.
Now I have an input for search and a user types "linux"
The profile foo should be returned because it contains the id "594ceeff6905e622425f523b" in the labels array.
This kind of search query through ObjectId between objects is possible through feathers?
Profiles
 Mongoose model
{
name: { type: String, trim: true, required: true },
labels: [{ type: ObjectId, ref: 'Labels' }],
}
Feathers api get response to profiles
get http://localhost:3030/profiles
{
"name" : "foo",
"labels" : [
"594ceeff6905e622425f523b",
"594ceeff6905e622425f523c",
"594ceeff6905e622425f523d"
],
}
{
"name" : "bar",
"labels" : [
"594ceeff6905e622425f523e",
"594ceeff6905e622425f523d"
],
}
Labels
 Mongoose model
{
name: { type: String, trim: true, unique: true, required: true },
}
Feathers api get response to labels
get http://localhost:3030/labels
{
"_id": "594ceeff6905e622425f523b",
"name": "linux"
},
{
"_id": "594ceeff6905e622425f523c",
"name": "systemd"
},
{
"_id": "594ceeff6905e622425f523d",
"name": "mongodb"
},
{
"_id": "594ceeff6905e622425f523e",
"name": "javascript"
}
Now I have to populate all the labels on the profiles response, send all the profiles and then filter them on the front with that value of the input for search.
As the database grows this is going to be very inefficient, it has to exist a better way of doing this right?

You can try code like this
Profile.find({}).populate({
path: 'labels',
match: {
name: {
$regex: new RegExp(searchText, 'i');
//searchText: passed from the front end.
}
}
}).then(function(profiles){
var filteredProfiles = profiles.forEach(function(profile){
return profile.labels; //will be null for items don't match the
//searching regex.
//resolve the filtered profiles to the front end.
})
},function(error){
//Error handling
})

Feathers does not restrict you on anything that you can do with Mongoose itself and for what you would like to do you can use the Mongoose query population.
The feathers-mongoose adapter supports this through the $populate query parameter so querying
http://localhost:3030/labels?$populate=labels
Should do what you are looking for.

I the end I just two calls to the api like this:
computed: {
...mapState('profiles', { profiles: 'keyedById' }),
...mapState('labels', { labels: 'keyedById' }),
},
methods: {
...mapActions('profiles', { findProfiles: 'find' }),
async fetch() {
const labels = this.labels
const search = this.search_input.toLowerCase()
// Generates an array of matched labels per profile
const labels_id = Object.keys(labels).filter(label => {
const name = labels[label].name.toLowerCase()
return name.includes(search)
})
// Searches profiles by name or labels
this.findProfiles({
query: {
$or: [
{
name: { $regex: search, $options: 'igm' },
},
{ labels: { $in: labels_id } },
],
$populate: ['user'],
$sort: { updatedAt: -1 },
},
})
},
},

How to Avoid Sequelize JS Associations Including both the Foreign Key and the Included Object

How can I avoid showing both the foreignKey that sequelize creates and the eagerly fetched object through includes?
I have the following model structure:
FormEntry:
owner: User
createdBy: User
modifiedBy: User
formEntryData: [FormEntryData]
I modeled it after reading through SequelizeJS docs and came up with the following:
const User = sequelize.define('user', {
id: {
type: Sequelize.BIGINT(20),
field: 'user_id',
primaryKey: true
},
emailAddress: {
type: Sequelize.STRING(256),
field: 'email_address'
}
}, {
tableName: 'users',
timestamps: false
});
const FormEntryData = sequelize.define('formEntryData', {
id: {
type: Sequelize.BIGINT(20),
field: 'id',
primaryKey: true
},
entryId: {
type: Sequelize.BIGINT(20),
field: 'entry_id'
},
...
}, {
tableName: 'formEntryData',
timestamps: false
});
const FormEntry = sequelize.define('formEntry', {
id: {
type: Sequelize.BIGINT(20),
field: 'entry_id',
primaryKey: true
},
...
}, {
tableName: 'formEntries',
timestamps: false
});
I then need to create the associations to tie the models together and after a lot of trial and error I came up with the following:
FormEntry.hasMany(FormEntryData, {foreignKey: 'entry_id', as: 'FormEntryData'});
FormEntry.belongsTo(User, {foreignKey: 'created_by', as: 'CreatedBy'});
FormEntry.belongsTo(User, {foreignKey: 'modified_by', as: 'ModifiedBy'});
FormEntry.belongsTo(User, {foreignKey: 'owner', as: 'Owner'});
I then was able to query the data by doing the following:
FormEntry.findByPrimary(1472280, {
include: [
{
model: FormEntryData,
as: "FormEntryData"
},
{
model: User,
as: "CreatedBy"
},
{
model: User,
as: "Owner"
},
{
model: User,
as: "ModifiedBy"
}
]
})
Unfortunately, my results seem kind of repetitive as it seems to be including both the foreign key and the object that is eagerly fetched.
{
"id": 1472280,
...
"created_by": 26508, <-- repetitive (don't want this)
"modified_by": 26508, <-- repetitive (don't want this)
"owner": null, <-- repetitive (don't want this)
"FormEntryData": [
{
"id": 27164476,
"entryId": 1472280, <-- repetitive (but I want this one)
...
"entry_id": 1472280 <-- repetitive (don't want this)
},
...
],
"CreatedBy": { <-- repetitive (but I want this one)
"id": 26508,
"emailAddress": "swaraj.kler#greywallsoftware.com"
},
"Owner": null, <-- repetitive (but I want this one)
"ModifiedBy": { <-- repetitive (but I want this one)
"id": 26508,
"emailAddress": "swaraj.kler#greywallsoftware.com"
}
}

You need to exclude specified fields from the query
FormEntry.findByPrimary(1472280, {
include: [
{
model: FormEntryData,
as: "FormEntryData",
attributes: { exclude: ['entry_id'] }
},
{
model: User,
as: "CreatedBy"
},
{
model: User,
as: "Owner"
},
{
model: User,
as: "ModifiedBy"
}
],
attributes: { exclude: ['owner', 'created_by', 'modified_by'] }
})