Hello i have a mysql query which is working fine in sequelize.query and the query is
select list_name from lists l where l.list_id not in
(SELECT sub.list_id from list_sub_activities sub left join.
Activities a on a.list_act_id = sub.list_act_id where a.agency_id = 2)
and i want to do the same using the sequelize model, i have tried but i think i am missing something.
List of Package ---> lists
List_of_Packages.findAll({
attributes: ['list_name'],
where: {
list_id: {
[Op.notIn]: [List_sub_Activities.findAll({
attributes: ['list_id'],
include: {
model: Activities,
required: false,
where: {
agency_id: 2
}
}
})
]
}
}
}).then((response) => {
console.log(response);
})
I appreciate that if you help me.
Thank you !!!
The findAll() (and other query methods) are asynchronous so you will need to resolve the promise (or use a callback) to resolve the value before you can pass the list_ids to Op.notIn. It will also return an array of objects with a property of list_id, so you will need to map this to an array of integers before you can use it. You can also pass in raw: true so that it will not generate Sequelize Instances from your results and will instead return plain javascript objects - this is more efficient than creating objects just to fetch a single property.
By setting required: false on the Activities include you will be returning all List_sub_Activities and not filtering on them (some will be null in your results). This is likely not what you intended.
This example uses async/await for clarity instead of thenables. Note that this is not the most efficient as it requires multiple database queries, the ideal solution would be to use a LEFT JOIN and then remove items where the package.list_id IS NULL (see second example).
// get an array of Activities with the list_id set
const activities = await List_sub_Activities.findAll({
attributes: ['list_id'],
include: {
model: Activities,
// don't use required: false to only return results where List_sub_Activities.Activities is not null
// required: false,
where: {
agency_id: 2,
},
},
raw: true,
});
// map the property to an array of just the IDs
const activityIds = activities.map((activity) => activity.list_id);
// now you can pass the activityIds to Op.notIn
const packages = await List_of_Packages.findAll({
attributes: ['list_name'],
where: {
list_id: {
[Op.notIn]: activityIds,
},
},
});
With thenables.
List_sub_Activities.findAll(...)
.then((activities) => activities.map((activity) => activity.list_id))
.then((activityIds) => List_of_Packages.findAll(...))
.then((packages) => {
console.log(packages);
});
This example LEFT JOINs List_of_Packages to List_sub_Activities which is JOINed to Activities with a WHERE setting the agency_id to 2, then only returns results from List_of_Packages where the List_sub_Activities.list_id is NULL (nothing was matched on the LEFT JOIN). This should return the same results as above in a single query.
// Get List_of_Packages where there is no match in List_sub_Activities after
// it is joined to Activities with the agency_id set.
const agencyId = 2;
const packages = await List_of_Packages.findAll({
attributes: ['list_name'],
include: {
model: List_sub_Activities,
// we don't need to actually fetch the list_id
attributes: [],
include: {
model: Activities,
where: {
agency_id: agencyId,
},
},
// uses a LEFT JOIN
required: false,
},
// only return results where the List_sub_Activities.list_id is null
where: sequelize.where(sequelize.col('List_sub_Activities.list_id'), 'IS', null),
});
Related
This question already has answers here:
mongodb/mongoose findMany - find all documents with IDs listed in array
(9 answers)
Closed 3 months ago.
I am trying to search using node.js, ejs and mongoose. All the filter parameters are working perfectly but only categoryIds is not (stored as a collection of ObjectIDs in the mongodb document, referring to the respective document in categories collection), always giving me the empty record set.
For example:
If I need to find the a movie called Cosmos (see the attached screenshot) then I can easily find it with all or any filter except categories. Once I select any category, the record-set will go blank even if the I have selected the one which it belongs to.
model.js
const Model = mongoose.model('Movie', new Schema({
...
categoryIds: [{
type: Schema.Types.ObjectId,
trim: true,
default: null,
ref: 'Category',
}],
copyrightId: {
type: Schema.Types.ObjectId,
trim: true,
default: null,
ref: 'Copyright',
},
...
}, {
timestamps: true
});
Controller.js
Router.get('/', (req, res) => {
const search = req.query;
const conditions = (() => {
let object = {};
['releaseYear', 'languageId', 'copyrightId'].forEach(filter => {
if (search[filter] != '') {
object[filter] = search[filter];
}
});
if (typeof search.categoryIds !== 'undefined') {
object.categoryIds = [];
search.categoryIds.forEach(item => object.categoryIds.push(item));
}
if (search.keywords != '') {
object.title = {
$regex: search.keywords,
$options: 'i'
};
}
return object;
})();
const count = await Model.count(conditions);
const items = await Model.find(conditions, {
__v: false,
imdb: false,
trailer: false,
createdAt: false,
updatedAt: false,
}).sort({
status: -1,
releaseYear: -1,
title: 1
})
.populate('languageId', ['title'])
.populate('copyrightId', ['title'])
.populate('categoryIds', ['title'])
.skip(serialNumber)
.limit(perPage);
...
});
All the fields in the search form
{
categoryIds: [
'6332a8a2a336e8dd78e3fe30',
'6332a899a336e8dd78e3fe2e',
'6332a87ba336e8dd78e3fe2c',
'634574ab339b1a6b09c1e144'
],
languageId: '',
copyrightId: '',
releaseYear: '',
rating: '',
seen: '',
status: '',
keywords: '',
submit: 'search' // button
}
filtered search parameters
{
categoryIds: [
'6332a8a2a336e8dd78e3fe30',
'6332a899a336e8dd78e3fe2e',
'6332a87ba336e8dd78e3fe2c',
'634574ab339b1a6b09c1e144'
]
}
Here is the screenshot of mongodb document.
...
if (typeof search.categoryIds !== 'undefined') {
object.categoryIds = {
$in: []
};
search.categoryIds.forEach(item => object.categoryIds.$in.push(
mongoose.Types.ObjectId(item))
);
}
console.log(object);
return object;
The is the final filter object
{
categoryIds: {
'$in': [
new ObjectId("6332a87ba336e8dd78e3fe2c"),
new ObjectId("634669f4a2725131e80d99f1")
]
}
}
Now, all the filters are working perfectly.
Thank you everyone.
The filter should contain all categoryIds and in the same order to match the document. It's not quite clear from the question if it is the intended functionality. If not, most popular usecases are documented at https://www.mongodb.com/docs/manual/tutorial/query-arrays/
I don't recall how mongoose handles types when you query with array function like $all, so you may need to convert string IDs to ObjectIDs manually, e.g.:
search.categoryIds.forEach(item => object.categoryIds.push(
mongoose.Types.ObjectId(item))
);
When i fetch new alerts, i want to check if the ID of the new alert was already recorded. The issue is that that ID is nested inside an array. There's the alertsDetails array, which contains objects and those objects have an _ID filed which is what i want to check. I am not sure how to achieve that. I got the code below but then i have to iterate over the result to check the exists value. Im sure there must be a better way.
const mongoose = require('mongoose');
const { Schema } = mongoose;
const G2AlertsSchema = new Schema(
{
status: { type: String, required: true },
openDate: { type: Date, required: true },
alertType: { type: Array, required: true },
severity: { type: Array, required: true },
locationName: { type: Array, required: true },
history: { type: Array, required: true },
alertDetails: { type: Array, required: false },
assignedTo: { type: Schema.Types.ObjectId, ref: 'user' },
},
{
timestamps: true,
},
);
const G2Alerts = mongoose.model('G2Alert', G2AlertsSchema);
module.exports = G2Alerts;
This is the code i found on mongodb's website. I just want to see if the ID exists only. Basically when i fetch the new alerts i get an array and i iterate over it, i want to check each item's ID against what's inside the Database. If it's there, skip and go to the next. If it's new, then create a new alert and save it.
const exists = await G2Alerts.aggregate([
{
$project: {
exists: {
$in: ['5f0b4f508bda3805754ab343', '$alertDetails._id'],
},
},
},
]);
EDIT: Another thing. I am getting a eslint warning saying i should use array iteration instead of a for loop. The issue is, i need to use await when looking up the Alert ID. If i use, reduce or filter, i can't use await. If i use async inside the reduce or filter function, then it will return promises in or just an empty array.
This below works, based on the answer provided by Tom Slabbaert
const newAlertsData = [];
for (let item of alertData.data.items) {
const exists = await G2Alerts.find({ 'alertDetails._id': `${item._id}` });
if (exists.length === 0) {
newAlertsData.push(item);
}
}
if (newAlertsData.length !== 0) {......
But this does not
const filteredAlerts = alertData.data.items.reduce((filtered, item) => {
const exists = await G2Alerts.find({ 'alertDetails._id': `${item._id}` });
if (exists.length === 0) {
filtered.push(item);
}
return filtered;
}, []);
You're not far off, here is an example using the correct syntax:
const exists = await G2Alerts.findOne({"alertDetails._id": '5f0b4f508bda3805754ab343'}});
if (!exists) {
... do something
}
This can also be achieve using aggregate with a $match stage instead of a $project stage or even better countDocuments which just returns the count instead of the entire object if you do not require it.
One more thing I'd like to add is that make sure alertDetails._id is string type as you're using string in you're $in. otherwise you'll need to cast them to ObjectId type in mongoose like so:
new mongoose.Types.ObjectId('5f0b4f508bda3805754ab343')
And for Mongo:
import {ObjectId} from "mongodb"
...
new ObjectId('5f0b4f508bda3805754ab343')
EDIT
Try something like this?
let ids = alertData.data.items.map(item => item._id.toString());
let existing = await G2Alerts.distinct("alertsDetails._id", {"alertsDetails._id": {$in: ids}});
const filteredAlerts = alertData.data.items.reduce((filtered, item) => {
if (!existing.includes(item._id.toString())) {
return [item].concat(filtered)
}
return filtered;
}, []);
This way you only need to call the db once and not multiple times.
Final code based on the provided answer.
const ids = alertData.data.items.map(item => item._id);
const existing = await G2Alerts.find({ 'alertDetails._id': { $in: ids } }).distinct(
'alertDetails._id',
(err, alerts) => {
if (err) {
res.send(err);
}
return alerts;
},
);
const filteredAlerts = alertData.data.items.reduce((filtered, item) => {
if (!existing.includes(item._id.toString()) && item.openDate > dateLimit) {
return [item].concat(filtered);
}
return filtered;
}, []);
I want to add the property type as billing to the result of a mongodb operation. I treid many answers from Is it possible to add dynamically named properties to JavaScript object?, like this one: https://stackoverflow.com/a/46512466/6655884 here:
let r1 = await Billing.find({
launchDate: {
"$gte": new Date(req.query.minDate),
"$lte": new Date(req.query.maxDate)
},
deleted: false,
status: "pago"
});
r1.map((obj)=>({...obj,['type']:"billing"}));
But when I print r1, there's no type property.
UPDATE:
What about
for (var i=0; i<r1.length; i++) {
r1[i]['type'] = "billing";
}
?
It doesn't work either
That could possible be happening because of mongoose .find() might be resulting in raw MongoDB docs rather than javascript objects, try to use .lean() to convert result into Js object in order to manipulate it :
let r1 = await Billing.find({
launchDate: {
"$gte": new Date(req.query.minDate),
"$lte": new Date(req.query.maxDate)
},
deleted: false,
status: "pago"
}).lean();
r1.map((obj)=>({...obj,['type']:"billing"}));
Or alternatively you can do it in query using aggregation-pipeline :
let r1 = await Billing.aggregate([
/** Filter docs with criteria */
{
$match: { }
},
/** Add new field to all returned docs */
{
$addFields: {
type: "Billing"
}
}
]);
I am trying to write a query with that would roughly do:
SELECT Challenge.id, Challenge.name, count(AcceptedChallenge.userId) AS attempts, count(a.met) AS met, count(b.active) AS active, count(c.id) AS WHOLE
FROM Challange
LEFT JOIN AcceptedChallenge ON Challenge.id = AcceptedChallenge.challengeId,
LEFT JOIN AcceptedChallenge AS a ON Challenge.id = a.challengeId
LEFT JOIN AcceptedChallenge AS b ON Challenge.id = b.challengeId
LEFT JOIN AcceptedChallenge AS c ON Challenge.id = c.challengeId
WHERE a.met = true
AND b.userId = id and b.active = true
AND c.userId = id;
Tried multiple versions, including the below:
const Sequelize = require('sequelize');
const ChallengeController = async ({ user: { id } }) => {
const challenges = await Challenge
.findAll({
attributes: {
include: [[Sequelize.fn('count', Sequelize.col('AcceptedChallenges.userId')), 'attempts'],
[Sequelize.fn('count', Sequelize.col('a.met')), 'met']]
},
include: [{
model: AcceptedChallenge, attributes: [],
required: false,
}, {
model: AcceptedChallenge, attributes: [],
as: 'a',
where: { userId: id, met: true },
required: false,
}],
group: ['Challenge.id']
})
.catch((e) => {
console.log("error:", e);
throw new HTTP404Error('Challenges not found');
});
return challenges;
};
It is not recognizing my associations. Please advise. The version here results in: SequelizeEagerLoadingError: AcceptedChallenge is associated to Challenge using an alias. You've included an alias (a), but it does not match the alias(es) defined in your association (AcceptedChallenges).
When including the AcceptedChallenge model just once, it calculates attempts just fine. I am perplexed as to how I could do the include/JOIN multiple times to get the result, which I need from a single SQL request.
This works for me when the association is repeated, once for each alias needed in your query (example below, but obviously your association may be different).
ChallengeController.hasMany(AcceptedChallenge, {as: 'a', foreignKey: 'challengeId'});
ChallengeController.hasMany(AcceptedChallenge, {as: 'b', foreignKey: 'challengeId'});
ChallengeController.hasMany(AcceptedChallenge, {as: 'c', foreignKey: 'challengeId'});
Are you doing something similar?
When at the top of my server-side code, this works fine and the results produced are correct:
var data_playlists = {};
models.Playlist.findAll({
attributes: ['id', 'name']
}).then(function (playlists){
data_playlists['playlists'] = playlists.map(function(playlist){
return playlist.get({plain: true})
});
addsongs(data_playlists, 1);
addsongs(data_playlists, 2);
addsongs(data_playlists, 3);
});
but when it's inside one of my Express methods, it isn't functioning properly; particularly, the addsongs method is not working as it should.
function addsongs(playlist_object, id_entered){
var arraysongs = [];
models.Playlist.findOne({
attributes: ['id'],
where: {
id: id_entered
}
})
.then(function(playlist) {
playlist.getSongs().then(function (thesongs){
for(var k = 0; k < thesongs.length ; k++){
arraysongs.push(thesongs[k].Songs_Playlists.SongId);
}
playlist_object.playlists[(id_entered - 1)]['songs'] = arraysongs;
});
});
}
I cannot for the life of me figure out why it works when the top segment of code is at the top, but doesn't work when inside my app.get() call.
From your code I have conducted that you want to return playlists (id and name) together with their songs (id). First of all your code will not work because the calls of addsongs(data_playlists, id) are run before data_playlists is filled with data by code above it. Moreover, the addsongs function performs asynchronous operations returning Promises, so calling them one by one will not give expected result. I suppose you can do it completely differently.
I suggest you use include attribute of options object that can be passed to findAll() method. include says which association model you also want to return from current query. In this case you want to return playlists together with their songs (M:M relation according to your code), so you need to include Song model in the query.
function getPlaylistsWithSongs() {
return models.Playlist.findAll({
attributes: ['id', 'name'],
include: [
{
model: models.Song,
as: 'Songs', // depends on how you have declare the association between songs and playlists
attributes: ['id'],
through: { attributes: [] } // prevents returning fields from join table
}
]
}).then((playlistsWithSongs) => {
return playlistsWithSongs;
});
}
Example result of getPlaylistsWithSongs result would be (after translating it to JSON e.g. like playlistsWithSongs.toJSON())
[
{
id: 1,
name: 'playlist #1',
Songs: [
{ id: 1 },
{ id: 2 }
]
}
]
Above code returns all playlists (their id and name) with their songs (only their id). Now in your route resolver you can simply call above function to return the result
app.get('/api/playlists', function (request, response) {
response.setHeader("Content-Type", "application/json; charset=UTF-8");
getPlaylistsWithSongs().then(function(playlistsWithSongs){
response.status(200).send(JSON.stringify(playlistsWithSongs));
});
});
EDIT
In order to simply return array of IDs instead array of objects with id (songs), you need to map the result. There is no simple sequelize way to return array of IDs in such a case.
}).then((playlistWithSongs) => {
let jsonPlaylists = playlistsWithSongs.map((singlePlaylist) => {
// return JSON representation of each playlist record
return singlePlaylist.toJSON();
});
jsonPlaylists.forEach((playlist) => {
// at every playlist record we map Songs to array of primitive numbers representing it's IDs
playlist.songs = playlist.Songs.map((song) => {
return song.id;
});
// when we finish we can delete the Songs property because now we have songs instead
delete playlist.Songs;
});
console.log(jsonPlaylists);
// example output: [{ id: 1, name: 'playlist #1', songs: [1, 2, 3] }]
return jsonPlaylists;
});