Related
I'm writing here because I'm completely lost. I would like to do a findall based on association and nested [Op.and], but I can't do it. Let me explain.
I have two tables (car and properties) with an association between these two tables (one car, can have several properties). The data looks like this :
{
"car": "BMW M5",
"properties": [
{
"name": "make",
"value": "bmw"
},
{
"name": "color",
"value": "blue"
}
]
},
{
"car": "AUDI A3",
"properties": [
{
"name": "make",
"value": "audi"
},
{
"name": "color",
"value": "black"
}
]
},
What I'm trying to do is a "findAll" of all cars of make BMW and with blue color. Logically, I would see something like this :
( properties.name = make & properties.value = audi ) & ( properties.name = color & properties.value = blue )
From this logic, I therefore tried to create the sequelize command below, but without success :
const cars = await models.Car.findAll({
include: [{
model: models.Properties,
required: false,
}],
where: {
[Sequelize.Op.and]:[
{[Sequelize.Op.and]: [{"$properties.name$": "make"}, {"$properties.value$": "bmw"}]},
{[Sequelize.Op.and]: [{"$properties.name$": "color"}, {"$properties.value$": "blue"}]},
]
});
Apparently when I do this it only takes the last [Op.and] ([Sequelize.Op.and]: [{"$properties.name$": "color"}, {"$properties.value$": "blue"}]), the others don't seem to be taken into consideration.
Maybe I'm wrong, but I tried several possibilities, but I don't know how to do it. Any help would be very appreciated, thank you in advance to everyone.
It seems you over-complicated the where condition:
const cars = await models.Car.findAll({
include: [{
model: models.Properties,
required: false,
}],
where: {
[Sequelize.Op.and]:[
{
"$properties.name$": "make",
"$properties.value$": "bmw"
},
{
"$properties.name$": "color",
"$properties.value$": "blue"
},
]
}
});
If you have different props in the same group of conditions then you can just use an object to combine them with AND operator.
I have a collection of product folders productfolders and a collection of products products.
const ProductFolderSchema = new Schema(
{
folderName: { type: String, required: true },
parent: { type: Schema.Types.ObjectId, ref: 'ProductFolder' },
children: [{ type: Schema.Types.ObjectId, ref: 'ProductFolder' }],
}
);
const ProductSchema = new Schema<TProductSchema>(
{
productName: String,
folder: { type: Schema.Types.ObjectId, ref: 'ProductFolder' },
},
);
I have a backend that receives query parameter folderId and should return all products whose folder property is equal to folderId or is a descendant of folder with _id of folderId (meaning folder is one of the children of folder with _id of folderId - children can be nested deep inside children's children).
For example, consider collections productfolders and products that look like this:
const productfolders = [
{
"_id": "62e74dac78c13b738874e1a9",
"folderName": "Weapons",
"children": [
{
"_id": "62e74dd278c13b738874e1ac",
"folderName": "Bows",
"parent": "62e74dac78c13b738874e1a9",
"children": [
{
"_id": "62e74ddb78c13b738874e1b1",
"folderName": "Long Bows",
"parent": "62e74dd278c13b738874e1ac",
"children": [],
},
{
"_id": "62e74de278c13b738874e1b7",
"folderName": "Short Bows",
"parent": "62e74dd278c13b738874e1ac",
"children": [],
}
],
},
]
}
];
const products = [
{
"productName": "Curved Bow",
"folder": "62e74de278c13b738874e1b7",
"_id": "62e237368fbde6ed77e3e489"
}
];
When I pass folderId of 62e74dac78c13b738874e1a9 ("folderName": "Weapons"), I want "Curved Bow" product to be found because its folder is a deep children of "Weapons" folder.
I think you can only search something in recursive structures using $graphLookup but I couldn't figure out how to pass the variable folderId to its startsWith operator(sorry if I'm using the wrong naming of things)
Here's example db: https://mongoplayground.net/p/Yxps44cfG28
Here's my code that doesn't find anything:
const products = await ProductModel.aggregate([
{
$graphLookup: {
from: 'productfolders',
startWith: folderId, // can only pass mongo expressions here, not working with variables
connectFromField: '_id',
connectToField: 'children',
as: 'output',
},
},
]);
How do I find all products whose folder property is equal to or is a deep children of folder with folderId?
Your search was quite close. I guess the confusion came from having both parent and children fields in your schema.
As I mentioned in the comment, I don't see how you keep children up to date for all parents when you add a new folder to one of the children, but I will leave it with you. For now I will just ignore the children array. parent is enough for $graphLookup:
db.products.aggregate([
{
"$graphLookup": {
"from": "productfolders",
"startWith": "$folder",
"connectFromField": "parent",
"connectToField": "_id",
"as": "path"
}
},
{
"$match": {
"path._id": "<Folder to search>"
}
}
])
Here $graphLookup builds a flat array of all parents for each product:
startWith is the folder from products document
connectToField is the corresponding field in productfolders collection
productfolders is thee field of the productfolders document to use in the next recursive call instead of the startWith
So the path array for the Fireworks ("folder": "62e7bead91041bdddf25dd4b") will be:
[
{
"_id": "62e7bead91041bdddf25dd4b",
"folderName": "Short swords\n",
"parent": "62e79c6191041bdddf25dd1c"
},
{
"_id": "62e79c6191041bdddf25dd1c",
"folderName": "Swords",
"parent": "62e74dac78c13b738874e1a9"
},
{
"_id": "62e74dac78c13b738874e1a9",
"folderName": "Weapons"
}
]
Do you see the chain - parent of the document matches _id of the next in the chain, right?
So after the $graphLookup stage you have full folder path from the root to the product's folder for each product. Now you just $match products that do have the folder in question anywhere in the chain.
There is a simplified example on https://mongoplayground.net/p/Cy-_SzzcdNT
Based on this question by #rickhg12hs, you can use $function for this recursive search to get an array of nested folders. Then all is left is to use a regular $lookup:
db.productfolders.aggregate([
{$project: {
folders: {
$function: {
body: "function drill(t, n) {if (n.length > 0) {for (let elem of n) {t.push(elem._id); drill(t, elem.children)}} return t};",
args: [["$_id"], "$children"],
lang: "js"
}
}
}
},
{$lookup: {
from: "products",
localField: "folders",
foreignField: "folder",
as: "products"
}
}
])
See how it works on the playground example
I have the following schema:
const LIST_EVENTS = {
"id": "/listEvents",
"type": "object",
"properties": {
"filter": {
"$ref": "/MarketFilter",
"required": true
},
"locale": {
"type": "string"
}
}
}
From debugging, I can see that the object being sent to the validation is:
{
marketFilter: {
eventTypeIds: [ '1' ],
marketStartTime: {
from: '2018-12-15T00:00:00+00:00',
to: '2018-12-15T23:59:59+00:00'
}
}
}
marketFilter does not match the name of filter in the schema. To my understanding, seeing as this is a required property, this should have been flagged in the errors array of the validation result but it is not. This is my validation result:
ValidatorResult {
instance:
{ marketFilter: { eventTypeIds: [Array], marketStartTime: [Object] } },
schema:
{ id: '/listEvents',
type: 'object',
properties: { filter: [Object], locale: [Object] } },
propertyPath: 'instance',
errors: [],
throwError: undefined,
disableFormat: false }
I thought that it was possible that it did not mind about the naming convention so I removed the property altogether and still, an error is not logged with this being the validation result:
ValidatorResult {
instance: {},
schema:
{ id: '/listEvents',
type: 'object',
properties: { filter: [Object], locale: [Object] } },
propertyPath: 'instance',
errors: [],
throwError: undefined,
disableFormat: false }
I have many schemas and they are all added via .addSchema method
You have two issues with your schema. The main issue is that your required keyword is ignored because it is next to $ref. When an object with a $ref keyword is encountered where a schema is expected, it's treated as a JSON Reference only. It's not treated as a schema. A JSON Reference only has semantics for the $ref keyword. Everything else is ignored. You could fix your problem by isolating the $ref in you schema.
"filter": {
"allOf": [{ "$ref": "/MarketFilter" }],
"required": true
}
The other problem is the use of the boolean form of the required keyword. This usage of the required keyword was removed from the JSON Schema specification years ago. Unless you are specifically writing JSON Schemas against the draft-03 specification (unlikely, it's long out of date), you should be using the array form of required. Some older implementations allow you to use both forms, but that's not a good idea. You should be targeting a single specification and not mix keywords from two different versions of specification.
{
"id": "/listEvents",
"type": "object",
"properties": {
"filter": {
"$ref": "/MarketFilter"
},
"locale": {
"type": "string"
}
},
"required": ["filter"]
}
For now, I have found a work-around that is also described in the docs. I have added the required array property to the schema and added filter to it. This now threw an error.
However, the documentation states that the required property on the property itself should work the same. Is this potentially an issue with the package or is there different behaviours if the property is a reference?
I'm creating a recipe-database (commonly known as a cookbook) where I need to have a many-to-many relationship between ingredients and recipes and I'm using sequelize.js in combination with postgresql.
When an ingredient is added to a recipe I need to declare the correct amount of that ingredient that goes into the recipe.
I've declared (reduced example)
var Ingredient = sequelize.define('Ingredient', {
name: Sequelize.STRING
}, {
freezeTable: true
});
var Recipe = sequelize.define('Recipe', {
name: Sequelize.STRING
}, {
freezeTable: true
});
var RecipeIngredient = sequelize.define('RecipeIngredient', {
amount: Sequelize.DOUBLE
});
Ingredient.belongsToMany(Recipe, { through: RecipeIngredient });
Recipe.belongsToMany(Ingredient, {
through: RecipeIngredient,
as: 'ingredients'
});
My problem is with how data is returned when one my REST endpoints do
router.get('/recipes', function(req, res) {
Recipe.findAll({
include: [{
model: Ingredient,
as: 'ingredients'
}]
}).then(function(r) {
return res.status(200).json(r[0].toJSON());
})
});
The resulting JSON that gets sent to the client looks like this (timestamps omitted):
{
"id": 1,
"name": "Carrots",
"ingredients": [
{
"id": 1,
"name": "carrot",
"RecipeIngredient": {
"amount": 12,
"RecipeId": 1,
"IngredientId": 1
}
}
]
}
While all I wanted was
{
"id": 1,
"name": "Carrots",
"ingredients": [
{
"id": 1,
"name": "carrot",
"amount": 12,
}
]
}
That is, I want the amount field from the relation-table to be included in the result instead of the entire RecipeIngredient object.
The database generated by sequelize looks like this:
Ingredients
id name
1 carrot
Recipes
id name
1 Carrots
RecipeIngredients
amount RecipeId IngredientId
12 1 1
I've tried to provide an attributes array as a property to the include like this:
include: [{
model: Ingredient,
as: 'ingredients',
attributes: []
}]
But setting either ['amount'] or ['RecipeIngredient.amount'] as the attributes-value throws errors like
Unhandled rejection SequelizeDatabaseError: column ingredients.RecipeIngredient.amount does not exist
Obviously I can fix this in JS using .map but surely there must be a way to make sequelize do the work for me?
I am way late to this one, but i see it been viewed quite a bit so here is my answer on how to merge
attributes
Some random examples in this one
router.get('/recipes', function(req, res) {
Recipe.findAll({
include: [{
model: Ingredient,
as: 'ingredients',
through: {
attributes: ['amount']
}
}]
})
.then(docs =>{
const response = {
Deal: docs.map(doc =>{
return{
cakeRecipe:doc.recipe1,
CokkieRecipe:doc.recipe2,
Apples:doc.ingredients.recipe1ingredient
spices:[
{
sugar:doc.ingredients.spice1,
salt:doc.ingredients.spice2
}
]
}
})
}
})
res.status(200).json(response)
})
You can use sequelize.literal. Using Ingredient alias of Recipe, you can write as follows. I do not know if this is the right way. :)
[sequelize.literal('`TheAlias->RecipeIngredient`.amount'), 'amount'],
I tested with sqlite3. Received result with alias "ir" is
{ id: 1,
name: 'Carrots',
created_at: 2018-03-18T04:00:54.478Z,
updated_at: 2018-03-18T04:00:54.478Z,
ir: [ { amount: 10, RecipeIngredient: [Object] } ] }
See the full code here.
https://github.com/eseom/sequelize-cookbook
I've gone over the documentation but I couldn't find anything that seems like it would let me merge the attributes of the join-table into the result so it looks like I'm stuck with doing something like this:
router.get('/recipes', function(req, res) {
Recipe.findAll({
include: [{
model: Ingredient,
as: 'ingredients',
through: {
attributes: ['amount']
}
}]
}).then(function(recipes) {
return recipes[0].toJSON();
}).then(function(recipe) {
recipe.ingredients = recipe.ingredients.map(function(i) {
i.amount = i.RecipeIngredient.amount;
delete i.RecipeIngredient;
return i;
});
return recipe;
}).then(function(recipe) {
return res.status(200).json(recipe);
});
});
Passing through to include lets me filter out which attributes I want to include from the join-table but for the life of me I could not find a way to make sequelize merge it for me.
The above code will return the output I wanted but with the added overhead of looping over the list of ingredients which is not quite what I wanted but unless someone comes up with a better solution I can't see another way of doing this.
I'm using Ydn-Db-Fulltext to allow users to search a local database of contacts in an HTML5 app. So far, when it comes to searching for names of people, it works great, is smart, and returns results instantly.
Here's an example of a contact object that contains an array of contact Methods:
{
"source": "COMPANY",
"ownerPin": "12345",
"name": "brian wilkins",
"dateUpdated": "2014-03-18T14:41:05.217Z",
"dateAdded": "2014-03-18T14:41:05.217Z",
"isFavorite": false,
"socialId": "54321",
"id": "1",
"deleted": false,
"edited": false,
"favorite": false,
"contactMethods": [
{
"id": "4321",
"contactKey": "12321",
"contactId": "1",
"value": "brian.wilkins#geemail.com",
"kind": "email",
"ownerPin": "12345",
"isPrimary": false
},
{
"id": "5432",
"contactKey": "2",
"contactId": "1",
"kind": "phone",
"ownerPin": "12345",
"isPrimary": false
},
{
"id": "23",
"contactKey": "333",
"contactId": "1",
"value": "112345",
"kind": "extension",
"ownerPin": "12345",
"isPrimary": false
}
]
}
To create the index on the "name" property, I setup the fullTextCatalog as follows:
fullTextCatalogs: [{
name: 'name',
lang: 'en',
sources: [
{
storeName: 'contacts',
keyPath: 'id',
weight: 1.0
}, {
storeName: 'contacts',
keyPath: 'name',
weight: 0.5
}
]
}],
stores: [
{
name: 'contacts',
keyPath: 'id',
autoIncrement: true
}
]
};
this.db = new ydn.db.Storage('thedatabase', db_schema);
I can search by name or by id (the key) and get a list of contacts that match. Little appears to be stored in memory. Every search queries the local backing indexedDB database.
The challenge is that I also want to be able to search based on email address and extension, which are stored in the contactMethods property inside an array of contactMethods. The "value" property is where we store the email address and/or extension depending on the contactMethod type.
I tried adding contactMethods as a secondary searchable object store, but this resulted in searches for "Brian" returning two results, both the contact containing the name, and the contactMethod containing the email address. Ideally, I'd want to take the contactId (foreign key to the contact) and use it to pull the actual contact object, but it seems like this could create very expensive overhead and negate the benefits of this great search tool.
Is there a way to index object properties that are not at the parent level? How can I approach this in a way that would scale and not eat up all of the resources?
this.db.get(entry.storeName, entry.primaryKey).done(function(x) {
this.textContent += ' [Full name: ' + x.name + ']'; // this is in the contact
this.textContent += ' [email: ' + x.value + ']'; // but this is in the contactMethod
}, span);
Is there a way to index object properties that are not at the parent level?
keyPath can refer to deep object property by using dotted notation. For example, you could specify contactMethods.value to index email, but unfortunately it does not work with array value - as in this case.
So, obvious choice is keeping contactMethods record in separate object store using parent-child relationship. Since ydn-db currently does not support embedded attribute in the schema, you will have to load all child records when loading parent object.
Alternatively, IndexedDB v2 may have virtual index generated by a function expression. You can use in ydn-db by generator in index schema, for example:
stores: [
{
name: 'contacts',
keyPath: 'id',
autoIncrement: true,
indexes: [{
name: '$emails',
multiEntry: true,
generator: function(record) {
return record.contactMethods.map(function(x) {return x.value};
})
}]
}
]
One thing to note though, the generated field $emails will appear when you load the data. It likely will be removed from the record so as to match with v2 spec.
We are using this generator index heavily in multiple projects, so I will fix bug.
Indexing id and email address in full text search is convenient, but does not make sense because phonetic base full text search will be index them as it is without normalization.