I'm having a problem within my application that is related to multiple tables/models arranged in a Many:Many relationship but also leverages polymorphic columns and is using the BelongsToManyAddAssociationMixin. I am unable to insert multiple records in a M:N relationship because I'm unable to tell Sequelize to change the unique fields to include roleableId
Essentially I have a User model, a Role model, a relationship table called UserRole and other models which are "roleable" (For the sake of this example, one model is a Facility)
So essentially my UserRole model looks like
export class UserRole extends Model {
public readonly id!: UUID
public RoleId!: UUID
public UserId!: UUID
public roleableType!: string
public roleableId!: string
public readonly User!: User
public readonly Role!: Role
public getUser!: BelongsToGetAssociationMixin<User>
public getRole!: BelongsToGetAssociationMixin<Role>
}
export const initializeUserRole = (sequelize: Sequelize) => {
UserRole.init(
{
RoleId: {
type: DataTypes.UUID,
allowNull: false,
validate: {
isUUID: 4,
},
field: 'role_id',
},
UserId: {
type: DataTypes.UUID,
allowNull: false,
validate: {
isUUID: 4,
},
field: 'user_id',
},
roleableId: {
type: DataTypes.UUID,
field: 'roleable_id',
validate: {
isUUID: 4,
},
},
roleableType: {
type: DataTypes.STRING,
field: 'roleable_type',
},
},
{
timestamps: false,
sequelize,
tableName: TableNames.USER_ROLES,
}
)
}
In my scenario, I would have a role named Sales Rep and it's primary key is a UUID
In theory, I would like to create a new record for the user to have the SalesRep role twice in my UserRole table, one with values for roleableId & roleableType and one where those values are null.
Using mixins, I'm able to add these roles with something like
const randomUser = await userFactory()
const salesRepRole = await roleFactory({ roleName: 'Sales Rep' })
await randomUser.addRole(salesRepRole)
^^^ This works properly
But if I try to do something like:
const randomUser = await userFactory()
const randomFacility = await facilityFactory()
const salesRepRole = await roleFactory({ roleName: 'Sales Rep' })
await randomUser.addRole(salesRepRole)
await randomuser.addRole(salesRepRole, { through: { roleableId: randomFacility.id, roleableType: 'facility' }})
What will happen is that instead of performing a new insert into my UserRole table, instead Sequelize will identify that there is already a record based upon User.id & Role.id and will perform an UPDATE instead, resulting in only one record being returned.
I'm not sure how to indicate to Sequelize that instead of putting a unique constraint on User.id & Role.id, I really need it to be User.id, Role.id & roleableId
My work around right now has been to simply just roll my own add<Model> function which will perform an INSERT properly (adding an index on my DB to check for unique instances of user_id,role_id,roleable that throw an error if this combination already exists) but this feels a bit dirty and I'd rather try to utilize the mixin as much as possible, especially if I also have to add a BelongsToManyAddAssociationsMixin version as well for addRoles
Does anyone have any ideas? I'm happy to elaborate more if that helps.
Thanks!
Well say I have a database that has a schema like:
CREATE TABLE public.mytable
(
datacolumn json,
id serial NOT NULL,
PRIMARY KEY (id)
)
Clearly datacolumn can be null: null is even an important value of this.
Now in sails I have a datamodel defined as:
moduel.exports = {
attributes: {
datacolumn: {
type: 'ref',
columnType: 'json',
allowNull: true,
},
},
id: {
type: 'number',
},
}
This should work right? - Well it doesn't sails refuses to lift:
Failed to lift app: userError: The attribute datacolumn on the mytable model contains invalid properties. The allowNull flag may not be used on attributes with type json or type ref.
I could of course just remove the line allowNull: but that would give lots of warnings that null is not a valid value (directly logged, and giving like 10 lines per warning). And I doubt it will correctly insert null values would it?
So how to handle above database schema?
I want to build a sequelize relationship that represents : An item is composed by a specific amount of others items.
Database tables
Item (itemId, name)
Ingredient (ingredientId, itemParentId, itemChildrenId, amount)
Sequelize models
// Item.js
class Item extends Sequelize.Model { }
Item.init({
itemId: {
type: Sequelize.INTEGER,
autoIncrement: true,
primaryKey: true
},
name: Sequelize.STRING,
}, {
sequelize: db,
})
// Ingredient.js
class Ingredient extends Sequelize.Model { }
Ingredient.init({
ingredientId: {
type: Sequelize.INTEGER,
autoIncrement: true,
primaryKey: true
},
amount: Sequelize.INTEGER,
}, {
sequelize: db,
})
And I am just trying to write the correct sequelize association to match my database logic, so I tried :
// Association.js
Item.belongsToMany(Item, { through: Ingredient, as: 'ingredients', foreignKey: 'itemParentId' })
But I'm having this error Unknown column 'ingredients->ingredient.ingredientItemId' in 'field list', which is true but I do not know how to specify the right keys/columns.
Any help, please!
I see few problems: First, you're performing Item.init within the Ingredient model. Probably a mistake on your part. Change it to Ingredient.init.(personally i never used this "init" api, i define models differently, so i'm not sure how it works)
Second, change the primary keys of both Ingredient and Item to be just "id", like:
id: {
type: Sequelize.INTEGER,
autoIncrement: true,
primaryKey: true
}
Also, your association isn't correct, it needs to be through a junction table:
Item.belongsToMany(Ingredient, { through: "ItemIngredient"})
Note that "through" referes here to a table name, that Sequelize will create automatically if you're using model.sync(), and if not- you will need to create it yourself(or with a migration, which i recommend), with columns: itemId, ingredientId.
You also need to add the "reverse" association, like that:
Ingredient.belongsToMany(Item, { through: "ItemIngredient"})
I'm using node v9.5, sequelize v4.33 (postgres dialect).
I have two first-class models: Driver (specific people) and Car (generic make+model combinations). Thus far, they've been connected by a many-to-many join table. Now I want to start tracking additional properties on that join table, but am having trouble declaring these relationships so they actually work.
const Driver = sqlz.define('Driver', {
id: { primaryKey: true, type: DataTypes.UUID },
name: DataTypes.string
})
const Car = sqlz.define('Car', {
id: { primaryKey: true, type: DataTypes.UUID },
make: DataTypes.string,
model: DataTypes.string
})
// old associations; worked great when requirements were simpler
Driver.belongsToMany(Car, {
through: 'DriverCar',
as: 'carList',
foreignKey: 'driverId'
})
Car.belongsToMany(Driver, {
through: 'DriverCar',
as: 'driverList',
foreignKey: 'carId'
})
Now I want to begin tracking more information about the relationship between a car and its driver, like the color of that specific car.
Step 1: I update the migration script, adding a new column to the join table like so:
queryInterface.createTable( 'DriverCar', {
driverId: {
type: sqlz.UUID,
allowNull: false,
primaryKey: true,
references: {
model: 'Driver',
key: 'id'
}
},
carId: {
type: sqlz.UUID,
allowNull: false,
primaryKey: true,
references: {
model: 'Car',
key: 'id'
}
},
createdAt: {
type: sqlz.DATE,
allowNull: false
},
updatedAt: {
type: sqlz.DATE,
allowNull: false
},
// new column for join table
color: {
type: Sequelize.STRING
}
})
Step 2: I define a new sqlz model for DriverCar:
const DriverCar = sqlz.define('DriverCar', {
color: DataTypes.string
})
(I assume I only need to define the interesting properties, and that driverId and carId will still be inferred from the associations that will be defined.)
Step 3: I need to update the associations that exist among Driver, Car, and DriverCar.
This is where I'm stuck. I have attempted updating the existing associations, like so:
Driver.belongsToMany(Car, {
through: DriverCar, // NOTE: no longer a string, but a reference to new DriverCar model
as: 'carList',
foreignKey: 'driverId'
})
Car.belongsToMany(Driver, {
through: DriverCar, // NOTE: no longer a string, but a reference to new DriverCar model
as: 'driverList',
foreignKey: 'carId'
})
This executes without error, but the new color property is not fetched from the join table when I try driver.getCarList(). (Sqlz is configured to log every SQL statement, and I have verified that no properties from the join table are being requested.)
So, instead, I tried spelling out this relationship more explicitly, by associating Driver to DriverCar, and then Car to DriverCar:
// Driver -> Car
Driver.hasMany(DriverCar, {
as: 'carList',
foreignKey: 'driverId'
})
// Car -> Driver
Car.hasMany(DriverCar, {
foreignKey: 'carId'
})
I also tell sqlz that DriverCar won't have a standard row id:
DriverCar.removeAttribute('id')
At this point, requesting a Driver's carList (driver.getCarList()) seems to work, because I can see join table props being fetched in SQL. But saving fails:
driverModel.setCarList([ carModel1 ])
UPDATE DriverCar
SET "driverId"='a-uuid',"updatedAt"='2018-02-23 22:01:02.126 +00:00'
WHERE "undefined" in (NULL)
The error:
SequelizeDatabaseError: column "undefined" does not exist
I assume this error is occurring because sqzl doesn't understand the proper way to identify rows in the join table, because I've failed to establish the necessary associations. And frankly, I'm not confident I've done this correctly; I'm new to ORMs, but I was expecting I'd need to specify 4 assocations:
Driver -> DriverCar
DriverCar -> Car
Car -> DriverCar
DriverCar -> Driver
To recap: I have 2 first-class entities, joined in a many-to-many relationship. I'm trying to add data to the relationship, have discovered that the ORM requires defining those associations differently, and am having trouble articulating the new associations.
A note about your aliases
Before going to the answer, I would like to point out that your choice of aliases (carList and driverList) could be better, because although the auto-generated sequelize methods .setCarList() and .setDriverList() do make sense, the methods .addCarList(), .addDriverList(), .removeCarList() and .removeDriverList() are nonsense, since they take only a single instance as a parameter, not a list.
For my answer, I won't use any aliases, and let Sequelize default to .setCars(), .setDrivers(), .addCar(), .removeCar(), etc, which make much more sense to me.
Example of working code
I've made a 100% self-contained code to test this. Just copy-paste it and run it (after running npm install sequelize sqlite3):
const Sequelize = require("sequelize");
const sequelize = new Sequelize({ dialect: 'sqlite', storage: 'db.sqlite' });
const Driver = sequelize.define("Driver", {
name: Sequelize.STRING
});
const Car = sequelize.define("Car", {
make: Sequelize.STRING,
model: Sequelize.STRING
});
const DriverCar = sequelize.define("DriverCar", {
color: Sequelize.STRING
});
Driver.belongsToMany(Car, { through: DriverCar, foreignKey: "driverId" });
Car.belongsToMany(Driver, { through: DriverCar, foreignKey: "carId" });
var car, driver;
sequelize.sync({ force: true })
.then(() => {
// Create a driver
return Driver.create({ name: "name test" });
})
.then(created => {
// Store the driver created above in the 'driver' variable
driver = created;
// Create a car
return Car.create({ make: "make test", model: "model test" });
})
.then(created => {
// Store the car created above in the 'car' variable
car = created;
// Now we want to define that car is related to driver.
// Option 1:
return car.addDriver(driver, { through: { color: "black" }});
// Option 2:
// return driver.setCars([car], { through: { color: "black" }});
// Option 3:
// return DriverCar.create({
// driverId: driver.id,
// carId: car.id,
// color: "black"
// });
})
.then(() => {
// Now we get the things back from the DB.
// This works:
return Driver.findAll({ include: [Car] });
// This also works:
// return car.getDrivers();
// This also works:
// return driver.getCars();
})
.then(result => {
// Log the query result in a readable way
console.log(JSON.stringify(result.map(x => x.toJSON()), null, 4));
});
The code above logs as expected (as I would expect, at least):
[
{
"id": 1,
"name": "name test",
"createdAt": "2018-03-11T03:04:28.657Z",
"updatedAt": "2018-03-11T03:04:28.657Z",
"Cars": [
{
"id": 1,
"make": "make test",
"model": "model test",
"createdAt": "2018-03-11T03:04:28.802Z",
"updatedAt": "2018-03-11T03:04:28.802Z",
"DriverCar": {
"color": "black",
"createdAt": "2018-03-11T03:04:28.961Z",
"updatedAt": "2018-03-11T03:04:28.961Z",
"driverId": 1,
"carId": 1
}
}
]
}
]
Note that there is no secret. Observe that the extra attribute that you're looking for, color, comes nested in the query result, not in the same nesting level of the Car or Driver. This is the correct behavior of Sequelize.
Make sure you can run this code and get the same result I do. My version of Node is different but I doubt that could be related to anything. Then, compare my code to your code and see if you can figure out what is causing you problems. If you need further help, feel free to ask in a comment :)
A note about many-to-many relationships with extra fields
Since I stumbled myself upon problems with this, and this is related to your situation, I thought I should add a section in my answer alerting you to the "trap" of setting up an overcomplicated many-to-many relationship (it's a lesson that I learned myself after struggling for a while).
Instead of repeating myself, I will just add a brief quote of what I said in Sequelize Issue 9158, and add links for further reading:
Junction tables, the tables that exist in relational databases to represent many-to-many relationships, initially have only two fields (the foreign keys of each table defining the many-to-many relationship). While it's true that it's possible to define extra fields/properties on that table, i.e. extra properties for the association itself (as you put in the issue title), care should be taken here: if it's getting overcomplicated, it's a sign that you should "promote" your junction table to a full-fledged entity.
Further reading:
My own self-answered question involving an overcomplicated setup of many-to-many relationships in sequelize: FindAll with includes involving a complicated many-to-(many-to-many) relationship (sequelizejs)
And its sibling question: Is it OK to have a many-to-many relationship where one of the tables involved is already a junction table?
I have postgres table name tests which contain few records, now I want to fetch all record of this table but unable because its provides only id,createdAt and updatedAt.
So either I have to provide an object which contain column name that I don't want, I wish it should be dynamic so after this if I pass another table name it will provide data of that.
I tried this but it returns null array of object
Project.findAll(attributes: ['*']
}).then(function(project) {
console.log("select_data: " + JSON.stringify(project));
})
I'm not totally clear on what the question / problem is here, but it sounds like when you're defining your model, you might not have included all the column names of the table in your model definition. Here's what it should look like:
module.exports = function(sequelize, DataTypes) {
const test = sequelize.define('test', {
my_column: DataTypes.STRING,
my_other_column: DataTypes.STRING,
my_boolean_column: DataTypes.BOOLEAN,
}, {
timestamps: true,
underscored: true,
tableName: 'tests',
})
return test
}
Notice each column of the table is explicitly defined in the model.