We Keep Coding

Mongoose deleteMany subdocuments and related subdocuments

I have a document Project with an array of subdocuments, with a schema Tasks. Tasks has an array of subdocuments with a schema Comments.
const projectSchema = new Schema({
_id: Schema.Types.ObjectId,
name: { type: String, required: true, unique: true },
description: { type: String, default: '' },
tasks: [{ type: Schema.Types.ObjectId, ref: 'Threads' }]
});
module.exports = mongoose.model('Project', projectSchema);
const tasksSchema = new Schema({
projectId: { type: Schema.Types.ObjectId },
_id: Schema.Types.ObjectId,
title: { type: String, required: true },
text: { type: String, required: true },
comments: [{ type: Schema.Types.ObjectId, ref: 'Replies' }]
})
module.exports = mongoose.model('Tasks', tasksSchema);
const commentSchema = new Schema({
taskId: { type: Schema.Types.ObjectId },
_id: Schema.Types.ObjectId,
text: { type: String, required: true }
})
module.exports = mongoose.model('Comment', commentSchema);
When I delete the Project document I want to delete every Task and every Comment relate to that project.
To delete the Project I use findOneAndDelete so I set up a post middleware to delete all the Tasks
projectSchema.post('findOneAndDelete', function(doc, next) {
mongoose.model('Tasks').deleteMany({ projectId: doc._id }).exec();
next();
})
But now I don’t know how to delete every comment, because deletemany returns an object with the result of the operation.
Should I map the array of Tasks and call findOneAndDelete every time and then delete every single comment? It looks very inefficient for a lot of tasks.

How about embedding comments in post? since its one to many(not huge) relation. So in your code where you delete a project, you first delete all posts, which contain all the comments, only after it succeeds you delete the project. It will also benefit your read performance significantly because you just have to return a single post document instead of multiple(1post + many comment) documents.
Embedding post to project could also be possible, but depending on the size and number of possible posts, its probably better to keep it as a separate document.
In this case you need some logic to ensure consistency.
Here you could use mongodb's new feature, transaction. But I think for this case a transaction is not necessary.(Also I find it quite unstable for now) You could go with the "eventual consistency" method.
Basically you just delete all the posts related to a project and then delete a project. And then you run batches to check for any inconsistency.(check if there are any posts where its project doesnt exist. If it doestnt then delete the posts)

Unhandled rejection SequelizeDatabaseError: relation "users" does not exist

I am getting started with Sequelize. I am following the documentation they are providing on their website :http://docs.sequelizejs.com/manual/installation/getting-started.html
const Sequelize = require('sequelize');
const sequelize = new Sequelize('haha', 'postgres', 'postgres', {
host: 'localhost',
dialect: 'postgres',
operatorsAliases: false,
pool: {
max: 5,
min: 0,
acquire: 30000,
idle: 10000
},
// SQLite only
storage: 'path/to/database.sqlite'
});
sequelize
.authenticate()
.then(() => {
console.log('Connection has been established successfully.');
})
.catch(err => {
console.error('Unable to connect to the database:', err);
});
const User = sequelize.define('user', {
firstName: {
type: Sequelize.STRING
},
lastName: {
type: Sequelize.STRING
}
});
// force: true will drop the table if it already exists
User.sync({force: true}).then(() => {
// Table created
return User.create({
firstName: 'John',
lastName: 'Hancock'
});
});
Up until here, everything works perfectly. And the table "user" is correctly built and populated. (Although I do not understand Sequelize appends an "s" automatically to "user", any explanation.)
However when I add the following portion of code:
User.findAll().then(users => {
console.log(users)
})
I get this error :
Unhandled rejection SequelizeDatabaseError: relation "users" does not
exist
So my questions are:
Why does Sequelize add an "s" to user. (I know it makes sense but shouldn't the developer decide that)
What is causing that error? I followed the documentation but it still didn't work?

When you are defining your model you can add configurations, in this case the option that you must add is freezeTableName prevents the names from being plural.
const User = sequelize.define('user', {
firstName: {
type: Sequelize.STRING
},
lastName: {
type: Sequelize.STRING
}
}, {
// disable the modification of table names; By default, sequelize will automatically
// transform all passed model names (first parameter of define) into plural.
// if you don't want that, set the following
freezeTableName: true,
});

There is another interesting way you can avoid this. But you need to really focus on this way of implementation.
const User = sequelize.define("user", {
firstname: {
type: Sequelize.STRING
},
lastname: {
type: Sequelize.STRING
}
});
you intentionally put user here and use users in other places of coding(Assume sequelize will automatically transform all passed model names (first parameter of define) into plural) . This way of coding will simplify your code.

This problem occurs because creating a table is an asynchronous function. The problem is, the findAll() function can get executed while the table has not been created.
to solve this, you can use:
(async ()=>{
await User.sync({force: true});
// Table created
const users=await User.findAll();
console.log(users);
})();

The problem, in my case, was that the table users was not created. You can create the table manually with CREATE TABLE IF NOT EXISTS (SQL) or add the tableName = "users" in the options object:
export const User = db.define('user',
{
id: {
type: DataTypes.UUIDV4,
autoIncrement: true,
primaryKey: true,
},
name: {
type: new DataTypes.STRING(128),
allowNull: false,
},
email: {
type: new DataTypes.STRING(128),
allowNull: true,
},
password: {
type: new DataTypes.STRING(128),
allowNull: true,
},
},
{
freezeTableName: true,
tableName: "users"
}
);

Run that code twice.
Before running the second time, comment out the following code,
// force: true will drop the table if it already exists
User.sync({force: true}).then(() => {
// Table created
return User.create({
firstName: 'John',
lastName: 'Hancock'
});
});

Maybe answer is not entirely connected with you question but I want to describe my experience with this error
Error: relation "users" does not exist.
It appears Sequelize make migrations based on migrations file names and it alphabetical order. My problem was my files naming was not sorted in order to create proper connections.
If you face with this problem make sure yours migration files are fired in proper (in alphabetically) order.
The proper order is to first migrate table without connections (eg. table_A) and then tables with connections to table_A.
As I said this may not be answer for your particular order but I want to share my experiences because I didn't find this information on the internet when I was looking for this error.

Simply append tableName: "Users" to your model configuration.
The easiest way I found to solve, is to explicitly set the tableName on the model. As others have mentioned, sequelize defaults to the plural form of a model as the table name. For instance User, becomes Users.
When you query, sequelize looks after a table with the same name as your model User. By defining the tableName in the model, sequelize should search the correct table. Append tableName: "Users" to your model configuration i.e:
User.init(
{
email: DataTypes.STRING,
password: DataTypes.STRING,
role: DataTypes.INTEGER,
},
{
sequelize,
modelName: 'User',
tableName: 'Users',
}
);

If you want Sequelize to use a singular word ('info') for a model and that same singular word for the table ('info'), you can name the model 'info' and also add tablename: 'info' to the definition of your model.
This is a way to control Sequelize's default behavior of pluralizing model names, or not, on a table-by-table basis.
info.js
module.exports = (sequelize, DataTypes) => {
const info = sequelize.define('info', {
firstname: DataTypes.STRING,
email: DataTypes.STRING,
phone: DataTypes.STRING,
}, {
tableName: 'info'
});
return info;
};

Sequelize Populate Relation Query Either Table

I have the following two tables in Sequelize
const Tokens = sequelize.define("Tokens", {
id: {
type: DataTypes.UUID,
defaultValue: DataTypes.UUIDV4,
primaryKey: true
},
active: {
type: DataTypes.BOOLEAN
}
});
and
const User = sequelize.define("Users", {
id: {
type: DataTypes.UUID,
defaultValue: DataTypes.UUIDV4,
primaryKey: true
},
first_name: {
type: DataTypes.STRING
}
});
User.associate = models => {
models["Users"].hasMany(models["Tokens"], {foreignKey: 'userID', as: 'tokens_userid'});
};
I'm trying to run the following query in Sequelize.
const token = await db.Tokens.findOne({
where: {
id,
active: true
},
include: ["tokens_userid"]
});
But I'm getting the following error.
Error: Association with alias "tokens_userid" does not exists
My main goal is to get the user based on a Token ID. Now I would just move that association to the User table, but the problem with that later on I will want to get all the tokens for a given User ID. So I will run into this problem either way.
I tried adding the following line, but it was complaining about circular relations or something like that.
models["Tokens"].hasOne(models["User"], {foreignKey: 'userID', as: 'tokens_userid'});
How can I query either the Users or Tokens table and have it populate correctly with the relation?

I was able to solve this by adding the following line to my table.
models["Tokens"].belongsTo(models["User"], {foreignKey: 'userID', as: 'tokens_userid_from_token'});
Basically what I tried before but changed hasOne to belongsTo.
Hopefully this helps someone else.

How do I ORM additional columns on a join table in sequelize?

I'm using node v9.5, sequelize v4.33 (postgres dialect).
I have two first-class models: Driver (specific people) and Car (generic make+model combinations). Thus far, they've been connected by a many-to-many join table. Now I want to start tracking additional properties on that join table, but am having trouble declaring these relationships so they actually work.
const Driver = sqlz.define('Driver', {
id: { primaryKey: true, type: DataTypes.UUID },
name: DataTypes.string
})
const Car = sqlz.define('Car', {
id: { primaryKey: true, type: DataTypes.UUID },
make: DataTypes.string,
model: DataTypes.string
})
// old associations; worked great when requirements were simpler
Driver.belongsToMany(Car, {
through: 'DriverCar',
as: 'carList',
foreignKey: 'driverId'
})
Car.belongsToMany(Driver, {
through: 'DriverCar',
as: 'driverList',
foreignKey: 'carId'
})
Now I want to begin tracking more information about the relationship between a car and its driver, like the color of that specific car.
Step 1: I update the migration script, adding a new column to the join table like so:
queryInterface.createTable( 'DriverCar', {
driverId: {
type: sqlz.UUID,
allowNull: false,
primaryKey: true,
references: {
model: 'Driver',
key: 'id'
}
},
carId: {
type: sqlz.UUID,
allowNull: false,
primaryKey: true,
references: {
model: 'Car',
key: 'id'
}
},
createdAt: {
type: sqlz.DATE,
allowNull: false
},
updatedAt: {
type: sqlz.DATE,
allowNull: false
},
// new column for join table
color: {
type: Sequelize.STRING
}
})
Step 2: I define a new sqlz model for DriverCar:
const DriverCar = sqlz.define('DriverCar', {
color: DataTypes.string
})
(I assume I only need to define the interesting properties, and that driverId and carId will still be inferred from the associations that will be defined.)
Step 3: I need to update the associations that exist among Driver, Car, and DriverCar.
This is where I'm stuck. I have attempted updating the existing associations, like so:
Driver.belongsToMany(Car, {
through: DriverCar, // NOTE: no longer a string, but a reference to new DriverCar model
as: 'carList',
foreignKey: 'driverId'
})
Car.belongsToMany(Driver, {
through: DriverCar, // NOTE: no longer a string, but a reference to new DriverCar model
as: 'driverList',
foreignKey: 'carId'
})
This executes without error, but the new color property is not fetched from the join table when I try driver.getCarList(). (Sqlz is configured to log every SQL statement, and I have verified that no properties from the join table are being requested.)
So, instead, I tried spelling out this relationship more explicitly, by associating Driver to DriverCar, and then Car to DriverCar:
// Driver -> Car
Driver.hasMany(DriverCar, {
as: 'carList',
foreignKey: 'driverId'
})
// Car -> Driver
Car.hasMany(DriverCar, {
foreignKey: 'carId'
})
I also tell sqlz that DriverCar won't have a standard row id:
DriverCar.removeAttribute('id')
At this point, requesting a Driver's carList (driver.getCarList()) seems to work, because I can see join table props being fetched in SQL. But saving fails:
driverModel.setCarList([ carModel1 ])
UPDATE DriverCar
SET "driverId"='a-uuid',"updatedAt"='2018-02-23 22:01:02.126 +00:00'
WHERE "undefined" in (NULL)
The error:
SequelizeDatabaseError: column "undefined" does not exist
I assume this error is occurring because sqzl doesn't understand the proper way to identify rows in the join table, because I've failed to establish the necessary associations. And frankly, I'm not confident I've done this correctly; I'm new to ORMs, but I was expecting I'd need to specify 4 assocations:
Driver -> DriverCar
DriverCar -> Car
Car -> DriverCar
DriverCar -> Driver
To recap: I have 2 first-class entities, joined in a many-to-many relationship. I'm trying to add data to the relationship, have discovered that the ORM requires defining those associations differently, and am having trouble articulating the new associations.

A note about your aliases
Before going to the answer, I would like to point out that your choice of aliases (carList and driverList) could be better, because although the auto-generated sequelize methods .setCarList() and .setDriverList() do make sense, the methods .addCarList(), .addDriverList(), .removeCarList() and .removeDriverList() are nonsense, since they take only a single instance as a parameter, not a list.
For my answer, I won't use any aliases, and let Sequelize default to .setCars(), .setDrivers(), .addCar(), .removeCar(), etc, which make much more sense to me.
Example of working code
I've made a 100% self-contained code to test this. Just copy-paste it and run it (after running npm install sequelize sqlite3):
const Sequelize = require("sequelize");
const sequelize = new Sequelize({ dialect: 'sqlite', storage: 'db.sqlite' });
const Driver = sequelize.define("Driver", {
name: Sequelize.STRING
});
const Car = sequelize.define("Car", {
make: Sequelize.STRING,
model: Sequelize.STRING
});
const DriverCar = sequelize.define("DriverCar", {
color: Sequelize.STRING
});
Driver.belongsToMany(Car, { through: DriverCar, foreignKey: "driverId" });
Car.belongsToMany(Driver, { through: DriverCar, foreignKey: "carId" });
var car, driver;
sequelize.sync({ force: true })
.then(() => {
// Create a driver
return Driver.create({ name: "name test" });
})
.then(created => {
// Store the driver created above in the 'driver' variable
driver = created;
// Create a car
return Car.create({ make: "make test", model: "model test" });
})
.then(created => {
// Store the car created above in the 'car' variable
car = created;
// Now we want to define that car is related to driver.
// Option 1:
return car.addDriver(driver, { through: { color: "black" }});
// Option 2:
// return driver.setCars([car], { through: { color: "black" }});
// Option 3:
// return DriverCar.create({
// driverId: driver.id,
// carId: car.id,
// color: "black"
// });
})
.then(() => {
// Now we get the things back from the DB.
// This works:
return Driver.findAll({ include: [Car] });
// This also works:
// return car.getDrivers();
// This also works:
// return driver.getCars();
})
.then(result => {
// Log the query result in a readable way
console.log(JSON.stringify(result.map(x => x.toJSON()), null, 4));
});
The code above logs as expected (as I would expect, at least):
[
{
"id": 1,
"name": "name test",
"createdAt": "2018-03-11T03:04:28.657Z",
"updatedAt": "2018-03-11T03:04:28.657Z",
"Cars": [
{
"id": 1,
"make": "make test",
"model": "model test",
"createdAt": "2018-03-11T03:04:28.802Z",
"updatedAt": "2018-03-11T03:04:28.802Z",
"DriverCar": {
"color": "black",
"createdAt": "2018-03-11T03:04:28.961Z",
"updatedAt": "2018-03-11T03:04:28.961Z",
"driverId": 1,
"carId": 1
}
}
]
}
]
Note that there is no secret. Observe that the extra attribute that you're looking for, color, comes nested in the query result, not in the same nesting level of the Car or Driver. This is the correct behavior of Sequelize.
Make sure you can run this code and get the same result I do. My version of Node is different but I doubt that could be related to anything. Then, compare my code to your code and see if you can figure out what is causing you problems. If you need further help, feel free to ask in a comment :)
A note about many-to-many relationships with extra fields
Since I stumbled myself upon problems with this, and this is related to your situation, I thought I should add a section in my answer alerting you to the "trap" of setting up an overcomplicated many-to-many relationship (it's a lesson that I learned myself after struggling for a while).
Instead of repeating myself, I will just add a brief quote of what I said in Sequelize Issue 9158, and add links for further reading:
Junction tables, the tables that exist in relational databases to represent many-to-many relationships, initially have only two fields (the foreign keys of each table defining the many-to-many relationship). While it's true that it's possible to define extra fields/properties on that table, i.e. extra properties for the association itself (as you put in the issue title), care should be taken here: if it's getting overcomplicated, it's a sign that you should "promote" your junction table to a full-fledged entity.
Further reading:
My own self-answered question involving an overcomplicated setup of many-to-many relationships in sequelize: FindAll with includes involving a complicated many-to-(many-to-many) relationship (sequelizejs)
And its sibling question: Is it OK to have a many-to-many relationship where one of the tables involved is already a junction table?

How to add a property to an entity dynamically?

How do you add a property to an entity dynamically? I've been looking, but haven't found anything.
For example, I have this model definition (I'm using the WebSQL provider):
$data.Entity.extend('$db.Types.Person', {
id: { type: 'int', key: true, computed: true },
name: { type: 'string' }
});
$data.EntityContext.extend('$db.Types.DBContext', {
Persons: { type: $data.EntitySet, elementType: $db.Types.Person},
});
At some point I need to extend my model with new properties. Initially I don't know these properties' names.

The syntax is very simple for this, but the background info is more important, please read the whole answer before you reuse the snippet.
The YourType can be extended with new fields using the YourType.addMember() function. See this example snippet:
$data.Entity.extend('Product', {
id: { type: 'int', key: true, computed: true },
Name: { type: 'string' }
});
$data.EntityContext.extend('Northwind', {
Products: { type: $data.EntitySet, elementType: Product},
});
Product.addMember('Description', {
type:'string',
key: false,
computed: false,
required: false
});
var context = new Northwind({provider: 'webSql', databaseName: 'Northwind'});
context.onReady(function() {
var product1 = new Product({ Name: 'Beer', Description: 'tasty'});
context.Products.add(product1);
context.saveChanges(function(result) {
//check the content of WebSQL DB
console.log(product1);
});
});
You can user the addMember() only before creating an instance of the context.
Important info:
There is no data migration/merge by in the library, and the default behavior on schema modification for webSql is to drop&re-create the DB. As IndexedDB isn't bound to a schema, the existing records won't be dropped. Make a try by running this code and adding more fields, here is a working JSFiddle.
The real solution is to use Schema Evolution module of JayData Pro to manage the changes in your data model.