We Keep Coding

How to handle circular dependencies in mongoose?

I have the below code in my application
const citySchema = new Schema({
cityName: {
type: String,
required: true,
},
citizen:[{
type: Schema.Types.ObjectId,
ref: "Citizen",
}],
});
module.exports = mongoose.model("City", citySchema);
const citizenSchema = new Schema({
citizenName: {
type: String,
required: true,
},
city:{
type: Schema.Types.ObjectId,
ref: "City",
},
});
module.exports = mongoose.model("Citizen", citizenSchema);
router.post('/', (req, res) => {
// req.body.cityName
// req.body.citizenName
})
In my POST request , I receive both city name(new city) and citizen name(new citizen) that are not present . But I want both these schemas to be updated properly .
City should contain Citizen references
Citizen should contain City reference
How can I do that ? Kindly help

Rather than doing that, I think you'll be much better to apply referencing via pre-hook middleware in your data model.
The code should be like this:
const citySchema = new Schema({
cityName: {
type: String,
required: true,
},
citizen:[{
type: Schema.Types.ObjectId,
ref: "Citizen",
}],
});
// Query middleware to populate the 'citizen' attribute every time the 'find' function is called.
citySchema.pre(/^find/, function (next) {
this.populate('citizen');
next();
});
module.exports = mongoose.model("City", citySchema);
const citizenSchema = new Schema({
citizenName: {
type: String,
required: true,
},
city:{
type: Schema.Types.ObjectId,
ref: "City",
},
});
citizenSchema.pre(/^find/, function (next) {
this.populate('city');
next();
});
module.exports = mongoose.model("Citizen", citizenSchema);
If you want to select just the ID and not the 'full data', you could do it like this for example:
citizenSchema.pre(/^find/, function (next) {
this.populate({
path: 'city',
select: '_id',
});
next();
});
Explanation:
By doing this, every time you call Mongoose's functions like findByIdAndUpdate, find, findOne, the referenced data will appear in the city and citizen attribute. This is actually more efficient rather than updating each time there is a new data.
The populate method is used to populate the attributes with data from another data model.
The object that I inserted into the populate method is used to get the 'name' of the model (in the path), and to select what kind of data to take from the referenced model. In this case, I only want to take the _id attribute.

How to use a mongoose model in another model?

I have the two following models. In the user model I want to use an array of Requests and in Request Model I want to use User as an attribute(without the password). How can I do it?
var userSchema = new Schema({
cartaoCidadao: {
type: String,
required: true,
index: {
unique: true,
},
match: /[0-9]{8}/,
},
password: { type: String, required: true },
role: { type: String },
estado: { type: String, enum: ["Infetado", "Suspeito"] },
});
var requestSchema = new Schema({
encaminhado: { type: String },
pessoaRisco: { type: String },
trabalhoRisco: { type: String },
estadoPedido: { type: String },
resultado: { type: String },
});

You can use the schema you defined as a type itself:
var userSchema = new Schema({
// ...
requests: {
type: [requestSchema] // this property type is: array of requests
}
// ...
});
If both models are stored in database and you probably want to go for their association. You can reference one model from another. (see the answer of Muhammad Lahin)
Then you query your parent model and associate the children models with it (https://mongoosejs.com/docs/populate.html)
And here is an example of how you can exclude some fields during the population:
https://mongoosejs.com/docs/populate.html#query-conditions
It will be something like:
User.
find(/* some query */).
populate({
path: 'requests',
select: 'fieldToSelect1 fieldToSelect2' // You can control which fields to include
}).
exec();

you can do something like this
var userSchema = new Schema({
requests: [
{
type: Schema.Types.ObjectId,
ref: "Request",
}
]
});
var requestSchema = new Schema({
user: {
type: Schema.Types.ObjectId,
ref: "User"
}
})

you can set the type of the user in the request model like this
type: schema.Types.ObjectID,
ref: 'users'
which users id schema of users and when you want to send the whole user data in the response, use populate and in populate you can omit the password too
now if you see the mondodb database in the request database the whole user object is not saved and only the user_id is saved so you dont have access to the user. you have to send another request to get the user data from the gived user_id which is not good. the solution is you set the schema as I said and when you are sending a response which have the data of the request use populate to add the userdata to the response
res.json({request: requests.find(//options to find a specified request).populate('users','name')})
first argument is the model which is users and the second argument is the wanted field of the users model for example this will return the name of the user. you can add more arguments to add all of them except the password

const mongoose = require('mongoose')
const productSchema = new mongoose.Schema({
user: {
type: mongoose.Types.ObjectId,
ref: 'User'
}
});

Mongoose deleteMany subdocuments and related subdocuments

I have a document Project with an array of subdocuments, with a schema Tasks. Tasks has an array of subdocuments with a schema Comments.
const projectSchema = new Schema({
_id: Schema.Types.ObjectId,
name: { type: String, required: true, unique: true },
description: { type: String, default: '' },
tasks: [{ type: Schema.Types.ObjectId, ref: 'Threads' }]
});
module.exports = mongoose.model('Project', projectSchema);
const tasksSchema = new Schema({
projectId: { type: Schema.Types.ObjectId },
_id: Schema.Types.ObjectId,
title: { type: String, required: true },
text: { type: String, required: true },
comments: [{ type: Schema.Types.ObjectId, ref: 'Replies' }]
})
module.exports = mongoose.model('Tasks', tasksSchema);
const commentSchema = new Schema({
taskId: { type: Schema.Types.ObjectId },
_id: Schema.Types.ObjectId,
text: { type: String, required: true }
})
module.exports = mongoose.model('Comment', commentSchema);
When I delete the Project document I want to delete every Task and every Comment relate to that project.
To delete the Project I use findOneAndDelete so I set up a post middleware to delete all the Tasks
projectSchema.post('findOneAndDelete', function(doc, next) {
mongoose.model('Tasks').deleteMany({ projectId: doc._id }).exec();
next();
})
But now I don’t know how to delete every comment, because deletemany returns an object with the result of the operation.
Should I map the array of Tasks and call findOneAndDelete every time and then delete every single comment? It looks very inefficient for a lot of tasks.

How about embedding comments in post? since its one to many(not huge) relation. So in your code where you delete a project, you first delete all posts, which contain all the comments, only after it succeeds you delete the project. It will also benefit your read performance significantly because you just have to return a single post document instead of multiple(1post + many comment) documents.
Embedding post to project could also be possible, but depending on the size and number of possible posts, its probably better to keep it as a separate document.
In this case you need some logic to ensure consistency.
Here you could use mongodb's new feature, transaction. But I think for this case a transaction is not necessary.(Also I find it quite unstable for now) You could go with the "eventual consistency" method.
Basically you just delete all the posts related to a project and then delete a project. And then you run batches to check for any inconsistency.(check if there are any posts where its project doesnt exist. If it doestnt then delete the posts)

How to Populate nested queries

I have a typical Project with Node.js - Express - Mongoose - MongoDB
My goal is to generate a News feed by calling all the recent articles of all the users that the main User follows.
User Model
var UserSchema = new Schema({
username: { type: String, required: true, unique: true },
password: { type: String, required: true },
email: String,
articles : [{ type: Schema.Types.ObjectId, ref: 'Article' }],
follows: [{ type: Schema.Types.ObjectId, ref: 'User' }],
followers: [{ type: Schema.Types.ObjectId, ref: 'User' }]
});
Article Model
var articleSchema = new Schema({
_userID: { type: Schema.Types.ObjectId, ref: 'user' },
url: String,
summary: String,
title: String,
content: String,
image_url: String,
});
The Problem:
I am trying to write a mongoose query that does the following
Search by userID and get a list of follows
populate those followers recent articles
I can figure out how to populate the follows but I do not know how to then also populate the follows's respected articles and return them in chronologically order (recent > oldest)
exports.getFeed = function(req, res) {
var id = mongoose.Types.ObjectId(req.params.id);
// mongoose query goes here
};
Any help would be greatly appreciated.

You'd need two queries.
The first query is the one you already figured out and need anyway (get the list of follows).
You basically need an array of the follows IDs and then you ask for all articles from this user array:
.where('_userID').in(arrayOfFollowIDs)
Hope that helps

Relational database design to mongoDB/mongoose design

I have recently started using mongoDB and mongoose for my new node.js application. Having only used relational databases before I am struggling to adapt to the mongoDB/noSQL way of thinking such as denormalization and lack of foreign key relationships. I have this relational database design:
**Users Table**
user_id
username
email
password
**Games Table**
game_id
game_name
**Lobbies Table**
lobby_id
game_id
lobby_name
**Scores Table**
user_id
game_id
score
So, each lobby belongs to a game, and multiple lobbies can belong to the same game. Users also have different scores for different games. So far for my user schema I have the following:
var UserSchema = new mongoose.Schema({
username: {
type: String,
index: true,
required: true,
unique: true
},
email: {
type: String,
required: true
},
password: {
type: String,
required: true
}
});
So my question is, how would I go about structing the relational design into mongoDB/mongoose schemas? Thanks!
EDIT 1
I have now tried to create all the schemas but I have no idea if this is the right approach or not.
var UserSchema = new mongoose.Schema({
_id: Number,
username: {
type: String,
index: true,
required: true,
unique: true
},
email: {
type: String,
required: true
},
password: {
type: String,
required: true
},
scores: [{ type: Schema.Types.ObjectId, ref: 'Score' }]
});
var GameSchema = new mongoose.Schema({
_id: Number,
name: String
});
var LobbySchema = new mongoose.Schema({
_id: Number,
_game: { type: Number, ref: 'Game' },
name: String
});
var ScoreSchema = new mongoose.Schema({
_user : { type: Number, ref: 'User' },
_game : { type: Number, ref: 'Game' },
score: Number
});

Mongoose is designed in such a way that you can model your tables relationally with relative ease and populate relational data based on the ref you defined in the schema. The gotcha is that you need to be careful with populating. If you populate too much or nest your populations you will run into performance bottle necks.
Your approach in Edit 1 is largely correct however you usually don't want to populate a remote ref based on a Number or set the _id of a model to a Number since mongo uses it's own hashing mechanism for managing the _id, this would usually be an ObjectId with _id implied. Example as shown below:
var ScoreSchema = new mongoose.Schema({
user : { type: Schema.Types.ObjectId, ref: 'User' },
game : { type: Schema.Types.ObjectId, ref: 'Game' },
score: Number
});
If for some reason you need to maintain a number id for your records consider calling it uid or something that won't conflict with mongo / mongoose internals. Good luck!

First of all, you are hitting on some good points here. The beauty of Mongoose is that you can easily connect and bind schemas to a single collection and reference them in other collections, thus getting the best of both relational and non-relational DBs.
Also, you wouldn't have _id as one of you properties, Mongo will add it for you.
I've made some changes to your schemas using the mongoose.Schema.Types.ObjectId type.
var UserSchema = new mongoose.Schema({
username: {
type: String,
index: true,
required: true,
unique: true
},
email: {
type: String,
required: true
},
password: {
type: String,
required: true
},
scores: [{ type: Schema.Types.ObjectId, ref: 'Score' }]
});
var GameSchema = new mongoose.Schema({
name: String
});
var LobbySchema = new mongoose.Schema({
_game: {
type: mongoose.Schema.Types.ObjectId,
ref: 'Game'
},
name: String
});
var ScoreSchema = new mongoose.Schema({
_user : {
type: mongoose.Schema.Types.ObjectId,
ref: 'User'
},
_game : {
type: mongoose.Schema.Types.ObjectId,
ref: 'Game'
},
score: Number
});
This will allow you to query your database and populate any referenced collections and objects.
For example:
ScoreSchema.find({_id:##userIdHere##})
.populate('_user')
.populate('_game')
.exec(function(err, foundScore){
if(err){
res.send(err)
} else {
res.send(foundScore)
}
}
This will populate the related user and game properties.

As you edited the post, I think it would be good. At least not bad :)
Check Mongoose Query Population. It's very useful to get related data.
var mongoose = require('mongoose'),
ObjectId = mongoose.Schema.Types.ObjectId
// code, code, code
function something(req, res) {
var id = req.params.id
// test id
return Lobby.findOne({_id: new ObjectId(id)})
.populate('_game')
.exec(function(error, lobby) {
console.log(lobby._game.name);
});
}

Two ways (that I know of). You store an id (that is indexed) and once you query the first table, you then query the second table to grab info from that, as there are no joins. This means that if you grab say, user id's from one table, you will then need to make multiple queries to the user table to get the user's data.
The other way is to store it all in one table, even if it's repetitive. If all you need to store is for example, a user's screen name with something else, then just store it with the other data, even if it's already in the user table. I'm sure others will know of better/different ways.