I have recently started using mongoDB and mongoose for my new node.js application. Having only used relational databases before I am struggling to adapt to the mongoDB/noSQL way of thinking such as denormalization and lack of foreign key relationships. I have this relational database design:
**Users Table**
user_id
username
email
password
**Games Table**
game_id
game_name
**Lobbies Table**
lobby_id
game_id
lobby_name
**Scores Table**
user_id
game_id
score
So, each lobby belongs to a game, and multiple lobbies can belong to the same game. Users also have different scores for different games. So far for my user schema I have the following:
var UserSchema = new mongoose.Schema({
username: {
type: String,
index: true,
required: true,
unique: true
},
email: {
type: String,
required: true
},
password: {
type: String,
required: true
}
});
So my question is, how would I go about structing the relational design into mongoDB/mongoose schemas? Thanks!
EDIT 1
I have now tried to create all the schemas but I have no idea if this is the right approach or not.
var UserSchema = new mongoose.Schema({
_id: Number,
username: {
type: String,
index: true,
required: true,
unique: true
},
email: {
type: String,
required: true
},
password: {
type: String,
required: true
},
scores: [{ type: Schema.Types.ObjectId, ref: 'Score' }]
});
var GameSchema = new mongoose.Schema({
_id: Number,
name: String
});
var LobbySchema = new mongoose.Schema({
_id: Number,
_game: { type: Number, ref: 'Game' },
name: String
});
var ScoreSchema = new mongoose.Schema({
_user : { type: Number, ref: 'User' },
_game : { type: Number, ref: 'Game' },
score: Number
});
Mongoose is designed in such a way that you can model your tables relationally with relative ease and populate relational data based on the ref you defined in the schema. The gotcha is that you need to be careful with populating. If you populate too much or nest your populations you will run into performance bottle necks.
Your approach in Edit 1 is largely correct however you usually don't want to populate a remote ref based on a Number or set the _id of a model to a Number since mongo uses it's own hashing mechanism for managing the _id, this would usually be an ObjectId with _id implied. Example as shown below:
var ScoreSchema = new mongoose.Schema({
user : { type: Schema.Types.ObjectId, ref: 'User' },
game : { type: Schema.Types.ObjectId, ref: 'Game' },
score: Number
});
If for some reason you need to maintain a number id for your records consider calling it uid or something that won't conflict with mongo / mongoose internals. Good luck!
First of all, you are hitting on some good points here. The beauty of Mongoose is that you can easily connect and bind schemas to a single collection and reference them in other collections, thus getting the best of both relational and non-relational DBs.
Also, you wouldn't have _id as one of you properties, Mongo will add it for you.
I've made some changes to your schemas using the mongoose.Schema.Types.ObjectId type.
var UserSchema = new mongoose.Schema({
username: {
type: String,
index: true,
required: true,
unique: true
},
email: {
type: String,
required: true
},
password: {
type: String,
required: true
},
scores: [{ type: Schema.Types.ObjectId, ref: 'Score' }]
});
var GameSchema = new mongoose.Schema({
name: String
});
var LobbySchema = new mongoose.Schema({
_game: {
type: mongoose.Schema.Types.ObjectId,
ref: 'Game'
},
name: String
});
var ScoreSchema = new mongoose.Schema({
_user : {
type: mongoose.Schema.Types.ObjectId,
ref: 'User'
},
_game : {
type: mongoose.Schema.Types.ObjectId,
ref: 'Game'
},
score: Number
});
This will allow you to query your database and populate any referenced collections and objects.
For example:
ScoreSchema.find({_id:##userIdHere##})
.populate('_user')
.populate('_game')
.exec(function(err, foundScore){
if(err){
res.send(err)
} else {
res.send(foundScore)
}
}
This will populate the related user and game properties.
As you edited the post, I think it would be good. At least not bad :)
Check Mongoose Query Population. It's very useful to get related data.
var mongoose = require('mongoose'),
ObjectId = mongoose.Schema.Types.ObjectId
// code, code, code
function something(req, res) {
var id = req.params.id
// test id
return Lobby.findOne({_id: new ObjectId(id)})
.populate('_game')
.exec(function(error, lobby) {
console.log(lobby._game.name);
});
}
Two ways (that I know of). You store an id (that is indexed) and once you query the first table, you then query the second table to grab info from that, as there are no joins. This means that if you grab say, user id's from one table, you will then need to make multiple queries to the user table to get the user's data.
The other way is to store it all in one table, even if it's repetitive. If all you need to store is for example, a user's screen name with something else, then just store it with the other data, even if it's already in the user table. I'm sure others will know of better/different ways.
Related
Currently working on a personal website for myself and ran in to some difficulties with mongodb & mongoose when trying to delete a user.
Right now I have two Schema's show below.
UserSchema:
const userSchema = new mongoose.Schema({
username: {
type:String,
required: true,
unique: true,
},
password: String,
firstName: String,
lastName: String,
eMail: String,
status: {type:Boolean, default: true},
hobbies: [
{
type: mongoose.Schema.Types.ObjectId,
ref: "Hobby"
}
]
HobbySchema:
const hobbySchema = new mongoose.Schema({
title: String,
user: {
id: {
type: mongoose.Schema.Types.ObjectId,
ref: "User"
},
username: String
},
desc: String,
iconURL: String
});
My issue right now is that when I delete the user using the following route with nodeJS:
router.delete('/delete/:id', async (req,res) => {
await User.findOneAndDelete(req.params.id, (err, foundUser) => {
if(err){
console.log(err);
}else{
res.redirect("/logout");
}
})
});
I run into the issue that the Hobby collection still contains all hobbies that the user created. I'm looking to have them all deleted when the user wants to delete their account.I understand that Mongodb/Mongoose is a non-relation database and is unlike SQL as I'm coming from SQL. Is there any way to delete all hobbies a user created? (Just creating hobbies as a very basic example).
What is the best alternative? Just going with SQL lite and changing the database entirely?
Thanks
I have the two following models. In the user model I want to use an array of Requests and in Request Model I want to use User as an attribute(without the password). How can I do it?
var userSchema = new Schema({
cartaoCidadao: {
type: String,
required: true,
index: {
unique: true,
},
match: /[0-9]{8}/,
},
password: { type: String, required: true },
role: { type: String },
estado: { type: String, enum: ["Infetado", "Suspeito"] },
});
var requestSchema = new Schema({
encaminhado: { type: String },
pessoaRisco: { type: String },
trabalhoRisco: { type: String },
estadoPedido: { type: String },
resultado: { type: String },
});
You can use the schema you defined as a type itself:
var userSchema = new Schema({
// ...
requests: {
type: [requestSchema] // this property type is: array of requests
}
// ...
});
If both models are stored in database and you probably want to go for their association. You can reference one model from another. (see the answer of Muhammad Lahin)
Then you query your parent model and associate the children models with it (https://mongoosejs.com/docs/populate.html)
And here is an example of how you can exclude some fields during the population:
https://mongoosejs.com/docs/populate.html#query-conditions
It will be something like:
User.
find(/* some query */).
populate({
path: 'requests',
select: 'fieldToSelect1 fieldToSelect2' // You can control which fields to include
}).
exec();
you can do something like this
var userSchema = new Schema({
requests: [
{
type: Schema.Types.ObjectId,
ref: "Request",
}
]
});
var requestSchema = new Schema({
user: {
type: Schema.Types.ObjectId,
ref: "User"
}
})
you can set the type of the user in the request model like this
type: schema.Types.ObjectID,
ref: 'users'
which users id schema of users and when you want to send the whole user data in the response, use populate and in populate you can omit the password too
now if you see the mondodb database in the request database the whole user object is not saved and only the user_id is saved so you dont have access to the user. you have to send another request to get the user data from the gived user_id which is not good. the solution is you set the schema as I said and when you are sending a response which have the data of the request use populate to add the userdata to the response
res.json({request: requests.find(//options to find a specified request).populate('users','name')})
first argument is the model which is users and the second argument is the wanted field of the users model for example this will return the name of the user. you can add more arguments to add all of them except the password
const mongoose = require('mongoose')
const productSchema = new mongoose.Schema({
user: {
type: mongoose.Types.ObjectId,
ref: 'User'
}
});
I have a document Project with an array of subdocuments, with a schema Tasks. Tasks has an array of subdocuments with a schema Comments.
const projectSchema = new Schema({
_id: Schema.Types.ObjectId,
name: { type: String, required: true, unique: true },
description: { type: String, default: '' },
tasks: [{ type: Schema.Types.ObjectId, ref: 'Threads' }]
});
module.exports = mongoose.model('Project', projectSchema);
const tasksSchema = new Schema({
projectId: { type: Schema.Types.ObjectId },
_id: Schema.Types.ObjectId,
title: { type: String, required: true },
text: { type: String, required: true },
comments: [{ type: Schema.Types.ObjectId, ref: 'Replies' }]
})
module.exports = mongoose.model('Tasks', tasksSchema);
const commentSchema = new Schema({
taskId: { type: Schema.Types.ObjectId },
_id: Schema.Types.ObjectId,
text: { type: String, required: true }
})
module.exports = mongoose.model('Comment', commentSchema);
When I delete the Project document I want to delete every Task and every Comment relate to that project.
To delete the Project I use findOneAndDelete so I set up a post middleware to delete all the Tasks
projectSchema.post('findOneAndDelete', function(doc, next) {
mongoose.model('Tasks').deleteMany({ projectId: doc._id }).exec();
next();
})
But now I don’t know how to delete every comment, because deletemany returns an object with the result of the operation.
Should I map the array of Tasks and call findOneAndDelete every time and then delete every single comment? It looks very inefficient for a lot of tasks.
How about embedding comments in post? since its one to many(not huge) relation. So in your code where you delete a project, you first delete all posts, which contain all the comments, only after it succeeds you delete the project. It will also benefit your read performance significantly because you just have to return a single post document instead of multiple(1post + many comment) documents.
Embedding post to project could also be possible, but depending on the size and number of possible posts, its probably better to keep it as a separate document.
In this case you need some logic to ensure consistency.
Here you could use mongodb's new feature, transaction. But I think for this case a transaction is not necessary.(Also I find it quite unstable for now) You could go with the "eventual consistency" method.
Basically you just delete all the posts related to a project and then delete a project. And then you run batches to check for any inconsistency.(check if there are any posts where its project doesnt exist. If it doestnt then delete the posts)
I am trying to create an event list where users can add and remove themselves from events and specify if they are bringing guests with them to that event.
So I have an event schema and a user schema, where the event schema is referencing the user schema. So when a new event is created users can add themselves to that event with their ids.
Now I'm trying to make it so that users can also include guests. How Do I achieve that?
Here's an example
User Schema:
let UserSchema = new mongoose.Schema({
email: {
type: String,
lowercase: true,
unique: true,
required: true
},
name:{
type: String,
require: true
},
password: {
type: String,
required: true
},
...
Event Schema:
let EventSchema = new mongoose.Schema({
date: {
type: Date,
unique: true,
timestamps: true,
required: true
},
title: {
type: String,
require: true
},
// Guest property is ignored
attending: [{
type: mongoose.Schema.Types.ObjectId,
guest: Number, //This is being ingored and never updated
ref: 'User'
}]
})
Second way of defining the relavant part in the schema:
...
//In this example the guest will be added but duplicates will occur
user:[{
guest: Number, // To make it clear this is not referencing anything it's just a number
attending: {
type: mongoose.Schema.Types.ObjectId,
ref: 'User'
}
}]
How do I update the type and guest properties with addToSet (to prevent duplication) in the above configuration?
Event.findOneAndUpdate({_id:req.body.eventId}, query)
I don't think you understand how mongoose schemas work, you might want to spend some more time on their documentation.
What you have provided as code is what appears to be a field called Events in your Schema which is an array of objects, each object of which has a single field called attending, which itself is required to be an ObjectId type and reference the 'User' collection. There is also a guest property on the field definition which will be ignored by Mongoose as it doesn't understand what you're asking for.
Realize that what this data structure is, is instructions to Mongoose on how to validate and persist your data. It won't generally be updated at runtime for most applications and will not store data directly, again its purpose is to give clues to Mongoose as to how you want the data stored.
/** Edit based on comments and updated question **/
As I said before, you can't directly embed another field into the definition of a field. What you can do is create a mixed type which has both pieces of information, but that will require you to manage things yourself to some degree.
let EventSchema = new mongoose.Schema({
date: {
type: Date,
unique: true,
timestamps: true,
required: true
},
title: {
type: String,
require: true
},
attendees: [{
user : {
type: mongoose.Schema.Types.ObjectId,
ref: 'User'
},
guests : Number
}]
})
Anytime anyone is added to the attending list, you'll need to call event.markModified() to make sure it gets saved. If you don't want to allow duplicate users, you'll also need to check that. One way to make sure that happens is to populate() that field when you fetch the event, then just check locally for matches.
/** Edit #2 **/
You can also explicitly create another schema to 'hold' your user and # guests information, which will then create models that Mongoose will watch, and you can apply validation to them via normal Mongoose methods and not worry about dirty checking. That'd look like this:
// in ./models/attendee.js
let AttendeeSchema = new Schema({
user : {
type: mongoose.Schema.Types.ObjectId,
ref: 'User',
unique : true
},
guests : Number
}
mongoose.model('Attendee', AttendeeSchema);
// in your Events definition
let Attendee = mongoose.model('Attendee');
let EventSchema = new mongoose.Schema({
date: {
type: Date,
unique: true,
timestamps: true,
required: true
},
title: {
type: String,
require: true
},
attendees: [Attendee]
})
/** Edit 3: Now, with queries **/
To insert a new attendee, given an existing event and a known user:
event.attendees.push(new Attendee({user: user, guests: 5}));
event.save(console.log);
To update an existing attendee, you'll need to find the one you're looking for first:
let attendee = event.attendees.find((attendee) => { return attendee._id.toString() === user._id.toString(); });
attendee.guests = 10;
event.save(console.log);
I have a typical Project with Node.js - Express - Mongoose - MongoDB
My goal is to generate a News feed by calling all the recent articles of all the users that the main User follows.
User Model
var UserSchema = new Schema({
username: { type: String, required: true, unique: true },
password: { type: String, required: true },
email: String,
articles : [{ type: Schema.Types.ObjectId, ref: 'Article' }],
follows: [{ type: Schema.Types.ObjectId, ref: 'User' }],
followers: [{ type: Schema.Types.ObjectId, ref: 'User' }]
});
Article Model
var articleSchema = new Schema({
_userID: { type: Schema.Types.ObjectId, ref: 'user' },
url: String,
summary: String,
title: String,
content: String,
image_url: String,
});
The Problem:
I am trying to write a mongoose query that does the following
Search by userID and get a list of follows
populate those followers recent articles
I can figure out how to populate the follows but I do not know how to then also populate the follows's respected articles and return them in chronologically order (recent > oldest)
exports.getFeed = function(req, res) {
var id = mongoose.Types.ObjectId(req.params.id);
// mongoose query goes here
};
Any help would be greatly appreciated.
You'd need two queries.
The first query is the one you already figured out and need anyway (get the list of follows).
You basically need an array of the follows IDs and then you ask for all articles from this user array:
.where('_userID').in(arrayOfFollowIDs)
Hope that helps