We Keep Coding

Filtering a query using Node JS and MongoDB

I am new to MangoDB and Node JS. I have always worked on SQL databases and I do not know the syntax of MongoDB well. I wanna try to filter the array that I receive from a MongoDB database. I know that JavaScript has a .filter() function to filter just the results that contain a string. Is it best practice to get all the objects from MongoDB and filter in Node or do I let MongoDB do the filtering?
My Node.JS project is a back-end project using Node.JS and Express to do CRUD operations on a MongoDB database. In the request I send a parameter called 'equalTo' that contains the value that should be filtered on.
var router = express.Router();
var Plot = require("./models/plot");
...
router.get("/plots", (req, res) => {
let query = "" + req.query.equalTo;
Plot.find((err, plots) => {
if (err) {
res.send(err);
}
res.json(plots);
});
});
The filtering should be an OR filter where all results where either the name or the cropName should CONTAIN the value of the string. If it is possible I would also like the comparison to ignore uppercase's. Here is a schema for the Plot object:
const plotSchema = mongoose.Schema({
area: {
type: String,
required: true
},
comments: {
type: String,
required: true
},
cropGroupName: {
type: String,
required: true
},
cropName: {
type: String,
required: true
},
name: {
type: String,
required: true
},
plotId: {
type: Number,
required: true
},
coords: {
type: [],
required: true
},
}, {
collection: "plots"
});

The format is the following:
Plot.find({$or:[{name: "anyname"},{cropName:"othername"}]})
For further information you can read here https://docs.mongodb.com/manual/reference/operator/query/or/
You may replace the strings above in your case with equalTo.

In mongoose, how to find records based on value in related collection?

In Mongoose, I have two collections, with one referencing the other. Is it possible to have a find query that selects records based on a value in the other. An example of what I am try to get at (not actual schemas):
const CarModelSchema = new mongoose.Schema({
name: String,
brand: { type: mongoose.Schema.Types.ObjectId, ref: 'CarBrand' }
});
const CarBrandSchema = new mongoose.Schema({
name: String,
country: String
});
I then want to perform a query of the form, without needing to do two queries:
CarModelSchema.find({ 'brand.country': 'GER' });
So far I haven't been able to make this work, so I am wondering whether this can be done in Mongo or whether I am approaching it wrong?

Yes it is possible.
I realize you don't have models for your schemas so add them like this:
const CarModel = mongoose.model('CarModel', CarModelSchema);
const CarBrand = mongoose.model('CarBrand', CarBrandSchema);
Also brands should be defined like this:
brand: [{ type: mongoose.Schema.Types.ObjectId, ref: 'CarBrand' }] //added the brackets
You can then run a find query to filter by country by doing the following:
CarModel.
find(...).
populate({
path: 'brand',
match: { country: { $eq: 'GER' }},
// You can even select the field you want using select like below,
select: 'name -_id',
//Even limit the amount of documents returned in the array
options: { limit: 5 }
}).
exec();
And that should do it, as long as the ObjectIds saved in brands array in the CarModel collection are valid or exist.

Using match in your population will do the work.
CarModel.find()
.populate({
path: 'brand',
model: CarBrandModel,
match: { country: { $eq: 'GER' }},
})
.exec()
Keep in mind you have to define CarModel and CarBrandModel like this:
const CarModel = mongoose.model('CarModel', CarModelSchema)
const CarBrandModel = mongoose.model('CarBrandModel', CarBrandSchema)

Yes, you are doing it wrong.
In CarModelSchema.brand there is not string saved, there is ObjectId saved, therefore you have to find that ObjectId (the reference).
You can do it manually - first finding the CarBrandSchema.find({ 'country': 'GER' }); and then use its ObjectId (=_id), or you can use https://mongoosejs.com/docs/populate.html to populate your CarModel with the CarBrand object.

Simple Mongodb embedded database structure

i want to build a Mongodb database (Mongoose/Node.js) structure but i face a problem right now. i have two entities. Users and Books and i want to use embedded system(because of lack of joins in mongodb). And my problem is that which of this entities shpuld be an inner value to other.
For example i wiil face this two type of query in my app:
1- Books of an specific user
2- Users of an specific book
Now, Books should be a inner value for Users or contrariwise?
i can do this two:
Users schema:
var schema = new mongoose.Schema({
use_name: String,
user_family: String,
user_books: { type: Schema.Types.ObjectId, ref: 'books' }
});
Or this:
Books Schema:
var schema = new mongoose.Schema({
book_name: String,
book_lang: String,
book_user: { type: Schema.Types.ObjectId, ref: 'users' }
});
which is better? which is standard approach?
if i use both of them, when saving i have to do two save operation. if i has a large database with a lots of collections its gets worse that this...
after a lot of research i find out i have to use embedded system rather that using relation like collections to connect entities to each other, because Mongodb doesn't support joins and has poor support of things like this. embedded system is the correct way for a NoSql database like Mongodb?

Firstly, there's a minor correction that your user_books needs to be an array [].
Secondly, you should only reference one schema into another, otherwise you'll have to add unnecessary complexity in keeping them in sync.
So here's what your schemas should look like:
var UserSchema = new mongoose.Schema({
use_name: String,
user_family: String,
user_books: [{ type: Schema.Types.ObjectId, ref: 'books' }]
});
var BookSchema = new mongoose.Schema({
book_name: String,
book_lang: String,
});
Now "to fetch users that reads an specific book", you'll query like this:
UserSchema.find({ user_books: book._id })
which will give you all users that have BOOK_ID as (at least) one of their books.
If that's all, I guess you don't need population at all then.
Updated on the issue with $elemMatch query not working:
So as it turned out, we don't actually need $elemMatch with referenced docs array, since it's a simple array of _ids.
user // =>
{
_id: 56351c611ca0d2e81274100a
name: ...
books: [56351c611ca0d2e81274100b, 56351c611ca0d2e81274100c, ...]
}
$elemMatch works with array of objects, and would've been in the case of embedded doc:
var BookSchema = new mongoose.Schema({
book_name: String,
book_lang: String,
});
var UserSchema = new mongoose.Schema({
use_name: String,
user_family: String,
user_books: [BookSchema]
});
Since in this case the document would be like this:
user // =>
{
_id: 56351c611ca0d2e81274100a
name: ...
books: [
{ // each book here has an _id:
_id: 56351c611ca0d2e81274100b,
// this is what `$elemMatch: {_id:` would match for
name: ...
// you could do `$elemMatch: {name:`
lang: ...
// or `$elemMatch: {lang:`
}, {
_id: 56351c611ca0d2e81274100c,
name: ...
lang: ...
}, ...
]
}
This is where $elemMatch query would be needed.
UserSchema.find({ user_books: {$elemMatch: {_id: book._id } } })

Query with OR conditions, mongodb/mognoose

This is (a part) of my model:
var materialSchema = new Schema({
ownerType: { type: String, required: true},
organization: {
type: Schema.ObjectId,
ref: 'organization'
},
user: {
type: Schema.ObjectId,
ref: 'users'
},
});
I want to make a query that returns:
ownerType = 'public'
organization = 321
The condition are 'OR'. So the material should be either ownerType 'public' or organization 321.
Can not find this in the docs. Do I need to make nested queries with "find" to do this? Or can it be done with a single query?
Some pseudo code:
mongoose.model('material').find({ownerType:'public' || organization:321}, function(err,materials){
...
}

Well presuming that your actual "Model" is named Material then you would come out to something like this in MongoDB parlance:
Material.find(
{
"$or": [
{ "ownerType": "public" },
{ "orginization._id": 123 }
]
},
function(err,docs) {
// results in here
}
);
So MongoDB has an $or operator, which makes sense since the query operands are represented in BSON ( from JSON conversion in the JavaScript case ). The purpose presents an "array" of possible arguments which are evaluated in a "short circuit" manner to determine if either case results in a true condition to match your query criteria.

Relational database design to mongoDB/mongoose design

I have recently started using mongoDB and mongoose for my new node.js application. Having only used relational databases before I am struggling to adapt to the mongoDB/noSQL way of thinking such as denormalization and lack of foreign key relationships. I have this relational database design:
**Users Table**
user_id
username
email
password
**Games Table**
game_id
game_name
**Lobbies Table**
lobby_id
game_id
lobby_name
**Scores Table**
user_id
game_id
score
So, each lobby belongs to a game, and multiple lobbies can belong to the same game. Users also have different scores for different games. So far for my user schema I have the following:
var UserSchema = new mongoose.Schema({
username: {
type: String,
index: true,
required: true,
unique: true
},
email: {
type: String,
required: true
},
password: {
type: String,
required: true
}
});
So my question is, how would I go about structing the relational design into mongoDB/mongoose schemas? Thanks!
EDIT 1
I have now tried to create all the schemas but I have no idea if this is the right approach or not.
var UserSchema = new mongoose.Schema({
_id: Number,
username: {
type: String,
index: true,
required: true,
unique: true
},
email: {
type: String,
required: true
},
password: {
type: String,
required: true
},
scores: [{ type: Schema.Types.ObjectId, ref: 'Score' }]
});
var GameSchema = new mongoose.Schema({
_id: Number,
name: String
});
var LobbySchema = new mongoose.Schema({
_id: Number,
_game: { type: Number, ref: 'Game' },
name: String
});
var ScoreSchema = new mongoose.Schema({
_user : { type: Number, ref: 'User' },
_game : { type: Number, ref: 'Game' },
score: Number
});

Mongoose is designed in such a way that you can model your tables relationally with relative ease and populate relational data based on the ref you defined in the schema. The gotcha is that you need to be careful with populating. If you populate too much or nest your populations you will run into performance bottle necks.
Your approach in Edit 1 is largely correct however you usually don't want to populate a remote ref based on a Number or set the _id of a model to a Number since mongo uses it's own hashing mechanism for managing the _id, this would usually be an ObjectId with _id implied. Example as shown below:
var ScoreSchema = new mongoose.Schema({
user : { type: Schema.Types.ObjectId, ref: 'User' },
game : { type: Schema.Types.ObjectId, ref: 'Game' },
score: Number
});
If for some reason you need to maintain a number id for your records consider calling it uid or something that won't conflict with mongo / mongoose internals. Good luck!

First of all, you are hitting on some good points here. The beauty of Mongoose is that you can easily connect and bind schemas to a single collection and reference them in other collections, thus getting the best of both relational and non-relational DBs.
Also, you wouldn't have _id as one of you properties, Mongo will add it for you.
I've made some changes to your schemas using the mongoose.Schema.Types.ObjectId type.
var UserSchema = new mongoose.Schema({
username: {
type: String,
index: true,
required: true,
unique: true
},
email: {
type: String,
required: true
},
password: {
type: String,
required: true
},
scores: [{ type: Schema.Types.ObjectId, ref: 'Score' }]
});
var GameSchema = new mongoose.Schema({
name: String
});
var LobbySchema = new mongoose.Schema({
_game: {
type: mongoose.Schema.Types.ObjectId,
ref: 'Game'
},
name: String
});
var ScoreSchema = new mongoose.Schema({
_user : {
type: mongoose.Schema.Types.ObjectId,
ref: 'User'
},
_game : {
type: mongoose.Schema.Types.ObjectId,
ref: 'Game'
},
score: Number
});
This will allow you to query your database and populate any referenced collections and objects.
For example:
ScoreSchema.find({_id:##userIdHere##})
.populate('_user')
.populate('_game')
.exec(function(err, foundScore){
if(err){
res.send(err)
} else {
res.send(foundScore)
}
}
This will populate the related user and game properties.

As you edited the post, I think it would be good. At least not bad :)
Check Mongoose Query Population. It's very useful to get related data.
var mongoose = require('mongoose'),
ObjectId = mongoose.Schema.Types.ObjectId
// code, code, code
function something(req, res) {
var id = req.params.id
// test id
return Lobby.findOne({_id: new ObjectId(id)})
.populate('_game')
.exec(function(error, lobby) {
console.log(lobby._game.name);
});
}

Two ways (that I know of). You store an id (that is indexed) and once you query the first table, you then query the second table to grab info from that, as there are no joins. This means that if you grab say, user id's from one table, you will then need to make multiple queries to the user table to get the user's data.
The other way is to store it all in one table, even if it's repetitive. If all you need to store is for example, a user's screen name with something else, then just store it with the other data, even if it's already in the user table. I'm sure others will know of better/different ways.