We Keep Coding

Mongoose search by array entries ($in) behavior not aligned with MongoDB Atlas

I'm running a Node.js server, connecting to a MongoDB database with mongoose.
Inside my controller, I have several methods that make operations to the database. One of them is this one:
async findMultiple(req, res) {
const [baseSkillsArray] = Array(req.body);
try {
// if there is not baseSkillsArray, skip
if (!baseSkillsArray) {
return res.status(200).send([]);
}
// find all baseSkills using the ids in the baseSkillsArray
const allBaseSkills = await BaseSkill.find({
_id: { $in: [baseSkillsArray.baseSkillArray] } //
});
console.log('test ' + allBaseSkills);
res.status(200).send(allBaseSkills);
} catch (error) {
console.error(error.message);
res.status(500).send('Server error find BaseSkills');
}
}
However, this returns me nothing. I did some debugging and I found the reason is the find id $in the array. So I tried hard coding a value, like '2', for instance.
// find all baseSkills using the ids in the baseSkillsArray
const allBaseSkills = await BaseSkill.find({ _id: { $in: ['2'] } });
No success. So I went to MongoDB Atlas, where my DB is stored. I tried filtering using the same line of code in my collections.
{ _id: { $in: ['2'] } }
Surprisingly, it returns my document as I wanted!
The issue is that I need to make it work with mongoose. Any ideas? Is this a known bug?

There is nothing wrong with the query, nor a bug regarding $in.
In fact, what's wrong is the actual collection name. I manually created a collection in MongoDB Atlas, called "baseSkills". However, mongoose by default transforms your collection name into lowercase and adds an "s" if your collection's name is not in the plural.
So every time I started my server, I noticed that there was a new collection called "baseskills". I assumed it was a bug and deleted it. Only after making this post that I realized the collection was there again.
So I exported the documents to this collection and my query was working fine.
FYI, there is a way to enforce the collection's name in mongoose. When you declare you model, add a second parameter to the Schema function called "collection". Here is an example:
const BaseSkillSchema = new mongoose.Schema({
_id: {
type: String,
required: true
}, ...
}, { collection: 'baseSkills' })
That's it! Sorry for the mess and thank you for your help!

you want to query over mongo db object ids. So you should create a new ObjectId to do that.
import {Types} from 'mongoose';
{ _id: { $in: [new Types.Object("2")] } }
Or if you have 2 ids one generated and one custom created as id then you can query without creating a new object.
{ id: { $in: ['2'] } }

Realtime database update or create functionality

I have a user object in firebases realtime database. I am querying realtime database and when successful I am wanting to write some new data to the user object under the users node.
My desired outcome: When a user has not fetched the information before, a new field called 'lastViewed' under the user object is created and if the field has already been created then we update the timeViewed keys value. A user can have multiple objects in the array corresponding to the uuid of the fetched data.
Please see the user object below
This may not need to be an array if using .push()
-N0X8VLHTw3xgvD2vJs- : { // this is the users unique key
name: 'myName',
lastViewed: {
[
{
timeViewed: 1651558791, // this is the field to update if exists
datasUniqueKey: 'N17ZmwIsbqaVSGh93Q0' // if this value exists update timeViewed else we create the entry.
},
{
timeViewed: 1651558952,
datasUniqueKey: 'N17ZmwIsbqaVSad3gad'
},
]
}
}
Please see my attempt below.
const getData = database()
.ref(`data/${uniqueKeyFromData}`)
.on('value', snapshot => {
if (snapshot.exists()) {
database()
.ref(`users/${currentFirebaseUserKey}/lastViewed`) // currentFirebaseUserKey = N0X8VLHTw3xgvD2vJs
.once('value', childSnapshot => {
if (childSnapshot.exists()) {
// update
database()
.ref(
`users/${currentFirebaseUserKey}/lastViewed`,
)
.update({
timeViewed: new Date(), // new data will not give us the corresponding date format in the user object above but don't worry about that
fetchedDatasUniqueKey: uniqueKeyFromData,
});
} else {
// create
database()
.ref(
`users/${currentFirebaseUserKey}/lastViewed`,
)
// Push creates a unique key which might not be required so maybe set?
.push({
timeViewed: new Date(),
fetchedDatasUniqueKey: uniqueKeyFromData,
});
}
});
}
});
Where I think I am going wrong
Above I am not creating an array, if I use push I would get a unique key generated from firebase but then would have to use that key when updating, something like
`users/${currentFirebaseUserKey}/lastViewed/${lastViewedUniqueKey}`
So the user object would look like so
-N0X8VLHTw3xgvD2vJs- : { // this is the users unique key
name: 'myName',
lastViewed: {
-N17i2X2-rKYXywbJGmQ: { // this is lastViewedUniqueKey
timeViewed: 1651558791,
datasUniqueKey: 'N17ZmwIsbqaVSGh93Q0'
},
}
}
then check for snapshot.key in the if?, any help would be appreciated.

Since you don't want a list of data, but a single set of properties for ``, you should just call set instead of push:
database()
.ref(`users/${currentFirebaseUserKey}/lastViewed`)
.set({ // 👈
timeViewed: new Date(),
fetchedDatasUniqueKey: uniqueKeyFromData,
});
I also don't think you need the two different cases here for create vs update, as both seem to do the exact same thing. If you do need both cases though, consider using a transaction.

Cloud firestore: Update field values in nested array object with dot notation

Please help me solve this, I would like to update the fields using dot notation, using set() but each time I run with the below implementation. I have the fields added to firestore as e.g studentInfo.0.course.0.courseId instead of updating the already existing ones.
Json sample as it sits in firestore
"schoolId": "school123",
"studentInfo": [
{
"studentId": "studentI23",
"regDate": "2020-04-18",
"course": [
{
"courseId": "cs123",
"regDate": "2020-05-28",
"status": "COMPLETED"
}
]
}
],
"registered":"yes"
}
Code logic
const query = firestore.collection('users').where('registered', '==', 'yes')
const students = await query.get()
students.forEach(student => {
firestore.doc(student.ref.path).set({
'studentInfo.0.studentId': '345','studentInfo.0.course.0.courseId': '555'
}, { merge: true })
})
On the docs https://firebase.google.com/docs/firestore/manage-data/add-data#update_fields_in_nested_objects I can only find updating nested objects but not nested array objects.

It is indeed not possible to update a single element in an array using dot notation, or otherwise. To update an array you'll need to:
Read the document
Get the current value of the array from it
Determine the new array contents
Write the entire updated array back to the database.
The only alternative array operations are array-union and array-remove, which add and remove unique elements to/from the array - essentially treating it as a mathematical set. But since you are looking to update an existing element, these operations are of no use here.
Also see:
Firestore Update single item in an array field
Firestore update specific element in array
How to update an "array of objects" with Firestore?

There is no direct way to update the as stated in the article. You can either run a transaction to get the latest array value and then updating the array with the final array value. That would be as below:
await firestore.runTransaction((transaction: Transaction) => {
const students: Array<Students> = firestore
.collection("users")
.where("registered", "==", "yes");
students.forEach((student) => {
const firebaseDoc = firestore.doc(student.ref.path);
transaction.set(
firebaseDoc,
{
"studentInfo.0.studentId": "345",
"studentInfo.0.course.0.courseId": "555",
},
{ merge: true }
);
});
});
Inside transaction I am getting the array first and then updating each values as per my need. This will make the whole operation atomic so the issues mentioned in the article will not come.
Alternatively, you can also model your firestore database as below
"schoolId": "school123",
"studentInfo": {
"studentI23": {
"studentId": "studentI23",
"regDate": "2020-04-18",
"course": [
{
"courseId": "cs123",
"regDate": "2020-05-28",
"status": "COMPLETED"
}
]
}
},
"registered":"yes"
}
Above I have changed the array to map, since in map you can update the each field based on dot notation fields(doc), hence. you can achieve your end result. This solution will avoid any transaction query and will be faster

document wont save using save() method with mongoose

I have this schema:
var UserSchema = mongoose.Schema({
analytic: {
type: Object,
default: {
today:[],
weekly:[],
monthly:[],
yearly:[],
allTime:[]
}
}
});
let User = mongoose.model("bloger", UserSchema);
module.exports = {User};
and I am trying to save some data into one of the arrays like so:
User.findOne({username:username}, (e, user) => {
if (e) {
res.send('error fetching post')
}
else if (!user) {
res.send('no user found')
}
else if (user) {
user.analytic.today.push(req.body.visitor) // push the data object to the array
user.save((e, doc) => {
if (e) {
res.send(e)
}
if (doc) {
console.log('user saved')
res.send(doc)
}
})
}
})
})
I am getting the doc object on save() and not the e so I though it should have save it but it wasn't.

I have had a similar issue before this is because I am not defining a new Model I am just passing a JSON object.
Instead of saving the object you need to create a new model and save that.
Try creating a new model passing the save into it like below;
var newUser = new User(user);
newUser.save((e, doc) {
if (e) {
res.send(e)
}
if (doc) {
console.log('user saved')
res.send(doc)
}
});
Making sure you require the User Model inside the script.

Performing deep modifications in objects not in your schema makes Mongoose oblivious to those changes, preventing it from knowing what to save (and from making efficient atomic updates). The end result is that, when calling .save, Mongoose thinks there's nothing modified and therefore does nothing.
In your particular scenario, you have two options:
1. Add your analytics sub-arrays to your schema
This is the best option and allows for finer control of everything:
const UserSchema mongoose.Schema({
analytic: {
today: [{}],
weekly: [{}],
monthly: [{}],
yearly: [{}],
allTime: [{}],
}
});
With this, those arrays are now known to Mongoose and everything should work correctly.
Note that you don't need defaults in this case, as Mongoose is smart enough to create the arrays as needed.
2. Manually mark modified object as modified
If for any reason you don't want or can't modify the schema, you can manually mark the analytic object as modifies so Mongoose knows of it:
user.analytic.today.push(req.body.visitor) // push the data object to the array
user.markModified('analytic'); // mark field as modified
user.save(...);
This signals Mongoose that analytic or any of its children have changed and triggers an update when calling .save. Note however that Mongoose views this as a full change in the object, while with option 1 it can use $push instead.

Sequelize Many to Many - How to create a new record and update join table

I'm building a simple database with node, express and sequelize. I have created my models, and sequelize created the tables in my database.
I have the models User and City, with a many to many relationship. Sequelize created the tables Users, Cities and a join table CitiesUsers: with UserId and CityId.
My question is when I create a new user how do I update that join table? The CityId property gets ignored on create.
//Models use
//City.hasMany(User);
//User.hasMany(City);
var user = User.build({
first_name: 'John',
last_name: 'Doe',
CityId: 5
});
user.save();

After digging further into the documentation, I believe I've found the answer.
When creating a many to many relationship sequelize creates get, set and add methods to each model.
From the docs assuming models User and Project with many to many:
http://docs.sequelizejs.com/en/latest/docs/associations/#belongs-to-many-associations
This will add methods getUsers, setUsers, addUsers to Project, and
getProjects, setProjects and addProject to User.
So in my case I did the following where "city" is a specific City model returned from City.find...
//user.setCities([city]);
models.User.find({ where: {first_name: 'john'} }).on('success', function(user) {
models.City.find({where: {id: 10}}).on('success', function(city){
user.setCities([city]);
});
});

You can create a new instance of the model used as the join table once both City and User models have been created.
const User = sequelize.define('user')
const City = sequelize.define('city')
const UserCity = sequelize.define('user_city')
User.belongsToMany(City, { through: UserCity })
City.belongsToMany(User, { through: UserCity })
const user = await User.create()
const city = await City.create()
const userCity = await UserCity.create({
userId: user.userId,
cityId: city.cityId,
})

Just to add on to the many excellent answers in this thread, I find generally that when I have one entity referencing another, I want to create the referenced entity if (and only if) it does not already exist. For this I like to use findOrCreate().
So imagine you were storing articles, and each article could have any number of tags. What you'd typically want to do is:
Iterate through all the desired tags, and check if they exist. Create them if they don't already exist.
Once all the tags have been found or created, create your article.
Once your article has been created, link it to the tags you looked up (or created) in step 1.
For me, this winds up looking like:
const { article, tags } = model.import("./model/article");
let tagging = [
tags.findOrCreate({where: {title: "big"}}),
tags.findOrCreate({where: {title: "small"}}),
tags.findOrCreate({where: {title: "medium"}}),
tags.findOrCreate({where: {title: "xsmall"}})
];
Promise.all(tagging).then((articleTags)=> {
article.create({
title: "Foo",
body: "Bar"
}).then((articleInstance) => {
articleInstance.setTags(articleTags.map((articleTag) => articleTag[0]));
})
})

From The docs v3:
// Either by adding a property with the name of the join table model to the object, before creating the association
project.UserProjects = {
status: 'active'
}
u.addProject(project)
// Or by providing a second argument when adding the association, containing the data that should go in the join table
u.addProject(project, { status: 'active' })
// When associating multiple objects, you can combine the two options above. In this case the second argument
// will be treated as a defaults object, that will be used if no data is provided
project1.UserProjects = {
status: 'inactive'
}
u.setProjects([project1, project2], { status: 'active' })
// The code above will record inactive for project one, and active for project two in the join table