mongodb changestream "pipeline" not working - javascript

I have some code that watches a mongodb collection for updates... I have it setup so that when anyone sends a message, the changestream will detect that... The issue is that when I try to add a pipeline, the updates do not get detected...
Here are some things that I've tried:
const pipeline = [
{ $match: { 'fullDocument.username_pair': 'HardCodedUsernameInDatabase' }}
];
changeStream = message_collection.watch(pipeline);
and
const pipeline = [
{ $match: { username_pair: 'HardCodedUsernameInDatabase' }}
];
changeStream = message_collection.watch(pipeline);
again, the code will detect all messages in the absence of any pipeline, ie:
changeStream = message_collection.watch();
...
UPDATE:
The reason why this is tricky is because the document I'm listening to looks like this:
_id:some_id
username_pair:"Test1234test1111"
messages:Array
The username_pair never updates, the messages array is what updates... I need to be looking at the document matching the username_pair, and I need to detect changes in the messages array that corresponds to the username_pair.

I guess the trick is to add option called fullDocument while opening the change stream on collection. see here
someCollection.watch([{
"$match": {
"operationType": {
"$in": ["update"]
},
"fullDocument.username_pair": "HardCodedUsernameInDatabase"
}
}], {"fullDocument": "updateLookup"})
Hope this helps

Related

Any way to insert fields only if they don't already exist in MongoDB?

I've tried some solutions i've found but im not getting how this is supposed to work (im new to mongo). i have a document that looks like this:
{
"_id": {
"$oid": "63ef95daf324e5a26354eb61"
},
"idCarga": "6182944",
"12121212121212": {
"info1": "hello",
"info2": "goodbye"
},
"13131313131313": {
"info1": "hello",
"info2": "goodbye"
}
}
My goal is to:
If i try to update the document with a new product ID field (like 15151515151515), it gets inserted as a new field.
If i try to update the document with an already existing prod ID (like 13131313131313), nothing happens (dont insert or update anything).
I got some results with the $exists operator paired with upsert option. But i have to insert so many product IDs it seems like an unviable option. Is there any way to do this with only one document per "idCarga"? Or do i have to separate this into 2 collections? Thanks in advance.
Simply do a $set with $ifNull
db.collection.update({
<your criteria>
},[
{
$set: {
"<your id>": {
"$ifNull": [
"$<your id>",
{
<your new sub doc>
}
]
}
}
}
])
Mongo Playground for new id case
Mongo Playground for existing id case

Mongoose search by array entries ($in) behavior not aligned with MongoDB Atlas

I'm running a Node.js server, connecting to a MongoDB database with mongoose.
Inside my controller, I have several methods that make operations to the database. One of them is this one:
async findMultiple(req, res) {
const [baseSkillsArray] = Array(req.body);
try {
// if there is not baseSkillsArray, skip
if (!baseSkillsArray) {
return res.status(200).send([]);
}
// find all baseSkills using the ids in the baseSkillsArray
const allBaseSkills = await BaseSkill.find({
_id: { $in: [baseSkillsArray.baseSkillArray] } //
});
console.log('test ' + allBaseSkills);
res.status(200).send(allBaseSkills);
} catch (error) {
console.error(error.message);
res.status(500).send('Server error find BaseSkills');
}
}
However, this returns me nothing. I did some debugging and I found the reason is the find id $in the array. So I tried hard coding a value, like '2', for instance.
// find all baseSkills using the ids in the baseSkillsArray
const allBaseSkills = await BaseSkill.find({ _id: { $in: ['2'] } });
No success. So I went to MongoDB Atlas, where my DB is stored. I tried filtering using the same line of code in my collections.
{ _id: { $in: ['2'] } }
Surprisingly, it returns my document as I wanted!
The issue is that I need to make it work with mongoose. Any ideas? Is this a known bug?
There is nothing wrong with the query, nor a bug regarding $in.
In fact, what's wrong is the actual collection name. I manually created a collection in MongoDB Atlas, called "baseSkills". However, mongoose by default transforms your collection name into lowercase and adds an "s" if your collection's name is not in the plural.
So every time I started my server, I noticed that there was a new collection called "baseskills". I assumed it was a bug and deleted it. Only after making this post that I realized the collection was there again.
So I exported the documents to this collection and my query was working fine.
FYI, there is a way to enforce the collection's name in mongoose. When you declare you model, add a second parameter to the Schema function called "collection". Here is an example:
const BaseSkillSchema = new mongoose.Schema({
_id: {
type: String,
required: true
}, ...
}, { collection: 'baseSkills' })
That's it! Sorry for the mess and thank you for your help!
you want to query over mongo db object ids. So you should create a new ObjectId to do that.
import {Types} from 'mongoose';
{ _id: { $in: [new Types.Object("2")] } }
Or if you have 2 ids one generated and one custom created as id then you can query without creating a new object.
{ id: { $in: ['2'] } }

How to push a document in an array nested in another array?

This is my problem : I have a document ( let's call it root) containing an array of another documents (stick), that contain another array of another documents (leaf).
Or simply said : root{ stickChain[leaveschain1[ leaf1, leaf2],leaveschain2[ leaf1, leaf2] ]}
I have access to root _id, stick _id, or whatever it is needed to identify each document.
Basically, the best result I've come so far is, when creating my leaves documents, is to store then at the same level tha sticks, or in another word I've come to create an array of leaves in root.
I'm working in javascript and using mongoose
This is the line I've used:
db.collection('root').findOneAndUpdate({ _id: root.id, "stick._id":expTemp._id },{stickChain:{$push: {"leavechain.$": leaf}}})
And I this gives me this result : root{ leavesChain[leaf1,leaf2], stickchain[1,2,3] }
I've come across something new to me (since Mongodb 3.6 there is a new way of handling array of arrays), here is what I've tried :
try{ db.collection('root').findOneAndUpdate(
{ _id: root.id, "stick._id":expTemp._id },
{$push: {"stickChain.$[xpc].leavesChain.$[qac]": leaf}},
{ arrayFilters: [ { "xpc._id": user._id } , { "qac._id": expTemp._id } ]})}
UPDATE
try{ db.collection('root').findAndModify(
{$push: {"root.$[cv].stickChain.$[xpc].leavesChain.$[qac]": leaf}},
{ arrayFilters: [ {"cv._id":user.cv} ,{ "xpc._id": user._id } , { "qac._id": expTemp._id } ],
upsert:true})
.catch(error => console.error(error))
}catch{}
And this gives me a new kind of error : MongoError: Either an update or remove=true must be specified
The thing is that I'm not familiar with how to do it, while I know what I want to do: I want to insert a "leaf" into a specific array in a document fetched in MongoDB. Maybe not the best practice, so any hint are welcome :)
I've splitted my dument like this :
root[stickChain _id]
stick[leavesChain[leaf]]
Thanks to Andrey Popov for his explications

Firebase rules: How to access document id when querying collections

I am stuck with this one and need help please 🙏
I have a firebase document like so:
"789" : {
slug: "document-slug-name",
owner_uid: "123",
}
which has a subcollection called "roles" that looks like this:
"345" : {
permission_type: 1,
}
Now I am querying the documents like this:
const snapshot = await firebaseApp.firestore()
.collection("documents")
.where("owner_uid", "==","123")
.where("slug", "==", "document-slug-name")
.limit(1)
.get()
const document = snapshot.docs[0]
my rules look like this:
match /documents/{documentId} {
allow list: if
get(/databases/$(database)/documents/documents/$(documentId)/roles/$(currentUser().uid)).permission_type == 1;
}
But for some reason, both the rules simulator and the actual firebase db tell me that documentId is not set?
I have also tried resource.data.uid and resource.id to get the documents id but both are not set :(
is there any other way to query the roles subcollection?

Migrate simple List Array key to another key with an extra attribute in MongoDB

Sorry if I'm not getting the terminology right. Here's what I have currently my MongoDB user docs db.users:
"liked" : [
"EBMKgrD4DjZxkxvfY",
"WJzAEF5EKB5aaHWC7",
"beNdpXhYLnKygD3yd",
"RHP3hngma9bhXJQ2g",
"vN7uZ2d6FSfzYJLmm",
"NaqAsFmMmnhqNbqbG",
"EqWEY3qkeJYQscuZJ",
"6wsrFW5pFdnQfoWMs",
"W4NmGXyha8kpnJ2bD",
"8x5NWZiwGq5NWDRZX",
"Qu8CSXveQxdYbyoTa",
"yLLccTvcnZ3D3phAs",
"Kk36iXMHwxXNmgufj",
"dRzdeFAK28aKg3gEX",
"27etCj4zbrKhFWzGS",
"Hk2YpqgwRM4QCgsLv",
"BJwYWumwkc8XhMMYn",
"5CeN95hYZNK5uzR9o"
],
And I am trying to migrate them to a new key that also captures the time that a user liked the post
"liked_time" : [
{
"postId" : "5CeN95hYZNK5uzR9o",
"likedAt" : ISODate("2015-09-23T08:05:51.957Z")
}
],
I am wondering if it might be possible to simply do this within the MongoDB Shell with a command that iterates over each user doc and then iterates over the liked array and then updates and $push the new postId and time.
Or would it be better to do this in JavaScript. I am using Meteor.
I almost got it working for individual users. But want to know if I could do all users at once.
var user = Meteor.users.findOne({username:"atestuser"});
var userLiked = user.liked;
userLiked.forEach(function(entry) {
Meteor.users.update({ username: "atestuser" },
{ $push: { liked_times: { postId: entry, likedAt: new Date() }}});
console.log(entry);
});
Still a bit of a newbie to MongoDB obviously......
Here is something i made real quick you should run this on the server side just put it into a file e.g. "migrate.js" in root meteor and run the meteor app
if (Meteor.isServer) {
Meteor.startup(function () {
var users = Meteor.users.find().fetch();
users.forEach(function (doc) {
liked.forEach(function (postId) {
Meteor.users.update(doc._id, { $push: { liked_times: { postId: postId, likedAt: new Date() } } });
});
});
console.log('finished migrating');
});
}
p.s I didn't test it
If this is a one time migration i would do something like this in a one time js script.
Get all users
Iterate over each user
Get all likes
Iterate over them, get likedAt
var liked_times = _.collect(likes, function (likeId) {
return {
'postId' : likeId,
'likedAt': // get post liked time from like id.
}
});
Insert the above in the collection of choice.
Note:
The above example makes use of lodash
I would rather just save likedAt as a timestamp.

Categories