MongoDB Bulk Insert Operation instead of For-Loop - javascript

Let's consider a social network to be built by NodeJS and MongoDB.
So, if a user creates a new post, it should be saved to his/her followers feed.
The straightforward implementation of this operation as follow (simplified):
var newPost="New Post";
//get list of followers of user 1
var listOfFollowers = followersCollection.find({u:"1"});
for(var i=0;i<listOfFollowers.length;i++){
var followerId = listOfFollowers[i]._id;
//insert new post of user 1 to every follower feed
feedsCollection.insertOne(
{ownerId:followerId,authorId:"1",content:newPost}
);
}
This, of course, has very bad performance in case of big numbers in followers count. How can do this with a single fast performing MongoDB query?

MongoDB provides bulk document insert functionality, check out this link - https://docs.mongodb.com/manual/reference/method/db.collection.initializeUnorderedBulkOp/
db.collection.initializeUnorderedBulkOp() It creates an unordered list of operations and mongodb executes this list in parallel, so it's fast and you don't have to take extra care of performance as mongo handles it.
For ordered insertions, you can use db.collection.initializeOrderedBulkOp().
e.g.
var newPost="New Post";
var bulk = db.followersCollection.initializeUnorderedBulkOp();
//get list of followers of user 1
var listOfFollowers = followersCollection.find({u:"1"});
for(var i=0;i<listOfFollowers.length;i++){
var followerId = listOfFollowers[i]._id;
//insert new post of user 1 to every follower feed
bulk.insert( {ownerId:followerId,authorId:"1",content:newPost});
}
bulk.execute();
If you are using Mongoose then checkout Mongoose docs for the same. In the above example, I have just trying to explain how you can do it using plain MongoDB.

Insert Many Read this document I think you will get the answer

Check this:
var newPost="New Post";
//Object Array
var collection = []
//get list of followers of user 1
var listOfFollowers = followersCollection.find({u:"1"});
for(var i=0;i<listOfFollowers.length;i++){
var followerId = listOfFollowers[i]._id;
collection.push({ownerId:followerId,authorId:"1",content:newPost})
}
feedsCollection.insert(collection); //Or use insertMany()
You can create your object array and insert it at once.
Check documentation :- https://docs.mongodb.com/manual/reference/method/db.collection.insert/#insert-multiple-documents
Even though this is a simple answer for your question, If the collection array has a large number of elements, there still might be performance issues. So the best way to handle this is using triggers. https://docs.mongodb.com/stitch/triggers/

Related

Firebase/Firestore start loop from certain document ID

I'm getting a collection from my Firestore database and adding the document values to an array in JS:
var data = myCollection();
var myArray = [];
data.forEach(function(data) {
var splitPath = data.name.split('/');
var documentId = splitPath[splitPath.length - 1];
var name = data.fields.name ? data.fields.name.stringValue : '';
var country = data.fields.vin ? data.fields.vin.stringValue : '';
myArray.push( [ documentId, name, country ] );
});
Suppose I know a document ID, is it possible to get the collection documents from that certain document ID?
I'm not sure if Firestore documents are ordered by date. I am trying to get the most recent documents from a certain document ID.
Suppose I know a document ID, is it possible to get the collection documents from that certain document ID?
When it comes to the Firebase console, all documents are sorted lexicographically by ID and this behavior can't be changed. When it comes to code, it's up to you to choose how to order the results.
I'm not sure if Firestore documents are ordered by date.
No, there is no time component inside the document IDs.
I am trying to get the most recent documents from a certain document ID.
In that case, the simplest solution would be to add a timestamp field in each document and order them according to that field.
However, Firebase Realtime Database pushed IDs do contain a time component. So if you want, you can add that data there. Both databases are working great together.
If you have multiple documents and you want to implement pagination, for example, you can use query cursors and use startAt() or starAfter() if you need that.
I don't know if this is exactly what you need but firebase docs has below example:Order and limit data
import { query, where, orderBy, limit } from "firebase/firestore";
const q = query(citiesRef, where("population", ">", 100000), orderBy("population"), limit(2));
if you adjust where part to your id, then sort by date it should work.

Firebase JavaScript Child Comparison

I need to compare some values in my Firebase Database using JavaScript, but I'm having a hard time doing that. See image above.
The way that is suppose to work is I need to check if the user likes the same id as the other user. So for example:
User XbsX0IskrHVcaEmEBgyeok9isiM2 liked 4 items with unique ID's. Now I need to check if user jBc2Ls32DgMUSgzKUkVSw38UjQD2 liked the same thing to see if it's a match.
I have this code:
var check = ref.child('likes').child(uid2).child(uid2);
but it's not working.
You could iterate over the likes of the first user and make sure that everyone exists on the list of likes of the second user:
// Gets the list of likes from both users
var likes_user_1 = ref.child('likes').child(uid1);
var likes_user_2 = ref.child('likes').child(uid2);
// Iterates over the list of likes of the first user
var isAMatch = Object.keys(likes_user_1).every(function(like) {
// Returns true if user_2 has this like
return likes_user_2[like];
});

Meteor How to get ONE element from MongoDB

In my MongoDatabase i have 5 elements.
First the _id then a follower with his idfollower and a following with his idfollowing.
now im trying to get only the IDFOLLOWING element
var userid = Relationships.findOne({follower: Meteor.user().username}).idfollowing;
this works!
Now i got 5 entries in the Database and the code only shows me 1 ID not 5
Give this a try:
// fetch an array of relationships where the current user is the follower
var selector = {follower: Meteor.user().username};
var relationships = Relationships.find(selector).fetch();
// create an array of ids? by extracting the idfollowing values from relationships
var userIds = _.pluck(relationships, 'idfollowing');
Here's the documentation for:
pluck
find
fetch

Mongoose update multiple documents

Actually i am using mongoDB, and i am able to update document with single ObjectID, so right now i want to perform update on multiple documents having different ObjectID, i did considerable search, came to know {multi:true} will help to update multiple documents.
Let's say i have
var uid = {'Userid': ObjectId(..)};
var id = {'Newid': Object(..)};
now i want to perform update on both of these different documents having different ObjectId' s total separatley, so far i tried this considering "xyz" and "abc" array's in Schema just for demo
var update_uid = {$push:{'xyz': some_id}};
var update_id = {$push:{'abc': some_other_id}};
Test.update(uid,id,update_uid,update_id,{multi:true},function(){..});
this is wrong i know, i mentioned it for the records just in case, Any help and suggestion would be much appreciated
If you have multiple ids that you want to match you can use the keywork $in which checks if there's any matches in the array. Now that you also have multiple properties that you want to check you can user the $or keyword, which will find a match if either of the objects in the array matches a document in the database.
var userIds = [{'Userid':ObjectId(..)}, {'Userid':ObjectId(..)}];
var newIds = [{'Newid': Object(..)}, {'Newid': Object(..)}];
var query = {
$or: [{$in: userIds}, {$in: newIds}]
}

How to get the row count from an azure database?

Am working on a windows store javascript application. The application uses data from azure mobile services.
Consider the below code:
var itemTable = mobileService.getTable('item');
//item is the table name stored in the azure database
The code fetches the entire table item and saves it to a variable itemTable.
What code will return the no of rows present in itemTable??
What you're looking for is the includeTotalCount method on the table/query object (unfortunately it's missing from the documentation, I'll file a bug to the product team to have it fixed).
When you call read on the query object, it will return by default 50 (IIRC, the number may be different) elements from it, to prevent a naïve call from returning all elements in a very large table (thus either incurring the outbound bandwidth cost for reserved services, or hitting the quota for free ones). So getting all the elements in the table, and getting the length of the results may not be accurate.
If all you want is the number of elements in the table, you can use the code below: returning zero elements, and the total count.
var table = client.getTable('tableName');
table.take(0).includeTotalCount().read().then(function (results) {
var count = results.totalCount;
new Windows.UI.Popups.MessageDialog('Total count: ' + count).showAsync();
});
If you want to query some elements, and also include the total count (i.e., for paging), just add the appropriate take() and skip() calls, and also the includeTotalCount as well.
If anybody comes here and interested in how to get the totalCount only on C# (like me), then this is how you do it:
var table = MobileService.GetTable<T> ();
var query = table.Take(0).IncludeTotalCount();
IList<T> results = await query.ToListAsync ();
long count = ((ITotalCountProvider)results).TotalCount;
Credit goes to this blog post here
You need to execute read() on the table query and then get the length of the results.
var items, numItems;
itemTable.read().then(function(results) { items = results; numItems = items.length; });
If you are only showing a record count and not the entire results - you should just select the ID column to reduce the amount of data transmitted. I don't see a count() method available yet in the JS Query API to fill this need.
var itemTable = mobileService.getTable('item').select('itemID');

Categories