We Keep Coding

mongoDB - Regex search against partial field values

I've a collection of countries with country calling code in the country object. How can I find a country using calling code with a mobile number?
const countries = [
{
name: 'UAE',
callingCode: 971
},
{
name: 'USA',
callingCode: 1
},
{
name: 'UK',
callingCode: 44
}
];
const number = '971524500000'; // Input
How can I find country for the given mobile using regex in mongoose javascript;

[https://en.wikipedia.org/wiki/List_of_country_calling_codes][1]
Take a look at the link above on country calling codes, and specifically see the section "Tree List".
One solution would be to implement a schema in Mongo based on this tree in order to decode the country codes.
So, a table could be created to store Mongo documents containing a field "1x" "2x" "21x" etc (the Y axis in the Tree List table).
Each of these documents could contain an array of sub-documents from x=0 to x=9 (the x axis in the Tree List table). The sub-document can contain the country name/code you are looking for. You can use a direct index into the array in the Mongo document for an efficient lookup.
I think you'll find this to be a pretty efficient implementation and should cover all the bases.

If you can restructure your array to an object this would be the fastest
const countries =
{
971: 'UAE',
1: 'USA',
44: 'UK',
}
;
var code = 44;
console.log(countries[code]);
const countries = [
{
name: 'UAE',
callingCode: 971
},
{
name: 'USA',
callingCode: 1
},
{
name: 'UK',
callingCode: 44
}
];
var myFound =countries.filter(myFunc.bind(this,44));
function myFunc(code,element) {
if(element.callingCode == code){
return element;
}
}
console.log(myFound);

On MongoDB v 4.2 - you can use $let & $regexFind to do this :
db.collection.aggregate([
{
$match: {
$expr: {
$eq: [
{
$let: {
vars: {
value: {
$regexFind: {
input: "971524500000", // pass in as string
regex: { $toString: "$callingCode" }
}
}
},
in: "$$value.idx",
}
},
0
]
}
}
}
]);
Test : MongoDB-Playground
Explanation :
General Use-case :
In general regex searches - Will have an input which will be sub-string of actual string, Ex.:-
Actual string in DB : 'We can do it in mongo'
Input : mongo (/mongo/ - will work)
Your Use-case :
From above case as mongo exists in actual string/value of db field then you can get that document, But your case is different :
Actual string in DB : mongo
Input : 'We can do it in mongo'
This doesn't work that way, So using normal /We can do it in mongo/ isn't going help you here (Also doing few tricks with regex). So we need to make a trick to $regexFind operator. Unlike mongo documentation we need take 971524500000 into input field & regex as string value of callingCode field which is vice-versa to what's given in documentation.
So once we do that, We would get something like below :
{
"match" : "971", /** value of callingCode field */
"idx" : 0, /** Index of `971` in '971524500000' */
"captures" : []
},{
"match" : "1",
"idx" : 2,
"captures" : []
},
null /** 3rd doc no match */
As country code has to be at first of given number we need docs where "idx" : 0 - So we're using $let to get index of returned object & checking against 0 & eventually getting respective docs using $match.
Note :
There is something you need to look into, Just in case if you've docs like below :
{
"_id": ObjectId("5e8f67091aa1cc3d2158ade1"),
"name": "USA",
"callingCode": 1.0
},
{
"_id": ObjectId("5e8f67091aa1cc3d2158ade3"),
"name": "somecountry",
"callingCode": 197.0
}
& input is 1971524500000, then this query will bring both docs in result. This will be the case you need to check on. Anyhow I would suggest to try this query, rather than getting all documents for collection to the code & extract required o/p this might be better to do.

Firebase: HowTo update the value item of a record without knowing its unique id?

So, I have a firebase structure as follows:
"Posts" : {
"0" : {
"code": 12345TG4
"Likes" : 59
},
"1" : {
"code": RT560451
"Likes" : 12
}
}
Ideally what I want to do is:
var updateData = {
Likes: 74
}
Posts.child(id).update(updateData);
But I don't know the UiD in advance, but I do have the value of the 'code', which is itself a unique value.
How do I create a query which can update the value 'Likes' based on the known value of 'code' in Firebase?

As Doug commented and Rodrigo showed, you'll have to first query for the item based on its code, before you can update it:
var ref = firebase.database().ref();
ref.child("Posts").orderByChild("code").equalTo("12345TG4").once("value", function(snapshot) {
snapshot.forEach(function(child) {
child.ref.update(updateData);
});
});
Change your model
Although the above will work, you should consider why you're storing your posts under an array index, then they also have a unique ID already. It is more idiomatic in NoSQL solutions such as Firebase to store the posts under the code:
"Posts" : {
"12345TG4" : {
"Likes" : 59
},
"RT560451" : {
"Likes" : 12
}
}
Which you'd update with:
ref.child("Posts").child("12345TG4").update({ Likes: 74 });
Or even:
"PostLikes" : {
"12345TG4" : 59,
"RT560451" : 12
}
And then:
ref.child("PostLikes/12345TG4/Likes").set(74);

Try to do this.
var ref = firebase.database().ref("Posts");
ref.orderByChild("code").on("child_added", function(snapshot) {
snapshot.ref.update(updateData);
});

Update object with given ID embed in array without restructuring Mongo database

I've got the following document named "clients" which includes id, name and list of projects (array of objects):
{
"_id": {
"$oid": "572225d997bb651819f379f7"
},
"name": "ppg",
"projects": [
{
"name": "aaa",
"job_description": "",
"projectID": 20
},
{
"name": "bbbb",
"job_description": "",
"projectID": 21
}
]
}
I would like to update "job_description" of project with given "projectID" like this:
module.exports.saveJobDesc = function(client, idOfProject, textProvided) {
db.clients.update({ name: client},
{ $set: {'projects.0.job_description': textProvided }});
};
But instead of hardcoded index "0" of array I want to find specific project using "projectID". Is there a way to achieve this without changing the structure of collection and/or document?

If you want to update the "job_description" where name="ppg" and project_id=20 then you can use below mongo query:-
db.clients.update({ "name":"ppg","projects.projectID":20 },{$set: {"projects.$.job_description": "abcd"}})
Please let me know if any thing else is required

You cannot update multiple array elements in single update operation, instead you can update one by one which takes time depends upon number of elements in array and number of such documents in collection. see New operator to update all matching items in an array
db.test2.find().forEach( function(doc) {
var projects = doc.projects;
for(var i=0;i<projects.length;i++){
var project = projects[i];
if(project.projectID == 20){
var field = "projects."+i+".job_description";
var query = {};
query[field] = "textasdsd";
db.test2.update({ _id: doc._id},{ $set:query});
}
}
})

How to improve or avoid find / fetch cycle in meteor's publication?

TL;DR:
Chat is one collection. ChatMess another one that has messages refering to a Chat's _id. How do I get the last messages from a list of chats with the less computation possible ? Here, find / fetch cycle in a loop is way too heavy and long.
I have this publication that is used to return a set of cursor to the user :
The chats sessions he takes part in (from Chat collection)
The last message from each of the chat session referenced in the first cursor (from ChatMess collection)
Currently, the logic is to :
Get the list of chat sessions from the user profile
Find the Chat sessions and loop through it
In the loop, I findOne the last message from this chat session and store its _id in an array. In addition, I store all the other users _ids.
Then, I find the messages which _id match the ones in my array.
Here is my main problem :
Isn't there a way more faster way to get the last messages from each of my chat session ? With that algo, I easily reach the 8000ms of response time, which is a way too heavy computation time, as much of this time is spent to find / fetch the chat messages's _id (cf linked screen from Kadira).
Meteor.publish("publishNewChat", function() {
this.unblock();
// we get a list of chat _id
let chatIdList = _get_all_the_user_chats_ids(this.userId);
if (!chatList)
return ;
// get the chat sessions objects
let chats_cursor = Modules.both.queryGet({
type : 'chat',
method : 'find',
query : { _id: { $in: chatIdList } },
projection : { sort: { _id: 1 }, limit : 1000 }
});
let array_of_fetched_chats = chats_cursor.fetch();
let chat_ids = [];
// and here we loop through the chat documents in order to get the last message that's been attached to each of them
array_of_fetched_chats.forEach(function(e) {
let lastMess = Modules.both.queryGet({
type : 'chatMess',
method : 'findOne',
query : { chatId: e._id },
projection : { sort: { date: -1 } }
});
if (lastMess)
chat_ids.push(lastMess._id);
});
return ([
chats_cursor,
Modules.both.queryGet({
type : 'chatMess',
method : 'find',
query : { _id: { $in: chat_ids } },
projection : { sort: { date: -1 }, limit: 1000 }
})
]);
});
Finally, it also add latence to all my DDP request that follows. I currently use a this.unblock() to avoid that, but I'd prefer not to use it here.
FYI, I have another publish that is updated each time the client change his current active chat session : on the client, routing to a new chat add its _id in a reactive array that update my getChatMess subscription in order to get on the client the messages from every chats the user visited in this since he connected. The goal is obviously to spare the server the sending of every message from every chat session the user have visited in his life.
Unfortunately, I lack ideas to improve that algo without breaking all my chat logic :S. Have you any idea ? How would you do ?
Thanks you.
EDIT: here is a screen from kadira that clearly show the problem :

Have you considered using the reywood/publishComposite package?
With this package you can publish related data in the same method without having to do a bunch of logic to get the correct data published.
The below code should get you started:
Meteor.publishComposite("publishNewChat", function() {
return [{
find:function(){
return Users.find({ _id: this.userId },{fields:{"profile.chat":1}});
},
children:[{
find:function(user){ //this function is passed each user returned from the cursor above.
return UserChats.find({userId:user._id},{fields:{blah:1,blah:1}}); //find the user chats using whatever query
},
children:[
//if there are any children of user chats that you need to publish, do so here...
{
find:function(userchat){
return Chats.find({_id:userchat.chatId})
},
children:[
{
find:function(chat){
return ChatMess.find({chatId:chat._id},{ sort: { date: -1 } });
},
children:[
{
find:function(chatMess){
var uids = _.without(chatMess.participants, this.userId);
return Users.find({_id:{$in:uids}});
}
}
]
}
]
}
]
},
]
}]
This will publish the cursors for all of the documents related to each of the parent documents. It is pretty fast, I use this package on a production platform high traffic and large datasets with no problems. On the client you could then query the documents as normal to get the ones you need to display.
Something like:
Users.findOne({_id:Meteor.userId()});
UserChats.find({userId:Meteor.userId()});
etc...

Here is a solution I developped :
Meteor.publish("publishNewChat", function() {
this.unblock();
let user = Modules.both.queryGet({
type : 'users',
method : 'findOne',
query : { _id: this.userId },
projection : { fields: { "profile.chat": true } }
});
let thisUserschats = tryReach(user, "profile", "chat").value;
if (!thisUserschats)
return ;
thisUserschats = thisUserschats.map(function(e) { return (e.chatId); });
let chats = Modules.both.queryGet({
type : 'chat',
method : 'find',
query : { _id: { $in: thisUserschats } },
projection : { sort : { _id: 1 },
limit : 1000
}
});
let chatArray = chats.fetch(),
uids = cmid = [];
let messages_id_list = [],
i = chatArray.length;
let _parallelQuery = index => {
Meteor.setTimeout(function () {
let tmp = Modules.both.queryGet({
type : 'chatMess',
method : 'find',
query : { chatId: chatArray[index]._id },
projection: { limit: 1, sort: { date: -1 } }
});
tmp.forEach(doc => {
messages_id_list.push((doc && doc._id) ? doc._id : null);
});
}, 1);
}
while (--i >= 0)
_parallelQuery(i);
let cursors = {
chats : chats,
chatMessages : null
}
let interval = Meteor.setInterval(function () {
if (messages_id_list.length === chatArray.length)
{
Meteor.clearInterval(interval);
cursors.chatMessages = Modules.both.queryGet({
type : 'chatMess',
method : 'find',
query : { _id: { $in: messages_id_list } },
projection : { sort: { date: -1 }, limit: 1000 }
});
cursors.chats.observeChanges({
// ...
});
cursors.chatMessages.observeChanges({
// ...
});
self.ready();
self.onStop(() => subHandle.stop(); );
}
}, 10);
});
I used async function with Meteor.setTimeout to parallelize the queries and save an index refering to a chat _id to look for. Then, when a query is finished, I add the last message to an array. With a Meteor.setInterval, I check the array length to know when all the queries are done. Then, as I can't return cursors anymore, I use the Meteor publication low level API to handle the publishing of the documents.
FYI : in a first attempt, I was using 'findOne' in my _parallelQueries, which divided my computation time by 2/3. But then, thanks to a friend, I tried the cursor.foreach() function, which allowed me to divide the computation time by 2 again !
In production, the benchmarks allowed me to go from a 7/8 second response time to an average response time of 1.6 second :)
Hope this will be usefull to you people ! :)

Updating MongoDB

Does anyone know how I can update in MongoDB.
I want to update the totalVisits with timesvisted
// Test data
var currentUser = "John"
var currentPage = "pageName"
var timesvisited = 59
Page.find({"_id" : currentUser}, [], {},function(err, pages) {
pages = pages.map(function(ud) {
return { userDetails: ud};
});
//database structure example
{ "_id" : "John",
"pageName" : { "totalVisits" : 58,
"timeOnPage" : 2432,
"lastVisitDate" : "10/11/2011",
"clickNoOnPage" : "5"
},
"anotherPageName" : { "totalVisits" : 18,
"timeOnPage" : 5362,
"lastVisitDate" : "01/10/2011",
"clickNoOnPage" : "15"
I am trying to update the totalVisits value and have tried something like
{$set : { pages[0].userDetails[currentPage].totalVisits : timesvisted}}
However I get a "SyntaxError: Unexpected token [" message
One of the problems I am having is with the [currentPage] section as currentPage can change so I can not hard code the pageName in.
Edit ***
I have modified this line
lastVisitedSiteDate = {$set : { "pageName.totalVisits" : timesvist}};
and this works fine. However, I need the pageName not to be hard coded in, its needs to be something like currentPage so different page names can be passed into it e.g. anotherPageName.

The second parameter to Mongo's find function specifies which fields to pull back. It needs to be a document, as in {}, not an array.