We have an application where we're storing two types of documents in a Mongo database:
contacts, which basically represent people
filters, which are essentially a stored MongoDB query that represents a "saved search" for a user.
Here's a simplified version of what the data models would look like:
contacts: [
{ id: 1, name: 'Phil', age: 40 },
{ id: 2, name: 'Bob', age: 34 }
]
filters: [
{ query: { name: 'Phil' } }
{ query: { age: { >: 30 } } }
]
Given a filter, it's relatively easy to list all contacts that match that filter:
db.contacts.find(filter.query);
What's harder is finding all filters that match a certain contact. Right now we have something like the following:
matchedFilters = []
_.each(filters, function(filter) {
if (db.contacts.find(_.extend(filter.query, {id: contact_id}).length > 0) {
matchedFilters.push(filter.id)
}
});
Essentially, we need to ask mongo about each filter individually. This results in a huge amount of queries to Mongo.
At the time that we are evaluating this query, we have all the relevant information about the contact we are trying to find. Is there any way to apply the Mongo query syntax to an in-memory Javascript object without needing to ask Mongo about it?
Alternatively, is there a way to ask Mongo to conduct a large number of queries in a single round trip?
Have a look at sift.js. I think, it's exactly what you're looking for.
And here is a blog post about it.
Related
I have a collection of persons with a group property. Group value is a random string, which I don't know when I make my query. Some persons group values will possibly be the same.
I want to write a query to retrieve only one person per group (which person is returned in the group is non relevant for the moment).
I suppose I have to write some kind of aggregation. But I only read about examples aggregating on known values... Instead I need to aggregate on unknown values...
My schema is something like:
const schemaPersons = new Schema({
name: {
type: String,
required: true,
},
group: {
type: String,
},
};
You can write aggregation query using $group stage:
$group stage, group by group field, and get only first person's root document in person field
Persons.aggregate([
{
$group: {
_id: "$group",
person: { $first: "$$ROOT" }
}
}
]);
Playground
I've a collection of countries with country calling code in the country object. How can I find a country using calling code with a mobile number?
const countries = [
{
name: 'UAE',
callingCode: 971
},
{
name: 'USA',
callingCode: 1
},
{
name: 'UK',
callingCode: 44
}
];
const number = '971524500000'; // Input
How can I find country for the given mobile using regex in mongoose javascript;
[https://en.wikipedia.org/wiki/List_of_country_calling_codes][1]
Take a look at the link above on country calling codes, and specifically see the section "Tree List".
One solution would be to implement a schema in Mongo based on this tree in order to decode the country codes.
So, a table could be created to store Mongo documents containing a field "1x" "2x" "21x" etc (the Y axis in the Tree List table).
Each of these documents could contain an array of sub-documents from x=0 to x=9 (the x axis in the Tree List table). The sub-document can contain the country name/code you are looking for. You can use a direct index into the array in the Mongo document for an efficient lookup.
I think you'll find this to be a pretty efficient implementation and should cover all the bases.
If you can restructure your array to an object this would be the fastest
const countries =
{
971: 'UAE',
1: 'USA',
44: 'UK',
}
;
var code = 44;
console.log(countries[code]);
const countries = [
{
name: 'UAE',
callingCode: 971
},
{
name: 'USA',
callingCode: 1
},
{
name: 'UK',
callingCode: 44
}
];
var myFound =countries.filter(myFunc.bind(this,44));
function myFunc(code,element) {
if(element.callingCode == code){
return element;
}
}
console.log(myFound);
On MongoDB v 4.2 - you can use $let & $regexFind to do this :
db.collection.aggregate([
{
$match: {
$expr: {
$eq: [
{
$let: {
vars: {
value: {
$regexFind: {
input: "971524500000", // pass in as string
regex: { $toString: "$callingCode" }
}
}
},
in: "$$value.idx",
}
},
0
]
}
}
}
]);
Test : MongoDB-Playground
Explanation :
General Use-case :
In general regex searches - Will have an input which will be sub-string of actual string, Ex.:-
Actual string in DB : 'We can do it in mongo'
Input : mongo (/mongo/ - will work)
Your Use-case :
From above case as mongo exists in actual string/value of db field then you can get that document, But your case is different :
Actual string in DB : mongo
Input : 'We can do it in mongo'
This doesn't work that way, So using normal /We can do it in mongo/ isn't going help you here (Also doing few tricks with regex). So we need to make a trick to $regexFind operator. Unlike mongo documentation we need take 971524500000 into input field & regex as string value of callingCode field which is vice-versa to what's given in documentation.
So once we do that, We would get something like below :
{
"match" : "971", /** value of callingCode field */
"idx" : 0, /** Index of `971` in '971524500000' */
"captures" : []
},{
"match" : "1",
"idx" : 2,
"captures" : []
},
null /** 3rd doc no match */
As country code has to be at first of given number we need docs where "idx" : 0 - So we're using $let to get index of returned object & checking against 0 & eventually getting respective docs using $match.
Note :
There is something you need to look into, Just in case if you've docs like below :
{
"_id": ObjectId("5e8f67091aa1cc3d2158ade1"),
"name": "USA",
"callingCode": 1.0
},
{
"_id": ObjectId("5e8f67091aa1cc3d2158ade3"),
"name": "somecountry",
"callingCode": 197.0
}
& input is 1971524500000, then this query will bring both docs in result. This will be the case you need to check on. Anyhow I would suggest to try this query, rather than getting all documents for collection to the code & extract required o/p this might be better to do.
I would like to know how to keep track of the values of a document in MongoDB.
It's a MongoDB Database with a Node and Express backend.
Say I have a document, which is part of the Patients collection.
{
"_id": "4k2lK49938d82kL",
"firstName": "John",
"objective": "Burn fat"
}
Then I edit the "objective" property, so the document results like this:
{
"_id": "4k2lK49938d82kL",
"firstName": "John",
"objective": "Gain muscle"
}
What's the best/most efficient way to keep track of that change? In other words, I would like to know that the "objective" property had the value "Burn fat" in the past, and access it in the future.
Thanks a lot!
Maintaining/tracking history in the same document is not all recommended. As the document size will keep on increasing leading to
probably if there are too many updates, 16mb document size limit
Performance degrades
Instead, you should maintain a separate collection for history. You might have use hibernates' Javers or envers for auditing for your relational databases. if not you can check how they work. A separate table (xyz_AUD) is maintained for each table (xyz). For each row (with primary key abc) in xyz table, there exist multiple rows in xyz_AUD table, where each row is version of that row.
Moreover, Javers also support MongoDB auditing. If you are using java you can directly use it. No need to write your own logic.
Refer - https://nullbeans.com/auditing-using-spring-boot-mongodb-and-javers/
One more thing, Javers Envers Hibernate are java libraries. But I'm sure for other programming languages also, similar libraries will be present.
There is a mongoose plugin as well -
https://www.npmjs.com/package/mongoose-audit (quite oudated 4 years)
https://github.com/nassor/mongoose-history#readme (better)
Maybe you can change the type of "objective" to array and track the changes in it. the last one of the array is the latest value.
Maintain it as a sub-document like below
{
"_id": "4k2lK49938d82kL",
"firstName": "John",
"objective": {
obj1: "Gain muscle",
obj2: "Burn fat"
}
}
You can also maintain it as an array field but remember, mongodb doesn't allow you to maintain uniqueness in an array field and if you plan to index the "objective" field, you'll have to create a multi key index
I think the simplest solution would be to use and update an array:
const patientSchema = new Schema({
firstName: { type: String, required: true },
lastName: { type: String, required: true },
objective: { type: String, required: true }
notes: [{
date: { type: Date, default: Date.now() },
note: { type: String, required: true }
}],
});
Then when you want to update the objective...
const updatePatientObjective = async (req, res) => {
try {
// check if _id and new objective exist in req.body
const { _id, objective, date } = req.body;
if (!_id || !objective) throw "Unable to update patient's objective.";
// make sure provided _id is valid
const existingPatient = await Patient.findOne({ _id });
if (!existingPatient) throw "Unable to locate that patient.";
// pull out objective as previousObjective
const { objective: previousObjective } = existingPatient;
// update patient's objective while pushing
// the previous objective into the notes sub document
await existingPatient.updateOne({
// update current objective
$set { objective },
// push an object with a date and note (previouseObjective)
// into a notes array
$push: {
notes: {
date,
note: previousObjective
},
},
}),
);
// send back response
res
.status(201)
.json({ message: "Successfully updated your objective!" });
} catch (err) {
return res.status(400).json({ err: err.toString() });
}
};
Document will look like:
firstName: "John",
lastName: "Smith",
objective: "Lose body fat.",
notes: [
{
date: 2019-07-19T17:45:43-07:00,
note: "Gain muscle".
},
{
date: 2019-08-09T12:00:38-07:00,
note: "Work on cardio."
}
{
date: 2019-08-29T19:00:38-07:00,
note: "Become a fullstack web developer."
}
...etc
]
Alternatively, if you're worried about document size, then create a separate schema for patient history and reference the user's id (or just store the patient's _id as a string instead of referencing an ObjectId, whichever you prefer):
const patientHistorySchema = new Schema({
_id: { type: Schema.Types.ObjectId, ref: "Patient", required: true },
objective: { type: String, required: true }
});
Then create a new patient history document when the objective is updated...
PatientHistory.create({ _id, objective: previousObjective });
And if you need to access to the patient history documents...
PatientHistory.find({ _id });
I'm writing a node.js app that uses the pg package for accessing a PostgreSQL database. The issue I'm running into is that if I do a query like this:
select * from posts p inner join blogs b on b.id = p.blog_id
When I get the results, they're all in the same namespace, so any field repeated in the blogs table will overwrite those in the posts table.
My question is, what's the best way of binding these results to objects?
Ideally, I'd like a result like:
{
id: 1,
name: 'A post name',
published_at: (some date object),
blog_id: 1,
b: {
id: 1,
name: 'A blog name'
}
}
But I'm open to any convenient solution short of adding an alias for every column manually.
http://www.postgresql.org/docs/9.3/static/functions-json.html
http://www.postgresql.org/docs/9.4/static/functions-aggregate.html
You may want to look at the json features of Postgres. If I'm understanding you right, and without a test database something like this may be close to what you're looking for:
SELECT
p.*, /* Select all the post fields */
row_to_json(blogs.*) as b /* Use the row_to_json function on the blogs results */
FROM
posts p
INNER JOIN
blogs ON (blogs.id=p.blog_id); /* Join blogs on the proper fields */
Returns:
{
id: 3,
name: 'test',
published_at: 2015-10-08,
blog_id: 2,
b: {
id:2,
name:"test 2"
}
}
Here's a great tutorial on them:
http://bender.io/2013/09/22/returning-hierarchical-data-in-a-single-sql-query/
If you change your query to
'SELECT * FROM posts, blogs WHERE posts.id = blogs.id;'
you should have your column names prefixed with either 'posts' or 'blogs'
If you want a nested result like above, you'll have to run some manual processing.
res.map(d => {
return {
id: d.posts_id
b : {
id: d.blogs_id
}
};
});
I am currently using StrongLoop as my API backend server and Mongodb as data storage engine.
Let's say there is a collection called article. It has two fields title, and content. And there are two frontend pages to display a list of articles and view a single article.
Obviously the data list page only need title field and the view page need both. Currently the GET method of StrongLoop API return all fields including content. It cost extra traffic. Is there any way that can just return specific field?
Mongodb support projection in find() method for this. How can I do the same thing by StrongLoop?
Have you taken a look at the filters offered. http://docs.strongloop.com/display/LB/Querying+models
Query for NodeAPI:
server.models.Student.findOne({where: {RFID: id},fields: {id: true,schoolId: true,classId: true}}, function (err, data) {
if (err)
callback(err);
else {
callback();
}
})
Query for RestAPI :
$http.get('http://localhost:3000/api/services?filter[fields][id]=true&filter[fields][make]=true&filter[fields][model]=true')
.then(function (response) {
}, function (error) {
});
You can use fields projections,
Sample Record:
{ name: 'Something', title: 'mr', description: 'some desc', patient: { name: 'Asvf', age: 20, address: { street: 1 }}}
First Level Projection:
model.find({ fields: { name: 1, description: 1, title: 0 } })
and I think Strong loop is not yet supporting for second-level object filter, does anyone know how to filter second-level object properties or is yet to implement?.
Second Level Projection: (Need help here)
Ex: 2
model.find({ fields: { name: 1, 'patient.name': 1, 'patient.age': 1, 'patient.address': 0 } })
// Which results { name } only