We Keep Coding

Mongoose - Deleting documents is unresponsive

I'm trying to use Mongoose (MongoDB JS library) to create a basic database, but I can't figure out how to delete the documents / items, I'm not sure what the technical term for them is.
Everything seems to work fine, when I use Item.findById(result[i].id), it returns a valid id of the item, but when I use Item.findByIdAndDelete(result[i].id), the function doesn't seem to start at all.
This is a snippet the code that I have: (Sorry in advance for bad indentation)
const testSchema = new schema({
item: {
type: String,
required: true
},
detail: {
type: String,
required: true
},
quantity: {
type: String,
required: true
}
})
const Item = mongoose.model("testitems", testSchema)
Item.find()
.then((result) => {
for (i in result) {
Item.findByIdAndDelete(result[i].id), function(err, result) {
if (err) {
console.log(err)
}
else {
console.log("Deleted " + result)
}
}
}
mongoose.connection.close()
})
.catch((err) => {
console.log(err)
})
I'm not sure what I'm doing wrong, and I haven't been able to find anything on the internet.
Any help is appreciated, thanks.

_id is a special field on MongoDB documents that by default is the type ObjectId. Mongoose creates this field for you automatically. So a sample document in your testitems collection might look like:
{
_id: ObjectId("..."),
item: "xxx",
detail: "yyy",
quantity: "zzz"
}
However, you retrieve this value with id. The reason you get a value back even though the field is called _id is because Mongoose creates a virtual getter for id:
Mongoose assigns each of your schemas an id virtual getter by default which returns the document's _id field cast to a string, or in the case of ObjectIds, its hexString. If you don't want an id getter added to your schema, you may disable it by passing this option at schema construction time.
The key takeaway is that when you get this value with id it is a string, not an ObjectId. Because the types don't match, MongoDB will not delete anything.
To make sure the values and types match, you should use result[i]._id.

What is the best way to keep track of changes of a document's property in MongoDB?

I would like to know how to keep track of the values of a document in MongoDB.
It's a MongoDB Database with a Node and Express backend.
Say I have a document, which is part of the Patients collection.
{
"_id": "4k2lK49938d82kL",
"firstName": "John",
"objective": "Burn fat"
}
Then I edit the "objective" property, so the document results like this:
{
"_id": "4k2lK49938d82kL",
"firstName": "John",
"objective": "Gain muscle"
}
What's the best/most efficient way to keep track of that change? In other words, I would like to know that the "objective" property had the value "Burn fat" in the past, and access it in the future.
Thanks a lot!

Maintaining/tracking history in the same document is not all recommended. As the document size will keep on increasing leading to
probably if there are too many updates, 16mb document size limit
Performance degrades
Instead, you should maintain a separate collection for history. You might have use hibernates' Javers or envers for auditing for your relational databases. if not you can check how they work. A separate table (xyz_AUD) is maintained for each table (xyz). For each row (with primary key abc) in xyz table, there exist multiple rows in xyz_AUD table, where each row is version of that row.
Moreover, Javers also support MongoDB auditing. If you are using java you can directly use it. No need to write your own logic.
Refer - https://nullbeans.com/auditing-using-spring-boot-mongodb-and-javers/
One more thing, Javers Envers Hibernate are java libraries. But I'm sure for other programming languages also, similar libraries will be present.
There is a mongoose plugin as well -
https://www.npmjs.com/package/mongoose-audit (quite oudated 4 years)
https://github.com/nassor/mongoose-history#readme (better)

Maybe you can change the type of "objective" to array and track the changes in it. the last one of the array is the latest value.

Maintain it as a sub-document like below
{
"_id": "4k2lK49938d82kL",
"firstName": "John",
"objective": {
obj1: "Gain muscle",
obj2: "Burn fat"
}
}
You can also maintain it as an array field but remember, mongodb doesn't allow you to maintain uniqueness in an array field and if you plan to index the "objective" field, you'll have to create a multi key index

I think the simplest solution would be to use and update an array:
const patientSchema = new Schema({
firstName: { type: String, required: true },
lastName: { type: String, required: true },
objective: { type: String, required: true }
notes: [{
date: { type: Date, default: Date.now() },
note: { type: String, required: true }
}],
});
Then when you want to update the objective...
const updatePatientObjective = async (req, res) => {
try {
// check if _id and new objective exist in req.body
const { _id, objective, date } = req.body;
if (!_id || !objective) throw "Unable to update patient's objective.";
// make sure provided _id is valid
const existingPatient = await Patient.findOne({ _id });
if (!existingPatient) throw "Unable to locate that patient.";
// pull out objective as previousObjective
const { objective: previousObjective } = existingPatient;
// update patient's objective while pushing
// the previous objective into the notes sub document
await existingPatient.updateOne({
// update current objective
$set { objective },
// push an object with a date and note (previouseObjective)
// into a notes array
$push: {
notes: {
date,
note: previousObjective
},
},
}),
);
// send back response
res
.status(201)
.json({ message: "Successfully updated your objective!" });
} catch (err) {
return res.status(400).json({ err: err.toString() });
}
};
Document will look like:
firstName: "John",
lastName: "Smith",
objective: "Lose body fat.",
notes: [
{
date: 2019-07-19T17:45:43-07:00,
note: "Gain muscle".
},
{
date: 2019-08-09T12:00:38-07:00,
note: "Work on cardio."
}
{
date: 2019-08-29T19:00:38-07:00,
note: "Become a fullstack web developer."
}
...etc
]
Alternatively, if you're worried about document size, then create a separate schema for patient history and reference the user's id (or just store the patient's _id as a string instead of referencing an ObjectId, whichever you prefer):
const patientHistorySchema = new Schema({
_id: { type: Schema.Types.ObjectId, ref: "Patient", required: true },
objective: { type: String, required: true }
});
Then create a new patient history document when the objective is updated...
PatientHistory.create({ _id, objective: previousObjective });
And if you need to access to the patient history documents...
PatientHistory.find({ _id });

How can I query by sort key in AWS DynamoDB?

I have a database of users, and I'd like to return all users with the last name X. I currently have "last" as the sort key for my table. I'm running the following code, but getting a "Query condition missed key schema element" error.
var params = { TableName: 'Patients',
KeyConditionExpression: '#id = :idNum',
ExpressionAttributeNames: {
'#id': 'last'
},
ExpressionAttributeValues: {
':idNum': 'Test'
}
};
docClient.query(params, function(err, data) {
if (err) {
console.error("Unable to query. Error:", JSON.stringify(err, null, 2));
} else {
console.log("Query succeeded.");
res.json(data);
}
});

To query you must provide a partition key, so with your table, as it is, your only option would be to do a Scan (which is expensive and almost never recommended).
However you can easily add Global Secondary Indexes to allow you to use another key as your partition.
In this case you can add a GSI with last as your partition key.
Then you would be able to Query the global secondary index (note the IndexName parameter) by last name.
If you want to go the scan route, however, you'll need to use:
docClient.scan({
TableName: 'Patients',
FilterExpression: '#l = :l',
ExpressionAttributeNames: { '#l': 'last' },
ExpressionAttributeValues: { ':l': 'Test' },
Limit: 1
}, (err, data) => { /* callback */ })
Just remember that using scans can get expensive and put a strain on your table quickly (more detail here)

This is not the way DynamoDB/NoSQL should be used. You either create a GSI as thomasmichaelwallace suggested or have the users lastname as partition key too. You can add "ln_" before all of your last name partitionkeys to separte them from your other partition keys. If you had firstname as partitions before your primary key should look like that now:
partition | sort
fn_hans | ln_zimmer
fn_steve | ln_jobs
ln_zimmer | fn_hans
ln_jobs | fn_steve
Now you can query all partition which start with ln_. And yes you are supposed to have douplicate data in noSql. NoSQL Design has some articles about how to design a dynamoDB table.

Using results from one RethinkDB query in another?

I want to send a http post with an address, which will grab a doc from the database. I would like to then use that doc's geodata in a getNearest function, ultimately returning the nearest four other locations. How do I go about stringing these two queries together?
r.table('dealer_locations').filter(function(dealer) {
return {
dealer : ("Address").match(request.body.Address)
.getNearest(dealer, {
index: 'geodata',
maxResults: 4
})
}
}).run(conn, function(error, cursor) {
if (error) {
handleError(response, error);
} else {
cursor.toArray(function(error, results) {
if (error) {
handleError(response, error);
} else {
response.send(results);
}
});
}
});

I'll reformulate the question just to make a bit more clear:
Problem
Given a specific document with geodata, I want to also return the four nearest locations to that location.
Solution
First, make sure that you've creating the geo index in the table you want to query:
r.table('dealer_locations').indexCreate('location', { geo: true })
After that, make sure that you've inserted a r.point object into the document. You can't just use geo indexes on just any property. They have to be r.points.
r.table('dealer_locations')
.insert({
name: 'San Francisco',
location: r.point(37.774929, -122.419416)
})
After you've inserted all your documents and they all have r.points property on the same property you created an index for, now you can start querying them.
If you want to get all the nearest locations for a location, you can do as follows:
r.table('dealer_locations')
.filter({ name: 'San Francisco' })(0)
.do(function (row) {
return r.table('dealer_locations')
.getNearest(row('location'), { index: 'location', maxResults: 4 })
})
If you want to append the closets locations to a document, so that you can return both the document and the nearest locations at the same time, you can do that using the merge method.
r.table('dealer_locations')
.filter({ name: 'San Francisco' })(0)
.merge(function (row) {
return {
nearest_locations: r.table('dealer_locations')
.getNearest(row('location'), { index: 'location', maxResults: 4 })
}
})
Finally, if you want to get all the nearest locations, based on an address (supposing your document has both an address property with a string and a location property with an r.point), you can do something like this:
r.table('dealer_locations')
.filter(r.row('address').match(request.body.Address))
(0) // Only get the first document
.do(function (row) {
// Return the 4 documents closest to the 'location' if the previous document
return r.table('dealer_locations')
.getNearest(row('location'), { index: 'location', maxResults: 4 })
})
That being said, the problem with this might be that you might match multiple addresses which are not necessarily the ones you want to match!

How to get result with specific fields in StrongLoop?

I am currently using StrongLoop as my API backend server and Mongodb as data storage engine.
Let's say there is a collection called article. It has two fields title, and content. And there are two frontend pages to display a list of articles and view a single article.
Obviously the data list page only need title field and the view page need both. Currently the GET method of StrongLoop API return all fields including content. It cost extra traffic. Is there any way that can just return specific field?
Mongodb support projection in find() method for this. How can I do the same thing by StrongLoop?

Have you taken a look at the filters offered. http://docs.strongloop.com/display/LB/Querying+models

Query for NodeAPI:
server.models.Student.findOne({where: {RFID: id},fields: {id: true,schoolId: true,classId: true}}, function (err, data) {
if (err)
callback(err);
else {
callback();
}
})
Query for RestAPI :
$http.get('http://localhost:3000/api/services?filter[fields][id]=true&filter[fields][make]=true&filter[fields][model]=true')
.then(function (response) {
}, function (error) {
});

You can use fields projections,
Sample Record:
{ name: 'Something', title: 'mr', description: 'some desc', patient: { name: 'Asvf', age: 20, address: { street: 1 }}}
First Level Projection:
model.find({ fields: { name: 1, description: 1, title: 0 } })
and I think Strong loop is not yet supporting for second-level object filter, does anyone know how to filter second-level object properties or is yet to implement?.
Second Level Projection: (Need help here)
Ex: 2
model.find({ fields: { name: 1, 'patient.name': 1, 'patient.age': 1, 'patient.address': 0 } })
// Which results { name } only