{
"INFO": {
"email": "test#example.com",
"password": "123"
},
"PK": "3a95eab0-57de-4e15-90ea-004082e53384",
"SK": "user"
}
Above is my dataset in dynamoDB. I am building login api with expressjs with dynamodb. I am able to scan and update data with PK & SK keys but i want to query inside my INFO set.
I am trying like this:
var params = {
TableName: "table",
FilterExpression: "contains (INFO, :sendToVal)",
ExpressionAttributeValues: {
":sendToVal": {
email: "test#example.com",
password: "123",
},
},
};
But its returning:
{ error: 'Error retrieving Event' }
{ error: 'Event not found' }
Anyone help guide me, how can i retrive the set.
The DynamoDB documentation explains that the contains() function in an expression only works for strings or sets. This isn't completely accurate - it also works for lists. But in any case, it doesn't work for maps, which is what your INFO is, so the comparison doesn't match anything.
If you intended for INFO to be a list, not a map, please make it so. Otherwise, if you really intended for it to be a map, and you wanted to test whether { email: "test#example.com", password: "123" } is in that map, then what you really need to check is whether the email and password entries in this map is equal to the desired value. So the filter condition can be something like INFO.email = :email AND INFO.password = :password. Or something like this (I'm not sure I understannd what your intention was here).
Related
I'm using Dynamoose to simplify my interactions with DynamoDB in a node.js application. I'm trying to write a query using Dynamoose's Model.query function that will search a table using an index, but it seems like Dynamoose is not including all of the info required to process the query and I'm not sure what I'm doing wrong.
Here's what the schema looks like:
const UserSchema = new dynamoose.Schema({
"user_id": {
"hashKey": true,
"type": String
},
"email": {
"type": String,
"index": {
"global": true,
"name": "email-index"
}
},
"first_name": {
"type": String,
"index": {
"global": true,
"name": "first_name-index"
}
},
"last_name": {
"type": String,
"index": {
"global": true,
"name": "last_name-index"
}
}
)
module.exports = dynamoose.model(config.usersTable, UserSchema)
I'd like to be able to search for users by their email address, so I'm writing a query that looks like this:
Users.query("email").contains(query.email)
.using("email-index")
.all()
.exec()
.then( results => {
res.status(200).json(results)
}).catch( err => {
res.status(500).send("Error searching for users: " + err)
})
I have a global secondary index defined for the email field:
When I try to execute this query, I'm getting the following error:
Error searching for users: ValidationException: Either the KeyConditions or KeyConditionExpression parameter must be specified in the request.
Using the Dynamoose debugging output, I can see that the query winds up looking like this:
aws:dynamodb:query:request - {
"FilterExpression": "contains (#a0, :v0)",
"ExpressionAttributeNames": {
"#a0": "email"
},
"ExpressionAttributeValues": {
":v0": {
"S": "mel"
}
},
"TableName": "user_qa",
"IndexName": "email-index"
}
I note that the actual query sent to DynamoDB does not contain KeyConditions or KeyConditionExpression, as the error message indicates. What am I doing wrong that prevents this query from being written correctly such that it executes the query against the global secondary index I've added for this table?
As it turns out, calls like .contains(text) are used as filters, not query parameters. DynamoDB can't figure out if the text in the index contains the text I'm searching for without looking at every single record, which is a scan, not a query. So it doesn't make sense to try to use .contains(text) in this context, even though it's possible to call it in a chain like the one I constructed. What I ultimately needed to do to make this work is turn my call into a table scan with the .contains(text) filter:
Users.scan({ email: { contains: query.email }}).all().exec().then( ... )
I am not familiar with Dynamoose too much but the following code below will do an update on a record using node.JS and DynamoDB. See the key parameter I have below; by the error message you got it seems you are missing this.
To my knowledge, you must specify a key for an UPDATE request. You can checks the AWS DynamoDB docs to confirm.
var params = {
TableName: table,
Key: {
"id": customerID,
},
UpdateExpression: "set customer_name= :s, customer_address= :p, customer_phone= :u, end_date = :u",
ExpressionAttributeValues: {
":s": customer_name,
":p": customer_address,
":u": customer_phone
},
ReturnValues: "UPDATED_NEW"
};
await docClient.update(params).promise();
I would like to know how to keep track of the values of a document in MongoDB.
It's a MongoDB Database with a Node and Express backend.
Say I have a document, which is part of the Patients collection.
{
"_id": "4k2lK49938d82kL",
"firstName": "John",
"objective": "Burn fat"
}
Then I edit the "objective" property, so the document results like this:
{
"_id": "4k2lK49938d82kL",
"firstName": "John",
"objective": "Gain muscle"
}
What's the best/most efficient way to keep track of that change? In other words, I would like to know that the "objective" property had the value "Burn fat" in the past, and access it in the future.
Thanks a lot!
Maintaining/tracking history in the same document is not all recommended. As the document size will keep on increasing leading to
probably if there are too many updates, 16mb document size limit
Performance degrades
Instead, you should maintain a separate collection for history. You might have use hibernates' Javers or envers for auditing for your relational databases. if not you can check how they work. A separate table (xyz_AUD) is maintained for each table (xyz). For each row (with primary key abc) in xyz table, there exist multiple rows in xyz_AUD table, where each row is version of that row.
Moreover, Javers also support MongoDB auditing. If you are using java you can directly use it. No need to write your own logic.
Refer - https://nullbeans.com/auditing-using-spring-boot-mongodb-and-javers/
One more thing, Javers Envers Hibernate are java libraries. But I'm sure for other programming languages also, similar libraries will be present.
There is a mongoose plugin as well -
https://www.npmjs.com/package/mongoose-audit (quite oudated 4 years)
https://github.com/nassor/mongoose-history#readme (better)
Maybe you can change the type of "objective" to array and track the changes in it. the last one of the array is the latest value.
Maintain it as a sub-document like below
{
"_id": "4k2lK49938d82kL",
"firstName": "John",
"objective": {
obj1: "Gain muscle",
obj2: "Burn fat"
}
}
You can also maintain it as an array field but remember, mongodb doesn't allow you to maintain uniqueness in an array field and if you plan to index the "objective" field, you'll have to create a multi key index
I think the simplest solution would be to use and update an array:
const patientSchema = new Schema({
firstName: { type: String, required: true },
lastName: { type: String, required: true },
objective: { type: String, required: true }
notes: [{
date: { type: Date, default: Date.now() },
note: { type: String, required: true }
}],
});
Then when you want to update the objective...
const updatePatientObjective = async (req, res) => {
try {
// check if _id and new objective exist in req.body
const { _id, objective, date } = req.body;
if (!_id || !objective) throw "Unable to update patient's objective.";
// make sure provided _id is valid
const existingPatient = await Patient.findOne({ _id });
if (!existingPatient) throw "Unable to locate that patient.";
// pull out objective as previousObjective
const { objective: previousObjective } = existingPatient;
// update patient's objective while pushing
// the previous objective into the notes sub document
await existingPatient.updateOne({
// update current objective
$set { objective },
// push an object with a date and note (previouseObjective)
// into a notes array
$push: {
notes: {
date,
note: previousObjective
},
},
}),
);
// send back response
res
.status(201)
.json({ message: "Successfully updated your objective!" });
} catch (err) {
return res.status(400).json({ err: err.toString() });
}
};
Document will look like:
firstName: "John",
lastName: "Smith",
objective: "Lose body fat.",
notes: [
{
date: 2019-07-19T17:45:43-07:00,
note: "Gain muscle".
},
{
date: 2019-08-09T12:00:38-07:00,
note: "Work on cardio."
}
{
date: 2019-08-29T19:00:38-07:00,
note: "Become a fullstack web developer."
}
...etc
]
Alternatively, if you're worried about document size, then create a separate schema for patient history and reference the user's id (or just store the patient's _id as a string instead of referencing an ObjectId, whichever you prefer):
const patientHistorySchema = new Schema({
_id: { type: Schema.Types.ObjectId, ref: "Patient", required: true },
objective: { type: String, required: true }
});
Then create a new patient history document when the objective is updated...
PatientHistory.create({ _id, objective: previousObjective });
And if you need to access to the patient history documents...
PatientHistory.find({ _id });
I'm trying to achieve a "getOrCreate" behavior using "findAndModify".
I'm working in nodejs using the native driver.
I have:
var matches = db.collection("matches");
matches.findAndModify({
//query
users: {
$all: [ userA_id, userB_id ]
},
lang: lang,
category_id: category_id
},
[[ "_id", "asc"]], //order
{
$setOnInsert: {
users: [userA_id, userB_id],
category_id: category_id,
lang: lang,
status: 0
}
},
{
new:true,
upsert:true
}, function(err, doc){
//Do something with doc
});
What i was trying to do is:
"Find specific match with specified users, lang and category... if not found, insert a new one with specified data"
Mongo is throwing this error:
Error getting/creating match { [MongoError: exception: cannot infer query fields to set, path 'users' is matched twice]
name: 'MongoError',
message: 'exception: cannot infer query fields to set, path \'users\' is matched twice',
errmsg: 'exception: cannot infer query fields to set, path \'users\' is matched twice',
code: 54,
ok: 0 }
Is there a way to make it work?
It's impossible?
Thank you :)
It's not the "prettiest" way to handle this, but current restrictions on the selection operators mean you would need to use a JavaScript expression with $where.
Substituting your vars for values for ease of example:
matches.findAndModify(
{
"$where": function() {
var match = [1,2];
return this.users.filter(function(el) {
return match.indexOf(el) != -1;
}).length >= 2;
},
"lang": "en",
"category_id": 1
},
[],
{
"$setOnInsert": {
"users": [1,2],
"lang": "en",
"category_id": 1
}
},
{
"new": true,
"upsert": true
},
function(err,doc) {
// do something useful here
}
);
As you might suspect, the "culprit" here is the positional $ operator, even though your operation does not make use of it.
And the problem specifically is because of $all which is looking for the possible match at "two" positions in the array. In the event that a "positional" operator was required, the engine cannot work out ( presently ) which position to present. The position should arguably be the "first" match being consistent with other operations, but it is not currently working like that.
Replacing the logic with a JavaScript expression circumvents this as the JavaScript logic cannot return a matched position anyway. That makes the expression valid, and you can then either "create" and array with the two elements in a new document or retrieve the document that contains "both" those elements as well as the other query conditions.
P.S Little bit worried about your "sort" here. You may have added it because it is "mandatory" to the method, however if you do possibly expect this to match "more than one" document and need "sort" to work out which one to get then your logic is slightly flawed.
When doing this to "find or create" then you really need to specifiy "all" of the "unique" key constraints in your query. If you don't then you are likely to run into duplicate key errors down the track.
Sort can in fact be an empty array if you do not actually need to "pick" a result from many.
Just something to keep in mind.
I am trying to add an index to a certain Schema with mongoose for text searches. If I add a text index to individual fields it works fine, also with compound indexes it is okay. For example the answer provided here is great:
Full text search with weight in mongoose
However, I am trying to add an index to fields which are references to other Schemas. For example my model looks like the following:
var Book = new Schema({
"title": String,
"createdAt": Date,
"publisher": {
"type": Schema.ObjectId,
"ref": "Publisher"
},
"author": {
"type": Schema.ObjectId,
"ref": "Author"
},
"isbn": String
});
So something like the following indexing doesn't work when you perform a search query as described below:
Indexing:
Book.index({"title": "text", "publisher.name": "text", "author.firstName": "text"});
Search query:
function searchBooks(req, res) {
var query = req.query.searchQuery;
Book.find({ $text: { $search: query } })
.populate('author publisher')
.limit(25)
.exec(function(err, books) {
if (err) {
res.json(err);
} else {
res.json(books);
}
}
);
}
Does anyone have any suggestions of how to add a text index in for the "publisher" and "author" fields, I am using the "populate" mongodb method to pull in the data for these schemas.
I think, what you are looking for is the ability to join tables of data and perform a query against the sum of that data. That is something you need a relational database for, which MongoDB isn't.
So I recommend you change your approach in how you would like to preform your search, e.g. search for your keyword within both author and title instead of attempting to search the whole dataset at the same time.
I know this question may be asked multiple times but I am not able to get the User object in its entirity. The problem is that the user object returns only the username, email and the phone number properties while I have certain fields added like first name and last name to the user object.
I'd like to know if there's a way to get the user object in its entirity
The object on parse cloud should look like
User:
{
"firstname": "firstname",
"lastname": "lastname",
"username": "username",
"email": "abc.xyz#jkl.pqr",
"phone": "0123456789",
"type": "admin/super-admin/user"
}
I use following query to get the object
var query = new Parse.Query(Parse.User);
query.equalTo("username", "username"); // Whatever be the username passed by search string
query.find({
success: function (user) {
console.log("Success:", user)
},
error: function (error) {
//Show if no user was found to match
}
});
I get the user object as follows:
user:{
createdAt:"2014-12-27T07:16:09.826Z"
email: "abc.xyz#pqr.mno"
objectId: "223tch5S6J"
phone: "0123456789"
updatedAt: "2014-12-27T07:16:09.826Z"
username: "username"
}
I do not get the other properties like type, firstname, lastname.
Note: I do not need to get the password which parse does not provide anyways so that's not the problem here.
user.attributes should hold all the user attributes saved server side