DynamoDB Update ExpressionAttributeValues - javascript

Sample test data:
"suppliers" : [
{
"supplierName": "xxx supplier"
},
{
"supplierName": "zzz supplier"
}
]
Excerpt of UpdateItem Params:
ExpressionAttributeValues:{
":sA" : {L: [event.suppliers]}
// ":sA" : {L: event.suppliers}
}
I encounter "UnexpectedParameter: Unexpected key '0' found in params". I tried changing the formatting of the AttributeValues but I get other errors like "UnexpectedParameter: Unexpected key 'supplierName' found in params".
I need help properly defining a list AttributeValue. Note that the input sample test data may contain x number of supplierName objects and hence defining the keys (ie. '0', '1', etc) in the AttributeValue is also not an option. I also prefer not to use the documentClient version of UpdateItem.

ExpressionAttributeValues is about to set a value to the parameter of the update expression, for example:
dynamoDb.update({
TableName: ...,
Key: ...,
UpdateExpression: 'set suppliers = :suppliers',
ExpressionAttributeValues: {':suppliers': event.suppliers}
}
You have to provide the key to the record and then you can change the suppliers attribute.

Related

how to query dynamodb GSI with multiple operators

Lets say i have an entity with this model:
{
id: 'apples',
createdAt: 'some date'
rate: 430,
side: 'SELL',
status: 'OPEN',
GSI1: 'SELL#OPEN#430'
GSI2: 'apples'
}
i want to query using the GSI attributes of GSI2 beign the hash and GSI1 being the range.
The query im looking for is get all apples(GSI2) where GSI1 begins with SELL#OPEN and >= SELL#OPEN#430 so basically im trying to get all apples being sold for 430 or greater and are open.
Please how do i go about this using dynamodb query?
what i have done is:
params = {
TableName: process.env.ORDERS_TABLE_NAME,
IndexName: "GSI2_GSI1",
KeyConditionExpression: `GSI2 = :item and ((begins_with(GSI2, :sideStatus) and >= :baseRate)`,
ExpressionAttributeValues: {
":item": `apple`,
":baseRate": `SELL#OPEN#${rate}`,
":sideStatus": "SELL#OPEN",
},
};
thanks
You can only operate on the Key attributes in the key condition expression. These parameters should do what you want because you have all the information in the GSI1 attribute
params = {
TableName: process.env.ORDERS_TABLE_NAME,
IndexName: "GSI2_GSI1",
KeyConditionExpression: 'GSI2 = :item and GSI1 BETWEEN :lower AND :upper',
ExpressionAttributeValues: {
":item": `apple`,
":lower": `SELL#OPEN#430`,
":upper": "SELL#OPEN#999", // you can probably also use "SELL#OPEN$" because $ is the character following # in ascii order
},
};
Note this assumes that your rate in the GSI1 attribute is left padded with 0s. You need the string to be sorted in the same order as the numbers so if the rate is 10 then you need to store SELL#OPEN#010. (Note you might need more leading 0s depending on the maximum rate.)

What is the proper syntax to get() entries where the sort key begins with a substring using AWS.DynamoDB.DocumentClient in NodeJS?

What is the proper syntax to query database entries whose sort key starts with a specific string?
I believe it's something along the lines of
const query_params = {
TableName: 'my_table',
Key: {
my_primary_key: 'a_primary_key_value',
},
FilterExpression: "my_sort_key begins_with :string",
ExpressionAttributeValues: {
":string": "starts_with_substring"
}
};
Followed by a dynamoDb.get(query_params, ..., but that is not quite right. I am getting an ValidationException: The provided key element does not match the schema error.
According to the SDK query documentation, your query params should look like this
{
TableName: "my-table",
KeyConditionExpression: "#pk= :pk And begins_with(#sk, :sk)",
ExpressionAttributeValues: {
":pk": "a_primary_key_value",
":sk": "starts_with_substring"
},
ExpressionAttributeNames: {
"#pk": "my_primary_key",
"#sk": "my_sort_key"
}
}
You'll also need to change dynamoDb.get() to dynamoDb.query().

mongoDB - Regex search against partial field values

I've a collection of countries with country calling code in the country object. How can I find a country using calling code with a mobile number?
const countries = [
{
name: 'UAE',
callingCode: 971
},
{
name: 'USA',
callingCode: 1
},
{
name: 'UK',
callingCode: 44
}
];
const number = '971524500000'; // Input
How can I find country for the given mobile using regex in mongoose javascript;
[https://en.wikipedia.org/wiki/List_of_country_calling_codes][1]
Take a look at the link above on country calling codes, and specifically see the section "Tree List".
One solution would be to implement a schema in Mongo based on this tree in order to decode the country codes.
So, a table could be created to store Mongo documents containing a field "1x" "2x" "21x" etc (the Y axis in the Tree List table).
Each of these documents could contain an array of sub-documents from x=0 to x=9 (the x axis in the Tree List table). The sub-document can contain the country name/code you are looking for. You can use a direct index into the array in the Mongo document for an efficient lookup.
I think you'll find this to be a pretty efficient implementation and should cover all the bases.
If you can restructure your array to an object this would be the fastest
const countries =
{
971: 'UAE',
1: 'USA',
44: 'UK',
}
;
var code = 44;
console.log(countries[code]);
const countries = [
{
name: 'UAE',
callingCode: 971
},
{
name: 'USA',
callingCode: 1
},
{
name: 'UK',
callingCode: 44
}
];
var myFound =countries.filter(myFunc.bind(this,44));
function myFunc(code,element) {
if(element.callingCode == code){
return element;
}
}
console.log(myFound);
On MongoDB v 4.2 - you can use $let & $regexFind to do this :
db.collection.aggregate([
{
$match: {
$expr: {
$eq: [
{
$let: {
vars: {
value: {
$regexFind: {
input: "971524500000", // pass in as string
regex: { $toString: "$callingCode" }
}
}
},
in: "$$value.idx",
}
},
0
]
}
}
}
]);
Test : MongoDB-Playground
Explanation :
General Use-case :
In general regex searches - Will have an input which will be sub-string of actual string, Ex.:-
Actual string in DB : 'We can do it in mongo'
Input : mongo (/mongo/ - will work)
Your Use-case :
From above case as mongo exists in actual string/value of db field then you can get that document, But your case is different :
Actual string in DB : mongo
Input : 'We can do it in mongo'
This doesn't work that way, So using normal /We can do it in mongo/ isn't going help you here (Also doing few tricks with regex). So we need to make a trick to $regexFind operator. Unlike mongo documentation we need take 971524500000 into input field & regex as string value of callingCode field which is vice-versa to what's given in documentation.
So once we do that, We would get something like below :
{
"match" : "971", /** value of callingCode field */
"idx" : 0, /** Index of `971` in '971524500000' */
"captures" : []
},{
"match" : "1",
"idx" : 2,
"captures" : []
},
null /** 3rd doc no match */
As country code has to be at first of given number we need docs where "idx" : 0 - So we're using $let to get index of returned object & checking against 0 & eventually getting respective docs using $match.
Note :
There is something you need to look into, Just in case if you've docs like below :
{
"_id": ObjectId("5e8f67091aa1cc3d2158ade1"),
"name": "USA",
"callingCode": 1.0
},
{
"_id": ObjectId("5e8f67091aa1cc3d2158ade3"),
"name": "somecountry",
"callingCode": 197.0
}
& input is 1971524500000, then this query will bring both docs in result. This will be the case you need to check on. Anyhow I would suggest to try this query, rather than getting all documents for collection to the code & extract required o/p this might be better to do.

Querying DynamoDB with node.js between values

Querying dynamoDB table with node.js. The DynamoDB table has key of Timestamp, represented by an integer. In this case, I left :timestampStart as 1 and timestampEnd as 10 as an example.
var params = {
TableName: "Table2",
KeyConditionExpression:"Timestamp = :ts BETWEEN :timestampStart AND :timestampEnd",
ExpressionAttributeValues: {
":ts":"Timestamp",
":timestampStart": 1,
":timestampEnd": 10
}
};
The :ts is not correct, I can see this. I want to return any rows found with a Timestamp value between timestampStart and timestampEnd.
Error message:
"errorMessage": "Invalid KeyConditionExpression: Syntax error;
token: \"BETWEEN\", near: \":ts BETWEEN :timestampStart\"",
If Timestamp is the partition key and not the sort key. Then, you have two problems:
In a Query operation, you cannot perform a comparison test (<, >, BETWEEN, ...) on the partition key. The condition must perform an equality test (=) on a single partition key value and, optionally, one of several comparison tests on a single sort key value. For example:
KeyConditionExpression: 'HashKey = :hkey and RangeKey > :rkey'
You have a syntax error in your KeyConditionExpression, obviously. Keep in mind that Timestamp is a reserved word in DynamoDB. So, you'll have to use ExpressionAttributeNames for that:
(Assuming you have an Id partition key and Timestamp sort key)
var params = {
TableName: "Table2",
KeyConditionExpression: "Id = :id AND #DocTimestamp BETWEEN :start AND :end",
ExpressionAttributeNames: {
'#DocTimestamp': 'Timestamp'
},
ExpressionAttributeValues: {
":id": "SOME VALUE",
":start": 1,
":end": 10
}
};

AWS Lambda: How to Add Numbers to a NS Set in Dynamodb

The Issue
I have tried several approaches, but haven't been able to find out how to add numbers to a NS set. This is all running inside a lambda function.
What I'm trying to accomplish
I am creating a dynamodb table where different colors in hex align to a set of ids. I am optimizing the table for fast reads and to avoid duplicates which is why I would like to maintain a set of ids for each hex.
How I'm adding items to the table:
let doc = require('dynamodb-doc');
let dynamo = new doc.DynamoDB();
var object = {
'TableName': 'Hex',
'Item': {
'hex': '#FEFEFE',
'ids': {
'NS': [2,3,4]
}
}
}
dynamo.putItem(object, callback);
Which results in
Then I try to add more ids to the set
Using the Dynamodb Update Item Documentation standards
var params = {
"TableName" : "Hex",
"Key": {
"hex": "#FEFEFE"
},
"UpdateExpression" : "ADD #oldIds :newIds",
"ExpressionAttributeNames" : {
"#oldIds" : "ids"
},
"ExpressionAttributeValues": {
":newIds" : {"NS": ["5", "6"]}
},
};
dynamo.updateItem(params, callback);
This returns the following error, so dynamo thinks :newIds is a map type instead of a set(?)
"errorMessage": "Invalid UpdateExpression: Incorrect operand type for operator or function; operator: ADD, operand type: MAP"
I have also tried these alternative approaches
Try 2:
var setTest = new Set([5, 6]);
var params = {
"TableName" : "Hex",
"Key": {
"hex": "#FEFEFE"
},
"UpdateExpression" : "ADD #oldIds :newIds",
"ExpressionAttributeNames" : {
"#oldIds" : "ids"
},
"ExpressionAttributeValues": {
":newIds" : setTest
},
};
dynamo.updateItem(params, callback);
Error 2 (same error):
"errorMessage": "Invalid UpdateExpression: Incorrect operand type for operator or function; operator: ADD, operand type: MAP"
Try 3:
var params = {
"TableName" : "Hex",
"Key": {
"hex": "#FEFEFE"
},
"UpdateExpression" : "ADD #oldIds :newIds",
"ExpressionAttributeNames" : {
"#oldIds" : "ids"
},
"ExpressionAttributeValues": {
":newIds" : { "NS" : { "L" : [ { "N" : "5" }, { "N" : "6" } ] }}
},
};
dynamo.updateItem(params, callback);
Error 3 (same error):
"errorMessage": "Invalid UpdateExpression: Incorrect operand type for operator or function; operator: ADD, operand type: MAP"
Try 4:
var params = {
"TableName" : "Hex",
"Key": {
"hex": "#FEFEFE"
},
"UpdateExpression" : "ADD #oldIds :newIds",
"ExpressionAttributeNames" : {
"#oldIds" : "ids"
},
"ExpressionAttributeValues": {
":newIds" : [5,6]
},
};
dynamo.updateItem(params, callback);
Error 4 (similar error, but dynamo thinks I'm adding a list this time)
"errorMessage": "Invalid UpdateExpression: Incorrect operand type for operator or function; operator: ADD, operand type: LIST"
Stack Overflow/Github Questions I've Tried
https://stackoverflow.com/a/37585600/4975772 (I'm adding to a set, not a list)
https://stackoverflow.com/a/37143879/4975772 (I'm using javascript, not python, but I basically need this same thing just different syntax)
https://github.com/awslabs/dynamodb-document-js-sdk/issues/40#issuecomment-123003444 (I need to do this exact thing, but I'm not using the dynamodb-document-js-sdk, I'm using AWS Lambda
How to Create a Set and Add Items to a Set
let AWS = require('aws-sdk');
let docClient = new AWS.DynamoDB.DocumentClient();
...
var params = {
TableName : 'Hex',
Key: {'hex': '#FEFEFE'},
UpdateExpression : 'ADD #oldIds :newIds',
ExpressionAttributeNames : {
'#oldIds' : 'ids'
},
ExpressionAttributeValues : {
':newIds' : docClient.createSet([1,2])
}
};
docClient.update(params, callback);
Which results in this dynamodb table:
If the set doesn't exist, then that code will create it for you. You can also run that code with a different set to update the set's elements. Super convenient.
Create a Set and Add Items to a Set (OLD API)
let doc = require('dynamodb-doc');
let dynamo = new doc.DynamoDB();
var params = {
TableName : 'Hex',
Key: {'hex': '#555555'},
UpdateExpression : 'ADD #oldIds :newIds',
ExpressionAttributeNames : {
'#oldIds' : 'ids'
},
ExpressionAttributeValues : {
':newIds' : dynamo.Set([2,3], 'N')
}
};
dynamo.updateItem(params, callback);
(Don't use this code for future development, I only include it to help anyone using the existing DynamoDB Document SDK)
Why the Original Was Failing
Notice how when I asked the question, the resulting set looked like a literal json map object (when viewed in the dynamodb screenshot), which would explain this error message
"errorMessage": "Invalid UpdateExpression: Incorrect operand type for operator or function; operator: ADD, operand type: MAP"
So I was using the wrong syntax. The solution is found in the (now depricated) AWS Labs dynamodb-document-js-sdk docs
The full documentation for the newer Document Client can be viewed here: http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/DynamoDB/DocumentClient.html
I've been struggling with this too. I discovered that there are actually 2 api's for DynamoDB.
AWS.DynamoDB
AWS.DynamoDBDocumentClient
Like you, I was not able to make the ADD function work. I tried to import and use the DynamoDB api instead of the DynamoDBDocumentClient api. It solved the problem for me. Below is my code snippet that works with the DynamoDB api but not with the DynamoDBDocumentClient api. I use a String Set instead of a Number Set, but that won’t make a difference I guess.
var AWS = require('aws-sdk');
var dynamo = new AWS.DynamoDB();
// var dynamo = new AWS.DynamoDB.DocumentClient();
...
var params = {};
params.TableName = “MyTable”;
params.ExpressionAttributeValues = { ':newIds': { "SS": ["someId"] } };
// :newIds represents a new dynamodb set with 1 element
params.UpdateExpression = "ADD someAttribute :newIds";
dynamoClient.updateItem(params, function(err, data) { ….}
This answer might be helpful to those who are using npm dynamoDB
module
We had the same issue . AS we were using npm dynamodb module for our queries rather than exporting from AWS.DynamoDB module, given solution of AWS.DynamoDBDocumentClint was implementable untill and unless we could shift from npm dynamoDb module to AWS.DynamoDB queries. So instead of shifting/transforming queries.
We tried to downgrade dynamoDB npm module to version ~1.1.0 and it worked.
Previous version was 1.2.X.

Categories