We Keep Coding

In DymanoDB/Dynamoose, how can I set a schema of a map of string keys and number values?

So I have this instructions:
Schema Defs:
Result object:
It's a map of string keys and number values
"result": { "M": { [STRING]: { "N": "401" } },
This is what I have so far
result: {
type: Object,
schema: {
// I am getting 2 errors:
// Parsing error: Unexpected token, expected "]"
// ',' expected.
[id: String]: Number
},
required: true
},
Any ideas?

[id: String] is a TypeScript thing. Not allowed in standard JavaScript.
This is not technically possible in Dynamoose. The only option here is to use the saveUnknown schema setting.
This was brought up in a conversation before, and the user who wanted to do this I told to create an issue on the GitHub repo but it doesn't look like that happened. If you'd like support for this in Dynamoose in the future, please submit a feature request on the GitHub repo.
Edit
In order to do this your schema would looks something like:
new dynamoose.Schema({
result: {
type: Object,
required: true
},
}, {
"saveUnknown": ["result.**"]
});
This will allow indefinitely nested objects within result.

Why am I getting an error that my conditions don't match the schema?

I have this query that works:
{
"TableName": "myTable",
"IndexName": "gsi1",
"Limit": 12,
"KeyConditionExpression": "#pk = :pk",
"ExpressionAttributeValues": {
":pk": "TOS"
},
"ExpressionAttributeNames": {
"#pk": "gsi1_pk"
}
}
I attempted to convert it into one using KeyConditions instead of an expression because that makes other things easier. So I changed it to this:
{
"TableName": "myTable",
"IndexName": "gsi1",
"Limit": 12,
"KeyConditions": {
"gsi1_pk": {
"ComparisonOperator": "EQ",
"AttributeValueList": [
{
"S": "TOS"
}
]
}
}
}
I have the GSI configured on my table:
But when I make the request I'm getting this error:
ValidationException: One or more parameter values were invalid: Condition parameter type does not match schema type
I don't understand why I'm getting this error.
My Partition key for this GSI is a string
The only attribute I'm querying against is a string
The name gsi1_pk matches the value I'm querying
Did I miss something in converting this query? I don't understand what I should change to resolve the error because it looks accurate to me.

From this aws doc
This is a legacy parameter. Use KeyConditionExpression instead.
'KeyConditions' >= 'KeyConditionExpression'
Changing your key var in your query should fix this.

DynamoDB Update ExpressionAttributeValues

Sample test data:
"suppliers" : [
{
"supplierName": "xxx supplier"
},
{
"supplierName": "zzz supplier"
}
]
Excerpt of UpdateItem Params:
ExpressionAttributeValues:{
":sA" : {L: [event.suppliers]}
// ":sA" : {L: event.suppliers}
}
I encounter "UnexpectedParameter: Unexpected key '0' found in params". I tried changing the formatting of the AttributeValues but I get other errors like "UnexpectedParameter: Unexpected key 'supplierName' found in params".
I need help properly defining a list AttributeValue. Note that the input sample test data may contain x number of supplierName objects and hence defining the keys (ie. '0', '1', etc) in the AttributeValue is also not an option. I also prefer not to use the documentClient version of UpdateItem.

ExpressionAttributeValues is about to set a value to the parameter of the update expression, for example:
dynamoDb.update({
TableName: ...,
Key: ...,
UpdateExpression: 'set suppliers = :suppliers',
ExpressionAttributeValues: {':suppliers': event.suppliers}
}
You have to provide the key to the record and then you can change the suppliers attribute.

AWS Lambda: How to Add Numbers to a NS Set in Dynamodb

The Issue
I have tried several approaches, but haven't been able to find out how to add numbers to a NS set. This is all running inside a lambda function.
What I'm trying to accomplish
I am creating a dynamodb table where different colors in hex align to a set of ids. I am optimizing the table for fast reads and to avoid duplicates which is why I would like to maintain a set of ids for each hex.
How I'm adding items to the table:
let doc = require('dynamodb-doc');
let dynamo = new doc.DynamoDB();
var object = {
'TableName': 'Hex',
'Item': {
'hex': '#FEFEFE',
'ids': {
'NS': [2,3,4]
}
}
}
dynamo.putItem(object, callback);
Which results in
Then I try to add more ids to the set
Using the Dynamodb Update Item Documentation standards
var params = {
"TableName" : "Hex",
"Key": {
"hex": "#FEFEFE"
},
"UpdateExpression" : "ADD #oldIds :newIds",
"ExpressionAttributeNames" : {
"#oldIds" : "ids"
},
"ExpressionAttributeValues": {
":newIds" : {"NS": ["5", "6"]}
},
};
dynamo.updateItem(params, callback);
This returns the following error, so dynamo thinks :newIds is a map type instead of a set(?)
"errorMessage": "Invalid UpdateExpression: Incorrect operand type for operator or function; operator: ADD, operand type: MAP"
I have also tried these alternative approaches
Try 2:
var setTest = new Set([5, 6]);
var params = {
"TableName" : "Hex",
"Key": {
"hex": "#FEFEFE"
},
"UpdateExpression" : "ADD #oldIds :newIds",
"ExpressionAttributeNames" : {
"#oldIds" : "ids"
},
"ExpressionAttributeValues": {
":newIds" : setTest
},
};
dynamo.updateItem(params, callback);
Error 2 (same error):
"errorMessage": "Invalid UpdateExpression: Incorrect operand type for operator or function; operator: ADD, operand type: MAP"
Try 3:
var params = {
"TableName" : "Hex",
"Key": {
"hex": "#FEFEFE"
},
"UpdateExpression" : "ADD #oldIds :newIds",
"ExpressionAttributeNames" : {
"#oldIds" : "ids"
},
"ExpressionAttributeValues": {
":newIds" : { "NS" : { "L" : [ { "N" : "5" }, { "N" : "6" } ] }}
},
};
dynamo.updateItem(params, callback);
Error 3 (same error):
"errorMessage": "Invalid UpdateExpression: Incorrect operand type for operator or function; operator: ADD, operand type: MAP"
Try 4:
var params = {
"TableName" : "Hex",
"Key": {
"hex": "#FEFEFE"
},
"UpdateExpression" : "ADD #oldIds :newIds",
"ExpressionAttributeNames" : {
"#oldIds" : "ids"
},
"ExpressionAttributeValues": {
":newIds" : [5,6]
},
};
dynamo.updateItem(params, callback);
Error 4 (similar error, but dynamo thinks I'm adding a list this time)
"errorMessage": "Invalid UpdateExpression: Incorrect operand type for operator or function; operator: ADD, operand type: LIST"
Stack Overflow/Github Questions I've Tried
https://stackoverflow.com/a/37585600/4975772 (I'm adding to a set, not a list)
https://stackoverflow.com/a/37143879/4975772 (I'm using javascript, not python, but I basically need this same thing just different syntax)
https://github.com/awslabs/dynamodb-document-js-sdk/issues/40#issuecomment-123003444 (I need to do this exact thing, but I'm not using the dynamodb-document-js-sdk, I'm using AWS Lambda

How to Create a Set and Add Items to a Set
let AWS = require('aws-sdk');
let docClient = new AWS.DynamoDB.DocumentClient();
...
var params = {
TableName : 'Hex',
Key: {'hex': '#FEFEFE'},
UpdateExpression : 'ADD #oldIds :newIds',
ExpressionAttributeNames : {
'#oldIds' : 'ids'
},
ExpressionAttributeValues : {
':newIds' : docClient.createSet([1,2])
}
};
docClient.update(params, callback);
Which results in this dynamodb table:
If the set doesn't exist, then that code will create it for you. You can also run that code with a different set to update the set's elements. Super convenient.
Create a Set and Add Items to a Set (OLD API)
let doc = require('dynamodb-doc');
let dynamo = new doc.DynamoDB();
var params = {
TableName : 'Hex',
Key: {'hex': '#555555'},
UpdateExpression : 'ADD #oldIds :newIds',
ExpressionAttributeNames : {
'#oldIds' : 'ids'
},
ExpressionAttributeValues : {
':newIds' : dynamo.Set([2,3], 'N')
}
};
dynamo.updateItem(params, callback);
(Don't use this code for future development, I only include it to help anyone using the existing DynamoDB Document SDK)
Why the Original Was Failing
Notice how when I asked the question, the resulting set looked like a literal json map object (when viewed in the dynamodb screenshot), which would explain this error message
"errorMessage": "Invalid UpdateExpression: Incorrect operand type for operator or function; operator: ADD, operand type: MAP"
So I was using the wrong syntax. The solution is found in the (now depricated) AWS Labs dynamodb-document-js-sdk docs
The full documentation for the newer Document Client can be viewed here: http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/DynamoDB/DocumentClient.html

I've been struggling with this too. I discovered that there are actually 2 api's for DynamoDB.
AWS.DynamoDB
AWS.DynamoDBDocumentClient
Like you, I was not able to make the ADD function work. I tried to import and use the DynamoDB api instead of the DynamoDBDocumentClient api. It solved the problem for me. Below is my code snippet that works with the DynamoDB api but not with the DynamoDBDocumentClient api. I use a String Set instead of a Number Set, but that won’t make a difference I guess.
var AWS = require('aws-sdk');
var dynamo = new AWS.DynamoDB();
// var dynamo = new AWS.DynamoDB.DocumentClient();
...
var params = {};
params.TableName = “MyTable”;
params.ExpressionAttributeValues = { ':newIds': { "SS": ["someId"] } };
// :newIds represents a new dynamodb set with 1 element
params.UpdateExpression = "ADD someAttribute :newIds";
dynamoClient.updateItem(params, function(err, data) { ….}

This answer might be helpful to those who are using npm dynamoDB
module
We had the same issue . AS we were using npm dynamodb module for our queries rather than exporting from AWS.DynamoDB module, given solution of AWS.DynamoDBDocumentClint was implementable untill and unless we could shift from npm dynamoDb module to AWS.DynamoDB queries. So instead of shifting/transforming queries.
We tried to downgrade dynamoDB npm module to version ~1.1.0 and it worked.
Previous version was 1.2.X.

Cannot infer query fields to set error on insert

I'm trying to achieve a "getOrCreate" behavior using "findAndModify".
I'm working in nodejs using the native driver.
I have:
var matches = db.collection("matches");
matches.findAndModify({
//query
users: {
$all: [ userA_id, userB_id ]
},
lang: lang,
category_id: category_id
},
[[ "_id", "asc"]], //order
{
$setOnInsert: {
users: [userA_id, userB_id],
category_id: category_id,
lang: lang,
status: 0
}
},
{
new:true,
upsert:true
}, function(err, doc){
//Do something with doc
});
What i was trying to do is:
"Find specific match with specified users, lang and category... if not found, insert a new one with specified data"
Mongo is throwing this error:
Error getting/creating match { [MongoError: exception: cannot infer query fields to set, path 'users' is matched twice]
name: 'MongoError',
message: 'exception: cannot infer query fields to set, path \'users\' is matched twice',
errmsg: 'exception: cannot infer query fields to set, path \'users\' is matched twice',
code: 54,
ok: 0 }
Is there a way to make it work?
It's impossible?
Thank you :)

It's not the "prettiest" way to handle this, but current restrictions on the selection operators mean you would need to use a JavaScript expression with $where.
Substituting your vars for values for ease of example:
matches.findAndModify(
{
"$where": function() {
var match = [1,2];
return this.users.filter(function(el) {
return match.indexOf(el) != -1;
}).length >= 2;
},
"lang": "en",
"category_id": 1
},
[],
{
"$setOnInsert": {
"users": [1,2],
"lang": "en",
"category_id": 1
}
},
{
"new": true,
"upsert": true
},
function(err,doc) {
// do something useful here
}
);
As you might suspect, the "culprit" here is the positional $ operator, even though your operation does not make use of it.
And the problem specifically is because of $all which is looking for the possible match at "two" positions in the array. In the event that a "positional" operator was required, the engine cannot work out ( presently ) which position to present. The position should arguably be the "first" match being consistent with other operations, but it is not currently working like that.
Replacing the logic with a JavaScript expression circumvents this as the JavaScript logic cannot return a matched position anyway. That makes the expression valid, and you can then either "create" and array with the two elements in a new document or retrieve the document that contains "both" those elements as well as the other query conditions.
P.S Little bit worried about your "sort" here. You may have added it because it is "mandatory" to the method, however if you do possibly expect this to match "more than one" document and need "sort" to work out which one to get then your logic is slightly flawed.
When doing this to "find or create" then you really need to specifiy "all" of the "unique" key constraints in your query. If you don't then you are likely to run into duplicate key errors down the track.
Sort can in fact be an empty array if you do not actually need to "pick" a result from many.
Just something to keep in mind.