We Keep Coding

How do I query an index properly with Dynamoose

I'm using Dynamoose to simplify my interactions with DynamoDB in a node.js application. I'm trying to write a query using Dynamoose's Model.query function that will search a table using an index, but it seems like Dynamoose is not including all of the info required to process the query and I'm not sure what I'm doing wrong.
Here's what the schema looks like:
const UserSchema = new dynamoose.Schema({
"user_id": {
"hashKey": true,
"type": String
},
"email": {
"type": String,
"index": {
"global": true,
"name": "email-index"
}
},
"first_name": {
"type": String,
"index": {
"global": true,
"name": "first_name-index"
}
},
"last_name": {
"type": String,
"index": {
"global": true,
"name": "last_name-index"
}
}
)
module.exports = dynamoose.model(config.usersTable, UserSchema)
I'd like to be able to search for users by their email address, so I'm writing a query that looks like this:
Users.query("email").contains(query.email)
.using("email-index")
.all()
.exec()
.then( results => {
res.status(200).json(results)
}).catch( err => {
res.status(500).send("Error searching for users: " + err)
})
I have a global secondary index defined for the email field:
When I try to execute this query, I'm getting the following error:
Error searching for users: ValidationException: Either the KeyConditions or KeyConditionExpression parameter must be specified in the request.
Using the Dynamoose debugging output, I can see that the query winds up looking like this:
aws:dynamodb:query:request - {
"FilterExpression": "contains (#a0, :v0)",
"ExpressionAttributeNames": {
"#a0": "email"
},
"ExpressionAttributeValues": {
":v0": {
"S": "mel"
}
},
"TableName": "user_qa",
"IndexName": "email-index"
}
I note that the actual query sent to DynamoDB does not contain KeyConditions or KeyConditionExpression, as the error message indicates. What am I doing wrong that prevents this query from being written correctly such that it executes the query against the global secondary index I've added for this table?

As it turns out, calls like .contains(text) are used as filters, not query parameters. DynamoDB can't figure out if the text in the index contains the text I'm searching for without looking at every single record, which is a scan, not a query. So it doesn't make sense to try to use .contains(text) in this context, even though it's possible to call it in a chain like the one I constructed. What I ultimately needed to do to make this work is turn my call into a table scan with the .contains(text) filter:
Users.scan({ email: { contains: query.email }}).all().exec().then( ... )

I am not familiar with Dynamoose too much but the following code below will do an update on a record using node.JS and DynamoDB. See the key parameter I have below; by the error message you got it seems you are missing this.
To my knowledge, you must specify a key for an UPDATE request. You can checks the AWS DynamoDB docs to confirm.
var params = {
TableName: table,
Key: {
"id": customerID,
},
UpdateExpression: "set customer_name= :s, customer_address= :p, customer_phone= :u, end_date = :u",
ExpressionAttributeValues: {
":s": customer_name,
":p": customer_address,
":u": customer_phone
},
ReturnValues: "UPDATED_NEW"
};
await docClient.update(params).promise();

webauthn authentication javascript formatting assistance

I have been trying to figure out how to do 2fa with webauthn and I have the registration part working. The details are really poorly documented, especially all of the encoding payloads in javascript. I am able to register a device to a user, but I am not able to authenticate with that device. For reference, I'm using these resources:
https://github.com/cedarcode/webauthn-ruby
https://www.passwordless.dev/js/mfa.register.js
And specifically, for authentication, I'm trying to mimic this js functionality:
https://www.passwordless.dev/js/mfa.register.js
In my user model, I have a webauthn_id, and several u2f devices, each of which has a public_key and a webauthn_id.
In my Rails app, I do:
options = WebAuthn::Credential.options_for_get(allow: :webauthn_id)
session[:webauthn_options] = options
In my javascript, I try to mimic the js file above and I do (this is embedded ruby):
options = <%= raw #options.as_json.to_json %>
options.challenge = WebAuthnHelpers.coerceToArrayBuffer(options.challenge);
options.allowCredentials = options.allowCredentials.map((c) => {
c.id = WebAuthnHelpers.coerceToArrayBuffer(c.id);
return c;
});
navigator.credentials.get({ "publicKey": options }).then(function (credentialInfoAssertion)
{
// send assertion response back to the server
// to proceed with the control of the credential
alert('here');
}).catch(function (err)
{
debugger
console.error(err); /* THIS IS WHERE THE ERROR IS THROWN */
});
The problem is, I cannot get past navigator.credentials.get, I get this error in the javascript console:
TypeError: CredentialsContainer.get: Element of 'allowCredentials' member of PublicKeyCredentialRequestOptions can't be converted to a dictionary
options at the time navigator.credentials.get is called looks like this:
I've tried every which way to convert my db-stored user and device variables into javascript properly encoded and parsed variables but cannot seem to get it to work. Anything obvious about what I'm doing wrong?
Thanks for any help,
Kevin
UPDATE -
Adding options json generated by the server:
"{\"challenge\":\"SSDYi4I7kRWt5wc5KjuAvgJ3dsQhjy7IPOJ0hvR5tMg\",\"timeout\":120000,\"allowCredentials\":[{\"type\":\"public-key\",\"id\":\"OUckfxGNLGGASUfGiX-1_8FzehlXh3fKvJ98tm59mVukJkKb_CGk1avnorL4sQQASVO9aGqmgn01jf629Jt0Z0SmBpDKd9sL1T5Z9loDrkLTTCIzrIRqhwPC6yrkfBFi\"},{\"type\":\"public-key\",\"id\":\"Fj5T-WPmEMTz139mY-Vo0DTfsNmjwy_mUx6jn5rUEPx-LsY51mxNYidprJ39_cHeAOieg-W12X47iJm42K0Tsixj4_Fl6KjdgYoxQtEYsNF-LPhwtoKwYsy1hZgVojp3\"}]}"

This is an example of the serialised JSON data returned by our implementation:
{
"challenge": "MQ1S8MBSU0M2kiJqJD8wnQ",
"timeout": 60000,
"rpId": "identity.acme.com",
"allowCredentials": [
{
"type": "public-key",
"id": "k5Ti8dLdko1GANsBT-_NZ5L_-8j_8TnoNOYe8mUcs4o",
"transports": [
"internal"
]
},
{
"type": "public-key",
"id": "LAqkKEO99XPCQ7fsUa3stz7K76A_mE5dQwX4S3QS6jdbI9ttSn9Hu37BA31JUGXqgyhTtskL5obe6uZxitbIfA",
"transports": [
"usb"
]
},
{
"type": "public-key",
"id": "nbN3S08Wv2GElRsW9AmK70J1INEpwIywQcOl6rp_DWLm4mcQiH96TmAXSrZRHciZBENVB9rJdE94HPHbeVjtZg",
"transports": [
"usb"
]
}
],
"userVerification": "discouraged",
"extensions": {
"txAuthSimple": "Sign in to your ACME account",
"exts": true,
"uvi": true,
"loc": true,
"uvm": true
}
}
This is parsed to an object and the code used to coerce those base64url encoded values is:
credentialRequestOptions.challenge = WebAuthnHelpers.coerceToArrayBuffer(credentialRequestOptions.challenge);
credentialRequestOptions.allowCredentials = credentialRequestOptions.allowCredentials.map((c) => {
c.id = WebAuthnHelpers.coerceToArrayBuffer(c.id);
return c;
});
Hope that helps. The JSON data is retreived via a fetch() call and the byte[] fields are encoded as base64url on the serverside.

how to access json data contained in a GET response

Continuing yesterday's saga, now I can retrieve json objects in a response but I can't extract the data from them.
The following node.js snippet is from the file "accounts.js" which is in an ETrade api library that exists in the path /lib. It returns json containing data about the accounts of the authenticated user. The authentication part is working great.
exports.listAccounts = function(successCallback,errorCallback)
{
var actionDescriptor = {
method : "GET",
module : "accounts",
action : "accountlist",
useJSON: true,
};
this._run(actionDescriptor,{},successCallback,errorCallback);
};
The ETrade website says this call will produce the following sample response:
{
"AccountListResponse": {
"Account": [
{
"accountDesc": "MyAccount-1",
"accountId": "83405188",
"marginLevel": "MARGIN",
"netAccountValue": "9999871.82",
"registrationType": "INDIVIDUAL"
},
{
"accountDesc": "MyAccount-3",
"accountId": "83405553",
"marginLevel": "CASH",
"netAccountValue": "100105468.99",
"registrationType": "INDIVIDUAL"
},
{
"accountDesc": "SIMPLE IRA",
"accountId": "83405188",
"marginLevel": "CASH",
"netAccountValue": "99794.13",
"registrationType": "IRA"
}
]
}
}
In my app.js file, I have the following:
var etrade = require('./lib/etrade');
var et = new etrade(configuration);
et.listAccounts(
function(res){
var listAccountsRes = res;
console.log('account list success!');
console.log(listAccountsRes)
},
function(error) {
console.log("Error encountered while attempting " +
"to retrieve account list: " +
error);
});
When I run this code, the console log shows the following message:
{ 'json.accountListResponse':
{ response:
[ [Object],
[ [Object],
[ [Object],
[ [Object],
[ [Object],
[ [Object],
[ [Object],
[ [Object] ] } }
Suppose in app.js I want to put the accounts data in a variable called myAccounts.
One of our members, Jack, solved yesterday's problem and when I commented that I still couldn't access the data in the response, he suggested this: "That property has a dot in it so you'll have to use [ ... ] rather than dot notation to access it. See what's inside the objects with a['json.accountListResponse'].response." So far I have not been able to get that to work, even when I use ['json.accountListResponse'].res like this:
var listAccountsRes = [json.accountListResponse].res;
This returns undefined when printed to the console.
Thanks to Adam for his suggestion which led to this which works:
var listAccountsRes = res['json.accountListResponse'];
var listAccounts = listAccountsRes['response'];
console.log('account list success!');
console.log(listAccounts)
Now the console log reports almost exactly what ETrade says I should get. (They appear to have changed the name "Account" to "response"). I presume my variable listAccounts now contains the json with eight sample accounts in it that I can see in my console log. But I still don't know how to access individual elements. There should be some simple code that will iterate over the json file and produce an array of arrays that I could actually use for something. I tried accessing it like an array: console.log(listAccounts[0]) but that returns undefined. Do I need to stringify it or something?

Syntax for JSON 'payload' in Facebook FB.api post

I'm trying to add users to a Custom Audience in Facebook, and I believe I have bungled the payload piece of the request below.
The error returned is:
(#100) Missing required parameter: payload
For reference, I'm generating the hash using Crypto-JS. Here's the code I tried:
var payload = { "payload": [{ "schema": "EMAIL_SHA256", "data": [hash] }]};
FB.api('/000000000/users', 'post', payload, function (response) {
if (response && !response.error) {
alert("This worked");
} else {
alert(response.error.message);
}});
The FB.api documentation shows that it expects 'payload' as a JSON object (https://developers.facebook.com/docs/marketing-api/custom-audience-targeting/v2.3#add). I just haven't been able to figure out the correct syntax yet. The example in the Facebook API documentation shows the following:
payload = {"schema":"EMAIL_SHA256","data":["HASH", "HASH", "HASH" ]}
Here's what I have so far (not working):
var payload = { "payload": [{ "schema": "EMAIL_SHA256", "data": [hash] }]};
Can anyone assist with the syntax? I've found plenty of examples of JSON objects and arrays, but I haven't seen anything that matches this format:
payload = {"schema":"EMAIL_SHA256","data":["HASH", "HASH", "HASH" ]}

For the benefit of any other JS/JSON novices, I finally figured it out after more experimentation:
var payload = { "payload": { "schema": "EMAIL_SHA256", "data": [hash] } };

Ember-Data: How do "mappings" work

I'm currently trying to put something together with ember + emberdata + router + asp.net web api. Most of it seem to work, however I stuck in an error message I get when ember-data tries to findAll through the adapter for my models.
In my backend I have a model like this (C#):
public class Genre {
[Key]
public int Id { get; set; }
[Required]
[StringLength(50, MinimumLength=3)]
public string Name { get; set; }
}
Which in my app I represent it like this using ember-data:
App.Genre = DS.Model.extend({
id: DS.attr("number"),
name: DS.attr("string")
}).reopenClass({
url: 'api/genre'
});
I have also a Store defined in my App using the RESTAdapter like so:
App.store = DS.Store.create({
revision: 4,
adapter: DS.RESTAdapter.create({
bulkCommit: false
})
});
And the store is used in my controller as below:
App.GenreController = Ember.ArrayController.extend({
content: App.store.findAll(App.Genre),
selectedGenre: null
});
The router is defined as
App.router = Em.Router.create({
enableLogging: true,
location: 'hash',
root: Ember.Route.extend({
//...
genre: Em.Route.extend({
route: '/genre',
index: Ember.Route.extend({
connectOutlets: function (router, context) {
router.get('applicationController').connectOutlet('genre');
}
})
}),
//...
})
})
When I run my application, I get the following message for every object that has this same structure:
Uncaught Error: assertion failed: Your server returned a hash with the
key 0 but you have no mappings
For reference, here's the json the service is returning:
[
{
"id": 1,
"name": "Action"
},
{
"id": 2,
"name": "Drama"
},
{
"id": 3,
"name": "Comedy"
},
{
"id": 4,
"name": "Romance"
}
]
I cannot tell exactly what the problem is and since the assertion is mentioning that I need mapping, I'd like to know:
What this mapping is and how to use it.
Since the returned json is an array, should I be using a different type of controller in my app ,or is there anything I should know about when working with this type of json in ember-data? or should I change the JsonFormatter options in the server?
Any help is welcome.
I can definitely add more information if you feel this isn't enough to understand the problem.
EDIT: I've changed a few things in my backend and now my findAll() equivalent action in the server serializes the the output as the following json:
{
"genres": [
{ "id": 1, "name": "Action" },
{ "id": 2, "name": "Drama" },
{ "id": 3, "name": "Comedy" },
{ "id": 4, "name": "Romance" }
]
}
But I still can't get it to populate my models in the client and my error message has changed to this:
Uncaught Error: assertion failed: Your server returned a hash with the
key genres but you have no mappings
Not sure what else I might be doing wrong.
The method that throws this exception is sideload and checks for the mappings like this:
sideload: function (store, type, json, root) {
var sideloadedType, mappings, loaded = {};
loaded[root] = true;
for (var prop in json) {
if (!json.hasOwnProperty(prop)) { continue; }
if (prop === root) { continue; }
sideloadedType = type.typeForAssociation(prop);
if (!sideloadedType) {
mappings = get(this, 'mappings');
Ember.assert("Your server returned a hash with the key " + prop + " but you have no mappings", !!mappings);
//...
This call sideloadedType = type.typeForAssociation(prop); returns undefined and then I get the error message. The method typeForAssociation() checks for the for 'associationsByName' key which returns an empty Ember.Map.
Still no solution for this at the moment.
By the way...
My action is now like this:
// GET api/genres
public object GetGenres() {
return new { genres = context.Genres.AsQueryable() };
}
// GET api/genres
//[Queryable]
//public IQueryable<Genre> GetGenres()
//{
// return context.Genres.AsQueryable();
//}
I had to remove the original implementation which gets serialized by json.NET as I could not find config options to produce a json output as Ember-Data expects ( as in {resource_name : [json, json,...]}). Side effect of this is that I've lost built-in OData support, but I'd like to keep it. Does anyone know how could I configure it to produce different json for a collection?

The mapping can be defined in the DS.RESTAdapter. I think you could try to define something like this:
App.Store = DS.Store.extend({
adapter: DS.RESTAdapter.create({
bulkCommit: true,
mappings: {
genres: App.Genre
},
// you can also define plurals, if there is a unregular plural
// usually, RESTAdapter simply add a 's' for plurals.
// for example at work we have to define something like this
plurals: {
business_process: 'business_processes'
//else it tries to fetch business_processs
}
}),
revision: 4
});
Hope this resolves your problem.
Update:
At this time, this is not well documented, I don't remember if we found it by ourself reading the code, or perhaps Tom Dale pointed on it.
Anyway, here is the point for plurals
For the mappings, I think we were driven by the same error as you, and either we tried, either Tom teached us about this.

The RESTAdapter expects the returned JSON to be of the form:
{
"genres": [{
"id": 1,
"name": "action"
},{
"id": 2,
"name": "Drama"
}]
}
The tests are a good source of documentation, see https://github.com/emberjs/data/blob/master/packages/ember-data/tests/unit/rest_adapter_test.js#L315-329

I'm using Ember Data rev. 11 and it seems that the plurals config in DS.RESTAdapter.create never works. I looked into the codes and found a solution as following:
App.Adapter = DS.RESTAdapter.extend({
bulkCommit: false
})
App.Adapter.configure('plurals', {
series: 'series'
})