I'm using Dynamoose to simplify my interactions with DynamoDB in a node.js application. I'm trying to write a query using Dynamoose's Model.query function that will search a table using an index, but it seems like Dynamoose is not including all of the info required to process the query and I'm not sure what I'm doing wrong.
Here's what the schema looks like:
const UserSchema = new dynamoose.Schema({
"user_id": {
"hashKey": true,
"type": String
},
"email": {
"type": String,
"index": {
"global": true,
"name": "email-index"
}
},
"first_name": {
"type": String,
"index": {
"global": true,
"name": "first_name-index"
}
},
"last_name": {
"type": String,
"index": {
"global": true,
"name": "last_name-index"
}
}
)
module.exports = dynamoose.model(config.usersTable, UserSchema)
I'd like to be able to search for users by their email address, so I'm writing a query that looks like this:
Users.query("email").contains(query.email)
.using("email-index")
.all()
.exec()
.then( results => {
res.status(200).json(results)
}).catch( err => {
res.status(500).send("Error searching for users: " + err)
})
I have a global secondary index defined for the email field:
When I try to execute this query, I'm getting the following error:
Error searching for users: ValidationException: Either the KeyConditions or KeyConditionExpression parameter must be specified in the request.
Using the Dynamoose debugging output, I can see that the query winds up looking like this:
aws:dynamodb:query:request - {
"FilterExpression": "contains (#a0, :v0)",
"ExpressionAttributeNames": {
"#a0": "email"
},
"ExpressionAttributeValues": {
":v0": {
"S": "mel"
}
},
"TableName": "user_qa",
"IndexName": "email-index"
}
I note that the actual query sent to DynamoDB does not contain KeyConditions or KeyConditionExpression, as the error message indicates. What am I doing wrong that prevents this query from being written correctly such that it executes the query against the global secondary index I've added for this table?
As it turns out, calls like .contains(text) are used as filters, not query parameters. DynamoDB can't figure out if the text in the index contains the text I'm searching for without looking at every single record, which is a scan, not a query. So it doesn't make sense to try to use .contains(text) in this context, even though it's possible to call it in a chain like the one I constructed. What I ultimately needed to do to make this work is turn my call into a table scan with the .contains(text) filter:
Users.scan({ email: { contains: query.email }}).all().exec().then( ... )
I am not familiar with Dynamoose too much but the following code below will do an update on a record using node.JS and DynamoDB. See the key parameter I have below; by the error message you got it seems you are missing this.
To my knowledge, you must specify a key for an UPDATE request. You can checks the AWS DynamoDB docs to confirm.
var params = {
TableName: table,
Key: {
"id": customerID,
},
UpdateExpression: "set customer_name= :s, customer_address= :p, customer_phone= :u, end_date = :u",
ExpressionAttributeValues: {
":s": customer_name,
":p": customer_address,
":u": customer_phone
},
ReturnValues: "UPDATED_NEW"
};
await docClient.update(params).promise();
Related
{
"INFO": {
"email": "test#example.com",
"password": "123"
},
"PK": "3a95eab0-57de-4e15-90ea-004082e53384",
"SK": "user"
}
Above is my dataset in dynamoDB. I am building login api with expressjs with dynamodb. I am able to scan and update data with PK & SK keys but i want to query inside my INFO set.
I am trying like this:
var params = {
TableName: "table",
FilterExpression: "contains (INFO, :sendToVal)",
ExpressionAttributeValues: {
":sendToVal": {
email: "test#example.com",
password: "123",
},
},
};
But its returning:
{ error: 'Error retrieving Event' }
{ error: 'Event not found' }
Anyone help guide me, how can i retrive the set.
The DynamoDB documentation explains that the contains() function in an expression only works for strings or sets. This isn't completely accurate - it also works for lists. But in any case, it doesn't work for maps, which is what your INFO is, so the comparison doesn't match anything.
If you intended for INFO to be a list, not a map, please make it so. Otherwise, if you really intended for it to be a map, and you wanted to test whether { email: "test#example.com", password: "123" } is in that map, then what you really need to check is whether the email and password entries in this map is equal to the desired value. So the filter condition can be something like INFO.email = :email AND INFO.password = :password. Or something like this (I'm not sure I understannd what your intention was here).
I have been trying to figure out how to do 2fa with webauthn and I have the registration part working. The details are really poorly documented, especially all of the encoding payloads in javascript. I am able to register a device to a user, but I am not able to authenticate with that device. For reference, I'm using these resources:
https://github.com/cedarcode/webauthn-ruby
https://www.passwordless.dev/js/mfa.register.js
And specifically, for authentication, I'm trying to mimic this js functionality:
https://www.passwordless.dev/js/mfa.register.js
In my user model, I have a webauthn_id, and several u2f devices, each of which has a public_key and a webauthn_id.
In my Rails app, I do:
options = WebAuthn::Credential.options_for_get(allow: :webauthn_id)
session[:webauthn_options] = options
In my javascript, I try to mimic the js file above and I do (this is embedded ruby):
options = <%= raw #options.as_json.to_json %>
options.challenge = WebAuthnHelpers.coerceToArrayBuffer(options.challenge);
options.allowCredentials = options.allowCredentials.map((c) => {
c.id = WebAuthnHelpers.coerceToArrayBuffer(c.id);
return c;
});
navigator.credentials.get({ "publicKey": options }).then(function (credentialInfoAssertion)
{
// send assertion response back to the server
// to proceed with the control of the credential
alert('here');
}).catch(function (err)
{
debugger
console.error(err); /* THIS IS WHERE THE ERROR IS THROWN */
});
The problem is, I cannot get past navigator.credentials.get, I get this error in the javascript console:
TypeError: CredentialsContainer.get: Element of 'allowCredentials' member of PublicKeyCredentialRequestOptions can't be converted to a dictionary
options at the time navigator.credentials.get is called looks like this:
I've tried every which way to convert my db-stored user and device variables into javascript properly encoded and parsed variables but cannot seem to get it to work. Anything obvious about what I'm doing wrong?
Thanks for any help,
Kevin
UPDATE -
Adding options json generated by the server:
"{\"challenge\":\"SSDYi4I7kRWt5wc5KjuAvgJ3dsQhjy7IPOJ0hvR5tMg\",\"timeout\":120000,\"allowCredentials\":[{\"type\":\"public-key\",\"id\":\"OUckfxGNLGGASUfGiX-1_8FzehlXh3fKvJ98tm59mVukJkKb_CGk1avnorL4sQQASVO9aGqmgn01jf629Jt0Z0SmBpDKd9sL1T5Z9loDrkLTTCIzrIRqhwPC6yrkfBFi\"},{\"type\":\"public-key\",\"id\":\"Fj5T-WPmEMTz139mY-Vo0DTfsNmjwy_mUx6jn5rUEPx-LsY51mxNYidprJ39_cHeAOieg-W12X47iJm42K0Tsixj4_Fl6KjdgYoxQtEYsNF-LPhwtoKwYsy1hZgVojp3\"}]}"
This is an example of the serialised JSON data returned by our implementation:
{
"challenge": "MQ1S8MBSU0M2kiJqJD8wnQ",
"timeout": 60000,
"rpId": "identity.acme.com",
"allowCredentials": [
{
"type": "public-key",
"id": "k5Ti8dLdko1GANsBT-_NZ5L_-8j_8TnoNOYe8mUcs4o",
"transports": [
"internal"
]
},
{
"type": "public-key",
"id": "LAqkKEO99XPCQ7fsUa3stz7K76A_mE5dQwX4S3QS6jdbI9ttSn9Hu37BA31JUGXqgyhTtskL5obe6uZxitbIfA",
"transports": [
"usb"
]
},
{
"type": "public-key",
"id": "nbN3S08Wv2GElRsW9AmK70J1INEpwIywQcOl6rp_DWLm4mcQiH96TmAXSrZRHciZBENVB9rJdE94HPHbeVjtZg",
"transports": [
"usb"
]
}
],
"userVerification": "discouraged",
"extensions": {
"txAuthSimple": "Sign in to your ACME account",
"exts": true,
"uvi": true,
"loc": true,
"uvm": true
}
}
This is parsed to an object and the code used to coerce those base64url encoded values is:
credentialRequestOptions.challenge = WebAuthnHelpers.coerceToArrayBuffer(credentialRequestOptions.challenge);
credentialRequestOptions.allowCredentials = credentialRequestOptions.allowCredentials.map((c) => {
c.id = WebAuthnHelpers.coerceToArrayBuffer(c.id);
return c;
});
Hope that helps. The JSON data is retreived via a fetch() call and the byte[] fields are encoded as base64url on the serverside.
I'm trying to create a custom graphql schema to use on my graphql yoga server. The graphql yoga server is just a proxy to another graphql API from which I have managed to retrieve a schema from in JSON format. Here is a preview of what that schema looks like:
{
"data": {
"__schema": {
"queryType": {
"name": "Query"
},
"mutationType": null,
"subscriptionType": null,
"types": [
{
"kind": "OBJECT",
"name": "Core",
"description": null,
"fields": [
{
"name": "_meta",
"description": null,
"args": [],
"type": {
"kind": "NON_NULL",
"name": null,
"ofType": {
"kind": "OBJECT",
"name": "Meta",
"ofType": null
}
},
"isDeprecated": false,
"deprecationReason": null
},
{
"name": "_linkType",
"description": null,
"args": [],
"type": {
"kind": "SCALAR",
"name": "String",
"ofType": null
},
"isDeprecated": false,
"deprecationReason": null
}
],
"inputFields": null,
"interfaces": [
{
I now want to take this generated JSON schema and use it to create a graphql schema to use in my graphql yoga server. I believe the correct way to do this is by using the new GraphQLSchema method from graphql along with a root query. Here is my code attempting this:
schema = new GraphQLSchema({
query: new GraphQLObjectType({
name: 'Query',
fields: schema.data.__schema
})
});
The above code gives me the following error:
Error: Query.mutationType field config must be an object
Not entirely sure where it's going wrong or if this is the proper approach to creating a graphql schema from generated JSON?
The JSON you have is the results of an introspection query. Unfortunately, introspection will not allow you to copy a remote schema. That's because while it does identify what fields exist in a schema, it does not tell you anything about how they should be executed. For example, based on the snippet you posted, we know the remote server exposes a _meta query that returns a Meta type -- but we don't know what code to run to resolve the value returned by the query.
Technically, it's possible to pass the results of an introspection query to buildClientSchema from the graphql/utilities module. However, the schema will not be executable, as the docs point out:
Given the result of a client running the introspection query, creates and returns a GraphQLSchema instance which can be then used with all GraphQL.js tools, but cannot be used to execute a query, as introspection does not represent the "resolver", "parse" or "serialize" functions or any other server-internal mechanisms.
If you want to create a proxy to another GraphQL endpoint, the easiest way is to use makeRemoteExecutableSchema from graphql-tools.
Here's the example based on the docs:
import { HttpLink } from 'apollo-link-http';
import fetch from 'node-fetch';
const link = new HttpLink({ uri: 'http://your-endpoint-url/graphql', fetch });
async function getRemoteSchema () {
const schema = await introspectSchema(link);
return makeRemoteExecutableSchema({
schema,
link,
});
}
The resulting schema is a GraphQLSchema object that can be used like normal:
import { GraphQLServer } from 'graphql-yoga'
async function startServer () {
const schema = await introspectSchema(link);
const executableSchema = makeRemoteExecutableSchema({
schema,
link,
});
const server = new GraphQLServer({ schema: executableSchema })
server.start()
}
startServer()
graphql-tools also allows you to stitch schemas together if you not only wanted to proxy the existing endpoint, but wanted to add on to it as well.
I have a Firebase structure like below. I want to get the message as a returned object. To do this in SQL like
select messages from HukMesssage
How can I do that?
Can anyone helps me how to change the SQL to Firebase query?
My json file looks like below
{
"HukMessages":
[
{
"To": 1,
"From": 2,
"messages": [
{
"name": "'Venkman'",
"message": "'You on your way?'",
"face": "'img/venkman.jpg'"
},
{
"name": "'Felix He'",
"message": "'Ionic comes with a set of colors to start with, but as a general rule colors are meant to be overridden. '",
"face": "'img/felix.jpg'"
}
]
}
]
}
When you execute a query against the Firebase Database, there will potentially be multiple results. So the snapshot contains a list of those results. Even if there is only a single result, the snapshot will contain a list of one result.
Your code will need to deal with the list. Something like:
var query = firebase.database().ref().child("HukMessages").orderByChild("From").equalTo('2');
query.once("value", function(snapshot) {
console.log(snapshot.key); // this will print HukMessages, because that's the location you queried
snapshot.forEach(function(child) {
console.log(child.key); // this will print the key of a message
});
});
I am trying to add an index to a certain Schema with mongoose for text searches. If I add a text index to individual fields it works fine, also with compound indexes it is okay. For example the answer provided here is great:
Full text search with weight in mongoose
However, I am trying to add an index to fields which are references to other Schemas. For example my model looks like the following:
var Book = new Schema({
"title": String,
"createdAt": Date,
"publisher": {
"type": Schema.ObjectId,
"ref": "Publisher"
},
"author": {
"type": Schema.ObjectId,
"ref": "Author"
},
"isbn": String
});
So something like the following indexing doesn't work when you perform a search query as described below:
Indexing:
Book.index({"title": "text", "publisher.name": "text", "author.firstName": "text"});
Search query:
function searchBooks(req, res) {
var query = req.query.searchQuery;
Book.find({ $text: { $search: query } })
.populate('author publisher')
.limit(25)
.exec(function(err, books) {
if (err) {
res.json(err);
} else {
res.json(books);
}
}
);
}
Does anyone have any suggestions of how to add a text index in for the "publisher" and "author" fields, I am using the "populate" mongodb method to pull in the data for these schemas.
I think, what you are looking for is the ability to join tables of data and perform a query against the sum of that data. That is something you need a relational database for, which MongoDB isn't.
So I recommend you change your approach in how you would like to preform your search, e.g. search for your keyword within both author and title instead of attempting to search the whole dataset at the same time.