We Keep Coding

Approach to selecting a document

I am using Couchbase in a node app. Every time I insert a document, I am using a random UUID.
It inserts fine and I could retrieve data based on this id.
But in reality, I actually want to search by a key called url in the document. To be able to get or update or delete a document.
I could possibly add the url as the id I suppose but that is not what I see in any database concepts. Ids are not urls
or any unique names. They are typically random numbers or incremental numbers.
How could I approach this so that I can use a random UUID as id but be able to search by url?
Cos lets say the id was 56475-asdf-7856, I am not going to know this value to search for right.
Whereas if the id was https://www.example.com I know about this url and searching for it would give me what I want.
Is it a good idea making the url the id.
This is in a node app using Couchbase.
databaseRouter.put('/update/:id', (req, res) => {
updateDocument(req)
.then(({ document, error }) => {
if (error) {
res.status(404).send(error);
}
res.json(document);
})
.catch(error => res.status(500).send(error));
});
export const updateDocument = async (req) => {
try {
const result = await collection.get(req.params.id); // Feels like id should be the way to do this, but doesn't make sense cos I won't know the id beforehand.
document.url = req.body.url || document.url;
await collection.replace(req.params.id, document);
return { document };
} catch (error) {
return { error };
}
};

I think it's okay to use URLs as IDs, especially if that's the primary way you're going to lookup documents, and you don't need to change the URL later. Yes, often times IDs are numbers or UUIDs, but there is no reason you have to be restricted to this.
However, another approach you can take is to use a SQL query (SQL++, technically, since this is a JSON database).
Something like:
SELECT d.*
FROM mybucket.myscope.mydocuments d
WHERE d.url = 'http://example.com/foo/baz/bar'
You'll also need an index with that, something like:
CREATE INDEX ix_url ON mybucket.myscope.mydocuments (url)
I'd recommend checking out the docs for writing a SQL++ query (sometimes still known as "N1QL") with Node.js: https://docs.couchbase.com/nodejs-sdk/current/howtos/n1ql-queries-with-sdk.html
Here's the first example in the docs:
async function queryPlaceholders() {
const query = `
SELECT airportname, city FROM \`travel-sample\`.inventory.airport
WHERE city=$1
`;
const options = { parameters: ['San Jose'] }
try {
let result = await cluster.query(query, options)
console.log("Result:", result)
return result
} catch (error) {
console.error('Query failed: ', error)
}
}

Dynamobd error: Scan Results don't filter to HASH even though I put in an Actual entry ID

I have connected to this local I created dynamo with a Lambda with this code
console.log('Starting Function Now');
const AWS = require("aws-sdk");
const docClient = new AWS.DynamoDB.DocumentClient({region: 'us-east-1'})
exports.handler = function(event,ctx,callback){
let scanParameters = {
"TableName": "Accommodation_Request",
"FilterExpression": "SSD_ID = :val",
"ExpressionAttributeValues": {":val": {"N": "2"}},
"ProjectionExpression": 'SSD_ID,AI_Org_ID'
};
console.log(scanParameters)
console.log("Your Student information");
docClient.scan(scanParameters, function(err,data){
if(err){
callback(err, null)
} else{
callback(null,data)
}
});
};
After running the Lambda this I get this outcome
{
"Items": [],
"Count": 0,
"ScannedCount": 5
}
Problem: I am unable to return the correct scan results with SSD_ID = 1

It could be in case you are using String instead of Number in your filter or you have composite partition key

I would guess that there is more data to look at. Is there a value in 'LastEvaluatedKey'? When you do a scan operation with a filter you may get no results on a particular request. The LastEvaluatedKey value should be passed into the next request's ExclusiveStartKey. You can potentially have many empty responses before getting any data. Scans should be used with caution, and generally not be part of production workloads (exceptions do exists, of course).

What consequences does spamming a DynamoDB table with invalid keys have?

I have a list of id and for each of them I'm fetching the corresponding item from a DynamoDB table using GetItem.
The thing is some ids are not present.
My question is: Let's say if I go through my list, there are 5000 ids that doesn't match any item in table, and I make each call with a 2 seconds delay between each of them.
What should I expect to happen to my table ?
const dynamo = new AWS.DynamoDB.DocumentClient();
const getItem = (key) => {
const getParams = {
TableName: 'my-table',
Key: {
id: key
}
};
return dynamo
.get(getParams)
.promise()
.then(result => {
const item = result.Item;
if(item){
return Promise.resolve(item);
}
return Promise.reject();
}).catch(error => {
console.log('Could not retrieve item with id', key);
return Promise.reject(error);
});
};

Well nothing will happen to DynamoDB Table. It will still serve normally. It's very scalable and fast. But, attention here if you missed that. This might trick you into increasing costs ->
If you perform a read operation on an item that does not exist,
DynamoDB still consumes provisioned read throughput: A strongly
consistent read request consumes one read capacity unit, while an
eventually consistent read request consumes 0.5 of a read capacity
unit.
See here

Set an property to object after query.exec()

I am using node express. I am trying to populate a field which is populated from another field. But I was having problem so I tried to get the value from another query and then set it to the object I was sending.
api.get('/:id', authenticate, (req, res) => {
let query = MobileTranscation.findById(req.params.id)
.populate('sender')
.populate('created_by');
query.exec((err, datas) => {
if(datas.sender && datas.sender.parent) {
User.findById(datas.sender.parent, (err, user) => {
datas['parentName'] = user.username;
console.log(datas);
res.json(datas);
});
} else {
res.json(datas);
}
});
});
Here, I want to get username from MobileTranscation.sender in which sender has a parent property which has the username property. I was trying to populate sender and then take the parent id and search User with that and get the username. Problem I am facing is that if I log datas.parentName I can see I got the username, but when I am sending datas, there is no parentName property. What I am doing wrong here?
And is it possible to populate a property and then populate one of that populated property?

Whenever you do findById mongoose actually sends some kind of immutable object in return. So you cannot add an extra key to the object. You actually can use the .lean function. Like query.lean().exec((err, data) => { /** Function body* */ }). Now you can add extra keys you want.

What are MongoDB Modifiers and Operators?

I am working on a Meteor application and one of the features I'm building is a form that inserts a new document into an array (inserts a shipping address to a user's profile where a user can have multiple addresses). The error I keep getting is:
Exception while invoking method 'addAddress' Error: When the modifier option is true, validation object must have at least one operator
I have been unsuccessfully trying to figure out the answer on Stackoverflow, Github, etc. but could not find a solution. I now want to take the approach of understanding exactly what the error means - so my question is what exactly are modifier options and operators in MongoDB? From what I understand, modifiers provide constraints on what type of data is returned from a query, and operators are used to modify data. Are these definitions correct?
Does anyone know what the error I'm getting might mean? Here is my sample code:
My click event to capture data on a form and call a method to add an address:
Template.editAddress.events({
'click .addAddress': function(e, tmpl) {
e.preventDefault();
var currentUserId = Meteor.userId();
console.log(currentUserId);
var addressDetails = {
address: {
streetAddress: $('#streetAddress').val()
}
};
console.log(addressDetails);
Meteor.call('addAddress', addressDetails, currentUserId, function(error) {
if (error) {
alert(error.reason);
} else {
console.log('success!');
Router.go('Admin');
}
});
}
});
My method to insert the address:
Meteor.methods({
'addAddress': function(addressDetails, currUserId) {
var currentUserId = currUserId;
console.log('user to add address to is ' + currUserId);
Meteor.users.update(currentUserId, {$addToSet:
{
'address.streetAddress': addressDetails.streetAddress
}
});
}
});
Note that when I type that query in the console, it works:
db.users.update({_id: 'Qdf89k3fd93jfdk'}, {$addToSet: {'address.streetAddress': '12345 fake st'}});
Thank you in advance!

Your addressDetails object doesn't have a field streetAddress, so addressDetails.streetAddress returns undefined. Use addressDetails.address.streetAddress instead in the update. And also, like Joshua pointed out, use an object as selector with { _id: currUserId }. So the whole function should be:
Meteor.users.update( { _id: currentUserId }, {$addToSet:
{
'address.streetAddress': addressDetails.address.streetAddress
}
});
}
One more thing, you should not pass the userId from the client. Any method you define is callable from the client and like that, I would be able to call your method 'addAddress' from the browser console with any userId to update their address. Instead, use the this.userId object in method calls (see here) and check that it is not null, i.e. user is logged in.
if (! this.userId)
throw new Meteor.Error(401, "You must be logged in!");
var currentUserId = this.userId;

It looks like you're passing in the document id directly into the MongoDB query method, rather than constructing an object with an _id property with a value of said document id.
i.e try
var currentUserId = { _id: currUserId };