I'm building an app where I need to delete items stored in the database. Here's a (shortened) example of user data I have in my DynamoDB table called 'registeredUsers':
{
"userId": "f3a0f858-57b4-4420-81fa-1f0acdec979d"
"aboutMe": "My name is Mary, and I just love jigsaw puzzles! My favourite jigsaw category is Architecture, but I also like ones with plants in them.",
"age": 27,
"email": "mary_smith#gmail.com",
"favourites": {
"imageLibrary": [
{
"id": "71ff8060-fcf2-4523-98e5-f48127d7d88b",
"name": "bird.jpg",
"rating": 5,
"url": "https://s3.eu-west-2.amazonaws.com/jigsaw-image-library/image-library/images/bird.jpg"
},
{
"id": "fea4fd2a-851b-411f-8dc2-1ae0e144188a",
"name": "porsche.jpg",
"rating": 3,
"url": "https://s3.eu-west-2.amazonaws.com/jigsaw-image-library/image-library/images/porsche.jpg"
},
{
"id": "328b913f-b364-47df-929d-925676156e97",
"name": "rose.jpg",
"rating": 0,
"url": "https://s3.eu-west-2.amazonaws.com/jigsaw-image-library/image-library/images/rose.jpg"
}
]
}
}
I want to be able to delete the item 'rose.jpg' in the user.favourites.imageLibrary array. In order to select the correct user, I can provide the userId as the primary key. Then, in order to select the correct image in the array, I can pass the AWS.DocumentClient the 'id' of the item in order to delete it. However, I'm having trouble understanding the AWS API Reference docs. The examples given in the developer guide do not describe how to delete an item by looking at one of it's attributes. I know I have to provide an UpdateExpression and an ExpressionAttributeValues object. When I wanted to change a user setting, I found it pretty easy to do:
const params = {
TableName: REGISTERED_USERS_TABLE,
Key: { userId },
UpdateExpression: "set userPreferences.difficulty.showGridOverlay = :d",
ExpressionAttributeValues: {
":d": !showGridOverlay
},
ReturnValues: "UPDATED_NEW"
};
To conclude, I need a suitable Key, UpdateExpression and ExpressionAttributeValues object to access the rose.jpg item in the favourites array.
Unfortunately, the UpdateExpression syntax is not as powerful as you would have liked. It supports entire nested documents inside the item, but not sophisticated expressions to search in them or to modify them. The only ability it gives you inside a list is to access or modify its Nth element. For example:
REMOVE #favorites.#imagelibrary[3]
Will remove the 3rd element of imagelibrary (note that the "#imagelibrary" will need to be defined in ExpressionAttributeNames), and you can also have a condition on #favorites.#imagelibrary[3].#id, for example, in ConditionExpression. But unfortunately, there is no way to specify more complex combinations of conditions and updates, such as "find me the i where #favorites.#imagelibrary[i].#id is equal something, and then REMOVE this specific element".
Your remaining option is to read the full value of the item (or with ProjectionExpression just the #favorties.#imagelibrary array), and then in your own code find which of the elements you want to remove (e.g., discover that it is the 3rd element), and then in a separate update, remove the 3rd element.
Note that if there's a possibility that some other parallel operation also changes the item, you must use a conditional update (both UpdateExpression and ConditionExpression) for the element removal, to ensure the element that you are removing still has the id you expected. If the condition fails, you need to repeat the whole operation again - read the modified item again, find the element again, and try to remove it again. This is an example of the so-called "optimistic locking" technique which is often used with DynamoDB.
Related
How do I:
Update an object in the reviews array, if an existing object with the same author exists
Insert a new object otherwise
(In other words: update a review if the author has already written one, and create a new one if they haven't)
Document structure:
{
"_id": "(book id)",
"title": "Title",
"author": "Author",
"reviews": [{
"author": "(user id)",
"rating": 4
}],
}
This is what I've tried so far -- it only updates:
db.books.update_one(
{"_id": ObjectId(book_id)},
{"$set": {"reviews.$[elem]": { # or $push etc.
"author": user_id, "rating": rating}}},
array_filters=[{"elem.author": user_id}])
Alternatively I have also tried result.modified_count == 0: then insert but this would insert a duplicate if the previous rating is the same as the new one (hence not updating anything and result.modified_count == 0)
Sorry for another similar question! Thank you in advance!
EDIT:
From the $addToSet operator page:
"...you cannot specify that MongoDB compare only a subset of the fields in the document to determine whether the document is a duplicate of an existing array element."
However I'm still interested if anyone has a way to do this (it doesn't have to be in one query)!
From reading the docs, I would try find and modify with a condition on reviews.author, then an update with an update operator expression using $push on reviews.
I am working on a search application that uses algolia for indexing. When the user types a search term into the text input box, we want to populate the autocompletion dropdown with events. Every event belongs to an event category as well.
Example:
{
"category": "Disney",
"events": [
{
"title": "Ice Skating"
},
{
"title": "Peter-Pan"
},
{
"title": "Roller Skating"
}
]
}
If someone searches for "skating", we want to pull back the parent category and the child events "Ice Skating" and "Roller Skating" but omit the "Peter-Pan" event.
Is this type of nested filtering possible with Algolia? If so, how would the filtering work? Would it need to be done with JS, will Algolia handle it for me or do we need to create separate indexes for Event Categories and then Individual Events?
Thanks!
Yes, Algolia will automatically filter out Peter Pan and return items with Skating.
As an example I've got some data setup like so:
announcements: [
{
id: ..,
..
author: {
id: ..,
name: 'Preston',
..
}
},
..
]
If I search Announcements for Preston it'll return any announcement that has Preston in the author attribute. This is the default for Algolia and will search the entire record for your search term. This can be slow.
You can go into the Algolia dashboard, or with the API, and under your index's Ranking tab define what you want to search by and ignore.
The first thing you need to do is adding events.title to the searchable attributes. This will make sure that when the end-user types skat, it will match one of the title.
Then you can check what parts of each result is matched using _highlightResult and more specifically filtering based on the matchLevel:
full is when you have a match and none is when you don't.
This filtering should be possible in JS, in the template you use to display the results.
I’m migrating from Mongo to Firebase with Algolia on top to provide the search. But hitting a snag coming up with a comparable way to search in individual elements of a record.
I have an object that stores when a room is available: from and to. Each record can have many individual from/to combos (see the sample below with 2). I want to be able to run a search something like:
roomavailable.from <= 1522195200 AND roomavailable.to >=1522900799
But only have the query search a match within each element, not any facet in all elements. An element query in Mongo works like that. But if I run that query on the record listed below, it will return the record, because the two roomavailable objects satisfy the .from and .to query. I think.
Is there a way to ensure the search is looking only at matching a pair of .from and .to in an individual object/element?
Below is the pertinent part of the record stored in Algolia so you can see the structure.
"roomavailable": [
{
"_id": "rJbdWvY9M",
"from": 1522195200,
"to": 1522799999
},
{
"_id": "r1H_-vKqz",
"from": 1523923200,
"to": 1524268799
}
],
And here is the Mongo (mongoose) equivalent where its searching inside individual elements (this works):
$elemMatch: {
from: {
$lte: moment(dateArray[0]).utc().startOf('day').format()
},
to: {
$gte: moment(dateArray[1]).utc().endOf('day').format()
}
}
I have also tried this query but it seems to still match either the .from AND .to but in any of the the individual roomavailable elements:
index.search({
query: '',
filters: filters,
facetFilters: [roomavailable.from: 1522195200, roomavailable.to: 1524268799],
attributesToRetrieve: [
"roomavailable",
],
restrictHighlightAndSnippetArrays: true
})
I found a couple posts on Algolia discussing using 1 bracket vs. 2 brackets in the facetFilters. I've tried both. Neither work.
Any suggestions would be awesome. Thanks!
Edit: See discussion on Algolia Discourse:
https://discourse.algolia.com/t/how-to-match-multiple-attributes-in-nested-object-with-numericfilters/4887/8
Hi #kanec, thanks for clarifying your question!
Indeed what #Alefort suggested (using roomavailable in a separate index) would be the easiest option since the query I mentioned above will definitely return the results you want. This will mean that you'll have to query the room availability index separately in order to get which IDs are available, so you'll have to use multiple-queries:
https://www.algolia.com/doc/api-reference/api-methods/multiple-queries/
That said, I asked our core API team to see if there's a more reasonable way to approach this issue, but I fear that this is a filter limit due to performance reasons with arrays. You could transform your data structure in the following and index your rooms as an object instead:
[
{
"roomavailable": {
"0": {
"_id": "rJbdWvY9M",
"from": 1522195200,
"to": 1522799999
},
"1": {
"_id": "r1H_-vKqz",
"from": 1523923200,
"to": 1524268799
}
}
}
]
So you can apply the following filter:
{
"filters": "roomavailable.0.from <= 1522195200 AND roomavailable.0.to >= 1522799999 AND roomavailable.1.from <= 1522195200 AND roomavailable.1.to >=1522900799"
}
The downside of this is that you'll need to know the length of roomavailable in order to build the search query on the front-end (you can do so at indexing time by adding a roomavailable_count property) and also this will probably will be less performant with a considerable number of rooms per item; in this case, switching to a dedicated index makes totally sense for the following reasons:
If in your backend you frequently update available rooms you won't impact the other indices' build time
Filters will perform better (as explained above)
Indexing strategy will be simpler to handle
Let me know what you think about this and if it helps you out.
This code is sorting by different fields within the document and some of those fields are arrays of objects with the key I want to sort by. I don't understand the behavior that I am seeing when I run the following queries. I don't have a lot of experience with mongodb and didn't write these queries.
const dbCursor = connection.collection('container').find({});
cursor.sort({ "titles.title": 1 });
// These don't happen one after the other as shown here. It's either or.
cursor.sort({ "dates.start": 1 });
Titles and dates are both arrays of objects containing the key passed. Dates appears to be sorting by start date even though there may be multiple object with the the key start in it. Title is not sorting alphabetically and appears to be very random. I don't understand what is actually happening when this type of sort is performed in MongoDB.
How is Mongodb handling the array?
Is it only checking the first element in the array?
Is it checking all the elements in the array?
Is there a better way to perform this type of sorting when dealing with arrays?
// one
{ "titles": [{ "title": "Zippy Mississippi Race" }, { "title": "Wacky
Races"}] }
// two
{ "titles": [{ "title": "New Looney Tunes" }, { "title": "Your Bunny
or Your Life/Misjudgment Day" }] }
// three
{ "titles": [{ "title": "Why Oh Why Wyoming" }, { "title": "Wacky Races" }] }
Returns in this order
Update:
So I have discovered that its sorts all the elements in the array. I just don't understand how it determines who the winner is. Can anyone explain why this order is correct based on sorting all the elements in the array?
I am writing a REST api which I want to make idempotent. I am kind of struggling right now with nested arrays and idempotency. I want to update an item in product_notes array in one atomic operation. Is that possible in MongoDB? Or do I have to store arrays as objects instead (see my example at the end of this post)? Is it for example possible to mimic the upsert behaviour but for arrays?
{
username: "test01",
product_notes: [
{ product_id: ObjectID("123"), note: "My comment!" },
{ product_id: ObjectID("124"), note: "My other comment" } ]
}
If I want to update the note for an existing product_node I just use the update command and $set but what if the product_id isn't in the array yet. Then I would like to do an upsert but that (as far as I know) isn't part of the embedded document/array operators.
One way to solve this, and make it idempotent, would be to just add a new collection product_notes to relate between product_id and username.
This feels like violating the purpose of document-based databases.
Another solution:
{
username: "test01",
product_notes: {
"123": { product_id: ObjectID("123"), note: "My comment!" },
"124": { product_id: ObjectID("124"), note: "My other comment" } }
}
Anyone a bit more experienced than me who have anything to share regarding this?
My understanding of your requirement is that you would like to store unique product ids (array) for an user.
You could create an composite unique index on "username" and "username.product_id". So that when the same product id is inserted in the array, you would an exception which you could catch and handle in the code as you wanted the service to be Idempotent.
In terms of adding the new element to an array (i.e. product_notes), I have used Spring data in which you need to get the document by primary key (i.e. top level attribute - example "_id") and then add a new element to an array and update the document.
In terms of updating an attribute in existing array element:-
Again, get the document by primary key (i.e. top level attribute -
example "_id")
Find the correct product id occurrence by iterating the array data
Replace the "[]" with array occurrence
product_notes.[].note