The internet is full of resources for dealing with arrays, but often objects are a more natural fit for data and seemingly more efficient.
I want to store key-value objects under dynamic field names like this:
project['en-US'] = { 'nav-back': 'Go back', ... }
project['pt-BR'] = { 'nav-back': 'Volte', ... }
Doing this seems like it would be more efficient than keeping an array of all languages and having to filter it to get all language entries for a given language.
My question is: How can I insert a key-value pair into an object with a dynamic name using mongoose? And would the object need to exist or can I create it if it doesn't in one operation?
I tried this:
await Project.update(
{ _id: projectId },
{
$set: {
[`${language}.${key}`]: value,
},
});
But no luck regardless of if I have an empty object there to begin with or not: { ok: 0, n: 0, nModified: 0 }.
Bonus: Should I index these objects and how? (I will want to update single items)
Thanks!
In mongoose, the schema is everything. It describe the data you gonna read/store from the database. If you wanna add dynamically a new key in the schema it's gonna be hard.
In this particulary case I would recommend to use the mongodb-native-driver which is way more permissive about the data manipulation. So you could read the data in a specific format and dynamically add your field into it.
To resume my thought, how should your dynamic change happen :
Use mongodb-native-driver to insert the new key into the database data
Modify the mongoose schema you have in the code (push a new key into it)
Use mongoose to manipulate the data afterward
Do not forget to dynamically update your mongoose model or you won't read the new key at the next find.
I solved this using the original code snippet unchanged, but adding { strict: false } to the schema:
const projectSchema = new Schema({ ...schema... }, { strict: false });
Related
I'm really new to firebase and to be honest I find queries hard to write. I'm working on a script in python using firebase_admin and i'd like the query/answer to be in python code, but in any other programming language is fine.
Here's my one of my document, one document contains photos set
photos: {
id1: true
id2: true
}
I want to be ale to retrieve all items where they have id1 in photos object, what would be the query for that?
As the Firebase documentation on simple queries shows:
# Create a reference to the photos collection
ref = db.collection('documents')
# Create a query against the collection
query_ref = ref.where(u'photos.id1', u'==', True)
For the . notation, see the documentation on fields in nested objects.
Note that you'll probably want to change your data model to use an array for this data, as arrays can now be used to model mathematical sets. With a an array like this in your document:
user: ['id1', 'id2']
You can then filter with:
photos_ref = db.collection('documents')
query = photos_ref.where(u'photos', u'array_contains', u'id1')
To add/remove items to this array-that-behaves-like-a-set, see the documentation on updating elements in an array.
Based on Frank van Puffelen's solution, I was also able to use the dot notation to achieve the nested effect. Querying for objects where the assignment contains the current user's ID.
objectModel: {
objectName: 'foo',
otherField: 'bar',
assignment: {
installerIds: ['1', '2', '3'],
otherfields: 'foobar'
},
}
I was able to query like this
p = { field: 'assignment.installerIds', operator: 'array-contains', value: this.currentInstallerId}
query = query.where(p.field, p.operator, p.value);
or in a more general format like Frank showed:
query_ref = ref.where('assignment.installerIds', 'array-contains', '2');
I have a program where I'm requesting weather data from a server, processing the data, and then saving it to an mlab account using mongoose. I'm gathering 10 years of data, but the API that I'm requesting the data from only allows about a year at a time to be requested.
I'm using findOndAndUpdate to create/update the document for each weather station, but am having trouble updating the arrays within the data object. (Probably not the best way to describe it...)
For example, here's the model:
const stnDataSchema = new Schema(
{
station: { type: String, default: null },
elevation: { type: String, default: null },
timeZone: { type: String, default: null },
dates: {},
data: {}
},
{ collection: 'stndata' },
{ runSettersOnQuery: true }
)
where the dates object looks like this:
dates: ["2007-01-01",
"2007-01-02",
"2007-01-03",
"2007-01-04",
"2007-01-05",
"2007-01-06",
"2007-01-07",
"2007-01-08",
"2007-01-09"]
and the data object like this:
"data": [
{
"maxT": [
0,
null,
4.4,
0,
-2.7,
etc.....
what I want to have happen is when I run findOneAndUpdate I want to find the document based on the station, and then append new maxT values and dates to the respective arrays. I have it working for the date array, but am running into trouble with the data array as the elements I'm updated are nested.
I tried this:
const update = {
$set: {'station': station, 'elevation': elevation, 'timeZone': timeZone},
$push: {'dates': datesTest, 'data.0.maxT': testMaxT}};
StnData.findOneAndUpdate( query, update, {upsert: true} ,
function(err, doc) {
if (err) {
console.log("error in updateStation", err)
throw new Error('error in updateStation')
}
else {
console.log('saved')
but got an output into mlab like this:
"data": {
"0": {
"maxT": [
"a",
"b",
the issue is that I get a "0" instead of an array of one element. I tried 'data[0].maxT' but nothing happens when I do that.
The issue is that the first time I run the data for a station, I want to create a new document with data object of the format in my third code block, and then on subsequent runs, once that document already exists, update the maxT array with new values. Any ideas?
You are getting this output:
"data": {
"0": {
"maxT": [
"a",
"b",
because you are upserting the document. Upserting gets a bit complicated when dealing with arrays of documents.
When updating an array, MongoDB knows that data.0 refers to the first element in the array. However, when inserting, MongoDB can't tell if it's meant to be an array or an object. So it assumes it's an object. So rather than inserting ["val"], it inserts {"0": "val"}.
Simplest Solution
Don't use an upsert. Insert a document for each new weather station then use findOndAndUpdate to push values into the arrays in the documents. As long as you insert the arrays correctly the first time, you will be able to push to them without them turning into objects.
Alternative Simple Solution if data just Contains one Object
From your question, it looks like you only have one object in data. If that is the case, you could just make the maxT array top-level, instead of being a property of a single document in an array. Then it would act just like dates.
More Complicated MongoDB 3.6 Solution
If you truly cannot do without upserts, MongoDB 3.6 introduced the filtered positional operator $[<identifier>]. You can use this operator to update specific elements in an array which match a query. Unlike the simple positional operator $, the new $[<identifier>] operator can be used to upsert as long as an exact match is used.
You can read more about this operator here: https://docs.mongodb.com/manual/reference/operator/update/positional-filtered/
So your data objects will need to have a field which can be matched exactly on (say name). An example query would look something like this:
let query = {
_id: 'idOfDocument',
data: [{name: 'subobjectName'}] // Need this for an exact match
}
let update = {$push: {'data.$[el].maxT': testMaxT}}
let options = {upsert: true, arrayFilters: [{'el.name': 'subobjectName'}]}
StnData.findOneAndUpdate(query, update, options, callbackFn)
As you can see this adds much more complexity. It would be much easier to forget about trying to do upserts. Just do one insert then update.
Moreover mLab currently does not support MongoDB 3.6. So this method won't be viable when using mLab until 3.6 is supported.
I am writing a REST api which I want to make idempotent. I am kind of struggling right now with nested arrays and idempotency. I want to update an item in product_notes array in one atomic operation. Is that possible in MongoDB? Or do I have to store arrays as objects instead (see my example at the end of this post)? Is it for example possible to mimic the upsert behaviour but for arrays?
{
username: "test01",
product_notes: [
{ product_id: ObjectID("123"), note: "My comment!" },
{ product_id: ObjectID("124"), note: "My other comment" } ]
}
If I want to update the note for an existing product_node I just use the update command and $set but what if the product_id isn't in the array yet. Then I would like to do an upsert but that (as far as I know) isn't part of the embedded document/array operators.
One way to solve this, and make it idempotent, would be to just add a new collection product_notes to relate between product_id and username.
This feels like violating the purpose of document-based databases.
Another solution:
{
username: "test01",
product_notes: {
"123": { product_id: ObjectID("123"), note: "My comment!" },
"124": { product_id: ObjectID("124"), note: "My other comment" } }
}
Anyone a bit more experienced than me who have anything to share regarding this?
My understanding of your requirement is that you would like to store unique product ids (array) for an user.
You could create an composite unique index on "username" and "username.product_id". So that when the same product id is inserted in the array, you would an exception which you could catch and handle in the code as you wanted the service to be Idempotent.
In terms of adding the new element to an array (i.e. product_notes), I have used Spring data in which you need to get the document by primary key (i.e. top level attribute - example "_id") and then add a new element to an array and update the document.
In terms of updating an attribute in existing array element:-
Again, get the document by primary key (i.e. top level attribute -
example "_id")
Find the correct product id occurrence by iterating the array data
Replace the "[]" with array occurrence
product_notes.[].note
I'm just getting started with JavaScript objects. I'm trying to store catalog inventory by id and locations. I think a single object would look something like this:
var item = {
id: number,
locations: ["location1", "location2"]
};
I've started to read a bit about it but am still trying to wrap my head around it. Not sure what is the fastest way add new items to a list with a location, add a new location to an existing item, all while checking for dupes. Performance of getting the locations later isn't as critical. This is part of a process that is running thousands of checks to eventually get items by id and location, so performance is key.
Final question, I'm not even sure if it's possible to store this in local storage. From another similar question, I'm not sure.
Using lodash, something like this should work to determine if an item id exists and append either a new item to the array, or just add a new location:
var item = [{
id: 1,
locations: ["location1", "location2"]
},{
id: 2,
locations: ["location2", "location4"]
}];
function findItem(id){
return _.findIndex(item, function(chr) {
return chr.id == id;
});
}
function addItem(id,locations) {
var position = findItem(id);
if (position<0) {
item.push({
id: id,
locations: locations
})
} else {
item[position].locations = _.uniq(item[position].locations.concat(locations));
}
}
addItem(2,['location292']);
addItem(3,['location23']);
console.log(item);
What it basically does is to search the array of objects (item) for an id as the one we are passing to the addItem() function, if it is found we add the new locations array to the existing item, if not it's creating a new object with a new id and location.
You've asked a question that contains some tradeoffs:
The simplest and fastest way to retrieve a list of locations is to store them in an array.
The fastest way to check something for a duplicates is not an array, but rather a map object that maintains an index of the key.
So, you'd have to discuss more about which set of tradeoffs you want. Do you want to optimize for performance of adding a non-duplicate or optimize for performance of retrieving the list of locations. Pick one or the other.
As for localStorage, you can store any string in LocalStorage and you can convert simply non-reference objects to a string with JSON.stringify(), so yes this type of structure can be stored in LocalStorage.
For example, if you want to use the array for optimized retrieval, then you can check for duplicates like this before adding:
function addLocation(item, newLocation) {
if (item.locations.indexOf(newLocation) === -1) {
item.locations.push(newLocation);
}
}
Also, you can store an array of items in LocalStorage like this:
localStorage.setItem("someKey", JSON.stringify(arrayOfItems));
And, then some time later, you can retrieve it like this:
var arrayOfItems = JSON.parse(localStorage.getItem("someKey"));
Hello I'd like to know if it's possible to create pseudo properties on Mondodb. This is, currently I have a collections users like this:
{_id: (_1), name: "user1", secret: "1"}
{_id: (_2), name: "user2", secret: "2"}
When I query the database. I do something like:
function getuser(objectId) {
db.users.find({_id : objectId}).toArray(function(err, result) {
x = result[0];
x.pseudoField1 = hash(secret);
return x;
});
}
Then I do some operations on the x object, and return to put on the database, but before I have to filter the not needed properties, so I do:
y = {}
y._id = x._id
y.name = x.name
y.secret = x.secret
db.users.update({_id: y._id}, y);
What I'd like to do is know if there is any way to make the databse automaticaly return an object with the pseudoField1 with the function I want, and furthermore, when I issue an update with x, only the fields _id, name and secret get updated.
When you want to calculate fields on the database, you can use an aggregation pipeline with a $project stage. The aggregation frameworks offers some simple arithmetic operators to create fields which are derived from values of other fields, but implementing a complex hash function is likely far too complicated to do on the database.
Your second requirement - telling MongoDB to ignore a certain field when inserting - isn't possible out-of-the-box. But what you can do is remove the field from the document before saving it. You can also use an object-document wrapper like Mongoose which allows you to define schemas and exclude certain fields from storing them in the database.