I am looking for alternative ways to parse a JSON tree. Given the following example:
{
"firstName": "John",
"lastName" : "doe",
"age" : 26,
"address" : {
"streetAddress": "naist street",
"city" : "Nara",
"postalCode" : "630-0192"
}
}
if I am trying to access branches, such as address.streetAddress or address.city then I can see two ways of getting that value:
1) JSON Path with the queries listed above
2) Manually split the query into "branches" (main branch address, sub-branch streetAddress) then check the object keys that way.
Are there any better ways of doing this? And by better I mean efficient, not necessarily "clean" in terms of code.
The application of said code would be to use this within an NGRX metareducer to automatically save the object to localstorage, so it is potentially going to be evaluating a large tree very often.
Thanks.
After some testing of various methods I ended up selecting _ug's answer.
Data below, as well as a gist of the script I used (sans data, which was sensitive). I ran each function 1 million times over 4 queries.
Executing test: empty test
Average time for 1,000,000 executions: 6.181699991226196 milliseconds.
Executing test: split and reduce
Average time for 1,000,000 executions: 429.71359902620316 milliseconds.
Executing test: lodash
Average time for 1,000,000 executions: 1091.8809990286827 milliseconds.
Executing test: json path
Average time for 1,000,000 executions: 32114.680999994278 milliseconds.
You can use lodash _.property method to do so:
https://lodash.com/docs/4.17.15#property
const _ = require('lodash');
let obj = {
"firstName": "John",
"lastName" : "doe",
"age" : 26,
"address" : {
"streetAddress": "naist street",
"city" : "Nara",
"postalCode" : "630-0192"
}
}
console.log(_.property('address.streetAddress')(obj));
// Or
console.log(_.propertyOf(obj)('address.streetAddress'));
Coincidentally just wrote something for this, it doesn't work for arrays but is very simple.
let obj = {
"firstName": "John",
"lastName" : "doe",
"age" : 26,
"address" : {
"streetAddress": "naist street",
"city" : "Nara",
"postalCode" : "630-0192"
}
}
function jsonPath(path, obj) {
return path.split('.').reduce((o, p) => o[p], obj);
}
console.log(jsonPath('address.streetAddress', obj));
console.log(jsonPath('address.city', obj));
You could also adapt it to run with arrays by changing the split to handle brackets split(/[\[\.\]]+/)
You can also make it return undefined instead of throwing an error when a path value is not found by changing the reduce to reduce((o, p) => o && o[p], obj)
Result would be:
let obj = {
"firstName": "John",
"lastName" : "doe",
"age" : 26,
"addresses" : [{
"streetAddress": "naist street",
"city" : "Nara",
"postalCode" : "630-0192"
}]
}
function jsonPath(path, obj) {
return path.split(/[\[\.\]]+/).reduce((o, p) => o && o[p], obj);
}
console.log(jsonPath('addresses[0].streetAddress', obj));
console.log(jsonPath('addresses[0].city', obj));
console.log(jsonPath('addresses[2].city', obj));
Related
I have a document structure something along the lines of the following:
{
"_id" : "777",
"someKey" : "someValue",
"someArray" : [
{
"name" : "name1",
"someNestedArray" : [
{
"name" : "value"
},
{
"name" : "delete me"
}
]
}
]
}
I want to delete the nested array element with the value "delete me".
I know I can find documents which match this description using nested $elemMatch expressions. What is the query syntax for removing the element in question?
To delete the item in question you're actually going to use an update. More specifically you're going to do an update with the $pull command which will remove the item from the array.
db.temp.update(
{ _id : "777" },
{$pull : {"someArray.0.someNestedArray" : {"name":"delete me"}}}
)
There's a little bit of "magic" happening here. Using .0 indicates that we know that we are modifying the 0th item of someArray. Using {"name":"delete me"} indicates that we know the exact data that we plan to remove.
This process works just fine if you load the data into a client and then perform the update. This process works less well if you want to do "generic" queries that perform these operations.
I think it's easiest to simply recognize that updating arrays of sub-documents generally requires that you have the original in memory at some point.
In response to the first comment below, you can probably help your situation by changing the data structure a little
"someObjects" : {
"name1": {
"someNestedArray" : [
{
"name" : "value"
},
{
"name" : "delete me"
}
]
}
}
Now you can do {$pull : { "someObjects.name1.someNestedArray" : ...
Here's the problem with your structure. MongoDB does not have very good support for manipulating "sub-arrays". Your structure has an array of objects and those objects contain arrays of more objects.
If you have the following structure, you are going to have a difficult time using things like $pull:
array [
{ subarray : array [] },
{ subarray : array [] },
]
If your structure looks like that and you want to update subarray you have two options:
Change your structure so that you can leverage $pull.
Don't use $pull. Load the entire object into a client and use findAndModify.
MongoDB 3.6 added $[] operator that facilitates updates to arrays that contain embedded documents. So the problem can be solved by:
db.test.update(
{ _id : "777" },
{$pull : {"someArray.$[].someNestedArray" : {"name":"delete me"}}}
)
As #Melkor has commented (should probably be an answer as itself),
If you do not know the index use:
{
_id: TheMainID,
"theArray._id": TheArrayID
},
{
$pull: {
"theArray.$.theNestedArray": {
_id: theNestedArrayID
}
}
}
From MongoDB 3.6 on you can use arrayFilters to do this:
db.test.update(
{ _id: "777" },
{ $pull: { "someArray.$[elem].someNestedArray": { name: "delete me" } } },
{ arrayFilters: [{ "elem.name": "name1"}] }
)
see also https://docs.mongodb.com/manual/reference/operator/update/positional-filtered/index.html#update-all-documents-that-match-arrayfilters-in-an-array
Other example and usage could be like this:
{
"company": {
"location": {
"postalCode": "12345",
"Address": "Address1",
"city": "Frankfurt",
"state": "Hessen",
"country": "Germany"
},
"establishmentDate": "2019-04-29T14:12:37.206Z",
"companyId": "1",
"ceo": "XYZ"
},
"items": [{
"name": "itemA",
"unit": "kg",
"price": "10"
},
{
"name": "itemB",
"unit": "ltr",
"price": "20"
}
]
}
DELETE : Mongodb Query to delete ItemB:
db.getCollection('test').update(
{"company.companyId":"1","company.location.city":"Frankfurt"},
{$pull : {"items" : {"name":"itemB"}}}
)
FIND: Find query for itemB:
db.getCollection('test').find(
{"company.companyId":"1","company.location.city":"Frankfurt","items.name":"itemB"},
{ "items.$": 1 }
)
3.UPDATE : update query for itemB:
db.getCollection('test').update
(
{"company.companyId":"1","company.location.city":"Frankfurt","items.name":"itemB"},
{ $set: { "items.$[].price" : 90 }},
{ multi: true });
This question already has answers here:
Convert a JavaScript string in dot notation into an object reference
(34 answers)
Closed 3 years ago.
I'm trying to extract some of the fields from a Json response and push them into a Javascript array. I want the selection of the fields to be configurable. Here is what I'm doing:
Consider this as my JSON string:
{
"id" : "1234",
"orderNumber" : "1196",
"createdOn" : "2019-07-02T12:03:39.697Z",
"modifiedOn" : "2019-07-02T12:25:52.126Z",
"testmode" : false,
"customerEmail" : "a#b.com",
"billingAddress" : {
"firstName" : "John",
"lastName" : "Doe",
"address1" : "Wall Street",
"address2" : null,
"city" : "NYC",
"state" : "NY",
"countryCode" : "US",
"postalCode" : "12345",
"phone" : "1122334455"
}
}
Say I want to extract some of the fields (defined in the FIELDS variable) and push them in an array.
# NOTE: the variable `data` here is my json object
var FIELDS = ['id', 'orderNumber', 'customerEmail', 'billingAddress.firstName', 'billingAddress.lastName']
var lineItem = []
# parse the data (my Json object)
for (var i = 0; i < FIELDS.length; i++){
lineItem.push(data[FIELDS[i]])
}
So, this seems to be working OK for the first level (the id, orderNumber and customerEmail) but not for the billingAddress.firstname, etc. This is what I'm wondering about.
My intent is to be able to modify the definition in the FIELDS variable without needing to make any change to the logic in the code.
Hope this makes sense.
Thanks!
As long as there are no periods in the key names this will work.
var data = {
"id" : "1234",
"orderNumber" : "1196",
"createdOn" : "2019-07-02T12:03:39.697Z",
"modifiedOn" : "2019-07-02T12:25:52.126Z",
"testmode" : false,
"customerEmail" : "a#b.com",
"billingAddress" : {
"firstName" : "John",
"lastName" : "Doe",
"address1" : "Wall Street",
"address2" : null,
"city" : "NYC",
"state" : "NY",
"countryCode" : "US",
"postalCode" : "12345",
"phone" : "1122334455"
}
};
var FIELDS = ['id', 'orderNumber', 'customerEmail', 'billingAddress.firstName', 'billingAddress.lastName'];
var lineItem = [];
for (var i = 0; i < FIELDS.length; i++){
let obj = data;
let parts = FIELDS[i].split(".");
while(parts.length) obj = obj[parts.shift()];
lineItem.push(obj)
}
console.log(lineItem);
You just need a function that will split that path on . and traverse the JSON data. With that you can just map() over your fields.
let data = {"id" : "1234","orderNumber" : "1196","createdOn" : "2019-07-02T12:03:39.697Z","modifiedOn" : "2019-07-02T12:25:52.126Z","testmode" : false,"customerEmail" : "a#b.com","billingAddress" : {"firstName" : "John","lastName" : "Doe","address1" : "Wall Street","address2" : null,"city" : "NYC","state" : "NY","countryCode" : "US","postalCode" : "12345","phone" : "1122334455"}}
var FIELDS = ['id', 'orderNumber', 'customerEmail', 'billingAddress.firstName', 'billingAddress.lastName']
// split an traverse
const getFromPath = (path, data) => path.split('.')
.reduce((curr, p) => curr && curr[p], data) // check for curr incase path is undefined
console.log(FIELDS.map(p => getFromPath(p, data)))
The function getFromPath will return undefined if any part of the traversal is not found in the object.
Suppose I have several nested objects(human1, human2, human3) in "human" object.
human: {
"human1": {
"name" : "John",
"sex" : "male",
"age" : 18
}
"human2": {
"name" : "Peter",
"sex" : "male",
"age" : 16
}
"human3": {
"name" : "May",
"sex" : "female",
"age" : 19
}
}
And I have another object called currentPlayer below, which I want it to be a vessel, in order to access the data from "human1", "human2", or "human3" for different use.
currentPlayer: {
"name" : "default",
"sex" : "default",
"age" : 0
}
Example: today I want currentPlayer to be John, and it goes
currentPlayer: {
"name" : "John",
"sex" : "male",
"age" : 18
}
And then I want currentPlayer to be Peter, and it goes:
currentPlayer: {
"name" : "Peter",
"sex" : "male",
"age" : 16
}
How do I iterate property values of currentPlayer like this with loop, not just key in one by one? Thanks...
Bellow code will iterate through all Properties of human object
listofhuman = Object.getOwnPropertyNames(human);
var currentPlayer;
for (var objHumanName in listofhuman) {
if (listofhuman[objHumanName].Name === "Jonh") {
currentPlayer = Object.create(listofhuman[objHumanName]);
break;
}
}
at the end of this loop you will get human which you wonted
if you do Object.getOwnPropertyNames(currentPlayer) this will return array of string which are the actual keys in object currentPlayer, and you can access those values by currentPlayer[arryofProp[0]]
Here is my document:
{
"_id" : "2",
"account" : "1234",
"positions" : {
"APPL" : { "quantity" : "13", "direction" : "long" },
"GOOG" : { "quantity" : "24", "direction" : "long" }
}
}
I would like to get the whole positions object, but only the quantity field and ignore the direction field. Is it possible to do that? Or should I consider this other schema (array of objects):
{
"_id" : "2",
"account" : "1234",
"positions" : [
{ "name" : "APPL", "quantity" : "13", "direction" : "long" },
{ "name" : "GOOG", "quantity" : "24", "direction" : "long" }
]
}
Many thanks!
For the "array" form, all you really need to do is specify the field using "dot notation" to the array member:
db.collection.find({}, { "positions.quantity": 1, })
Which would return:
{
"_id" : "2",
"positions" : [
{ "quantity" : "13" },
{ "quantity" : "24" }
]
}
Or for multiple fields but excluding the "direction" just use both in projection:
db.collection.find({},{ "positions.name": 1, "positions.quantity": 1 })
Which returns the named fields still:
{
"_id" : "2",
"positions" : [
{
"name" : "APPL",
"quantity" : "13"
},
{
"name" : "GOOG",
"quantity" : "24"
}
]
}
For the "named keys" form you need to specify each path:
db.collection.find({},{ "positions.APPL.quantity": 1, "positions.GOOG.quantity": 1 })
Which would return of course:
{
"_id" : "2",
"positions" : {
"APPL" : {
"quantity" : "13"
},
"GOOG" : {
"quantity" : "24"
}
}
}
And that kind of "nastiness" is pervasive with basically ALL MongoDB operations, query or projection or otherwise. When you used "named keys" the "database" has no sane option other than to require you to "name the path". Doing that is of course not really a practical exercise, when the names of keys are likely to differ between documents in the collection.
Traversing keys can only really be done in JavaScript evaluation from a MongoDB standpoint. Since JavaScript evaluation requires interpreter cost in launching and translating data from BSON to a workable JavaScript format, and not to mention the actual cost of evaluating the coded expressions themselves, that is not an ideal approach.
Moreover, from a "query" perspective such handling requires the use of $where to evaluate such an expression where you just want to look for things under each "key" of the "positions" data. This is a "bad" thing, since such an expression cannot possible use an "index" to optimize the query search. Only with a "directly named path" can you actually use or even "create" an index under those conditions.
From a "projection" perspective, the usage of "named keys" means that by similar "traversal" concepts, you actually need JavaScript processing again to do so. And the only mechanism in which MongoDB can use a JavaScript expression to "alter" the output document is by using mapReduce so again this is "super horrible" and you would be using this "aggregation method" for nothing more than document manipulation in this case:
db.collection.mapReduce(
function() {
var id = this._id;
delete this._id;
Object.keys(this.positions).forEach(function(el) {
delete el.direction;
});
emit(id,this);
},
function() {}, // reducer never gets called when all are unique
{ "out": { "inline": 1 } }
)
Even after you did that to just avoid naming paths, the output of mapReduce cannot be a "cursor". So this limits you to either the size of a BSON document in response or actually outputting to a "collection" instead. So this is as far from "practical" as you can get.
There are numerous reasons "why" using an array with "common paths" is so much better than a "named keys" structure that are also far too broad to go into here. The one thing you should accept is that "named keys are bad, okay!" and just move forward with consistent object naming that actually makes quite a lot of sense.
This is my first time using lodash and I'm trying to figure out how to update each record within a JSON object. For example, let's say I have a person object:
"person":[
{
"firstName" : "Alex",
"lastName" : "Smith"
},
{
"firstName" : "Madison",
"lastName" : "Matthews"
}
]
I would like to add a new field called "fullName" to each record in person that is a combination of their first and last names. Is there a way I can do this using lodash and/or javascript? I don't want to use any other tool.
Thanks!
Here if you have persons collection(Array of json objects) then you can add one more property by traversing the collection with lodash function. There are other several ways also.
var persons = [
{
"firstName" : "Alex",
"lastName" : "Smith"
},
{
"firstName" : "Madison",
"lastName" : "Matthews"
}
];
var personsWithFullName = _(persons).forEach(function(person) {
person.fullName = person.firstName + person.lastName;
return person;
});
or
var newPersonsWithFullName = _.map(persons , function(person) {
return _.assign({}, person, {fullName: (person.firstName + person.lastName)});
});