I want to get familiar with indexedDB to built my Firefox WebExtension.
My sample data is structured like this:
const sampleDataRaw = [
{
"ent_seq" : 1413190,
"att1" : [ {
"sub11" : "content1",
"sub12" : [ "word" ]
}, {
"sub11" : "content2"
} ],
"att2" : [ {
"sub21" : "other content",
"sub22" : [ "term" ]
} ]
}, {
"ent_seq" : 1000010,
"att2" : [ {
"sub21" : "more content"
}, {
"sub22" : "more words"
} ]
}
] // end sampleRawData
I got as far as opening/creating my database, adding this sample data and querying it by the ent_seq key using objectStore.get() and objectStore.openCursor().
The problem arises when I want to search the sub11 or sub21 fields using indexes I should have created for these like this:
objectStore.createIndex("sub11Elements", "att1.sub11", { unique: false });
objectStore.createIndex("sub21Elements", "att2.sub21", { unique: false });
When I want to search, say, fields sub11 as here:
var index = objectStore.index("sub11Elements");
index.get("content1").onsuccess = function(event) {
// I should have the first object of my data now, alas the result is undefined instead
};
It certainly does succeed, but the returned value is undefined since the get() didn't actually find anything.
I want to know why it doesn't find the entry and how to make it find it. I figured it might be because the keypath is wrong, but as stated, if I instead search by the key (ent_seq) I can successfully get the result.att1[i].sub11 values.
On mozilla's websites it's stated that keys can be of type string and array (or array within array etc) amongst others, and keypath parts are supposed to be concatenated using dots.
From searching on stackexchange I've so far found that it's not possible to have variable keys inside the keypath, but that shouldn't be the case here anyway.
Therefore, I really don't see what might be causing the search to not find the object inside the database.
It looks like the second level of objects are arrays, not properties of the first level of objects. The . accessor accesses sub properties, not indices of an array.
IDBObjectStore.prototype.get always yields success when there is no error, and is not indicative of whether a match was found.
A bit more on point 1. Look at "att1":[{"sub11" : "content1","sub12" : [ "word" ]}.... Pretend this was was an actual basic JavaScript object. Would you be able to use att1.sub11? No. Because the value of att1 is an array, not an object.
Related
I am getting stuck on a project where I am trying to pull out relevant data from a super massive Google Maps Timeline json, and running into the problem of the structure not being as orderly as I thought it was. Essentially, I am trying to pull out an address, time, date and mileage out of this json for every trip in my car. to use this data, I pasted it into a normal javascript file and named it so I can use it as an object. I then take this data and create a string that will format that info like a CSV file.
From going over the structure of the json by looking at only a few trips, I was able to determine the following general structure:
const google = {
timelineObjects: [
0: {activitySegment: {A NUMBER OF OBJECTS}},
1: {placeVisit : {A NUMBER OF OBJECTS}},
2: {activitySegment: {A NUMBER OF OBJECTS}},
3: {placeVisit : {A NUMBER OF OBJECTS}}
]
}
activitySegment has all the travelling info, like distance, travel times, etc. placeVisit has info about the destination. In my small sample, I was able to just loop through each using an if statement with i%2=0, and just change what I wanted to pull out from each, which worked well.
When I tried adding a larger sample, I was finding that Google occasionally did not create a activitySegment object and only had a placeVisit, which was throwing "Uncaught TypeError: Cannot read property 'distance' of undefined".
I am sure that the even/odd sorting will not work out any more. Is there a way to use a conditional to show if google[timelineObjects][i] is either a {activitySegment} or {placeVisit}? Or, is there a better way to figuring out what object is next in the array?
You can test to see if the Object at the particular array index has a given property using Object.prototype.hasOwnProperty():
const google = {
timelineObjects: [
{activitySegment: {}},
{placeVisit : {}},
{activitySegment: {}},
{placeVisit : {}}
]
};
console.log(google.timelineObjects[0].hasOwnProperty("activitySegment")); // true
console.log(google.timelineObjects[1].hasOwnProperty("activitySegment")); // false
If your objective to see what type of object you get. You can iterate over each object, see what the key of the object is and process the data depending on the key value. Something like this.
Object.entries(google).forEach(([key, value]) => {
if(key==='activitySegment') {
//process activeSegment here
}else {
//process placeVisit here
}
})
i have following issue, i have part of json returned by api:
...
"name": "NEWS",
"products": {
"10432471": {
"id": 10432471,
"productConfig": [
...
]
},
"10432481": {
"id": 10432481
},
"17300241": {
"id": 17300241
},
"17300251": {
"id": 17300251,
"productConfig": [
...
]
},
"18420501": {
"id": 18420501
}
}
...
and i want to get random simple product id (simple product means that it dont have productConfig field inside)
for (i=0;i<res.body.menu.categories.length;i++){
if(res.body.menu.categories[i].name=='NEWS'){
availableProducts = Object.keys(res.body.menu.categories[i].products);
console.log ('avbl prods from news category: ',availableProducts);
for (j=0;;j++){
random=Math.floor(Math.random()*Object.keys(res.body.menu.categories[i].products).length);
if(assert.notInclude(res.body.menu.categories[i].products[random],'productConfig')){
availableProductIdForDelivery = res.body.menu.categories[i].products[random].id;
break;
}
}
}
}
This code gives me error in line with second if (if(assert.notInclude....)) Uncaught AssertionError: object tested must be an array, a map, an object, a set, a string, or a weakset, but undefined given.
So, inside "products" we dont have array, becouse there're {}, not []?
There's any other way to iterate beetween products (their numerical names are changed everyday) and save id to availableProductIdForDelivery
Take a look at the documentation: http://www.chaijs.com/api/assert/#method_notinclude
You see that the first parameter must be Array, String or haystack. Let's take a look at the error message you have given:
Uncaught AssertionError: object tested must be an array, a map, an
object, a set, a string, or a weakset, but undefined given.
This error message indicates that something was undefined instead of the possible expected types listed in the message. I guess it was the first parameter, but it might be the third as well, since you did not pass anything as the third parameter, so it is undefined.
So, inside "products" we dont have array, becouse there're {}, not []?
Correct. As far as I can tell, you are trying to apply an assertion to res.body.menu.categories[i].products[random], but because products is an object (and not a list) attempting to retrieve an index (as oppose to one of the keys defined in the object) with products[random] returns undefined.
If you want to create a list of products in order to evaluate their contents, you may be able to use Object.values(res.body.menu.categories[i].products)[random]. As far as I am aware, this feature is only standard as of ES2017. You may also be able to apply a map function to your availableProducts object to retrieve the corresponding values for each product key.
Ok, thanks for help, now its works as i want. Below my solution, maybe it help someone in the future:
for (i=0;i<res.body.menu.categories.length;i++){
if(res.body.menu.categories[i].name=='NEWS'){ //random simple product from 'news' category
availableProducts = Object.keys(res.body.menu.categories[i].products);
console.log ('avbl prods from news category: ',availableProducts);
while(availableProductIdForDelivery==''){
random=Math.floor(Math.random()*Object.keys(res.body.menu.categories[i].products).length);
if((Object.values(res.body.menu.categories[i].products)[random].productConfig).length==0)
availableProductIdForDelivery = Object.values(res.body.menu.categories[i].products)[random].id;
}
console.log('random simple product: '+availableProductIdForDelivery) ;
}
}
I am trying to take a JSON list that is formatted as such: (real list has over 2500 entries).
[
['fb.com', 'http://facebook.com/']
['ggle.com', 'http://google.com/']
]
The JSON list represents: ['request url', 'destination url']. It is for a redirect audit tool built on node.js.
The goal is to put those JSON value pairs in a javascript object with a key value array pair as such:
var importedUrls = {
requestUrl : [
'fb.com',
'ggle.com'
],
destinationUrl : [
'https://www.facebook.com/',
'http://www.google.com/'
]
}
Due to the sheer amount of redirects, I do prefer a nonblocking solution if possible.
You first need to create your object:
var importedUrls = {
requestUrl: [],
destinationUrl: []
}
Now, let's say you have your data in an array called importedData for lack of a better name. You can then iterate that array and push each value to its proper new array:
importedData.forEach(function(urls){
importedUrls.requestUrl.push(urls[0]);
importedUrls.destinationUrl.push(urls[1]);
});
This will format your object as you want it to be formatted, I hope.
I will propose it to you that you take another approach.
Why not have an array of importedUrls, each one with its correspondent keys?
You could have something like:
importedUrls = [
{
requestUrl: 'req',
destinationUrl: 'dest'
},
{
requestUrl: 'req2',
destinationUrl: 'dest2'
},
]
I'm sure you can figure out how to tweak the code I showed to fit this format if you want to. What you gain with this is a very clear separation of your urls and it makes the iterations a lot more intuitive.
I have the following example client side object:
var obj = {
"locations": [
[
37.502917,
-122.501335
],
[
37.494473,
-122.499619
],
[
37.484394,
-122.455673
]
],
"types": [
[
"type1"
],
[
"type2"
],
[
"type3"
]
]
};
Locations could contain up to 50 values. An ajax request returns a set of new locations and I need to evaluate if they are already within obj.locations. Each new returned location is a string e.g:
var test = 37.502917 + ',' + -122.501335;
For each location I can iterate through the current ones and check if it is present:
for(var i = 0; i < obj.locations.length; i++) {
if(obj.locations[i] == test){
console.log('Found!');
}
}
Is there a more efficient way of doing this as iterating through the object for each new location seems inefficient?
EDIT: My Solution:
I decided to take the locations object and turn in to a string, then evaluate each of the incoming strings:
var test = -121.60183 + ',' + 38.025783;
var cords = [].concat([], obj.locations).toString();
if( cords.indexOf(test) !== -1) {
console.log('found! ');
}
This is perhaps one of the oldest problems in computer science--looking something up.
You first have to ask yourself if it's worth worrying about. Perhaps it will take 1ms to find the location with a linear search, but 0.5ms with some kind of optimized search. So, is it worth the trouble?
The next approach would be to sort the list of locations, and do a binary search into it.
Another approach is to create some kind of hash table. You could use JavaScript objects for this, with properties as hash keys. The simplest approach would be to use lat+long as the property key, but now you've just shifted the efficiency problem to the efficiency of JS looking up keys in large objects.
You could design your own custom hash-like approach, where all locations with the same integral portion of latitude are stored as an array under a hash of 37. Then the performance is governed by the time taken to find the hash key in the table, and then looking through the smaller number of locations within its array.
Proceeding further, if performance is truly an issue, you could build some kind of tree structure for optimal lookup. At some point, you have to start trading off between the cost of building and updating the tree, and the savings from looking things up using the tree.
It is for sure inefficient, but unless you have to deal with thousands of those objects it will not hang your browser.
However, you can index the locations in an associative array and then use that to check for presence or absence of an element.
For example, you could add a locations_index object to your object, like this :
var obj = {
"locations": [
[
37.502917,
-122.501335
],
[
37.494473,
-122.499619
],
[
37.484394,
-122.455673
]
],
"locations_index" : {
"37.502917,-122.501335" : true,
"37.494473,-122.499619" : true,
// ...
},
"types": [
[
Then you can check if it is or not in the location_index with :
if (obj.locations_index["37.502917,-122.501335"]) {
// It's already there
} else {
Obviously, you need to take care of adding the new locations (and removing the ones you remove) from both the "real" array and the "index".
I have a document in Mongo that is structured in the following way:
{
"_id" : ObjectId("4eea7237d0ba3a04f20008fb"),
"code" : "b2677c2809c844cc9d7e3e4ff8d95b46",
"city_id" : 4,
"datetime" : ISODate("2011-12-13T18:41:44.062Z"),
"plays" : [
{
"play_id" : 717224,
"clicks" : [ ],
"order" : 1,
"mysql_id" : 145
}
I want to query for docs whose plays.clicks attribute is a non-empty list. I've tried exists with no luck. I thought that something like this might work:
db.collection.find({plays.clicks.0: {$exists:true}})
But I believe that this would return only docs whose first element in the plays array contains a non-empty clicks list.
Any thought on how I might get this done?
Thanks
db.collection.find({plays.clicks.0: {$exists:true}})
is the right syntax, however as plays is a list the query will match any document that has clicks in plays. There is no way to retrieve a subset of an Array for subelements in this way[1]. There is a ticket for sub / virtual collections[2]
[1] http://www.mongodb.org/display/DOCS/Retrieving+a+Subset+of+Fields#RetrievingaSubsetofFields-RetrievingaSubrangeofArrayElements
[2] https://jira.mongodb.org/browse/SERVER-828
Save the size of the list as a separate attribute (e.g. num_plays). Then you can query for documents where num_plays is greater than 0:
Haven't tested, but I think the query you want is
{ "plays.clicks" : { "$size" : 0 } }
http://www.mongodb.org/display/DOCS/Advanced+Queries#AdvancedQueries-%24size