I'm looking for a way to take a bunch of JSON objects and store them in a data structure that allows both fast lookup and also fast manipulation which might change the position in the structure for a particular object.
An example object:
{
name: 'Bill',
dob: '2014-05-17T15:31:00Z'
}
Given a sort by name ascending and dob descending, how would you go about storing the objects so that if I have a new object to insert, I know very quickly where in the data structure to place it so that the object's position is sorted against the other objects?
In terms of lookup, I need to be able to say, "Give me the object at index 12" and it pulls it quickly.
I can modify the objects to include data that would be helpful such as storing current index position etc in a property e.g. {_indexData: {someNumber: 23, someNeighbour: Object}} although I would prefer not to.
I have looked at b-trees and think this is likely to be the answer but was unsure how to implement using multiple sort arguments (name: ascending, dob: descending) unless I implemented two trees?
Does anyone have a good way to solve this?
First thing you need to do is store all the objects in an array. That'll be your best bet in terms of lookup considering you want "Give me the object at index 12", you can easily access that object like data[11]
Now coming towards storing and sorting them, consider you have the following array of those objects:
var data = [{
name: 'Bill',
dob: '2014-05-17T15:31:00Z'
},
{
name: 'John',
dob: '2013-06-17T15:31:00Z'
},
{
name: 'Alex',
dob: '2010-06-17T15:31:00Z'
}];
The following simple function (taken from here) will help you in sorting them based on their properties:
function sortResults(prop, asc) {
data = data.sort(function(a, b) {
if (asc) return (a[prop] > b[prop]);
else return (b[prop] > a[prop]);
});
}
First parameter is the property name on which you want to sort e.g. 'name' and second one is a boolean of ascending sort, if false, it will sort descendingly.
Next step, you need to call this function and give the desired values:
sortResults('name', true);
and Wola! Your array is now sorted ascendingly w.r.t names. Now you can access the objects like data[11], just like you wished to access them and they are sorted as well.
You can play around with the example HERE. If i missed anything or couldn't understand your problem properly, feel free to explain and i'll tweak my solution.
EDIT: Going through your question again, i think i missed that dynamically adding objects bit. With my solution, you'll have to call the sortResults function everytime you add an object which might get expensive.
Related
I am using IE 11.
I have an object array that is grouped using the lodash library. I want to be able to query the object and based on certain conditions come up with sums/counts. So for example, I have this object array.
I would like to have the result seen below but in an object like the image above
As you can see, each company in the group should have certain values based on the following criteria
How many times does 'company x' have a Total Count >3?
How many times does 'company x' have expectingFunding eq ‘Yes’>
How many times does 'company x' have fundedOnIKNS eq ‘No’?
I've tried quite a bit in the last couple of days but not success. I first declared 2 arrays so I can capture the unique values of company name and program. I also created an object to update when conditions were met. The only successful thing I was able to get was to keep it in an grouped object. All the values in the new object were wrong.
Here's an excerpt of the code:
const companiesSummary = {};
for (const company of Object.keys(myData)) {
companiesSummary[company] = {
totalCount: 0,
expectedFunding: 0,
IKNSFunding: 0,
};
for (const { TotalCount, expectedFunding, fundedOnIKNS } of myData[company]) {
companiesSummary[company].totalCount += TotalCount;
companiesSummary[company].expectedFunding += expectedFunding === "Yes";
companiesSummary[company].fundedOnIKNS += fundedOnIKNS === "Yes";
}
}
I get the error,
TypeError: myData[company] is not iterable
Here's a link to the pen
I would still like the result to be in an object array, so I can create an html table later. Any help would be much appreciated. Thank you!
Your code isn't working because you're taking myData, an array, accessing myData[company], an object (company is 0, 1, ...), and you can't iterate through an object with for...of. myData is definitely not the same object in your screenshot.
Your code excerpt might work if your myData object were the object in your screenshot.
I have a table called subcategories with columns 'id' and 'name' and a table called goals with columns 'id', 'name' and foreign key 'subcategory_id'.
I want a query that results in an array of subcategory objects, which has a property 'goals' which is an array of goal objects.
Too give an example of how the result would look In JS code:
result = [
{id: 1, name: "name", goals: [{id: 1, name: "goalName"}, {...}, {...}]},
{...},
{...}
]
But (with a different syntax) the result would be the same for other languages..
Thusfar I tried to do this with left-join, like this:
SELECT sc.ID as subcatId, sc.name as subcatName, g.ID as ID, g.name as name
FROM needs_subcategories as sc
LEFT JOIN needs_goals as g
ON sc.ID=g.subcategory_id
But the goals aren't grouped under a single subcategory.. I feel like it should be possible to do with a query, but I can't figure out/google how to do it because I wouldn't know how to phrase the question due to my lack of SQL knowledge..
Hope you guys can help me!
Thanks in advance.
You won't be able to acheive that with a query. MySQL can't do that.
You are currently fetching all goals, each one with their subcategory (subcategories will repeat).
You can convert it to the desired array with some code (example in php, you can translate this to any other language).
$result=array();
$lastSubcatId=null;
$goals=array();
while($row=$query->fetch_object()) { //assuming $query is the resultset
if($lastSubcatId&&$lastSubcatId!=$row->subcatId) {
$row->goals=$goals;
$result[]=$row; //or you could assign each desired property
$goals=array();
}
$goals[]=$row; //or you could assign each desired property
}
//surely, there are items left in $goals
if($lastSubcatId) {
$row->goals=$goals;
$result[]=$row; //or you could assign each desired property
}
But a more efficient way would be, I think, with multiple queries:
$result=array();
$subcats=$db->query("SELECT * FROM needs_subcategories");
while($subcat=$subcats->fetch_object()) {
//you might want to use prepared statements, I'm just simplifying
//it will not only be safer, but reusing the prepared statement will increase the performance considerably
$goals=$db->query("select * from needs_goals where subcategory_id=".$subcat->ID);
$temp=array();
while($goal=$goals->fetch_object()) $temp[]=$goal;
$subcat->goals=$temp;
$result[]=$subcat;
}
In the end I solved this using groupBy as #tadman suggested in his comment.
I created a function (based on the information in this answer) that looks like this:
function processResults(collection, groupKey) {
var result = _.chain(collection)
.groupBy(groupKey)
.toPairs()
.map(function (currentItem) {
// 'text' and 'children' are the keys I want in my resulting object
// children being the property that contains the array of goal objects
return _.zipObject(['text', 'children'], currentItem);
})
.value();
return result;
}
Which results in the array of objects with grouped goals! As I structured the function now (with hard-coded key names) it only works for my specific case, if you want to generalize the function you could add parameters amd replace the hard-coded key names with those.
I'm just getting started with JavaScript objects. I'm trying to store catalog inventory by id and locations. I think a single object would look something like this:
var item = {
id: number,
locations: ["location1", "location2"]
};
I've started to read a bit about it but am still trying to wrap my head around it. Not sure what is the fastest way add new items to a list with a location, add a new location to an existing item, all while checking for dupes. Performance of getting the locations later isn't as critical. This is part of a process that is running thousands of checks to eventually get items by id and location, so performance is key.
Final question, I'm not even sure if it's possible to store this in local storage. From another similar question, I'm not sure.
Using lodash, something like this should work to determine if an item id exists and append either a new item to the array, or just add a new location:
var item = [{
id: 1,
locations: ["location1", "location2"]
},{
id: 2,
locations: ["location2", "location4"]
}];
function findItem(id){
return _.findIndex(item, function(chr) {
return chr.id == id;
});
}
function addItem(id,locations) {
var position = findItem(id);
if (position<0) {
item.push({
id: id,
locations: locations
})
} else {
item[position].locations = _.uniq(item[position].locations.concat(locations));
}
}
addItem(2,['location292']);
addItem(3,['location23']);
console.log(item);
What it basically does is to search the array of objects (item) for an id as the one we are passing to the addItem() function, if it is found we add the new locations array to the existing item, if not it's creating a new object with a new id and location.
You've asked a question that contains some tradeoffs:
The simplest and fastest way to retrieve a list of locations is to store them in an array.
The fastest way to check something for a duplicates is not an array, but rather a map object that maintains an index of the key.
So, you'd have to discuss more about which set of tradeoffs you want. Do you want to optimize for performance of adding a non-duplicate or optimize for performance of retrieving the list of locations. Pick one or the other.
As for localStorage, you can store any string in LocalStorage and you can convert simply non-reference objects to a string with JSON.stringify(), so yes this type of structure can be stored in LocalStorage.
For example, if you want to use the array for optimized retrieval, then you can check for duplicates like this before adding:
function addLocation(item, newLocation) {
if (item.locations.indexOf(newLocation) === -1) {
item.locations.push(newLocation);
}
}
Also, you can store an array of items in LocalStorage like this:
localStorage.setItem("someKey", JSON.stringify(arrayOfItems));
And, then some time later, you can retrieve it like this:
var arrayOfItems = JSON.parse(localStorage.getItem("someKey"));
I have a json object that looks something like:
{
posts: [
{id: 56, title: 'something', date: '10-10-10 00:00:00'},
{id: 57, title: 'Apples', date: '10-10-16 00:00:00'},
]
}
And I am trying to learn how to manipulate data structures in Javascript. I would like to use jquery to reorder these based on the posts attributes, so you might reorder based on date, title or id (default is id).
How ever being a novice, I have no idea where to start I have seen this answer andI dont think its the direction I want to go.
Would the resulting out put be in the same structure? How could I create a collections of models based off this kind of "re-ordering".
The idea is to allow for editing and publishing of one model in a collection, based off the attribute used to re-order or "sort" the collection, thus allowing for faster processing - so I don't always have to pass the whole collection to the server when I really just want to update one model in the collection.
Take a look at the JavaScript array.sort method. It may do what you want.
var posts = [{"id": 57, "title": "something"}, {"id": 56, "title": "something else"}];
posts.sort(function(a, b) { if (a.id < b.id) { return -1; } else { return 1; } });
for (var i=0; i<posts.length-1; i++) {
console.log(posts[i].title);
}
// prints "something else" followed by "something"
The sort method takes a function which is passed two elements, a and b, and you can return -1 if a is less than b, 0 if they are equal, and 1 if a is greater than b.
Note that my posts variable above is just a simplified version of what would be o.posts if your described JSON was stored at o. Also note that sort sorts in place, meaning it will change the array it operates over.
Can you suggest me an algorithm for filtering out data.
I am using javascript and trying to write out a filter function which filters an array of data.I have an array of data and an array of filters, so in order to apply each filter on every data, I have written 2 for loops
foreach(data)
{
foreach(filter)
{
check data with filter
}
}
this is not the proper code, but in short that what my function does, the problem is this takes a huge amount of time, can someone suggest a better method.
I am using the Mootools library and the array of data is JSON array
Details of data and Filter
Data is JSON array of lets say user, so it will be
data = [{"name" : "first", "email" : "first#first", "age" : "20"}.
{"name" : "second", "email" : "second#second", "age" : "21"}
{"name" : "third", "email" : "third#third", "age" : "22"}]
Array of filters is basically self define class for different fields of data
alFilter[0] = filterName;
alFilter[1] = filterEmail;
alFilter[2] = filterAge;
So when I enter the first for loop, I get a single JSON opbject (first row) in the above case.
When I enter the second for loop (filters loop) I have a filter class which extracts the exact field on which the current filter would work and check the filter with the appropriate field of the data.
So in my example
foreach(data)
{
foreach(filter)
{
//loop one - filter name
// loop two - filter email
// loop three - filter age
}
}
when the second loop ends i set a flag denoting if the data has been filtered or not and depending on it the data is displayed.
You're going to have to give us some more detail about the exact structure of your data and filters to really be able to help you out. Are the filters being used to select a subset of data, or to modify the data? What are the filters doing?
That said, there are a few general suggestions:
Do less work. Is there some way you can limit the amount of data you're working on? Some pre-filter that can run quickly and cut it down before you do your main loop?
Break out of the inner loop as soon as possible. If one of the filters rejects a datum, then break out of the inner loop and move on to the next datum. If this is possible, then you should also try to make the most selective filters come first. (This is assuming that your filters are being used to reject items out of the list, rather than modify them)
Check for redundancy in the computation the filters perform. If each of them performs some complicated calculations that share some subroutines, then perhaps memoization or dynamic programming may be used to avoid redundant computation.
Really, it all boils down to the first point, do less work, at all three levels of your code. Can you do less work by limiting the items in the outer loop? Do less work by stopping after a particular filter and doing the most selective filters first? Do less work by not doing any redundant computation inside of each filter?
That's pretty much how you should do it. The trick is to optimize that "check data with filter"-part. You need to traverse all your data and check against all your filters - you'll not going to get any faster than that.
Avoid string comparisons, use data models as native as possible, try to reduce the data set on each pass with filter, etc.
Without further knowledge, it's hard to optimize this for you.
You should sort the application of your filters, so that two things are optimized: expensive checks should come last, and checks that eliminate a lot of data should come first. Then, you should make sure that checking is cut short as soon as an "out" result occurs.
If your filters are looking for specific values, a range, or start of a text then jOrder (http://github.com/danstocker/jorder) will fit your problem.
All you need to do is create a jOrder table like this:
var table = jOrder(data)
.index('name', ['name'], { grouped: true, ordered: true })
.index('email', ['email'])
.index('age', ['age'], { grouped: true, ordered: true, type: jOrder.number });
And then call table.where() to filter the table.
When you're looking for exact matches:
filtered = table.where([{name: 'first'}, {name: 'second'}]);
When you're looking for a certain range of one field:
filtered = table.where([{age: {lower: 20, upper: 21}}], {mode: jOrder.range});
Or, when you're looking for values starting with a given string:
filtered = table.where([{name: 'fir'}], {mode: jOrder.startof});
Filtering will be magnitudes faster this way than with nested loops.
Supposing that a filter removes the data if it doesn't match, I suggest, that you switch the two loops like so:
foreach(filter) {
foreach(data) {
check data with filter
}
}
By doing so, the second filter doesn't have to work all data, but only the data that passed the first filter, and so on. Of course the tips above (like doing expensive checks last) are still true and should additionally be considered.