Prepending that a solution only needs to work in the latest versions of Chrome, Firefox, and Safari as a bonus.
-
I am trying to use an associative array for a large data set with knockout. My first try made it a true associative array:
[1: {Object}, 3: {Object},...,n:{Object}]
but knockout was not happy with looping over that. So I tried a cheating way, hoping that:
[undefined, {Object}, undefined, {Object},...,{Object}]
where the location in the array is the PK ID from the database table. This array is about 3.2k items large, and would be iterated over around every 10 seconds, hence the need for speed. I tried doing this with a splice, e.g.
$.each(data, function (index, item) {
self.myArray.splice(item.PKID, 0, new Object(item));
}
but splice does not create indices, so since my first PKID is 1, it is still inserted at myArray[0] regardless. If my first PK was 500, it would start at 0 still.
My second thought is to initialize the array with var myArray = new Array(maxSize) but that seems heavy handed. I would love to be able to use some sort of map function to do this, but I'm not really sure how to make the key value translate into an index value in javascript.
My third thought was to keep two arrays, one for easy look up and the other to store the actual values. So it combines the first two solutions, almost, by finding the index of the object in the first example and doing a lookup with that in the second example. This seems to be how many people manage associative arrays in knockout, but with the array size and the fact that it's a live updating app with a growing data set seems memory intensive and not easily manageable when new information is added.
Also, maybe I'm hitting the mark wrong here? We're putting these into the DOM via knockout and managing with a library called isotope, and as I mentioned it updates about every 10 seconds. That's why I need the fast look up but knockout doesn't want to play with my hash table attempts.
--
clarity edits:
so on initial load the whole array is loaded up (which is where the new Array(maxLength) would go, then every 10 seconds anything that has changed is loaded back. That is the information I'm trying to quickly update.
--
knockout code:
<!-- ko foreach: {data: myArray(), afterRender: setInitialTileColor } -->
<div class="tile" data-bind="attr: {id: 'tileID' + $data.PKID()}">
<div class="content">
</div>
</div>
<!-- /ko -->
Then on updates the hope is:
$.each(data.Updated, function (index, item) {
var obj = myModel.myArray()[item.PKID];
//do updates here - need to check what kind of change, how long it's been since a change, etc
}
Here is a solution how to populate array items with correct indexes, so it doesn't start from the first one (0 (zero) I meant)
just use in loop
arr[obj.PKID] = obj;
and if your framework is smart (to use forEach but not for) it will start from your index (like 500 in case below)
http://jsfiddle.net/0axo9Lgp/
var data = [], new_data = [];
// Generate sample array of objects with index field
for (var i = 500; i < 3700; i++) {
data.push({
PKID: i,
value: '1'
});
}
data.forEach(function(item) {
new_data[item.PKID] = item;
});
console.log(new_data);
console.log(new_data.length); // 3700 but real length is 3200 other items are undefined
It's not an easy problem to solve. I'm assuming you've tried (or can't try) the obvious stuff like reducing the number of items per page and possibly using a different framework like React or Mithril.
There are a couple of basic optimizations I can suggest.
Don't use the framework's each. It's either slower than or same as the native Array method forEach, either way it's slower than a basic for loop.
Don't loop over the array over and over again looking for every item whose data has been updated. When you send your response of data updates, send along an array of the PKIds of the updated item. Then, do a single loop:
.
var indexes = []
var updated = JSON.parse(response).updated; // example array of updated pkids.
for(var i=0;i<allElements.length;i++){
if(updated.indexOf(allElements[i].pkid)>-1)
indexes.push(i);
}
So, basically the above assumes you have a simple array of objects, where each object has a property called pkid that stores its ID. When you get a response, you loop over this array once, storing the indexes of all items that match a pk-id in the array of updated pk-ids.
Then you only have to loop over the indexes array and use its elements as indexes on the allElements array to apply the direct updates.
If your indexes are integers in a reasonable range, you can just use an array. It does not have to be completely populated, you can use the if binding to filter out unused entries.
Applying updates is just a matter of indexing the array.
http://jsfiddle.net/0axo9Lgp/2/
You may want to consider using the publish-subscribe pattern. Have each item subscribe to its unique ID. When an item needs updating it will get the event and update itself. This library may be helpful for this. It doesn't depend upon browser events, just arrays so it should be fairly fast.
Related
So this follows on from my previous question:
knockout js, add additional elements to an array
Basically I have an app where a user fills in certain data, clicks next and this is added to an array. However, what I'd like to do is add some items into the array before the user even begins using the app (these items I get from a database). The idea being that at the start they can view each item in the array and then choose and an item and edit this item. I've got a feeling I'm missing something blindingly obvious but I cannot seem to figure it out
Knockout observable arrays have equivalent functions to native JavaScript arrays. See: http://knockoutjs.com/documentation/observableArrays.html
So you need just to use arr.pop(item) or arr.push(item).
In case you need to replace all items and want to avoid multiple events to raise, use observableArray.valueWillMutate() and valueHasMutated() functions. See sample where I do swap the entire array:
ko.observableArray.fn.replaceWith = function (valuesToPush) {
// NOTE: base on - ko.observableArray.fn.pushAll
var underlyingArray = this();
var oldItemcount = underlyingArray.length;
this.valueWillMutate();
// adding new items to obs. array
ko.utils.arrayPushAll(underlyingArray, valuesToPush);
// removing old items (using KO observablearray fnc.)
if (oldItemcount > 0)
this.removeAll(underlyingArray.slice(0, oldItemcount));
this.valueHasMutated();
return this; //optional
};
I need help with a loop... it's probably simple but I'm having difficulty coding it up.
Basically, I need to check existing Ids for their number so I can create a unique id with a different number. They're named like this: id="poly'+i'" in sequence with my function where i is equal to the number of existing elements. Example: Array 1, Array 2, Array 3 corresponding with i=1 for the creation of Array 1, i=2 for Array 2, etc.
Right now i is based on the total number of existing elements, and my "CreateNew" function is driven off x=i+1 (so the example above, the new element will be named Array 4). The problem is that if you delete one of the middle numbers, the "Create" function will duplicate the high number. i.e. Array 1, 2, 3 delete 2, create new-> Array 1, 3, 3.
I need an if() statement to check if the array already exists then a for() loop to cycle through all i's until it validates. Not sure how to code this up.
The code I'm trying to correct is below (note I did not write this originally, I'm simply trying to correct it with my minimal JS skills):
function NewPanel() {
var i = numberOfPanels.toString();
var x = (parseInt(i)+1).toString();
$('#items').append('<div onclick="polygonNameSelected(event)" class="polygonName" id="poly'+i+'"> Array '+ x +' </div>');
$('div[id*=poly]').removeClass('selected');
$('#poly'+i).addClass('selected');
$('#poly'+i).click(function() {
selectedPolygon = i;
$('div[id*=poly]').removeClass('selected');
$(this).addClass('selected');
});
}
THANK YOU! :)
Please clarify "The problem is that if you delete one of the middle numbers, ". What do you mean by delete? Anyway, the simplest solution is to create two arrays. Both arrays will have the same created id's. Whenever an id is created in the first array, an id will be added to the second array. So when it is deleted from first array, check your second array's highest value and then create this id in first array. I hope this did not confuse you.
Well it is hard to tell why you cannot just splice the array down. It seems to me there is a lot of extra logic involved in the tracking of element numbers. In other words, aside from the index being the same, the ids become the same as well as other attributes due to the overlapping 1, 3, 3 (from the example). If this is not the case then my assumption is incorrect.
Based on that assumption, when I encounter a situation where I want to ensure that the index created will always be an appending one, I usually take the same approach as I would with a database primary key. I set up a field:
var primaryKeyAutoInc = 0;
And every time I "create" or add an element to the data store (in this case an array) I copy the current value of the key as it's index and then increment the primaryKeyAutoInc value. This allows for the guaranteed unique indexing which I am assuming you are going for. Moreover, not only will deletes not affect future data creation, the saved key index can be used as an accessor.
All the knockout examples I have found seem to add a new item to the end of an ObservableArray using something like:
viewModel.SomeItems.push(someNewItem);
This of course places the item at the end of the array.
How to I add an item to the ObservableArray at a certain position?
eg. something like:
viewModel.SomeItems.push(someNewItem, indexToInsertItAt);
You should be able to use the native JavaScript splice method -
viewModel.SomeItems.splice(2,0,someNewItem);
Docs here - https://developer.mozilla.org/en/JavaScript/Reference/Global_Objects/Array/splice
Example here (not Knockout specific) - How to insert an item into an array at a specific index?
From the Knockout docs -
For functions that modify the contents of the array, such as push and
splice, KO’s methods automatically trigger the dependency tracking
mechanism so that all registered listeners are notified of the change,
and your UI is automatically updated.
For knockout use
viewModel.SomeItems.unshift(someNewItem);
See also: http://knockoutjs.com/documentation/observableArrays.html
I made this extension function which worked nicely for me. Splice wasn't working for me if I was adding to the end of a sparse array.
ko.observableArray.fn.setAt = function(index, value) {
this.valueWillMutate();
this()[index] = value;
this.valueHasMutated();
}
This even works with:
var a = ko.observableArray(['a', 'b', 'c']);
a.setAt(42, 'the answer');
I am coding a lot of annual data in JavaScript, and I was considering adding it to arrays, using the year as the array index and putting the data into the array. However, Firebug seems to be indicating that JavaScript handles this by populating two thousand odd entries in the array with "undefined." With hundreds of such arrays kicking around in active memory, I'm worried the overhead of hundreds of thousands of useless array items could start to slow the program down. Will it?
When you set the value of a numeric index higher than the current length of your array, the length property is affected.
In brief, you should use an Object:
var data = {};
data[year] = "some data";
// or
var data = {
2009: "2009 data",
2010: "2010 data"
};
Now I answer the question title: "Does JavaScript populate empty array items?"
No, as I said before, only the length property is changed, (if necessary, only if the index added is larger than the current length), length is incremented to be one more than the numeric value of that index.
The Array.prototype methods work assuming that the array object will have its indexes starting from zero.
The previous indexes don't really exist in the Array object, you can test it:
var array = [];
array[10] = undefined;
array.hasOwnProperty(10); // true
array.hasOwnProperty(9); // false
In conclusion, arrays are meant to contain sequential indexes, starting from zero, if your properties don't meet those requirements, you should simply use an object.
Yes, most likely. You should consider using a JavaScript object instead:
var years = {2009: 'Good', 2010: 'Better'};
Well, if you iterate over many thousands of undefined, it will affect overall program speed, not sure if you'll notice it though.
On the other hand, sometimes a sparse array is simpler to use than a custom object,
and arrays have such handy methods available.
In a calendar application I begin with objects for each year in use, but each year consists of a twelve member (months array) and each 'month' is a sparse array of significant dates, whose lengths depend on the highest date of that month that has any data.
Can you suggest me an algorithm for filtering out data.
I am using javascript and trying to write out a filter function which filters an array of data.I have an array of data and an array of filters, so in order to apply each filter on every data, I have written 2 for loops
foreach(data)
{
foreach(filter)
{
check data with filter
}
}
this is not the proper code, but in short that what my function does, the problem is this takes a huge amount of time, can someone suggest a better method.
I am using the Mootools library and the array of data is JSON array
Details of data and Filter
Data is JSON array of lets say user, so it will be
data = [{"name" : "first", "email" : "first#first", "age" : "20"}.
{"name" : "second", "email" : "second#second", "age" : "21"}
{"name" : "third", "email" : "third#third", "age" : "22"}]
Array of filters is basically self define class for different fields of data
alFilter[0] = filterName;
alFilter[1] = filterEmail;
alFilter[2] = filterAge;
So when I enter the first for loop, I get a single JSON opbject (first row) in the above case.
When I enter the second for loop (filters loop) I have a filter class which extracts the exact field on which the current filter would work and check the filter with the appropriate field of the data.
So in my example
foreach(data)
{
foreach(filter)
{
//loop one - filter name
// loop two - filter email
// loop three - filter age
}
}
when the second loop ends i set a flag denoting if the data has been filtered or not and depending on it the data is displayed.
You're going to have to give us some more detail about the exact structure of your data and filters to really be able to help you out. Are the filters being used to select a subset of data, or to modify the data? What are the filters doing?
That said, there are a few general suggestions:
Do less work. Is there some way you can limit the amount of data you're working on? Some pre-filter that can run quickly and cut it down before you do your main loop?
Break out of the inner loop as soon as possible. If one of the filters rejects a datum, then break out of the inner loop and move on to the next datum. If this is possible, then you should also try to make the most selective filters come first. (This is assuming that your filters are being used to reject items out of the list, rather than modify them)
Check for redundancy in the computation the filters perform. If each of them performs some complicated calculations that share some subroutines, then perhaps memoization or dynamic programming may be used to avoid redundant computation.
Really, it all boils down to the first point, do less work, at all three levels of your code. Can you do less work by limiting the items in the outer loop? Do less work by stopping after a particular filter and doing the most selective filters first? Do less work by not doing any redundant computation inside of each filter?
That's pretty much how you should do it. The trick is to optimize that "check data with filter"-part. You need to traverse all your data and check against all your filters - you'll not going to get any faster than that.
Avoid string comparisons, use data models as native as possible, try to reduce the data set on each pass with filter, etc.
Without further knowledge, it's hard to optimize this for you.
You should sort the application of your filters, so that two things are optimized: expensive checks should come last, and checks that eliminate a lot of data should come first. Then, you should make sure that checking is cut short as soon as an "out" result occurs.
If your filters are looking for specific values, a range, or start of a text then jOrder (http://github.com/danstocker/jorder) will fit your problem.
All you need to do is create a jOrder table like this:
var table = jOrder(data)
.index('name', ['name'], { grouped: true, ordered: true })
.index('email', ['email'])
.index('age', ['age'], { grouped: true, ordered: true, type: jOrder.number });
And then call table.where() to filter the table.
When you're looking for exact matches:
filtered = table.where([{name: 'first'}, {name: 'second'}]);
When you're looking for a certain range of one field:
filtered = table.where([{age: {lower: 20, upper: 21}}], {mode: jOrder.range});
Or, when you're looking for values starting with a given string:
filtered = table.where([{name: 'fir'}], {mode: jOrder.startof});
Filtering will be magnitudes faster this way than with nested loops.
Supposing that a filter removes the data if it doesn't match, I suggest, that you switch the two loops like so:
foreach(filter) {
foreach(data) {
check data with filter
}
}
By doing so, the second filter doesn't have to work all data, but only the data that passed the first filter, and so on. Of course the tips above (like doing expensive checks last) are still true and should additionally be considered.