We Keep Coding

How to count a huge list of items

I have a huge list of items about almost all the crops and these data is to be plotted using maps and charts. I would like to count the number of each crop, say how many times was cabbage planted. I use Firebase database to store the data and I retrieve it using this function below:
database = firebase.database()
var ref = database.ref('Planting-Calendar-Entries');
ref.on('value', gotData, errData);
function gotData(data){
console.log(data.val())
var veggie = data.val();
var keys = Object.keys(veggie);
console.log(keys);
let counter = 0
for (var i = 0; i < keys.length; i++){
var k = keys[i];
var Veg_planted = veggie[k].Veg_planted;
var coordinates = veggie[k].coordinates;
if (Veg_planted == 'Cabbage'){
counter++;
}
// vegAll = Veg_planted.count()
console.log(Veg_planted, coordinates)
}
console.log(counter)
}
function errData(err){
console.log('Error!');
console.log(err)
}
This data I retrieve it from the database where it gets updated whenever someone submits their planting information. The code I used above will only apply if my list is small, but I have a list of about 170 items and it would be hard to write code to count each crop individually using something like let counter = 0, counter++. Is there a way I could navigate around this?

I'm assuming data.val() returns an array, not an object, and you're misusing Object.keys() on an array instead of just looping over the array itself. If that's true, then it sounds like you want to group by the Veg_planted key and count the groupings:
const counts = Object.values(veggie).reduce((counts, { Veg_planted }) => ({
...counts,
[Veg_planted]: (counts[Veg_planted] || 0) + 1
}), {});
Usage:
const veggie = [{ Veg_planted: 'Cabbage' }, { Veg_planted: 'Cabbage' }, { Veg_planted: 'Corn' }];
// result of counts:
// {Cabbage: 2, Corn: 1}

Actually: the code to count the items is probably going to be the same, no matter how many items there are. The thing that is going to be a problem as you scale though is the amount of data that you have to retrieve that you're not displaying to the user.
Firebase does not support aggregation queries, and your approach only works for short lists of items. For a more scalable solution, you should store the actual count itself in the database too.
So:
Have a blaCount property for each bla that exists.
Increment/decrement the counter each time your write/remove a bla to/from the database.
Now you can read only the counters, instead of having to read the individual items.

Firestore would be better option. You can query based on the field value.
var plantingRef = db.collection("PlantingCalendarEntries");
var query = plantingRef.where("Veg_planted", "==", "Cabbage");
if you still want to stuck with realtime database.
Save Counters to database.
Or use cloud dunctions to count.

Paging objects within an object

I have a long list of chat rooms
let chatRooms = {
"general": ChatRoom,
"myRoomA": ChatRoom,
"bobsRoom": ChatRoom,
...
}
ChatRoom has a serialize method
ChatRoom.serialize = function(){
return {
name: this.name,
clients: this.clients,
...
}
}
In order to list all ChatRooms to a user, I must send this data to them
ChatRoomManager.serialize = function(){
let serializedObjects = [];
Util.each(this.chatRooms, function(i, e){
if(e.serialize){
serializedObjects.push(e.serialize());
}
});
return serializedObjects;
}
This becomes a performance issue as people regularly request to list all chat rooms and it gets serialized so often so I want to do paging. But if an object has no guaranteed order, how can I possibly say "here are the next 10 chat rooms"? Even if I could guarantee order, how could I start at index 11 without looping through all of the objects? Imagine if I was at index 1000, etc..
TLDR: is it possible to do paging with an object of objects efficiently and accurately.

You coulf just take the values of the objects which returns an array, so the order is guaranteed:
const ordered = Object.values(chatRooms);
You could now also apply a custom sort order, e.g.:
ordered.sort((roomA, roomB) => roomA.name.localeCompare(roomB.name));
To now serialize only one chunk it is as easy as:
let index = 0, chunk = 100;
const result = ordered.slice(index * chunk, (index + 1) * chunk).map(room => room.serialize());

Separate strings in an array in a certain order

I have a list that I took from a converted CHANGELOG.md file, and it looks like this:
["[3.0.0]","Features", "changes done in file","[2.0.1]", "Bug Fixes", "fixed login"]
What I want to do is to separate each version into its own list, like this:
["[3.0.0]", "Features", "changes done in file"],
["[2.0.1]", "Bug Fixes", "fixed login"]
Obviously, because it's a changelog, there can be multiple features and multiple bugfixes in a single version, so I want to a piece of code that separates the code appropriately.
I tried using if (string.startsWith('[')) but i couldn't manage to fit it in a loop.
Any help is appreciated.

Here's something I came up with. The code basically loops through the input array and adds each string to a currentArray variable. Everytime it hits a [ it puts the currentArray into the output and clears currentArray. At the end it removes the first element as the first element of the output will always be an empty array (since the first element of the input starts with a [)
var input = ["[3.0.0]","Features", "changes done in file","[2.0.1]", "Bug Fixes", "fixed login"];
var output = [];
var currentArray = [];
for (var i = 0; i < input.length; i++) {
if (input[i].charAt(0) == '[') {
output.push(currentArray);
currentArray = [];
}
currentArray.push(input[i]);
}
output.push(currentArray);
currentArray = [];
//Since it will take the first one, and put empty one, need to do last step.
output.splice(0, 1);
console.log(output);
// ["[3.0.0]", "Features", "changes done in file"],
// ["[2.0.1]", "Bug Fixes", "fixed login"]

Assuming that you're always working in sets of three, this is a quick and ugly approach
var data = ["[3.0.0]","Features", "changes done in file","[2.0.1]", "Bug Fixes", "fixed login"],
items = [];
data.map( (el, idx) => {
var last = items.length;
if( idx % 3 === 0 ) {
items.push( [] );
last += 1;
}
last = items[ last - 1 ];
last.push( el );
} );
console.log( JSON.stringify( items ) );

Here's an alternative solution should you prefer it:
const arr = ["[3.0.0]","Features", "changes done in file","[2.0.1]", "Bug Fixes", "fixed login"];
const newArr = [];
let tempArr = [];
arr.forEach(function(v, i) {
if(/^\[\d+.\d+.\d\]$/.test(v) && i > 0) {
newArr.push(tempArr);
tempArr = [v];
} else {
tempArr.push(v)
}
});
newArr.push(tempArr);
console.log(newArr);
This snippet loops through the items one-by-one. It uses two arrays, one to hold the final result and one to populate with items for the current version.
I am using a regex to check if the item contains one [ followed by a number, then a period, number, period, number and finally the trailing ]. This allows the other strings that are not version tags to contain that character.
If the current item is a version tag, we push tempArr (which contains the changes of the current version that we've previously filled in our loop) to our result array newArr. Then, we empty the tempArr and give it the starting value of the next version tag.
If it is not, we just push the current item to our temporary array.

It would be interesting to know if you were guaranteed to get this data in triplets, as your example seems to imply. If you knew this up front, there are many creative solutions that could emerge. For just creating a 2D Array, however, I like this approach (you can run this directly in node.js to try it out):
const original = ['[3.0.0]', 'Features', 'changes done in file', '[2.0.1]', 'Bug Fixes', 'fixed login']
function transformToChangeLog (originalArray) {
const changeLog = originalArray.reduce((newList, element) => {
element.charAt(0) === '[' // check for version string
? newList.push([element]) // If version string, then push a new Array containing that string
: newList[newList.length - 1].push(element) // If something else, tack it onto the last Array in the changelog list
return newList // whatever is returned in the reduce function is passed to the next iteration, allowing us to build this 2D array one element at a time.
}, [])
return changeLog
}
console.log(transformToChangeLog(original))
I hope that helps! I like the reduce Array method, because of it's versatility and succinctness.

Cycling through an array modularly in Javascript

I've got an ordered array of objects, myBros. I want each one to get its own index in the array (which I'm storing as myPlace) and store the ID of the following one, with the last object storing the ID of the first.
The code below results in each object storing the ID of the last object, not the next one. On a positive note, the last object stores the ID of the first one.
EDIT: Sorry, I should have been more specific-this function lives in an object (a React component) which has some props, one of which is an id. getNextBroId returns a value based on the component's index.
What am I doing wrong?
EDIT: reworked the code in accordance with Daniel Beck's suggestion, still having the same problem.
for (let i = 0; i < myBros.length - 1; i++) {
myBros[i].nextBroId = myBros[i + 1]._id;
}
myBros[myBros.length - 1].nextBroId = myBros[0]._id;
const myPlace = myBros.findIndex(p => p._id === id);
const getNextBroId = () => {
return myBros[myPlace].nextBroId;
};
EDIT: I've posted the entire component here: React-cycling through components in an array and am considering closing this question to avoid redundancy.

This is one of those cases where boring old iteration is going to be a lot easier (and more performant) than the more exciting new techniques.
var myBros = [
{_id: "a"},
{_id: "b"},
{_id: "c"},
{_id: "d"},
{_id: "e"}
]
// step through all but last element in the array, link each one to the next
for (var i = 0; i < myBros.length - 1; i++) {
myBros[i].nextBroId = myBros[i + 1]._id;
}
// link the last one back to the first
myBros[myBros.length - 1].nextBroId = myBros[0]._id;
// and we're done
console.log(myBros);
(I'm not entirely sure why you would want to turn an array into a pseudo linked list, but I assume you have your reasons...)

Sorting strings in reverse order with backbone.js

I'm trying to sort a Backbone.js collection in reverse order. There are previous replies on how to do this with integers, but none with strings.
var Chapter = Backbone.Model;
var chapters = new Backbone.Collection;
chapters.comparator = function(chapter) {
return chapter.get("title");
};
chapters.add(new Chapter({page: 9, title: "The End"}));
chapters.add(new Chapter({page: 5, title: "The Middle"}));
chapters.add(new Chapter({page: 1, title: "The Beginning"}));
alert(chapters.pluck('title'));
The above code sorts the chapters from A -> Z, but how do I write a comparator that sorts it from Z -> A?

You could:
grab the char code for each character in the string,
subtract each value from 0xffff (the maximum return value of string.charCodeAt),
use String.fromCharCode to turn that back into string of "negated" characters
and that will be your sorting key.
chapters.comparator = function(chapter) {
return String.fromCharCode.apply(String,
_.map(chapter.get("title").split(""), function (c) {
return 0xffff - c.charCodeAt();
})
);
}
And voila:
> console.log(chapters.pluck('title'));
["The Middle", "The End", "The Beginning"]
Note: if your comparison strings are long (as in 65 kb or more), you may run into trouble (see Matt's comment below). To avoid this, and speed up comparisons a bit, just use a shorter slice of your comparison string. (In the above example, you could go for chapter.get("title").slice(0, 100).split("") instead.) How long a slice you need will depend on your application.

There are two versions of the comparator function that you can use, either the sortBy version - which was shown in the example, which takes one parameter, or sort - which you can return a more standard sort function, which the documentation says:
"sortBy" comparator functions take a model and return a numeric or string value by which the model should be ordered relative to others. "sort" comparator functions take two models, and return -1 if the first model should come before the second, 0 if they are of the same rank and 1 if the first model should come after.
So in this case, we can write a different comparator function:
var Chapter = Backbone.Model;
var chapters = new Backbone.Collection;
chapters.comparator = function(chapterA, chapterB) {
if (chapterA.get('title') > chapterB.get('title')) return -1; // before
if (chapterB.get('title') > chapterA.get('title')) return 1; // after
return 0; // equal
};
chapters.add(new Chapter({page: 9, title: "The End"}));
chapters.add(new Chapter({page: 5, title: "The Middle"}));
chapters.add(new Chapter({page: 1, title: "The Beginning"}));
alert(chapters.pluck('title'));
So you should get as a response:
"The Middle", "The End", "The Beginning"

If you're working with non-numerical values, there is no obvious way to do a reverse sort. Backbone makes use of the _.sortBy() and _.sortedIndex() methods from Underscore to order the models based on the comparator, and these methods automatically sort in ascending order. The naive way to do this would be to use chapters.pluck('title').reverse(), as the result of pluck will be an array. But calling reverse on some Collection methods will reverse the Collection models in place, so next time you call it, the models will be back in ascending order. You could always do something like:
var results = [],
titles = chapters.pluck('title');
for(var i=0, len=titles.length; i<len; i++) {
results.push(titles[i]);
}
results.reverse();
This would not affect the models array in your Backbone collection, as it would create a completely new results array in memory, but retain references to the original models, so calling things like save would still update the Collection state.
But that's not very elegant, and creates a lot of extra coding throughout your project any time you want to reverse the results. I think we can do better.
In order to make this work, you'll need to perform a bit of unwieldy JavaScript ninjary in your comparator method to make this work - note this is untested:
chapters.comparator = function(chapter) {
var alphabet = '0123456789abcdefghijklmnopqrstuvwxyz',
title = chapter.get('title').toLowerCase(),
inverse_title = '',
index;
for(var i=0, len=title.length; i<len; i++) {
index = alphabet.indexOf(title.charAt(i));
if(index === -1) {
inverse_title += title.charAt(i);
continue;
}
inverse_title += alphabet.charAt(alphabet.length - index - 1);
}
return inverse_title;
};
This concept probably needs improving to take into account symbols, etc., but essentially it inverts the comparator string in such a way that "Z" becomes "0", "Y" becomes "1", etc., which should produce the reverse sort you're after.

As Backbone merely uses the .sortBy method, simply proxy in your own logic:
collectionInQuestion.sortBy = function () {
var models = _.sortBy(this.models, this.comparator);
if (forSomeReason) {
models.reverse();
}
return models;
};
..or add it somewhere else..
TweakedCollection = Backbone.Collection.extend({ sortBy: [...] })

I just solved a similar problem with table sorting and I wanted to share the code since I didn't find much help in these answers:
events: {
'click th.sortable': function(e) {
var $this = $(e.target),
order = $this.hasClass('asc') ? 'desc' : 'asc',
field = $this.data('field'); /* this is a string */
$this.siblings().addBack().removeClass('asc desc');
$this.addClass( order );
this.bodyView.collection.comparator = field;
this.bodyView.collection.sort();
if ( order === 'desc' ) this.bodyView.collection.models.reverse();
this.bodyView.render();
}
},
in this case I simply set comparator to string instead of a function; the string has to be the name of the property you want to sort by.
Then I just call reverse on the models if the order has to be inverse.

Just add minus before chapter.get
chapters.comparator = function(chapter) {
    return -chapter.get("title");
};