We Keep Coding

Why does iterating over a Set's values allocate and create garbage?

Yesterday I asked this question on how to iterate over a Map without allocating. A v8 developer replied with the following:
But if you're only interested in processing the values anyway, you can avoid it being created by iterating over just the values: for (let v of map.values()) {...} does not allocate any short-lived objects. The same is true for iterating over map.keys().
I'm trying to replicate this in a test but I'm having trouble. Here's a demo I made that reproduces the problem I'm having with a simple Set. Paste the following code in an empty HTML file in its script tags:
let a = new Set([ 1, 2, 3, 4, 5 ]);
let b = [ 1, 2, 3, 4, 5 ];
let sum = 0;
function setIterate() {
for (let item of a.values()) {
sum += item;
}
}
function arrayIterate() {
for (let i = 0; i < b.length; i++) {
sum += b[i];
}
}
setInterval(setIterate, 1);
// setInterval(arrayIterate, 1);
If you open this HTML file and then hit Shift+Escape it should open the Chrome Task Manager. If you then look at the Javascript Memory column, you'll see the memory slowly growing over time.
However, if you comment the line setInterval(setIterate, 1); and uncomment the line setInterval(arrayIterate, 1); and refresh the page you'll see that the memory usage stays constant.
That is, iterating over a Set produces garbage build-up over-time while iterating over an array does not.
Is there any way to iterate a Set without producing this garbage build-up over time? I'm developing a browser game and iterating over many Sets and Maps in my render loop and getting occasional frame-spikes due to the GC running.

Why does iterating over a Set's values allocate and create garbage?
It doesn't. Doing a million calls to setIterate with --trace-gc (in d8 or node) shows zero activity. If the function allocated even a single byte per iteration (which it can't; the minimum allocation granularity is two pointers, i.e. 8 bytes), then the young generation would have filled up at least once in that time.
Is there any way to iterate a Set without producing this garbage build-up over time?
Absolutely; for example how you're doing it.
why does the Task Manager show memory increasing over time?
I don't see it doing that. I've copy-pasted your snippet exactly as-is and let it sit for a couple of minutes; the "JavaScript memory" column of Chrome's task manager for that tab hasn't budged by a single kilobyte. (Which actually only proves that this test isn't really a stress test: it will start allocating like crazy once sum exceeds 1<<30, but that's not because of the set or the array...)
Wild guess: maybe you have some extension installed that injects code into every page and causes allocations? At any rate, whatever it is, it's not the for..of-loops over Sets.

In JS which one makes sense: searching for a value in object collection by foreach vs keeping multiple collections with different keys

I'm working on some kind of 1:1 chat system, the environment is Node.JS
For each country, there is a country room (lobby), for each socket client there is a js class/object is being created and each object is in a list with their unique user id.
This unique id is preserved even users logged in from different browser tabs etc..
Each object stored in collections like: "connections" (all of them), "operators"(only operators), "{countryISO}_clients" (users) and the reference key is their unique id.
In some circumstances, I need to access these connections by their socket ids.
At this point, I can think of 2 resolutions.
Using a for each loop to find the desired object
Creating another collection, this time instead of unique id use socket id (or something else.)
Which one makes sense? Because in JS since this collection will be a reference list instead of a copy, it feels like it makes sense (and beautiful looking) but I can't be sure. Which one is expensive in memory/performance terms?
I can't make thorough tests since I don't know how to create dummy (simultaneous) socket connections.
Expected connected socket client count: 300 - 1000 (depends on the time of the day)
e.g. user:
"qk32d2k":{
"uid":"qk32d2k",
"name":"Josh",
"socket":"{socket.io's socket reference}",
"role":"user",
"rooms":["room1"],
"socketids":["sid1"]
"country":"us",
...
info:() => { return gatherSomeData(); },
update:(obj) => { return updateSomeData(obj); },
send:(data)=>{ /*send data to this user*/ }
}
e.g. Countries collection:
{
us:{
"qk32d2k":{"object above."}
"l33t71":{"another user object."}
},
ca:{
"asd231":{"other user object."}
}
}

Pick Simple Design First that Optimizes for Most Common Access
There is no ideal answer here in the absolute. CPUs are wicked fast these days, so if I were you I'd start out with one simple mechanism of storing the sockets that you can access both ways you want, even if one way is kind of a brute force search. Pick the data structure that optimizes the access mechanism that you expect to be either most common or most sensitive to performance.
So, if you are going to be looking up by userID the most, then I'd probably store the sockets in a Map object with the userID as the key. That will give you fast, optimized access to get the socket for a given userID.
For finding a socket by some other property of the socket, you will just iterate the Map item by item until you find the desired match on some other socket property. I'd probably use a for/of loop because it's both fast and easy to bail out of the loop when you've found your match (something you can't do on a Map or Array object with .forEach()). You can obviously make yourself a little utility function or method that will do the brute force lookup for you and that will allow you to modify the implementation later without changing much calling code.
Measure and Add Further Optimization Later (if data shows you need to)
Then, once you get up to scale (or simulated scale in pre-production test), you take a look at the performance of your system. If you have loads of room to spare, you're done - no need to look further. If you have some operations that are slower than desired or higher than desired CPU usage, then you profile your system and find out where the time is going. It's most likely that your performance bottlenecks will be elsewhere in your system and you can then concentrate on those aspects of the system. If, in your profiling, you find that the linear lookup to find the desired socket is causing some of your slow-down, then you can make a second parallel lookup Map with the socketID as the key in order to optimize that type of lookup.
But, I would not recommend doing this until you've actually shown that it is an issue. Premature optimization before you have actual metrics that prove it's worth optimizing something just add complexity to a program without any proof that it is required or even anywhere close to a meaningful bottleneck in your system. Our intuition about what the bottlenecks are is often way, way off. For that reasons, I tend to pick an intelligent first design that is relatively simple to implement, maintain and use and then, only when we have real usage data by which we can measure actual performance metrics would I spend more time optimizing it or tweaking it or making it more complicated in order to make it faster.
Encapsulate the Implementation in Class
If you encapsulate all operations here in a class:
Adding a socket to the data structure.
Removing a socket from the data structure.
Looking up by userID
Looking up by socketID
Any other access to the data structure
Then, all calling code will access this data structure via the class and you can tweak the implementation some time in the future (to optimize based on data) without having to modify any of the calling code. This type of encapsulation can be very useful if you suspect future modifications or change of modifications to the way the data is stored or accessed.
If You're Still Worried, Design a Quick Bench Measurement
I created a quick snippet that tests how long a brute force lookup is in a 1000 element Map object (when you want to find it by something other than what the key is) and compared it to an indexed lookup.
On my computer, a brute force lookup (non-indexed lookup) takes about 0.002549 ms per lookup (that's an average time when doing 1,000,000 lookups. For comparison an indexed lookup on the same Map takes about 0.000017 ms. So you save about 0.002532 ms per lookup. So, this is fractions of a millisecond.
function addCommas(str) {
var parts = (str + "").split("."),
main = parts[0],
len = main.length,
output = "",
i = len - 1;
while(i >= 0) {
output = main.charAt(i) + output;
if ((len - i) % 3 === 0 && i > 0) {
output = "," + output;
}
--i;
}
// put decimal part back
if (parts.length > 1) {
output += "." + parts[1];
}
return output;
}
let m = new Map();
// populate the Map with objects that have a property that
// you have to do a brute force lookup on
function rand(min, max) {
return Math.floor((Math.random() * (max - min)) + min)
}
// keep all randoms here just so we can randomly get one
// to try to find (wouldn't normally do this)
// just for testing purposes
let allRandoms = [];
for (let i = 0; i < 1000; i++) {
let r = rand(1, 1000000);
m.set(i, {id: r});
allRandoms.push(r);
}
// create a set of test lookups
// we do this ahead of time so it's not part of the timed
// section so we're only timing the actual brute force lookup
let numRuns = 1000000;
let lookupTests = [];
for (let i = 0; i < numRuns; i++) {
lookupTests.push(allRandoms[rand(0, allRandoms.length)]);
}
let indexTests = [];
for (let i = 0; i < numRuns; i++) {
indexTests.push(rand(0, allRandoms.length));
}
// function to brute force search the map to find one of the random items
function findObj(targetVal) {
for (let [key, val] of m) {
if (val.id === targetVal) {
return val;
}
}
return null;
}
let startTime = Date.now();
for (let i = 0; i < lookupTests.length; i++) {
// get an id from the allRandoms to search for
let found = findObj(lookupTests[i]);
if (!found) {
console.log("!!didn't find brute force target")
}
}
let delta = Date.now() - startTime;
//console.log(`Total run time for ${addCommas(numRuns)} lookups: ${delta} ms`);
//console.log(`Avg run time per lookup: ${delta/numRuns} ms`);
// Now, see how fast the same number of indexed lookups are
let startTime2 = Date.now();
for (let i = 0; i < indexTests.length; i++) {
let found = m.get(indexTests[i]);
if (!found) {
console.log("!!didn't find indexed target")
}
}
let delta2 = Date.now() - startTime2;
//console.log(`Total run time for ${addCommas(numRuns)} lookups: ${delta2} ms`);
//console.log(`Avg run time per lookup: ${delta2/numRuns} ms`);
let results = `
Total run time for ${addCommas(numRuns)} brute force lookups: ${delta} ms<br>
Avg run time per brute force lookup: ${delta/numRuns} ms<br>
<hr>
Total run time for ${addCommas(numRuns)} indexed lookups: ${delta2} ms<br>
Avg run time per indexed lookup: ${delta2/numRuns} ms<br>
<hr>
Net savings of an indexed lookup is ${(delta - delta2)/numRuns} ms per lookup
`;
document.body.innerHTML = results;

JavaScript: multiple array conditions in if statement

I'm pretty new here but i'm posting this cause i haven't found a single answer on the internet to this question.
How can I use multiple arrays as conditions to an if statement. the reason i would need this is simply for creating a 2D game. But i'm learning that even a simple 2D game has tons of variables because of all the objects involved. But here is a simple example for what I've started with.
var a = 27;
var test = 0;
if(a in {18:1, 27:1, 36:1}) {
test = 1;
}
This tests an array of variables against one variable. I've found that this returns true but this is only half the battle.
The only place I've found any close reference to this is here.
How to shorten my conditional statements
Now the hard part is getting two arrays as conditions instead of just a variable and an array. So basically i need this idea made shorter.
var a = 27;
var b = 27;
var c = 50;
var test = 0;
if(a in {18:1, 27:1, 36:1} || b in {18:1, 27:1, 36:1} || c in {18:1, 27:1, 36:1}) {
test = 1;
}
even though i'm a noob my bible is the hacker's standard:P. Which basically means i think that when i'm creating something with the process of doing something over and over without very good reason "IT IS THE DEVIL"(kudos to whoever got the references). So let me explain this again but very specific so there's no confusion. Say i create a lot of NPC(non player character) and i want a system that can detect if the individual NPC has been in contact by lets say a projectile. i want that individual to vanish and give a point to a score board. well creating functions for such characters requires a LOT of if statements. So ideally i want an if statement that somehow uses 2 or more arrays for it's conditions but look almost as short as using two variables.
maybe something that looks like this.
var test = 0;
var a = [5,6,8];
var b = [10,30,8];
if(a in b){
test = 1;
}
NOTE: I've actually already tried this but it only took the index of b and not the numbers inside. I believe this topic deserves attention unless there's already someone out there that posted a solution(in which case it NEEDS to be advertised).
EDIT: After a long while i've come to realize that the proper(more efficient and readable) solution is to use both OOP and game engine design. I was just too young to understand how to work with data. So to anyone who see's this wondering the same thing should simply try to more thoroughly study array and class logic. In honesty javascript is NOT the place to learn this. I recommending taking a trip to processing.org. and learning the ways of using classes. if Your having trouble there you can try openFrameworks and learn OOP in c++. But the biggest part is understanding proper array mechanics. The OOP just makes it easier.

var test = false;
var a = [5, 6, 8];
var b = { 10:1, 30:1, 8:1 };
for (var i = 0; i < a.length; i++) {
if (a[i] in b) {
test = true;
break;
}
}
If you're using a library like jQuery or Underscore.js, they have convenience functions like $.any() that can be used to replace the loop. You can also use the built-in Array#some method, but it's not compatible with IE8. Ex:
return a.some(function(x) {
return x in b;
});

Methods of writing large amount of data and formulas for that data in JS

I'm working on a mini html5 game and I've stumbled upon an interesting problem, for me at least. And that's how to write the formulas for spells. For example:
SpellOne = baseDmg + level*dmgPerLvl + ratio*someStat;
that would be only one formula, but my point is, obviously if I write it like that it will calculate the damage from the start and SpellOne will become just a number, not a formula anymore. I can only think of two ways to do it, please lend me your help and give me any advice you have on this.
SpellOne.baseDmg = 50;
SpellOne.dmgPerLvl = 30;
SpellOne.baseCd = 15; // cd = spell cooldown
SpellOne.CdPerLvl = 2; // cd decreases by 2 every level
SpellOne.baseCost = 50; // mana or whatever resource
SPellOne.CostPerLvl = 20; // increases by 20 each level
and then once the player uses the spell it calculates the latest update:
SpellOneDamage = baseDmg + level*dmgPerLevel
The other method I have is something that came to me recently. I have around 100 heroes, each with like a dozen stats and then several spells that also have their own stats. But only 1 or 2 heroes will be loaded ever at one time, so am I wasting 'resources' or adding to many useless variables to the dom tree? Since only 1% of them are being used? Which made me think of the following solution:
Make each hero one long function, and when a person changes their hero, it runs the function and for example the 'currentHero' variable takes upon itself ALL the stats from the hero, spell stats too. So when the person changes heroes again, all that's really happening is 'currentHero' variable changes its values and thus you wont have to load ALL the hero stats pointlessly? It only loads a hero when selected. Any suggestions you guys have are greatly appreciated, thank you in advance!

Unless you're recalculating the damage for each spell many times per second, why not just make it a function and add it to the prototype of your character?
Character.prototype.getSpellDamage = function(name) {
var spell = Spells[name];
return spell.baseDamage
+ this.level * spell.damagePerLevel
+ spell.ratio * this[spell.someStat];
}
Now, you can just store the spells in a simple object:
var Spells = {
'foo bar': {
baseDamage: 40,
damagePerLevel: 20,
ratio: 0.5,
someStat: 'foo'
},
...
};

JavaScript anti-flood spam protection?

I was wondering if it were possible to implement some kind of crude JavaScript anti-flood protection.
My code receives events from a server through AJAX, but sometimes these events can be quite frequent (they're not governed by me).
I have attempted to come up with a method of combating this, and I've written a small script: http://jsfiddle.net/Ry5k9/
var puts = {};
function receiverFunction(id, text) {
if ( !puts[id] ) {
puts = {};
puts[id] = {};
}
puts[id].start = puts[id].start || new Date();
var count = puts[id].count = puts[id].count + 1 || 0;
var time = (new Date() - puts[id].start) * 0.001;
$("text").set("text", (count / time.toFixed()).toString() + " lines/second");
doSomethingWithTextIfNotSpam(text);
}
};
which I think could prove effective against these kinds of attacks, but I'm wondering if it can be improved or perhaps rewritten?
So far, I think everything more than 3 or 2.5 lines per second seems like spam, but as time progresses forward (because start mark was set... well... at the start), an offender could simply idle for a while and then commence the flood, effectively never passing 1 line per minute.
Also, I would like to add that I use Mootools and Lo-Dash libraries (maybe they provide some interesting methods), but it would be preferable if this can be done using native JS.
Any insight is greatly appreciated!

If you are concerned about the frequency a particular javascript function fires, you could debounce the function.
In your example, I guess it would be something like:
onSuccess: function(){ _.debounce(someOtherFunction, timeOut)};
where timeout is the maximum frequency you want someOtherFunction to be called.

I know you asked about native JavaScript, but maybe take a look at RxJS.
RxJS or Reactive Extensions for JavaScript is a library for
transforming, composing, and querying streams of data. We mean all
kinds of data too, from simple arrays of values, to series of events
(unfortunate or otherwise), to complex flows of data.
There is an example on that page which uses the throttle method to "Ignores values from an observable sequence which are followed by another value before dueTime" (see source).
keyup = Rx.Observable.fromEvent(input, 'keyup').select(function(ev) {
return ev.target.value;
}).where(function(text) {
return text.length > 2;
}).throttle(500)
.distinctUntilChanged()
There might be a similar way to get your 2.5-3 per second and ignore the rest of the events until the next second.

I've spent many days pondering on effective measures to forbid message-flooding, until I came across the solution implemented somewhere else.
First, we need three things, penalty and score variables, and a point in time where last action occured:
var score = 0;
var penalty = 200; // Penalty can be fine-tuned.
var lastact = new Date();
Next, we decrease score by the distance between the previous message and current in time.
/* The smaller the distance, more time has to pass in order
* to negate the score penalty cause{d,s}.
*/
score -= (new Date() - lastact) * 0.05;
// Score shouldn't be less than zero.
score = (score < 0) ? 0 : score;
Then we add the message penalty and check if it crosses the threshold:
if ( (score += penalty) > 1000 ) {
// Do things.
}
Shouldn't forget to update last action afterwards:
lastact = new Date();