Sometimes i amaze myself by writting a piece of code which can be worst performant like the one below i guess. Code simply reads all files (files are text only) in directory and prints each line of each file on console after interval of one 1/10 thsecond. Now with below approach , if there are 1 million lines in the files , there will be 1 million setTimeout functions defined. and every 1/10th second one setTimeout will call its respective function. I was just curious how below code will impact the performance ? Is it ok to have millions of callbacks defined in nodejs env ? what are your thoughts about below piece of code.
function scanDir(dir){
fs.readdir(dir , function(err , list){
var interval = 0;
list.forEach(function(file, index){
lineReader.eachLine(dir + "/" + file, function(line, last) {
interval += 100;
(function(line){
setTimeout(function(){
console.log(line+"\n\r");
},interval)
})(line);
if (last) {
return false; // stop reading
}
});
})
})
}
You can't have millions of callbacks defined, you'll run out of space on the call stack after about 12,000 or so. This is the error that you'll get:
RangeError: Maximum call stack size exceeded
Now, you can increase the call stack size if you really need to, by using the following command line argument, but I don't think it would ever support millions of callbacks.
node --max-stack-size=val
Without really understanding what you are trying to accomplish, it's hard to suggest additional improvements in your code. If you really have a million lines, printing out the lines is going to take more than a day at 1 every 1/10 sec, why would you want to do this?
Related
I am writing a Whack-A-Mole game for class using HTML5, CSS3 and JavaScript. I have run into a very interesting bug where, at seemingly random intervals, my moles with stop changing their "onBoard" variables and, as a result, will stop being assigned to the board. Something similar has also happened with the holes, but not as often in my testing. All of this is completely independent of user interaction.
You guys and gals are my absolute last hope before I scrap the project and start completely from scratch. This has frustrated me to no end. Here is the Codepen and my github if you prefer to have the images.
Since Codepen links apparently require accompanying code, here is the function where I believe the problem is occuring.
// Run the game
function run() {
var interval = (Math.floor(Math.random() * 7) * 1000);
if(firstRound) {
renderHole(mole(), hole(), lifeSpan());
firstRound = false;
}
setTimeout(function() {
renderHole(mole(), hole(), lifeSpan());
run();
}, interval);
}
What I believe is happening is this. The function runs at random intervals, between 0-6 seconds. If the function runs too quickly, the data that is passed to my renderHole() function gets overwritten with the new data, thus causing the previous hole and mole to never be taken off the board (variable wise at least).
EDIT: It turns out that my issue came from my not having returns on my recursive function calls. Having come from a different language, I was not aware that, in JavaScript, functions return "undefined" if nothing else is indicated. I am, however, marking GameAlchemist's answer as the correct one due to the fact that my original code was convoluted and confusing, as well as redundant in places. Thank you all for your help!
You have done here and there in your code some design mistakes that, one after another, makes the code hard to read and follow, and quite impossible to debug.
the mole() function might return a mole... or not... or create a timeout to call itself later.. what will be done with the result when mole calls itself again ? nothing, so it will just be marked as onBoard never to be seen again.
--->>> Have a clear definition and a single responsibility for mole(): for instance 'returns an available non-displayed mole character or null'. And that's all, no count, no marking of the objects, just KISS (Keep It Simple S...) : it should always return a value and never trigger a timeout.
Quite the same goes for hole() : return a free hole or null, no marking, no timeout set.
render should be simplified : get a mole, get a hole, if either couldn't be found bye bye. If a mole+hole was found, just setup the new mole/hole couple + event handler (in a separate function). Your main run function will ensure to try again and again to spawn moles.
I have a piece of text that uses Webkit's background-clip:text feature and for pure aesthetics purposes, I want to include a script to scroll through the image on the text. The one way I have thought of doing this is using setInterval and running the function I have included below. I want to know if I will run into any problems of using setInterval since I have it looping at 50ms.
Function:
function movePos(){
var obj = document.querySelector('h1 span');
var pos = parseInt(obj.style.backgroundPositionY);
obj.style.backgroundPositionY = (pos + 1) + 'px';
}
Any modern browser with even modest hardware should be able to handle executing these three lines of code 20 times per second. You should be fine.
If you want to optimize, and the obj variable won't change, you could cache that value so you're not running a selector 20 times per second. I'm not certain, but this is probably the most CPU intensive of the three lines of code.
Disclaimer - I've tried finding an answer to this via google/stackoverflow, but I don't know how to define the problem (I don't know the proper term)
I have many small AI snippets such as what follows. There is an ._ai snippet (like below) per enemy type, with one function next() which is called by the finite state machine in the main game loop (fyi: the next function doesn't get called every update iteration, only when the enemy is shifted from the queue).
The question: How do I test every case (taking into account some enemy AI snippets might be more complex, having cases that may occur 1 in 1000 turns) and ensure the code is valid?
In the example below, if I added the line blabla/1 under count++, the error might not crop for a long time, as the Javascript interpreter won't catch the error until it hits that particular path. In compiled languages, adding garbage such as blabla/1 would be caught at compile time.
// AI Snippet
this._ai = (function(commands){
var count = 0;
return {
next: function(onDone, goodies, baddies) {
// If the internal counter reaches
// 2, launch a super attack and
// reset the count
if(count >= 2) {
commands.super(onDone);
count = 0;
}
else {
// If not performing the super attack
// there is a 50% chance of calling
// the `attack` command
if(chance(50)) {
var target = goodies[0];
commands.attack(onDone, target);
}
// Or a 50% chance of calling the
// `charge` command
else {
commands.charge(onDone);
count++;
}
}
}
};
})(this._commands);
I could rig the random generator to return a table of values from 0-n and run next 1000's of times against each number. I just don't feel like that is will concretely tell me every path is error free.
As you say, unit tests must test every path so you will be sure all works well.
But you should be able to decide which path the method will follow before calling it on your tests, so you're be able to know if the method behaviour is the expected one, and if there is any error.
So, for example, if there is a path that will be followed in only one of every 1000 executions, you shouldn't need to test all 0, 1, 2 ... 999 cases. You only one combination of results that behave distinctly.
For example, in the snippet shown you have these cases:
the counter has reached 2
the counter has not reached 2 and chance returns true
the counter has not reached 2 and chance returns false
One way to archieve this is taking control of the counter and of the chance method by mocking them.
If you want to know what happens when the counter has reached 2 and the next method is called, just pass a counter with 2 and call next. You don't need to reach 2 on the counter by really passing for all the code.
As for the randomizer, you don't need to try until the randomizer returns the value you want to test. Make it a mock and configure it to behave as you need for each case.
I hope this helps.
I'm doing a lot of recursion in Javascript, and to keep the stack from overflowing, I've been using setTimeout. Here's a quick theoretical example:
go(){
setTimeout(function(){
x++;
go();
},1);
}
I've also got a function logging x to the console every few seconds, but that isn't the concern. What I'm seeing is that no matter what value I put in for the timeout, for which I've used 1 in the example, the script can only run 1000 times per second. I'm doing recursion on the level of hundreds of millions, so this isn't fast enough. When I set the timeout value to 0, or .1, or 1/10, I still only get approximately 1000 times per second. I've tried using 32 and 64 bit browsers (Chrome and Firefox) to no avail.
How can I kick the speed up a notch? Also, I'm relatively new at all of this, so it'd be awesome if the solution was a simple one.
Oh, forgot to mention: if I remove the setTimeout altogether, I overflow the stack every time.
Thanks for the help!
Your solution doesn't lie in making your current code run, but to rethink the code.
I don't know how you are using recursion in your code, but clearly you are using it wrong.
For any reasonable use of recursion, you would be far from overflowing the stack. If you are making recursive calls to a level of hundreds of millions, that is at least a million times too much.
A common approach when using recursion is to divide the work in half for each level. That way you can handle all the items that you can fit in memory without going deeper than about 30 levels.
I tried something like you did and found the solution! You don't need recursion and the function setTimeout, but all what you need is to use the setInterval function on the function you want by 1 interval repeatedly in for loop. For example if the for loop repeats itself 10 times, then 10 timers will execute the same function every 4 ms. The code will be executed repeatedly more and more quickly.
Your code should look like this for example:
function onload() {
for (var i = 0; i < 10; i++)
setInterval(go, 1);
}
function go() {
x++;
}
JavaScript is single threaded, and setTimeout will put your operation at the end of the queue. Even if you reduce the delay, you still have to wait for the previous operations to complete before the one you added kicks in.
It's not possible to make setTimeout wait less than 4 milliseconds. That is how setTimeout is defined in the HTML standard (official spec here). More likely your problem is with how your code is structured. Show us the rest of your code, maybe we can help sort it out.
I have wrote this code to make seconds (with decisec & centisec) counting up.
You've wasted time <span id="alltime">0.00</span> seconds.
<script type="text/javascript">
function zeroPad(num,count)
{
var numZeropad = num + '';
while(numZeropad.length < count) { numZeropad = "0" + numZeropad; }
return numZeropad; }
function counttwo() {
tall = document.getElementById('alltime').innerHTML;
if(parseFloat(tall) < 1.00) { tnew2 = tall.replace('0.0','').replace('0.',''); }
else { tnew2 = tall.replace('.',''); }
tnum = parseInt(tnew2) + 1;
//check if have to add zero
if(tnum >= 100) { tstr1 = tnum + ''; }
else { tstr1 = zeroPad(tnum,3); }
tlast = tstr1.substr(0,tstr1.length - 2) + '.' + tstr1.substr(tstr1.length - 2);
document.getElementById("alltime").innerHTML = tlast;
}
var inttwo=setInterval("counttwo()",10);
</script>
In HTML document and run.
It works well but when I use Firefox 4 and run the code. Seems like it LAG a bit (stop a bit before counting up) when it's on some numbers (randomly like 12.20, 4.43). I've tried change "counttwo()" to counttwo but that doesn't help.
I have told some of my friends to run on Firefox 4 too. They said it doesn't lag at all. This cause because of my computer ? or My Firefox ? or something else ?
Thanks in advance!
PS. Fiddle here: http://jsfiddle.net/XvkGy/5/ Mirror: http://bit.ly/hjVtXS
When you use setInterval or setTimeout the time interval is not exact for several reasons. It is dependent on other javascript running, the browser, the processor etc. You have to take a reliability of +- 15ms for granted afaik. See also ...
That's a lot of counting, so on some computer, yes, it might lag (if it's a prehistoric one or the user's got his processor really busy with something), also if I'm right, that thing won't work with Chrome's V8, since that script would freeze if you switched tabs, and resume executing only when you return to that tab.
If you're just seeing pauses every so often, you're probably seeing garbage collection or cycle collection pauses.
You can test this by toggling your javascript.options.mem.log preference to true in about:config and then watching the error console's "Messages" tab as your script runs. If the GC/CC messages are correlated with your pauses, then they're the explanation for what you see.
As for why you see it but others don't... do you see the problem if you disable all your extensions?
The problem with setInterval is that it can eventually lead to a back-up. This happens because the JavaScript engine tries to execute the function on the interval (in your case, 10ms), but if ever that execution takes longer than 10ms, the JS engine starts trying to execute the next interval before the current one stops (which really just means it queues it up to run as soon as the previous callback finishes).
Since JavaScript executes single-threaded (with the exception of web workers in HTML 5), this can lead to pauses in your UI or DOM updates because it is continuously processing JavaScript callbacks from your setInterval. In worst case scenarios, the whole page can become permanently unresponsive because your stack of uncompleted setInterval executions gets longer and longer, never fully finishing.
With a few exceptions, it is generally considered a safer bet to use setTimeout (and invoking the setTimeout again after execution of the callback) instead of setInterval. With setTimeout, you can ensure that one and only one timeout is ever queued up. And since the timers are only approximate anyway (just because you specify 10ms doesn't mean it will happen at exactly 10ms), you don't typically gain anything from using setInterval over setTimeout.
An example using setTimeout:
var count = function(){
// do something
// queue up execution once again
setTimeout(count, 10);
};
count();
One reason why you may see pauses on some browsers, and not others, is because not all JavaScript engines are created equal :). Some are faster than others, and as such, less likely to end up with a setInterval backup.
Different browsers use different JavaScript engines, so it's possible that this code just finds a spot where Firefox's JägerMonkey scripting engine has some problems. There doesn't seem to be any obvious inefficiencies in the counting itself..
If it's working on your friends' installs of FF4, then it's probably just an isolated problem for you, and there isn't much you'll be able to do by changing the code.