We Keep Coding

Why is array.includes an order of magnitude faster than set.has in javascript?

Well having grown up in C++, I am always conscious about what algorithm would fit what. So when I notice the application started to behave sluggish on mobile phones I immediately started looking at the data structures and how they are represented.
I am noticing a very strange effect Array.includes is an order of Magnitude faster than Set.has. Even though Set.has much more potential to be optimized for lookup: it's the whole idea of using a set.
My initialize code is (this code is outside the timing for the tests):
function shuffle(a) {
for (let i = a.length - 1; i > 0; i--) {
const j = Math.floor(Math.random() * (i + 1));
[a[i], a[j]] = [a[j], a[i]];
}
}
const arr = []
for (let i = 0; i < 1000; i+=1) {
arr.push(i);
};
shuffle(arr);
const prebuildset=new Set(arr);
And the tests are:
(new Set(arr)).has(-1); //20.0 kOps/s
arr.includes(-1); //632 kOps/s
(new Set(arr)).has(0); //20.0 kOps/s
arr.includes(0); //720 kOps/s
prebuildset.has(-1); //76.7 kOps/s
prebuildset.has(0); //107 kOps/s
Tested with chrome 73.0.3683.103 on Ubuntu 18.04 using https://jsperf.com/set-array-has-test/1
I kind of can expect the versions that create a set on the fly to be slower than directly testing an array for inclusion. (Though I'd wonder why chrome doesn't JIT optimize the array away - I've also tested using a literal array and a literal array vs using a variable doesn't matter at all in speed).
However even prebuild sets are an order of magnitude slower than an array-inclusion test: even for the most negative case (entry not inside the array).
Why is this? What kind of black magic is happening?
EDIT: I've updated the tests to shuffle the results so as to not skew too much to an early stoppage for array.includes()- While no longer 10 times as slow it is still many times slower, very relevant and out of what I expect it to be.

I'll start by stating that I'm not an expert on JavaScript engine implementations and performance optimization; but in general, you should not trust these kind of tests to give you a reliable assessment of performance.
Time complexity of the underlying algorithm only becomes a meaningful factor over very (very) large numbers, and as a rule of thumb, 1000 is certainly not such a large number, especially for a simple array of integer values.
Over a small amount of millisecond-timed operations, you are going to have many other things happening in the engine at a similar time scale that will throw your measurements off wildly. Optimizations, unexpected overheads, and so on.
As an example, I edited your tests by simply increasing the size of the array to 100,000. The results on my poor old laptop look like this:
arr.includes(-1); //3,323 Ops/s
arr.includes(0); //6,132 Ops/s
prebuildset.has(-1); //41,923,084 Ops/s
prebuildset.has(0); //39,613,278 Ops/s
Which is, clearly, extremely different from your results. My point is, don't try to measure microperformance for small tasks. Use the data structure that makes the most sense for your project, keep your code clean and reasonable, and if you need to scale, prepare accordingly.

canvas - change perspective of the camera in a 2d setup

TLDR:
I need to change the perspective over an object in a 2d cavas.
Let's say I make a photo of my desk while I sit next to it. I want to change that image, to look as if I would see it from above. Basically the view would change like in the image below (yes, my paint skills are awesome :)) ).
I am using canvas and easeljs (createjs).
I am totally aware that this is not a 3rd object, I don't have a stage and a camera. I am also aware that easeljs doesn't support this as a feature.
What I am actually trying is to emulate this (I am also not interested in quality of the result at this point).
What I tried and it worked (I noticed I am not the first to try to do this and didn't actually found a real answer so this is my current approach):
I take the original image. Divide it in a high number of images (the higher the better) as in the image below.
Then I scale each of the "mini-images" on x axis with a different factor (that I'm computing depending on the angle the picture was made). Part of relevant code below (excuse the hardcodings and creepy code - is from a proof of concept I made in like 10 minutes):
for (var i = 0; i < 400; i++) {
crops[i] = new createjs.Bitmap(baseImage);
crops[i].sourceRect = new createjs.Rectangle(0, i, 700, i + 1);
crops[i].regX = 350;
crops[i].y = i - 1;
crops[i].x = 100;
crops[i].scaleX = (400 - i / 2) * angleFactor;
stage.addChild(crops[i]);
}
Then you crop again only the relevant part.
Ok... so this works but the performance is... terrible - you basically generate 400 images - in this case - then you put them in a canvas. Yes, I know it can be optimized a bit but is still pretty bad. So I was wondering if you have any other (preferably better) approaches.
I also tried combining a skewing transformation with a scale and a rotation but I could not achieve the right result (but I actually still think there may still be something here... I just can't put my finger on it and my "real" math is a bit rusty).

How does 2D drawing frameworks such as Pixi.js make canvas drawing faster?

I found a bunnymark for Javascript canvas here.
Now of course, I understand their default renderer is using webGL but I am only interested in the native 2D context performance for now. I disabled webGL on firefox and after spawning 16500 bunnies, the counter showed a FPS of 25. I decided to wrote my own little very simple rendering loop to see how much overhead Pixi added. To my surprise, I only got a FPS of 20.
My roughly equivalent JSFiddle.
So I decided to take a look into their source here and it doesn't appear to be that the magic is in their rendering code:
do
{
transform = displayObject.worldTransform;
...
if(displayObject instanceof PIXI.Sprite)
{
var frame = displayObject.texture.frame;
if(frame)
{
context.globalAlpha = displayObject.worldAlpha;
context.setTransform(transform[0], transform[3], transform[1], transform[4], transform[2], transform[5]);
context.drawImage(displayObject.texture.baseTexture.source,
frame.x,
frame.y,
frame.width,
frame.height,
(displayObject.anchor.x) * -frame.width,
(displayObject.anchor.y) * -frame.height,
frame.width,
frame.height);
}
}
Curiously, it seems they are using a linked list for their rendering loop and a profile on both app shows that while my version allocates the same amount of cpu time per frame, their implementation shows cpu usage in spikes.
My knowledge ends here unfortunately and I am curious if anyone can shed some light on whats going on.

I think, in my opinion, that it boils down to how "compilable" (cache-able) the code is. Chrome and Firefox uses two different JavaScript "compilers"/engines as we know which optimizes and caching code differently.
Canvas operations
Using transform versus direct coordinates should not have an impact as setting a transform merely updates the matrix which is in any case is used with what-ever is in it.
The type of position values can affect performance though, float versus integer values, but as both your fiddle and PIXI seem to use floats only this is not the key here.
So here I don't think canvas is the cause of the difference.
Variable and property caching
(I got unintentionally too focused on the prototypal aspect in the first version of this answer. The essence I was trying to get at was mainly object traversing, so here the following text is re-worded a bit -)
PIXI uses object properties as the fiddle but these custom objects in PIXI are smaller in size so the traversing of the object tree takes less time compared to what it takes to traverse a larger object such as canvas or image (a property such as width would also be at the end of this object).
It's a well known classic optimization trick to cache variables due to this very reason (traverse time). The effect is less today as the engines has become smarter, especially V8 in Chrome which seem to be able to predict/cache this better internally, while in Firefox it seem to still have a some impact not to cache these variables in code.
Does it matter performance-wise? For short operations very little, but drawing 16,500 bunnies onto canvas is demanding and do gain a benefit from doing this (in FF) so any micro-optimization do actually count in situations such as this.
Demos
I prototyped the "renderer" to get even closer to PIXI as well as caching the object properties. This gave a performance burst in Firefox:
http://jsfiddle.net/AbdiasSoftware/2Dbys/8/
I used a slow computer (to scale the impact) which ran your fiddle at about 5 FPS. After caching the values it ran at 6-7 fps which is more than 20% increase on this computer showing it do have an effect. On a computer with a larger CPU instruction cache and so forth the effect may be less, but it's there as this is related to the FF engine itself (disclaimer: I am not claiming this to be a scientific test however, only a pointer :-) ).
/// cache object properties
var lastTime = 0,
w = canvas.width,
h = canvas.height,
iw = image.width,
ih = image.height;
This next version caches these variables as properties on an object (itself) to show that also this improves performance compared to using large global objects directly - result about the same as above:
http://jsfiddle.net/AbdiasSoftware/2Dbys/9/
var RENDER = function () {
this.width = canvas.width;
this.height = canvas.height;
this.imageWidth = image.width;
this.imageHeight = image.height;
}
In conclusion
I am certain based on the results and previous experience that PIXI can run the code faster due to using custom small-sized objects rather than getting the properties directly from large objects (elements) such as canvas and image.
The FF engine seem not yet to be as "smart" as the V8 engine in regard to object traversing of tree and branches so caching variables do have an impact in FF which comes to display when the demand is high (such as when drawing 16,500 bunnies per "frame").

One difference I noticed between your version and Pixi's is this:
You render image at certain coordinates by passing x/y straight to drawImage function:
drawImage(img, x, y, ...);
..whereas Pixi translates entire canvas context, and then draws image at 0/0 (of already shifted context):
setTransform(1, 0, 0, 1, x, y);
drawImage(img, 0, 0, ...);
They also pass more arguments to drawImage; arguments that control "destination rectangle" — dx, dy, dw, dh.
I suspected this is where speed difference hides. However, changing your test to use same "technique" doesn't really make things better.
But there's something else...
I clocked bunnies to 5000, disabled WebGL, and Pixi actually performs worse than the custom fiddle version.
I get ~27 FPS on Pixi:
and ~32-35 FPS on Fiddle:
This is all on Chrome 33.0.1712.4 dev, Mac OS X.

I'd suspect that this is some canvas compositing issue. Canvas is transparent by default, so the page background needs to be combined with the canvas contents...
I found this in their source...
// update the background color
if (this.view.style.backgroundColor != stage.backgroundColorString &&
!this.transparent) {
this.view.style.backgroundColor = stage.backgroundColorString;
}
Maybe they set the canvas to be opaque for this demo (the fiddle doesn't really work for me, seems like most of the bunnys jump out with an extremely large dt most of the time)?
I don't think it's an object property access timing / compilability thing: The point is valid, but I don't think it can explain that much of a difference.

Performance concerns when storing data in large arrays with Javascript

I have a browser-based visualization app where there is a graph of data points, stored as an array of objects:
data = [
{x: 0.4612451, y: 1.0511} ,
... etc
]
This graph is being visualized with d3 and drawn on a canvas (see that question for an interesting discussion). It is interactive and the scales can change a lot, meaning the data has to be redrawn, and the array needs to be iterated through quite frequently, especially when animating zooms.
From the back of my head and reading other Javascript posts, I have a vague idea that optimizing dereferences in Javascript can lead to big performance improvements. Firefox is the only browser on which my app runs really slow (compared to IE9, Chrome, and Safari) and it needs to be improved. Hence, I'd like to get a firm, authoritative answer the following:
How much slower is this:
// data is an array of 2000 objects with {x, y} attributes
var n = data.length;
for (var i=0; i < n; i++) {
var d = data[i];
// Draw a circle at scaled values on canvas
var cx = xs(d.x);
var cy = ys(d.y);
canvas.moveTo(cx, cy);
canvas.arc(cx, cy, 2.5, 0, twopi);
}
compared to this:
// data_x and data_y are length 2000 arrays preprocessed once from data
var n = data_x.length;
for (var i=0; i < n; i++) {
// Draw a circle at scaled values on canvas
var cx = xs(data_x[i]);
var cy = ys(data_y[i]);
canvas.moveTo(cx, cy);
canvas.arc(cx, cy, 2.5, 0, twopi);
}
xs and ys are d3 scale objects, they are functions that compute the scaled positions. I mentioned the above that the above code may need to run up to 60 frames per second and can lag like balls on Firefox. As far as I can see, the only differences are array dereferences vs object accessing. Which one runs faster and is the difference significant?

It's pretty unlikely that any of these loop optimizations will make any difference. 2000 times through a loop like this is not much at all.
I tend to suspect the possibility of a slow implementation of canvas.arc() in Firefox. You could test this by substituting a canvas.lineTo() call which I know is fast in Firefox since I use it in my PolyGonzo maps. The "All 3199 Counties" view on the test map on that page draws 3357 polygons (some counties have more than one polygon) with a total of 33,557 points, and it loops through a similar canvas loop for every one of those points.

Thanks to the suggestion for JsPerf, I implemented a quick test. I would be grateful for anyone else to add their results here.
http://jsperf.com/canvas-dots-testing: results as of 3/27/13:
I have observed the following so far:
Whether arrays or objects is better seems to depend on the browser, and OS. For example Chrome was the same speed on Linux but objects were faster in Windows. But for many they are almost identical.
Firefox is just the tortoise of the bunch and this also helps confirm Michael Geary's hypothesis that its canvas.arc() is just super slow.

Invalid blending results across all browsers with HTML5 canvas

Summary
When repeatedly drawing anything (apparently with a low alpha value) on a canvas, regardless of if it's with drawImage() or a fill function, the resulting colors are significantly inaccurate in all browsers that I've tested. Here's a sample of the results I'm getting with a particular blend operation:
Problem Demonstration
For an example and some code to play with, check out this jsFiddle I worked up:
http://jsfiddle.net/jMjFh/2/
The top set of data is a result of a test that you can customize by editing iters and rgba at the top of the JS code. It takes whatever color your specify, RGBA all in range [0, 255], and stamps it on a completely clean, transparent canvas iters number of times. At the same time, it keeps a running calculation of what the standard Porter-Duff source-over-dest blend function would produce for the same procedure (ie. what the browsers should be running and coming up with).
The bottom set of data presents a boundary case that I found where blending [127, 0, 0, 2] on top of [127, 0, 0, 63] produces an inaccurate result. Interestingly, blending [127, 0, 0, 2] on top of [127, 0, 0, 62] and all [127, 0, 0, x] colors where x <= 62 produces the expected result. Note that this boundary case is only valid in Firefox and IE on Windows. In every other browser on every other operating system I've tested, the results are far worse.
In addition, if you run the test in Firefox on Windows and Chrome on Windows, you'll notice a significant difference in the blending results. Unfortunately we're not talking about one or two values off--it's far worse.
Background Information
I am aware of the fact that the HTML5 canvas spec points out that doing a drawImage() or putImageData() and then a getImageData() call can show slightly varying results due to rounding errors. In that case, and based on everything I've run into thus far, we're talking about something miniscule like the red channel being 122 instead of 121.
As a bit of background material, I asked this question a while ago where #NathanOstgard did some excellent research and drew the conclusion that alpha premultiplication was to blame. While that certainly could be at work here, I think the real underlying issue is something a bit larger.
The Question
Does anyone have any clue how I could possibly end up with the (wildly inaccurate) color values I'm seeing? Furthermore, and more importantly, does anyone have any ideas how to circumvent the issue and produce consistent results in all browsers? Manually blending the pixels is not an option due to performance reasons.
Thanks!
Edit: I just added a stack trace of sorts that outputs each intermediate color value and the expected value for that stage in the process.
Edit: After researching this a bit, I think I've made some slight progress.
Reasoning for Chrome14 on Win7
In Chrome14 on Win7 the resulting color after the blend operation is gray, which is many values off from the expected and desired reddish result. This led me to believe that Chrome doesn't have an issue with rounding and precision because the difference is so large. Instead, it seems like Chrome stores all pixel values with premultiplied alpha.
This idea can be demonstrated by drawing a single color to the canvas. In Chrome, if you apply the color [127, 0, 0, 2] once to the canvas, you get [127, 0, 0, 2] when you read it back. However, if you apply [127, 0, 0, 2] to the canvas twice, Chrome gives you [85, 0, 0, 3] if you check the resulting color. This actually makes some sense when you consider that the premultiplied equivalent of [127, 0, 0, 2] is [1, 0, 0, 2].
Apparently, when Chrome14 on Win7 performs the blending operation, it references the premultiplied color value [1, 0, 0, 2] for the dest component and the non-premultiplied color value [127, 0, 0, 2] for the source component. When these two colors are blended together, we end up with [85, 0, 0, 3] using the Porter-Duff source-over-dest approach, as expected.
So, it seems like Chrome14 on Win7 inconsistently references the pixel values when you work with them. The information is stored in a premultiplied state internally, is presented back to you in non-premultiplied form, and is manipulated using values of both forms.
I'm thinking it might be possible to circumvent this by doing some additional calls to getImageData() and/or putImageData(), but the performance implications don't seem that great. Furthermore, this doesn't seem to be the same issue that Firefox7 on Win7 is exhibiting, so some more research will have to be done on that side of things.
Edit: Below is one possible approach that seems likely to work.
Thoughts on a Solution
I haven't gotten back around to working on this yet, but one WebGL approach that I came up with recently was to run all drawing operations through a simple pixel shader that performs the Porter-Duff source-over-dest blend itself. It seems as though the issue here only occurs when interpolating values for blending purposes. So, if the shader reads the two pixel values, calculates the result, and writes (not blends) the value into the destination, it should mitigate the problem. At the very least, it shouldn't compound. There will definitely still be minor rounding inaccuracies, though.
I know in my initial question I mentioned that manually blending the pixels wouldn't be viable. For a 2D canvas implementation of an application that needs real-time feedback, I don't see a way to fix this. All of the blending would be done on the CPU and would prevent other JS code from executing. With WebGL, however, your pixel shader runs on the GPU so I'm pretty sure there shouldn't be any performance hit.
The tricky part is being able to feed the dest canvas in as a texture to the shader because you also need to render it on a canvas to be viewable. Ultimately, you want to avoid having to generate a WebGL texture object from your viewable canvas every time it needs to be updated. Maintaining two copies of the data, with one as an in-memory texture and one as a viewable canvas (that's updated by stamping the texture on as needed), should solve this.

Oh geez. I think we just stepped into the twilight zone here. I'm afraid I don't have an answer but I can provide some more information at least. Your hunch is right that at least something is larger here.
I'd prefer an example that is drastically more simple (and thus less prone to any sort of error - I fear you might have a problem with 0-255 somewhere instead of 0-1 but didnt look thoroughly) than yours so I whipped up something tiny. What I found wasn't pleasant:
<canvas id="canvas1" width="128" height="128"></canvas>
<canvas id="canvas2" width="127" height="127"></canvas>
+
var can = document.getElementById('canvas1');
var ctx = can.getContext('2d');
var can2 = document.getElementById('canvas2');
var ctx2 = can2.getContext('2d');
var alpha = 2/255.0;
ctx.fillStyle = 'rgba(127, 0, 0, ' + alpha + ')';
ctx2.fillStyle = 'rgba(127, 0, 0, ' + alpha + ')'; // this is grey
//ctx2.fillStyle = 'rgba(171, 0, 0, ' + alpha + ')'; // this works
for (var i = 0; i < 32; i++) {
ctx.fillRect(0,0,500,500);
ctx2.fillRect(0,0,500,500);
}
Yields this:
See for yourself here:
http://jsfiddle.net/jjAMX/
It seems that if your surface area is too small (not the fillRect area, but the canvas area itself) then the drawing operation will be a wash.
It seems to work fine in IE9 and FF7. I submitted it as a bug to Chromium. (We'll see what they say, but their typical M.O. so far is to ignore most bug reports unless they are security issues).
I'd urge you to check your example - I think you might have a 0-255 to 0-1 issue somewhere. The code I have here seems to work just dandy, save for maybe the expected reddish color, but it is a consistent resultant color across all browsers with the exception of this Chrome issue.
In the end Chrome is probably doing something to optimize canvases with a surface area smaller than 128x128 that is fouling things up.

Setting globalCompositeOperation to "lighter" (the operation where overlapping pixels values are added together, it is not the default operation for many browsers) yields same color and results in ie9, chrome 14 and latest firefox aurora:
Expected: [126.99999999999999, 0, 0, 63.51372549019608]
Result: [127, 0, 0, 64]
Expected: [126.99999999999999, 0, 0, 63.51372549019608]
Result: [127, 0, 0, 64]
http://jsfiddle.net/jMjFh/11/
edit: you may learn what the different composite operations mean here:
https://developer.mozilla.org/en/Canvas_tutorial/Compositing#globalCompositeOperation
for example, "copy" was useful to me when I needed to do fade animation in certain parts of a canvas