We Keep Coding

How can I make my JavaScript programs run faster?

Ok, so I am working on a sort of detection system where I will be pointing the camera at a screen, and it will have to find the red object. I can successfully do this, with pictures, but the problem is that it takes several seconds to load. I want to be able to do this to live videos, so I need it to find the object immediately. Here is my code:
video.addEventListener('pause', function () {
let reds = [];
for(x=0; x<= canvas.width; x++){
for(y=0; y<= canvas.height; y++){
let data = ctx.getImageData(x, y, 1, 1).data;
let rgb = [ data[0], data[1], data[2] ];
if (rgb[0] >= rgb[1] && rgb[0] >=rgb[2] && !(rgb[0]>100 && rgb[1]>100 && rgb[2]>100) && rgb[1]<100 && rgb[2]<100 && rgb[0]>150){
reds[reds.length] = [x, y]
}
let addedx = 0
let addedy = 0
for(i=0; i<reds.length; i++){
addedx = addedx + reds[i][0]
addedy = addedy + reds[i][1]
}
let center = [addedx/reds.length, addedy/reds.length]
ctx.rect(center[0]-5, center[1]-5, 10, 10)
ctx.stroke()
}, 0);
Ya, I know its messy. Is there something about the for loops that are slow? I know I'm looping through thousands of pixels but that's the only way I can think of to do it.

As it has been said, Javascript is not the most performant for this task. However, here are some things I noticed, which could slow you down.
You grab the image data one pixel at a time. Since this method can return the whole frame, you can do this once.
Optimize your isRed condition:
rgb[0] >= rgb[1] && // \
rgb[0] >= rgb[2] && // >-- This is useless
!(rgb[0] > 100 && rgb[1] > 100 && rgb[2] > 100) && // /
rgb[1] < 100 && // \
rgb[2] < 100 && // >-- These 3 conditions imply the others
rgb[0] > 150 // /
You calculate the center inside your for loop after each pixel, but it would only make sense after processing the whole frame.
Since the video feed is coming from a camera, maybe you don't need to look at every single pixel. Maybe every 5 pixels is enough? That's what the example below does. Tweak this.
Demo including these optimizations
Node: This demo includes an adaptation of the code from this answer, to copy the video onto the canvas.
const video = document.getElementById("video"),
canvas = document.getElementById("canvas"),
ctx = canvas.getContext("2d");
let width,
height;
// To make this demo work
video.crossOrigin = "Anonymous";
// Set canvas to video size when known
video.addEventListener("loadedmetadata", function() {
width = canvas.width = video.videoWidth;
height = canvas.height = video.videoHeight;
});
video.addEventListener("play", function() {
const $this = this; // Cache
(function loop() {
if (!$this.paused && !$this.ended) {
ctx.drawImage($this, 0, 0);
const reds = [],
data = ctx.getImageData(0, 0, width, height).data,
len = data.length;
for (let i = 0; i < len; i += 5 * 4) { // 4 because data is made of RGBA values
const rgb = data.slice(i, i + 3);
if (rgb[0] > 150 && rgb[1] < 100 && rgb[2] < 100) {
reds.push([i / 4 % width, Math.floor(i / 4 / width)]); // Get [x,y] from i
}
}
if (reds.length) { // Can't divide by 0
const sums = reds.reduce(function (res, point) {
return [res[0] + point[0], res[1] + point[1]];
}, [0,0]);
const center = [
Math.round(sums[0] / reds.length),
Math.round(sums[1] / reds.length)
];
ctx.strokeStyle = "blue";
ctx.lineWidth = 10;
ctx.beginPath();
ctx.rect(center[0] - 5, center[1] - 5, 10, 10);
ctx.stroke();
}
setTimeout(loop, 1000 / 30); // Drawing at 30fps
}
})();
}, 0);
video, canvas { width: 250px; height: 180px; background: #eee; }
<video id="video" src="https://shrt-statics.s3.eu-west-3.amazonaws.com/redball.mp4" controls></video>
<canvas id="canvas"></canvas>

I would run the detection algorithm in a webassembly module. Since it is just pixel data, thats right up its alley.
You could then pass individual frames to a different instance of the wasm module.
As far as answering your question directly, I would grab the whole frame, not 1 pixel at a time, or you might get pixels sampled from different frames. You can then submit that frame to a worker, you could even divide up the frame and send them to different workers (or as previously mentioned a wasm module)
Also since you have an array you can use Arrray.map and Array.reduce to get you to just the red values, and how big they are by testing for adjacent pixels, instead of all the comparison. Not sure if it will be faster but worth a try.

For the speed, you should consider all your process:
more your language is near the machine language, better your result will be. Saying so, C++ is better for the algorithm.
CPU speed is your friend. Launching your code on an Atom processor or on an i7 processor, is like night and day. Moreover, some type of processor is dedicated for vision like VPU
For your code:
You try to rewrite code that already exists. You can find good examples of detection in the great OpenCV library: https://www.learnopencv.com/invisibility-cloak-using-color-detection-and-segmentation-with-opencv
Hope it help you :)

Display value in canvas during mouseover

Dear JavaScript users,
I need to be able to:
load a png image and display it on a canvas
store its original pixel values
transform the pixel values to represent new colours
display the original values on the screen when the user moves their cursor over the image
This may sound like a slightly odd thing to do, but the original pixel values in my live system will contain encoded data that I need to retain, and display on the screen after whatever pixel value manipulation is subsequently carried out. I need to change the colour mapping after the initial loading of the image to make it more pleasing to the eye, but need to display the original values on the screen.
My method works when displaying some simple geometrical shapes on the canvas, but as soon as I try to use a png image it stops working. Can anyone help me to understand why this is?
An example (without the pixel value transformation) that works with the simple shapes is here:
http://jsfiddle.net/DV9Bw/1219/
If you comment out lines 24 - 29, and uncomment lines 32 - 40 so that it loads in a png, it stops working. The png file loads, but the data values are no longer shown on the screen.
Any suggestions as to why it breaks when I use a png would be welcome; any suggestions on how to fix it would be even more welcome!
Many thanks in advance for any help.
David
function findPos(obj) {
var curleft = 0, curtop = 0;
if (obj.offsetParent) {
do {
curleft += obj.offsetLeft;
curtop += obj.offsetTop;
} while (obj = obj.offsetParent);
return { x: curleft, y: curtop };
}
return undefined;
}
function rgbToHex(r, g, b) {
if (r > 255 || g > 255 || b > 255)
throw "Invalid color component";
return ((r << 16) | (g << 8) | b).toString(16);
}
var example = document.getElementById('example');
var context = example.getContext('2d');
// The squares works
context.fillStyle = "rgb(255,0,0)";
context.fillRect(0, 0, 50, 50);
context.fillStyle = "rgb(0,0,255)";
context.fillRect(55, 0, 50, 50);
// End of squares
/*
// Replacing the squares section above with this
// png image stops the mouseOver from working, Why?
var imageObj = new Image();
imageObj.onload = function() {
context.drawImage(imageObj, 0, 0, imageObj.width, imageObj.height, 0, 0, imageObj.width*4, imageObj.height*4);
};
imageObj.src = 'http://dplatten.co.uk/mouseOver/skin_dose_map.png';
*/
var originalValues = new Array();
originalValues = context.getImageData(0,0,280,360).data;
$('#example').mousemove(function(e) {
var pos = findPos(this);
var x = e.pageX - pos.x;
var y = e.pageY - pos.y;
var coord = "x=" + x + ", y=" + y;
var c = this.getContext('2d');
var r = originalValues[(y*280 + x)*4];
var g = originalValues[(y*280 + x)*4+1];
var b = originalValues[(y*280 + x)*4+2];
$('#status').html(coord + "<br>" + r + "," + g + "," + b);
});
<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
<canvas id="example" width="280" height="360"></canvas>
<div id="status"></div>

The problem is two-fold
First of all, these two line:
var originalValues = new Array();
originalValues = context.getImageData(0,0,280,360).data;
are being run before you paint the image since it's waiting for the image to load still.
Then, if would move the context.getImageData() to inside the imgObject.onload-function, you'd run into another problem namely, you can't run getImageData() on an image that is not on the same location as the file that is running the script. You will get the following message:
Uncaught SecurityError: Failed to execute 'getImageData' on 'CanvasRenderingContext2D': The canvas has been tainted by cross-origin data.
If you put the image on the same server and move the getImageData() call inside the onload-function, it should work.

how to see each pixel being changed on the canvas html5

I've created a filter in javascript which inverts the colors of image i.e creates a negative image, now when I run it in the browser it takes time to process and then returns the final negative image. How can I see each pixel of the image being inverted and not just the final inverted image? Instead of waiting for the the code to be implemented on the whole pixel array and then see its effects, I want to see each pixel being changed by the code till the last pixel.
var imgData = ctx.getImageData(0,0,x.width,x.height);
var d = imgData.data;
for (var i=0; i< d.length; i+=4) {
d[i] = 255 - d[i];
d[i+1] = 255 - d[i+1];
d[i+2] = 255 - d[i+2];
}
ctx.putImageData(imgData,0,0);
NEW CODE
invert(d,0);
function invert(d,i){
if(i < d.length){
d[i] = 255 - d[i];
d[i+1] = 255 - d[i+1];
d[i+2] = 255 - d[i+2];
d[i+3] = d[i+3];
//alert(i);
var n=i/4;
var h=parseInt(n/x.width);
var w = n - h*x.width;
ctx.fillStyle='rgba('+d[i]+','+d[i+1]+','+d[i+2]+','+d[i+3]/255+')';
ctx.fillRect(w,h,1,1);
//if(i>91000){alert(i);}
setTimeout(invert(d,i+4),50);
}
else{return ;}
}

You need to use an asynchronous "loop" so you can update some pixels, let the display update the result, then continue.
JavaScript is single threaded so nothing will be updated until the current loop finishes as the thread is occupied with that.
Watching the paint dry
Here is one approach you can use. You define a "batch" size in number of pixels you want to invert (1 is valid if you want to see each pixel, but this can take a long time:
ie. 16.67ms x total number of pixels.
So if you want to display each single pixel update with an image of 640x400 (as in the demo below) then it would take:
640 x 400 x 16.67ms = 4,267,520 ms, or more than an hour
Something to have in mind (the display cannot update faster than per 16.67ms = 60 fps). Below we use 128 pixels per batch.
Live example
Note that the batch value must match the width of the image. F.ex. if your image is 640 pixels wide you can use 1, 5, 10, 20, .. 64, 128 etc.
If you want widths that do not necessarily divide on anything but 1 or fractional values, you have to do a simple calculation to limit the last batch of one line as getImageData() require the arguments to define the area inside an image. Or, just do line by line...
You can also use a "batch" value for vertical tiles (box is probably a better term in that case).
var img = new Image();
img.crossOrigin = "";
img.onload = painter;
img.src = "http://i.imgur.com/Hl3I0cx.jpg";
function painter() {
// setup canvas and image
var canvas = document.querySelector("canvas"),
ctx = canvas.getContext("2d");
// set canvas size = image size
canvas.width = this.naturalWidth; canvas.height = this.naturalHeight;
// draw in image
ctx.drawImage(this, 0, 0);
// prepare loop
var batch = 128,
x = 0, y = 0,
w = canvas.width - batch,
h = canvas.height;
(function asyncUpdate() {
// do one batch only
var idata = ctx.getImageData(x, y, batch, 1), // get a bacth of pixels
data = idata.data,
i = 0, len = data.length;
while(i < len) { // invert the batch
data[i] = 255 - data[i++];
data[i] = 255 - data[i++];
data[i] = 255 - data[i++];
i++
}
ctx.putImageData(idata, x, y); // update bitmap
x += batch;
if (x > w) { // check x pos
x = 0;
y++;
}
if (y < h) { // new batch?
requestAnimationFrame(asyncUpdate); // let display update before next
}
else {
// use a callback here...
console.log("Done");
}
})();
}
<canvas></canvas>

You could context.fillRect each newly changed pixel on the canvas as it's calculated.
Something like this (warning: untested code, may need tweeks!):
var n=i/4;
var y=parseInt(n/canvas.width);
var x=n-y*canvas.width;
context.fillStyle='rgba('+d[i]+','+d[i+1]+','+d[i+2]+','+d[i+3]/255')';
context.fillRect(x,y,1,1);
Here's demo code that first does the inverting and then shows the effect over time:
var canvas=document.getElementById("canvas");
var ctx=canvas.getContext("2d");
var cw=canvas.width;
var ch=canvas.height;
var nextTime=0;
// a new line of converted image will be displayed
// after this delay
var delay=1000/60*2;
var y=0;
var imgData,d;
var img=new Image();
img.crossOrigin='anonymous';
img.onload=start;
img.src="https://dl.dropboxusercontent.com/u/139992952/multple/sun.png";
function start(){
cw=canvas.width=img.width;
ch=canvas.height=img.height;
ctx.drawImage(img,0,0);
imgData=ctx.getImageData(0,0,cw,ch);
d=imgData.data;
for (var i=0; i< d.length; i+=4) {
d[i] = 255 - d[i];
d[i+1] = 255 - d[i+1];
d[i+2] = 255 - d[i+2];
}
requestAnimationFrame(animate);
}
function animate(time){
if(time<nextTime){requestAnimationFrame(animate); return;}
nextTime=time+delay;
for(var x=0;x<cw;x++){
var i=(y*cw+x)*4;
ctx.fillStyle='rgba('+d[i]+','+d[i+1]+','+d[i+2]+','+d[i+3]/255+')';
ctx.fillRect(x,y,1,1);
}
if(++y<ch){
requestAnimationFrame(animate);
}else{
ctx.putImageData(imgData,0,0);
}
}
body{ background-color: ivory; }
#canvas{border:1px solid red;}
<canvas id="canvas" width=300 height=300></canvas>

How to get the average or main color from an image with javascript?

what i want is to the the HEX or the RGB average value from an image to the another div background this color.
So if i upload an image with a ot of red i get something like #FF0000 just as an example.
Let Me know if this is posible :)
Many thanks.

First, draw the image on a canvas:
function draw(img) {
var canvas = document.createElement("canvas");
var c = canvas.getContext('2d');
c.width = canvas.width = img.width;
c.height = canvas.height = img.height;
c.clearRect(0, 0, c.width, c.height);
c.drawImage(img, 0, 0, img.width , img.height);
return c; // returns the context
}
You can now iterate over the image's pixels. A naive approach for color-detection is to simply count the frequency of each color in the image.
// returns a map counting the frequency of each color
// in the image on the canvas
function getColors(c) {
var col, colors = {};
var pixels, r, g, b, a;
r = g = b = a = 0;
pixels = c.getImageData(0, 0, c.width, c.height);
for (var i = 0, data = pixels.data; i < data.length; i += 4) {
r = data[i];
g = data[i + 1];
b = data[i + 2];
a = data[i + 3]; // alpha
// skip pixels >50% transparent
if (a < (255 / 2))
continue;
col = rgbToHex(r, g, b);
if (!colors[col])
colors[col] = 0;
colors[col]++;
}
return colors;
}
function rgbToHex(r, g, b) {
if (r > 255 || g > 255 || b > 255)
throw "Invalid color component";
return ((r << 16) | (g << 8) | b).toString(16);
}
getColors returns a map of color names and counts. Transparent pixels are skipped. It should be trivial to get the most-frequently seen color from this map.
If you literally want an average of each color component, you could easily get that from the results of getColors, too, but the results aren't likely to be very useful. This answer explains a much better approach.
You can use it all like this:
// nicely formats hex values
function pad(hex) {
return ("000000" + hex).slice(-6);
}
// see this example working in the fiddle below
var info = document.getElementById("info");
var img = document.getElementById("squares");
var colors = getColors(draw(img));
for (var hex in colors) {
info.innerHTML += "<li>" + pad(hex) + "->" + colors[hex];
}
See a working example.

Put image on canvas.
Get 2D context.
Loop through pixels, and store each r,g,b value. If you find the same, increment it once.
Loop through stored r,g,b values and take note of largest r,g,b value.
Convert r,g,b to hex.

This is only possible using the canvas tag as described here :
http://dev.opera.com/articles/view/html-5-canvas-the-basics/#pixelbasedmanipulation
Of course this is only available in newer browsers

You might consider using the convolution filters css allows you to apply. This might be able to get the effect you're going for ( assuming you're wanting to present it back into the html). So you could display the image twice , one convolved.
That being said, doesn't really work if you need the information yourself for some purpose.

For finding that average color:
Put Image on Canvas
Resize image to 1px by 1px
Get the color of the resulting pixel(This pixel will be the calculated average)

Why is setting HTML5's CanvasPixelArray values ridiculously slow and how can I do it faster?

I am trying to do some dynamic visual effects using the HTML 5 canvas' pixel manipulation, but I am running into a problem where setting pixels in the CanvasPixelArray is ridiculously slow.
For example if I have code like:
imageData = ctx.getImageData(0, 0, 500, 500);
for (var i = 0; i < imageData.length; i += 4){
imageData.data[i] = buffer[i];
imageData.data[i + 1] = buffer[i + 1];
imageData.data[i + 2] = buffer[i + 2];
}
ctx.putImageData(imageData, 0, 0);
Profiling with Chrome reveals, it runs 44% slower than the following code where CanvasPixelArray is not used.
tempArray = new Array(500 * 500 * 4);
imageData = ctx.getImageData(0, 0, 500, 500);
for (var i = 0; i < imageData.length; i += 4){
tempArray[i] = buffer[i];
tempArray[i + 1] = buffer[i + 1];
tempArray[i + 2] = buffer[i + 2];
}
ctx.putImageData(imageData, 0, 0);
My guess is that the reason for this slowdown is due to the conversion between the Javascript doubles and the internal unsigned 8bit integers, used by the CanvasPixelArray.
Is this guess correct?
Is there anyway to reduce the time spent setting values in the CanvasPixelArray?

Try caching a reference to the data pixel array. Your slowdown could be attributed to the additional property accesses to imageData.data. See this article for more explanation.
E.g. This should be faster that what you currently have.
var imageData = ctx.getImageData(0, 0, 500, 500),
data = imageData.data,
len = data.length;
for (var i = 0; i < len; i += 4){
data[i] = buffer[i];
data[i + 1] = buffer[i + 1];
data[i + 2] = buffer[i + 2];
}
ctx.putImageData(imageData, 0, 0);

I don't know if this helps you because you want to manipulate pixels, but for me, in Firefox 3.6.8, just the call to putImageData was very, very slow, without doing any pixel manipulation. In my case, I just wanted to restore a previous version of the image that had been saved with getImageData. Too slow.
Instead, I got it to work well using toDataUrl/drawImage instead. For me it's working fast enough that I can call it within handling a mousemove event:
To save:
savedImage = new Image()
savedImage.src = canvas.toDataURL("image/png")
The to restore:
ctx = canvas.getContext('2d')
ctx.drawImage(savedImage,0,0)

Looks like you're doing some kind of "blitting", so maybe drawImage or all-at-once putImageData could help. Looping a quarter million times to copy pixels individually, rather than using massive "blitting" operations, tends to be much slower -- and not just in Javascript;-).

Oddly, loops through 2d object arrays are faster than a 1d array offset calcs and no objects. Format accordingly and see if that helps (in my tests, it was 20x faster).
(heads up: this script could crash your browser! If you run it, sit tight for few minutes and let it do its thing)
http://jsfiddle.net/hc52jx04/16/
function arrangeImageData (target) {
var imageCapture = target.context.getImageData(0, 0, target.width, target.height);
var imageData = {
data: []
};
imageData.data[0] = [];
var x = 0;
var y = 0;
var imageLimit = imageCapture.data.length;
for (var index = 0; index < imageLimit; index += 4) {
if (x == target.width) {
y++;
imageData.data[y] = [];
x = 0;
}
imageData.data[y][x] = {
red: imageCapture.data[index],
green: imageCapture.data[index + 1],
blue: imageCapture.data[index + 2],
alpha: imageCapture.data[index + 3]
};
x++;
}
return imageData;
}
function codifyImageData (target, data) {
var imageData = data.data;
var index = 0;
var codedImage = target.context.createImageData(target.width, target.height);
for (var y = 0; y < target.height; y++) {
for (var x = 0; x < target.width; x++) {
codedImage.data[index] = imageData[y][x].red;
index++;
codedImage.data[index] = imageData[y][x].green;
index++;
codedImage.data[index] = imageData[y][x].blue;
index++;
codedImage.data[index] = imageData[y][x].alpha;
index++;
}
}
return codedImage;
}
More information: http://discourse.wicg.io/t/why-a-straight-array-for-canvas-getimagedata/1020/6