We Keep Coding

What's the fastest way to draw an image to the screen in JavaScript from array of RGB pixels?

I'm working on a graphics frontend that renders server-side and pushes screen updates to a browser by sending compressed images to the client (think VNC). I've decided the overhead of encoding PNGs is too high, so currently I'm sending raw blobs of 8-bit RGB pixel values through a websocket (with compression enabled). This is actually very fast and I'm seeing great compression gains (75K -> 2.7k for example).
On the client however, I have to take the raw pixels and then draw them onto a canvas. This is my current best code performance wise:
// receive message from server
self.ws.onmessage = function (evt) {
// get image offset
var dv = new DataView(evt.data);
var dx = dv.getInt16(0);
var dy = dv.getInt16(2);
var ww = dv.getInt16(4);
var hh = dv.getInt16(6);
var offset = 8;
// get context to canvas and create image
var ctx = self.canvas.getContext("2d");
var img = ctx.createImageData(ww, hh);
// unpack image data
var start = performance.now();
var dst = 0, src = offset;
for (var ii=0; ii < ww*hh; ii++) {
img.data[dst++] = dv.getUint8(src++);
img.data[dst++] = dv.getUint8(src++);
img.data[dst++] = dv.getUint8(src++);
img.data[dst++] = 255;
}
// draw image
ctx.putImageData(img, dx, dy, 0, 0, ww, hh);
var end = performance.now();
console.log("draw took " + (end-start) + " milliseconds");
The aforementioned 75K image (which is 1000x500 pixels) takes ~53ms to render in this way though, which is a long time. What's the fastest way to do this drawing operation? I can output RGBA pixels instead of that makes life easier.
Edit: Seems like this might be more of a Firefox issue, Chrome runs this same decode loop in ~2.5ms on average.

Switching to full RGBA output (which doesn't add much overhead thanks to the compression), and using this code to directly wrap the websocket buffer is significantly faster:
// receive message from server
self.ws.onmessage = function (evt) {
// get image offset
var dv = new DataView(evt.data);
var dx = dv.getInt16(0);
var dy = dv.getInt16(2);
var ww = dv.getInt16(4);
var hh = dv.getInt16(6);
// get context to canvas and create image
var ctx = self.canvas.getContext("2d");
// draw image data
var start = performance.now();
ctx.putImageData(
new ImageData(new Uint8ClampedArray(evt.data, 8), ww, hh),
dx, dy,
0, 0,
ww, hh
);
var end = performance.now();
console.log("draw took " + (end-start) + " milliseconds");
}
Now the large image size takes ~1ms to render in Firefox and 350us in Chrome. Going to call that good enough.

HTML5 Canvas works and disappear when click anywhere

In my following code, Canvas works perfectly and it shows on my map then it disappears when I tried to click anywhere.
I tried all the possible ways which submitted in (StackOverflow) here as solutions but no way, maybe my code has something error which causes that.
html code:
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/openlayers/4.4.2/ol.css" type="text/css">
<script src="https://cdnjs.cloudflare.com/ajax/libs/openlayers/4.4.2/ol.js"></script>
<canvas id="myCanvas" width="1000" height="500"></canvas>
<button type="button" onclick="evt();">resolution</button>
js code:
image = new ol.layer.Tile({
source: new ol.source.XYZ({
projection: 'EPSG:4326',
wrapX: false,
url: image/{z}/{x}/{-y}.png'
})
});
map.addLayer(image);
function evt() {
var canvasContext = $('.ol-unselectable')[0].getContext('2d');
var canvas = document.getElementById('myCanvas');
var imgData = canvasContext.getImageData(0, 0, canvas.width, canvas.height);
var imageWidth = imgData.width;
var imageHeight = imgData.height;
var pix = imgData.data;
var l = pix.length;
var i;
for (i = 0; l > i; i += 4) {
if (pix[i] >= 100 && pix[i] <= 200 && pix[i + 1] >= 100 && pix[i + 1] <= 200 && pix[i + 2] >= 100 && pix[i + 2] <= 200) {
pix[i] = 255;
pix[i + 1] = 0;
pix[i + 2] = 0;
}
}
canvasContext.putImageData(imgData, 0, 0);
};

We have an incomplete view of your application so some of the following remarks are going to rely on assumptions.
You're selecting the first element with a class .ol-unslectable, which is presumably a HTML Canvas.
Then you're running through the pixels of that Canvas and changing colour values depending on whether a particular condition is met.
Then you're putting the pixels back onto the canvas you took them from.
There is much in your code which is unnecessary.
The following variables..
canvas
imageWidth
imageHeight
red
green
blue
x
y
... play no part in the function.
Your imgData variable holds the pixels of an image at the same dimensions as the #myCanvas elemnt, where more usually one would take those dimensions from the source image - after all it is these pixels which are being manipulated here.
If we remove all extraneous lines from the function it will look like this and still perform the same task.
/*
manipulate the pixels of $('.ol-unselectable')[0]
*/
function evt (e) {
var cnv = $('.ol-unselectable')[0],
ctx = cnv.getContext('2d'),
imgData = ctx.getImageData(0, 0, cnv.width, cnv.height),
pix = imgData.data,
l = pix.length,
i = 0;
for (; i < l; i += 4) {
[
pix[i],
pix[i + 1],
pix[i + 2]
].every(function (val) {
return val >= 100 && val <= 200;
}) && (
pix[i] = 255,
pix[i + 1] = 0,
pix[i + 2] = 0
);
}
ctx.putImageData(imgData, 0, 0);
}
I would also suggest that you register an eventListener to your button element rather than relying on an onclick HTML attribute. Assign it a unique id ...
<button id="reso">resolution</button>
... and then register an eventListener once the DOM is ready (the JQuery way)...
$("#reso").on("click", evt);
Now, I'm aware that this may not be the help you were looking for, but the irregular effects you describe are not apparent from your example code. I'm pretty sure you need those extra variables to do something - but from the question it isn't clear exactly what that 'something' might be.
Nonetheless, at least you now have a clean function that performs it's single task a little more efficiently. ;)
More on Array.prototype.every # MDN.
Canvas Pixel manipulation # MDN

I got the solution, for Canvas on OpenLayers Map, should add to the end of the function
map.on('postcompose', function(event){
evt(event.context);
});
Thank you all..

Check an image for off-white background

I'm writing a script for work where we have a bunch of images of jewelry 200x200 and the script gets all of the images on a page and creates a canvas and then checks the pixels on the edge for discoloration (they're supposed to be pure white) due to them not being edited correctly.
I started off checking the upper left and upper right corners for accuracy, but now i'm running into items where part of the necklace or whatever can go all the way to the corner or off the side which makes this inaccurate.
How do you recommend I go about this? What I'm doing now is checking if the sum of the rgba values are 1020 for both pixels, and if they aren't, then the image isn't pure white.
There are two possible defects with images: total background discoloration and a grey border around the edge. checking the corner pixels works for the grey border but not for the background if the item extends to the corners/sides.

Check all 4 corners of the image. If at least 1 of the 4 corners is white / 255,255,255 / #FFFFFF, the image is probably okay. (The discolouration should be consistent across the image, right?)
Other than that, there's not a lot you can do to check for the discolouration. However, you could count colours in the image, and check if the colour that occurs most, is in fact white:
<canvas id="canvas" width="300px" height="300px"></canvas>
var canvas = document.getElementById("canvas"),
canvasWidth = canvas.width,
canvasHeight = canvas.height,
c = canvas.getContext("2d"),
img = new Image();
img.src = '/images/favicon.png';
img.onload = drawImage;
function drawImage(){
// Prepare the canvas
var ptrn = c.createPattern(img, 'repeat');
c.fillStyle = "white";
c.fillRect(0,0,canvasWidth,canvasHeight);
c.fillStyle = ptrn;
c.fillRect(0,0,canvasWidth,canvasHeight);
// Get img data
var imgData = c.getImageData(0, 0, canvasWidth, canvasHeight),
data = imgData.data,
colours = {};
// Build an object with colour data.
for (var y = 0; y < canvasHeight; ++y) {
for (var x = 0; x < canvasWidth; ++x) {
var index = (y * canvasWidth + x) * 4,
r = data[index], // Red
g = data[++index], // Green
b = data[++index], // Blue
// a = data[++index], // Alpha
rgb = rgbToHex(r,g,b);
if(colours[rgb]){
colours[rgb]++;
}else{
colours[rgb] = 1;
}
}
}
// Determine what colour occurs most.
var most = {
colour:'',
amount:0
};
for(var colour in colours){
if(colours[colour] > most.amount){
most.amount = colours[colour];
most.colour = colour;
}
}
console.log("Highest occurence:",most,
"\nColours: ",colours);
}
function rgbToHex(r, g, b) {
return "#" + ((1 << 24) + (r << 16) + (g << 8) + b).toString(16).slice(1);
}

Recognize a red ball's position in an image with JavaScript?

Say I have this image:
I'd like to recognize the position of the red ball in the image, I could measure the size of the ball(in pixel) in ahead.
I know that I could draw the image to a canvas, then I could get the pixel color data with context.getImageData, but then what should I do? which algorithm sould I use? I'm new to image processing, thanks a lot.

Here's code dedicated to getting that ball position. The output position will logged to the console so have your JS console open! This code has some values in it that you can play with. I chose some that work for your image such as the rough diameter of the ball being 14 pixels and the threshold for each colour component.
I saved the image as "test.jpg" but you can change the code to the correct image path on line 11.
<!DOCTYPE html>
<html>
<body>
<canvas width="800" height="600" id="testCanvas"></canvas>
<script type="text/javascript">
var img = document.createElement('img');
img.onload = function () {
console.log(getBallPosition(this));
};
img.src = 'test.jpg';
function getBallPosition(img) {
var canvas = document.getElementById('testCanvas'),
ctx = canvas.getContext('2d'),
imageData,
width = img.width,
height = img.height,
pixelData,
pixelRedValue,
pixelGreenValue,
pixelBlueValue,
pixelAlphaValue,
pixelIndex,
redThreshold = 128,
greenThreshold = 40,
blueThreshold = 40,
alphaThreshold = 180,
circleDiameter = 14,
x, y,
count,
ballPosition,
closestBallCount = 0,
closestBallPosition;
// Draw the image to the canvas
canvas.width = width;
canvas.height = height;
ctx.drawImage(img, 0, 0);
// Get the image data
imageData = ctx.getImageData(0, 0, width, height);
pixelData = imageData.data;
// Find the ball!
for (y = 0; y < height; y++) {
// Reset the pixel count
count = 0;
// Loop through the pixels on this line
for (x = 0; x < width; x++) {
// Set the pixel data starting point
pixelIndex = (y * width * 4) + (x * 4);
// Grab the red pixel value
pixelRedValue = pixelData[pixelIndex];
pixelGreenValue = pixelData[pixelIndex + 1];
pixelBlueValue = pixelData[pixelIndex + 2];
pixelAlphaValue = pixelData[pixelIndex + 3];
// Check if the value is within out red colour threshold
if (pixelRedValue >= redThreshold && pixelGreenValue <= greenThreshold && pixelBlueValue <= blueThreshold && pixelAlphaValue >= alphaThreshold) {
count++;
} else {
// We've found a pixel that isn't part of the red ball
// so now check if we found any red data
if (count === circleDiameter) {
// We've found our ball
return {
x: x - Math.floor(circleDiameter / 2),
y: y
};
} else {
// Any data we found was not our ball
if (count < circleDiameter && count > closestBallCount) {
closestBallCount = count;
closestBallPosition = {
x: x - Math.floor(circleDiameter / 2),
y: y
};
}
count = 0;
}
}
}
}
return closestBallPosition;
}
</script>
</body>
</html>

Well i would go and cluster pixels of that color. For example, you could have a look up table where you store red (or in the range of a treshold) pixels (coordinates being the look up key) and an integer value being the cluster id whenever you encounter a pixel without any known red neighbours it starts a new cluster, all other red pixels get the cluster id of a red pixel they are the neighbour of. Depending of you algorithms kernel:
A) XXX B) X
XOX XOX
XXX X
you might need to deal (case B) with a pixel connecting two prior not connected clusters. You would have to replace the cluster id of one of that clusters.
After that you have clusters of pixels. These you can analyse. In case of a round shape i would look for the median in x and y for each cluster and check if all the pixels of that cluster are in the radius.
This will fail if the red ball (or part of it) is in front of another red object. You would than need more complex algorithms.

How to get the average or main color from an image with javascript?

what i want is to the the HEX or the RGB average value from an image to the another div background this color.
So if i upload an image with a ot of red i get something like #FF0000 just as an example.
Let Me know if this is posible :)
Many thanks.

First, draw the image on a canvas:
function draw(img) {
var canvas = document.createElement("canvas");
var c = canvas.getContext('2d');
c.width = canvas.width = img.width;
c.height = canvas.height = img.height;
c.clearRect(0, 0, c.width, c.height);
c.drawImage(img, 0, 0, img.width , img.height);
return c; // returns the context
}
You can now iterate over the image's pixels. A naive approach for color-detection is to simply count the frequency of each color in the image.
// returns a map counting the frequency of each color
// in the image on the canvas
function getColors(c) {
var col, colors = {};
var pixels, r, g, b, a;
r = g = b = a = 0;
pixels = c.getImageData(0, 0, c.width, c.height);
for (var i = 0, data = pixels.data; i < data.length; i += 4) {
r = data[i];
g = data[i + 1];
b = data[i + 2];
a = data[i + 3]; // alpha
// skip pixels >50% transparent
if (a < (255 / 2))
continue;
col = rgbToHex(r, g, b);
if (!colors[col])
colors[col] = 0;
colors[col]++;
}
return colors;
}
function rgbToHex(r, g, b) {
if (r > 255 || g > 255 || b > 255)
throw "Invalid color component";
return ((r << 16) | (g << 8) | b).toString(16);
}
getColors returns a map of color names and counts. Transparent pixels are skipped. It should be trivial to get the most-frequently seen color from this map.
If you literally want an average of each color component, you could easily get that from the results of getColors, too, but the results aren't likely to be very useful. This answer explains a much better approach.
You can use it all like this:
// nicely formats hex values
function pad(hex) {
return ("000000" + hex).slice(-6);
}
// see this example working in the fiddle below
var info = document.getElementById("info");
var img = document.getElementById("squares");
var colors = getColors(draw(img));
for (var hex in colors) {
info.innerHTML += "<li>" + pad(hex) + "->" + colors[hex];
}
See a working example.

Put image on canvas.
Get 2D context.
Loop through pixels, and store each r,g,b value. If you find the same, increment it once.
Loop through stored r,g,b values and take note of largest r,g,b value.
Convert r,g,b to hex.

This is only possible using the canvas tag as described here :
http://dev.opera.com/articles/view/html-5-canvas-the-basics/#pixelbasedmanipulation
Of course this is only available in newer browsers

You might consider using the convolution filters css allows you to apply. This might be able to get the effect you're going for ( assuming you're wanting to present it back into the html). So you could display the image twice , one convolved.
That being said, doesn't really work if you need the information yourself for some purpose.

For finding that average color:
Put Image on Canvas
Resize image to 1px by 1px
Get the color of the resulting pixel(This pixel will be the calculated average)