We Keep Coding

Best way to transform mouse coordinates to HTML5 Canvas's transformed context

I am testing to see if the mouse is located on an object. The problem is the object has been transformed. I have graph of objects, mainly the camera, then the slider object, and finally the shape object. I need to be able to see if the mouse coordinates are inside a specified rectangle relative to the shape object.
Here I have my game loop which transforms the clears the canvas then transforms the camera. I then go into a for loop and loop through all the objects calling their specific "draw" method, passing in the context that has been transformed.
Game.prototype.gameLoop = function()
{
this.context.clearRect(0,0,this.canvas.width, this.canvas.height);
this.context.save();
this.context.translate(this.canvas.width/2, this.canvas.height/2);
this.context.scale(this.camera.scale,this.camera.scale);
this.context.rotate(this.camera.rotate);
this.context.translate(this.camera.x,this.camera.y);
for(var i=0;i<this.objects.length;i++)
{
this.objects[i].update();
this.objects[i].draw(this.context);
}
this.context.restore();
}
Here is one of the objects draw method. The object is called a Slider. It successfully is called and performs a transformation based on it's x,y, and rotate values.
Slider.prototype.draw = function(ctx)
{
ctx.save();
ctx.translate(this.x,this.y);
ctx.rotate(this.rotate);
this.pointer.draw(ctx);
ctx.fillStyle = "black";
ctx.beginPath();
ctx.moveTo(-(this.width/2),0);
ctx.lineTo((this.width/2),0);
ctx.lineTo((this.width/2),5);
ctx.lineTo(-(this.width/2),5);
ctx.fill();
ctx.restore();
}
Finally I have the Shape's draw method which successfully is called and transforms the context yet again.
Shape.prototype.draw = function(ctx)
{
ctx.save();
ctx.translate(this.x,this.y);
ctx.rotate(this.rotate);
if(this.isMouseOver)
ctx.fillStyle = this.color;
else
ctx.fillStyle = this.mouseOverFillColor;
ctx.fill(this.shape);
ctx.restore();
}
And lastly, here is the method that gets called when the mouse moves called "mouseEventListener". I need to be able to transform the coordinates to see them relative to the shape.
Shape.prototype.mouseEventListener = function(evt,type)
{
console.log(evt.clientX+" "+evt.clientY);
}
Any ideas? If needed I can create a parent pointer object and have the shape point to the slider and the slider point to the camera to access each parent's x,y, rotate vales.
I am kind of looking for the equivalent of Android's mappoints method, which transforms points based off a matrix. In this case the context has been transformed multiple times and I need a way to capture that state for each object, and then transform some points.
I would also like to do all this easily without any other libraries.
Thank you.

Extract objects of specific colors and convert to paths

I have an idea for a project but I've hit a wall with the development of it. Essentially I would like to use JavaScript to analyze an image (any image), take a specific color and map it onto a HTML5 canvas as a path. All of the analyzed images will be basic shapes of different colors. This is an example of what I would like it to do.
This example is what I would like the script to do when passed the purple hex reference. Passing it the blue one would change the canvas output to just show the 2 blue SHAPES. Does anyone have any ideas on how this can be achieved without the use of a plugin? The canvas output will at some point be manipulated so each shape will need to be its own separate path (as they won't always be 4 sided shapes).

Since the question is quite broad as it stands I will not provide code, but a general approach on how to achieve this.
These are the steps that needs to be taken:
First pass: reduce the image based on a color and tolerance. If the color is absolute just iterate and create alpha channel where the pixel does not match the color. For tolerance a better approach would be to use RGB-HSL conversion, then define a radius and check if the color read is within the radius at the target color. Also consider alpha channel values.
This will leave an image with an alpha channel and only the colors that you are after.
Second pass: Run the image through a solution using Marching Squares algorithm (shameless plug: I made my own here (MIT license) inspired by this question, and it seem to be faster than the others incl. the D3 plugin - but anyone will do!). Extract the paths by iterating over the image, for each iteration remove the traced part. You do this by stroking+filling the obtained path using composition mode destination-out. Use a line width of about 3-5 specific to your scenario.
You can use Ramer-Douglas-Peucker to reduce points or leave them as they are. No point-reduction will allow for an accurate path but will also perform worse.
Third pass: Now you have path data that you can use to clip the parts from the original image. Add all the path data (use sub-paths by using moveTo() for each path), then composition mode to remove pixels you don't want. Or, if you're only after the paths: you're done!

To get color components of the pixel of coordinates (x, y) from a canvas, just do:
// assuming second canvas is same dimensions as first one
var secondCanvasId = context.createImageData(canvas.width, canvas.height);
for (var x = 0; x < canvas.width; x++) {
for (var y = 0; y < canvas.height; y++) {
var pix = context.getImageData(x, y, 1, 1).data;
var r = pix[0];
var g = pix[1];
var b = pix[2];
var a = pix[3];
// set canvas2 (x, y) pixel with this color, if it matches the choosen color
if (r === color.r && g === color.g && b === color.b && a === color.a) {
secondCanvasId.data[0] = r;
secondCanvasId.data[1] = g;
secondCanvasId.data[2] = b;
secondCanvasId.data[3] = a;
context.putImageData(secondCanvasId, x, y);
}
}
}

Is it possible to detect if a mouse is over a text in html5 canvas? [duplicate]

I am building a web application that draws a set of letters in different fonts on an HTML 5 Canvas using fillText. The user will click somewhere on that canvas and I need to check which letter they clicked on (or if they clicked on a letter at all).
I think I will need to:
Get the vector path for each letter (I have no clue how to do this).
Check if the click point is inside the letter path using some simple collision-detection algorithm.
Is there some easy function to do this that I am missing? Or maybe a library for things like this? If there aren't any libraries, how do I get the path for the letter in a specific font to do the checking myself?
I need to use the actual shape of the letter and not just its bounding box as I don't want the user to be able to click in the middle of an O and it register as a hit.
Any hints in this direction would be appreciated.

Logic
You can't handle separate letters on canvas without providing custom logic for it. Everything drawn to the canvas is merged to a soup of pixels.
And unfortunately you can't add text as pure path so you will have to check pixel values. Otherwise you could simply add the text to a new path and use the isPointInPath method for each letter.
One approach
We can't provide full solutions here on SO but here is a basis you can hopefully built on top of to provide basic logic to click single letters on a canvas:
Each letter is stored as object incl. its position, size, font and char, but also with a rectangle hit region (see below)
Define an array with these objects and then pass them to a render function
When you register a click iterate through the array and test against the rectangular hit-region and if inside check the pixel (*)
*) To distinguish between overlapping letters you need to check by priority. You can also render this char onto a separate canvas so you get pixels of only this char. I am not showing this in the demo but you'll get the idea.
Demo
var ltrs = []; /// stores the letter objects
/// Create some random objects
for(;i < 20; i++) {
/// build the object
var o = {char: alpha[((alpha.length - 1) * Math.random())|0],
x: ((w - 20) * Math.random())|0,
y: ((h - 20) * Math.random())|0,
size: (50 * Math.random() + 16)|0,
font: fonts[((fonts.length - 1) * Math.random())|0]};
/// store other things such as color etc.
/// store it in array
ltrs.push(o);
}
Then we have some function to render these characters (see demo).
When we then handle clicks we iterate through the object array and check by boundary first to check what letter we are at (picking just a pixel here won't enable us to ID the letter):
demo.onclick = function(e) {
/// adjust mouse position to be relative to canvas
var rect = demo.getBoundingClientRect(),
x = e.clientX - rect.left,
y = e.clientY - rect.top,
i = 0, o;
/// iterate
for(;o = ltrs[i]; i++) {
/// is in rectangle? "Older" letters has higher priority here...
if (x > o.x && x < (o.x + o.rect[2]) &&
y > o.y && y < (o.y + o.rect[3])) {
/// it is, check if we actually clicked a letter
/// This is what you would adopt to be on a separate canvas...
if (checkPixel(x, y) === true) {
setLetterObject(o, '#f00')
return;
}
}
}
}
The pixel check is straight forward, it picks a single pixel at x/y position and checks its alpha value (or color if you use solid backgrounds):
function checkPixel(x, y) {
var data = ctx.getImageData(x, y, 1, 1).data;
return (data[3] !== 0);
}
CLICK HERE FOR ONLINE DEMO
Updated check pixel function:
This updated check is capable of checking letters even if they are overlapping in the same region.
We create a separate canvas to draw the letter in. This isolates the letter and when we pick a pixel we can only get a pixel from that specific letter. It also doesn't matter what background color is as we our off-screen canvas only set pixels for the letter and not background during the check. The overhead is minimal.
function checkPixel(o, x, y) {
/// create off-screen canvas
var oc = document.createElement('canvas'),
octx = oc.getContext('2d'),
data,
oldX = o.x,
oldY = o.y;
/// default canvas is 300x150, adjust if letter size is larger *)
//oc.width = oc.height = 200;
/// this can be refactored to something better but for demo...
o.x = 0;
o.y = 0;
setLetterObject(octx, o, '#000');
o.x = oldX;
o.y = oldY;
data = octx.getImageData(x - oldX, y - oldY, 1, 1).data;
return (data[3] !== 0);
}
*) When we create a canvas the default size is 300x150. To avoid re-allocating a new bitmap we just leave it as it is as the memory is already allocated for it and we only need to pick a single pixel from it. If letters has larger pixel size than the default size we will of course need to re-allocate to make the letter fit.
In this demo we temporary override the x and y position. For production you should enable the setLetterObject method to somehow override this as that would be more elegant. But I'll leave it as-is here in the demo as the most important thing is to understand the principle.

I'd say that the best option is to actually use pixels that by the way are the most accurate thing you can do (remember that the user is seeing pixels when clicking, nothing more).
Since you cannot just use the color directly (because there can be many text objects with the same color (and may be also other primitives with the same color) you could use instead a separate "pick" canvas for that.
Basically when you draw your objects on the main canvas on the repaint function you also draw them in another hidden canvas with the exact same size, but you draw them using a unique color for each entity.
You can therefore have up to 16 millions entity (24-bit) on the canvas and know instantly which one was clicked by keeping a map between color code and the entity itself. By the way this sort of map is often used in CAD applications exactly to speed up picking.
The only somewhat annoying part is that there's no portable way to disable antialiasing when drawing on a canvas so it's possible that the color you'll get back from the pick canvas is not one of the colors you used or, even worse, it's possible that by clicking on the border of an entity a different unrelated entity is considered selected.
This should be a very rare event unless your display is really really crowded and picking is basically random anyway.

Events attached to object inside canvas

I have simply canvas code which draw rect on the canvas
var x=document.getElementById("canvas");
var ctx=x.getContext("2d");
ctx.rect(20,20,150,100);
ctx.stroke();
is it possible to add eventListener on said rect? For example, if I click on rect, it will turn red.

Regions
Depending on how well you want to support various and older browsers, there is addHitRegion() that you can use by enabling it through flags in Firefox and Chrome (at the moment of this being written):
Firefox: about:config -> search "hitregions" and set to true
Chrome: chrome://flags/ -> Enable experimental canvas features
This is the only technique that integrates directly with the event system. I would not recommend it for production quite yet though, and AFAIK there is not a polyfill for it either - but to show how easy it is to use:
var x=document.getElementById("canvas");
var ctx=x.getContext("2d");
ctx.rect(20,20,150,100);
ctx.addHitRegion({id: "demo"}); // enable in flags in Chrome/Firefox
ctx.stroke();
x.addEventListener("click", function(e) {
if (e.region && e.region === "demo") alert("Hit!");
})
<canvas id="canvas"></canvas>
Path: isPointInPath
The other techniques require one to manually implement a mechanism for hit-detection. One is by using isPointInPath(). You simply rebuild the paths you want to test, one by one, then run your (adjusted) x/y mouse coordinate against it:
var x=document.getElementById("canvas");
var ctx=x.getContext("2d");
generatePath();
ctx.stroke();
x.addEventListener("click", function(e) {
var r = this.getBoundingClientRect(),
x = e.clientX - r.left,
y = e.clientY - r.top;
// normally you would loop through your paths:
generatePath();
if (ctx.isPointInPath(x, y)) alert("Hit!");
})
function generatePath() {
ctx.beginPath(); // reset path
ctx.rect(20,20,150,100); // add region to draw/test
}
<canvas id="canvas"></canvas>
Path: Path2D objects
For the latter example there is also the new Path2D objects which can hold a path on their own - the advantage here is that you don't need to rebuild the paths, just pass in the path object with x/y to the isPointInPath() method.
The problem is that Path2D is not supported in all browsers yet, but there is this polyfill that will fix that for you,
var x=document.getElementById("canvas");
var ctx=x.getContext("2d");
var path1 = new Path2D();
path1.rect(20,20,150,100); // add rect to path object
ctx.stroke(path1);
x.addEventListener("click", function(e) {
var r = this.getBoundingClientRect(),
x = e.clientX - r.left,
y = e.clientY - r.top;
// normally you would loop through your paths objects:
if (ctx.isPointInPath(path1, x, y)) alert("Hit!");
})
<canvas id="canvas"></canvas>
Manually check boundary
And of course, there is the old technique of using manual boundary checks. This will work in all browsers. Here the advisable thing to do is to create objects that holds the bounds and can also be used to render it. This typically limits you to rectangular areas - more complex shapes will require more complex algorithms (such as the isPointInPath() embeds).
var x=document.getElementById("canvas");
var ctx=x.getContext("2d");
ctx.rect(20,20,150,100);
ctx.stroke();
x.addEventListener("click", function(e) {
var r = this.getBoundingClientRect(),
x = e.clientX - r.left,
y = e.clientY - r.top;
// normally you would loop through your region objects:
if (x >= 20 && x < 20+150 && y >= 20 && y < 20+100) alert("Hit!");
})
<canvas id="canvas"></canvas>

Shapes and paths are drawn to the canvas as side-effects, so there is no element to add an event listener to; you could, however, add an event listener to the entire canvas or to an element that shares a location with the canvas, and when it is clicked then redraw the canvas with the rectangle, but red (or anything else changed). (make sure to clear the canvas before redrawing it with the .clearRect() method).

If you draw something to a canvas, the shape that is drawn is not a javascript object, but rather changes the particular state that the canvas is in. Therefore, you cannot attach an event listener to it, and should instead attach the event to the canvas itself.
Your javascript could then check the co-ordinates of the click, and find whether or not it is inside the rectangle. Bear in mind that if you draw something on top of the rectangle or shape, the code will have to be adjusted to check the new area formed. You might also find it difficult to check the area if it is not a rectangle, but it will still be possible.
If you want to redraw the rectangle as red, you should repaint the canvas, changing the colour of the new rectangle that you redraw (the rectangle is not an object, so you cannot change the colour directly). This would also involve repainting all the other shapes on the canvas.

Text Collision Detection

I am building a web application that draws a set of letters in different fonts on an HTML 5 Canvas using fillText. The user will click somewhere on that canvas and I need to check which letter they clicked on (or if they clicked on a letter at all).
I think I will need to:
Get the vector path for each letter (I have no clue how to do this).
Check if the click point is inside the letter path using some simple collision-detection algorithm.
Is there some easy function to do this that I am missing? Or maybe a library for things like this? If there aren't any libraries, how do I get the path for the letter in a specific font to do the checking myself?
I need to use the actual shape of the letter and not just its bounding box as I don't want the user to be able to click in the middle of an O and it register as a hit.
Any hints in this direction would be appreciated.

Logic
You can't handle separate letters on canvas without providing custom logic for it. Everything drawn to the canvas is merged to a soup of pixels.
And unfortunately you can't add text as pure path so you will have to check pixel values. Otherwise you could simply add the text to a new path and use the isPointInPath method for each letter.
One approach
We can't provide full solutions here on SO but here is a basis you can hopefully built on top of to provide basic logic to click single letters on a canvas:
Each letter is stored as object incl. its position, size, font and char, but also with a rectangle hit region (see below)
Define an array with these objects and then pass them to a render function
When you register a click iterate through the array and test against the rectangular hit-region and if inside check the pixel (*)
*) To distinguish between overlapping letters you need to check by priority. You can also render this char onto a separate canvas so you get pixels of only this char. I am not showing this in the demo but you'll get the idea.
Demo
var ltrs = []; /// stores the letter objects
/// Create some random objects
for(;i < 20; i++) {
/// build the object
var o = {char: alpha[((alpha.length - 1) * Math.random())|0],
x: ((w - 20) * Math.random())|0,
y: ((h - 20) * Math.random())|0,
size: (50 * Math.random() + 16)|0,
font: fonts[((fonts.length - 1) * Math.random())|0]};
/// store other things such as color etc.
/// store it in array
ltrs.push(o);
}
Then we have some function to render these characters (see demo).
When we then handle clicks we iterate through the object array and check by boundary first to check what letter we are at (picking just a pixel here won't enable us to ID the letter):
demo.onclick = function(e) {
/// adjust mouse position to be relative to canvas
var rect = demo.getBoundingClientRect(),
x = e.clientX - rect.left,
y = e.clientY - rect.top,
i = 0, o;
/// iterate
for(;o = ltrs[i]; i++) {
/// is in rectangle? "Older" letters has higher priority here...
if (x > o.x && x < (o.x + o.rect[2]) &&
y > o.y && y < (o.y + o.rect[3])) {
/// it is, check if we actually clicked a letter
/// This is what you would adopt to be on a separate canvas...
if (checkPixel(x, y) === true) {
setLetterObject(o, '#f00')
return;
}
}
}
}
The pixel check is straight forward, it picks a single pixel at x/y position and checks its alpha value (or color if you use solid backgrounds):
function checkPixel(x, y) {
var data = ctx.getImageData(x, y, 1, 1).data;
return (data[3] !== 0);
}
CLICK HERE FOR ONLINE DEMO
Updated check pixel function:
This updated check is capable of checking letters even if they are overlapping in the same region.
We create a separate canvas to draw the letter in. This isolates the letter and when we pick a pixel we can only get a pixel from that specific letter. It also doesn't matter what background color is as we our off-screen canvas only set pixels for the letter and not background during the check. The overhead is minimal.
function checkPixel(o, x, y) {
/// create off-screen canvas
var oc = document.createElement('canvas'),
octx = oc.getContext('2d'),
data,
oldX = o.x,
oldY = o.y;
/// default canvas is 300x150, adjust if letter size is larger *)
//oc.width = oc.height = 200;
/// this can be refactored to something better but for demo...
o.x = 0;
o.y = 0;
setLetterObject(octx, o, '#000');
o.x = oldX;
o.y = oldY;
data = octx.getImageData(x - oldX, y - oldY, 1, 1).data;
return (data[3] !== 0);
}
*) When we create a canvas the default size is 300x150. To avoid re-allocating a new bitmap we just leave it as it is as the memory is already allocated for it and we only need to pick a single pixel from it. If letters has larger pixel size than the default size we will of course need to re-allocate to make the letter fit.
In this demo we temporary override the x and y position. For production you should enable the setLetterObject method to somehow override this as that would be more elegant. But I'll leave it as-is here in the demo as the most important thing is to understand the principle.

I'd say that the best option is to actually use pixels that by the way are the most accurate thing you can do (remember that the user is seeing pixels when clicking, nothing more).
Since you cannot just use the color directly (because there can be many text objects with the same color (and may be also other primitives with the same color) you could use instead a separate "pick" canvas for that.
Basically when you draw your objects on the main canvas on the repaint function you also draw them in another hidden canvas with the exact same size, but you draw them using a unique color for each entity.
You can therefore have up to 16 millions entity (24-bit) on the canvas and know instantly which one was clicked by keeping a map between color code and the entity itself. By the way this sort of map is often used in CAD applications exactly to speed up picking.
The only somewhat annoying part is that there's no portable way to disable antialiasing when drawing on a canvas so it's possible that the color you'll get back from the pick canvas is not one of the colors you used or, even worse, it's possible that by clicking on the border of an entity a different unrelated entity is considered selected.
This should be a very rare event unless your display is really really crowded and picking is basically random anyway.