Pixel perfect 2D mouse picking with Canvas - javascript

I'm writing a 2D game in html5 using Canvas which requires mouse click and hover events to be detected. There are 3 problems with this: detections must be pixel-perfect, objects are not rectangular (houses, weird-shaped UI buttons...), and it is required to be fast and responsive. (Obviously brute force is not an option)
So what I want to ask is how do I find out which object the mouse is on, and what are the possible optimizations.
P.S: I did some investigation and found a guy who used QuadTree here.

I have a (dated) tutorial that explains the concept of a ghost canvas which is decent for pixel-perfect hit detection. The tutorial is here. Ignore the warning about a newer tutorial, the newer one does not use the ghost canvas concept.
The idea is to draw the image in question to an in-memory canvas and then use getImageData to get the single pixel of the mouse click. Then you see if that single pixel is fully transparent or not.
If its not fully transparent, well, you've got your target.
If it is fully transparent, draw the next object to the in-memory canvas and repeat.
You only have to clear the in-memory canvas at the end.
getImageData is slow but it is your only option if you want pixel-perfect hit detection and aren't pre-computing anything.
Alternatively you could precompute a path or else an array of pixels with an offset. This would be a lot of work but might be faster. For instance if you have a 40x20 image with some transparency you'd compute an array[40][20] that would have true or false corresponding to transparent or not. Then you'd test that against the mouse position, with some offset, if the image is drawn at (25, 55) you'd want to subtract that from the mouse position and then test if the new position is true when you look at array[posx][posy].
That's my answer to your question. My Suggestion? Forget pixel-perfect detection if this is a game.
Seriously.
Instead make paths (not in canvas, in plain javascript code) that represent the objects but are not pixel perfect, for instance a house might be a square with a triangle on the top that is a very close approximation of the image but is used in its stead when it comes to hit testing. It is comparatively extremely fast to compute if a point is inside a path than it is to do pixel-perfect detection. Look up point in polygon winding number rule detection. That's your best bet, honestly.

The common solution in traditional game development is to build a click mask. You can re-render everything onto a separate off-screen canvas in a solid color (the rendering should be very quick). When you want to figure out what was clicked on, you simply sample the color at the x/y co-ordinate on the off-screen canvas. You end up building a color-->obj hash, akin to:
var map = {
'#000000' : obj1
, '#000001' : obj2
, ...
};
You can also optimize the rendering to the secondary canvas to only happen when the user clicks on something. And using various techniques, you can further optimize it to only draw the part of the canvas that the user has clicked on (for example, you can split you canvas into an NxN grid, e.g. a grid of 20x20 pixel squares, and flag all of the objects in that square -- you'd then only need to re-draw a small number of objects)

HTML5 Canvas is just a drawing plane, where you can set different transforms before calling each drawing API function. Objects cannot be created and there is no display list. So you have to build these features yourself or you can use different libraries available for this.
http://www.kineticjs.com/
http://easeljs.com/
A few months before I got interested in this and even wrote a library for this purpose. You can see it here : http://exsprite.com. Ended up facing a lot of performance issues, but because of lack of time I couldn't optimize it. It was really interesting, so waiting for some time to make it perfect.

I believe the comments should suffice. This is how I determine user intention in my 2d isometric scroller, currently located at http://untitled.servegame.com
var lastUp = 0;
function mouseUp(){
mousedown = false; //one of my program globals.
var timeNow = new Date().getTime();
if(mouseX == xmouse && mouseY == ymouse && timeNow > lastUp + 100){//if it was a centralized click. (mouseX = click down point, xmouse = mouse's most recent x) and is at least 1/10th of a second after the previous click.
lastUp = new Date().getTime();
var elem = document.elementFromPoint(mouseX, mouseY); //get the element under the mouse.
var url = extractUrl($(elem).css('background-image')); // function I found here: http://webdevel.blogspot.com/2009/07/jquery-quick-tip-extract-css-background.html
imgW = $("#hiddenCanvas").width(); //EVERY art file is 88px wide. thus my canvas element is set to 88px wide.
imgH = $(elem).css('height').split('p')[0]; //But they vary in height. (currently up to 200);
hiddenCanvas.clearRect(0, 0, imgW, imgH); //so only clear what is necessary.
var img = new Image();
img.src = url;
img.onload = function(){
//draw this elements image to the canvas at 0,0
hiddenCanvas.drawImage(img,0,0);
///This computes where the mouse is clicking the element.
var left = $(elem).css('left').split('p')[0]; //get this element's css absolute left.
var top = $(elem).css('top').split('p')[0];
offX = left - offsetLeft; //left minus the game rendering element's absolute left. gives us the element's position relative of document 0,0
offY = top - offsetTop;
offX = mouseX - offX; //apply the difference of the click point's x and y
offY = mouseY - offY;
var imgPixel = hiddenCanvas.getImageData(offX, offY, 1, 1); //Grab that pixel. Start at it's relative X and it's relative Y and only grab one pixel.
var opacity = imgPixel.data[3]; //get the opacity value of this pixel.
if(opacity == 0){//if that pixel is fully transparent
$(elem).hide();
var temp = document.elementFromPoint(mouseX, mouseY); //set the element right under this one
$(elem).show();
elem = temp;
}
//draw a circle on our hiddenCanvas so when it's not hidden we can see it working!
hiddenCanvas.beginPath();
hiddenCanvas.arc(offX, offY, 10, 0, Math.PI*2, true);
hiddenCanvas.closePath();
hiddenCanvas.fill();
$(elem).css("top", "+=1"); //apply something to the final element.
}
}
}
In conjunction with this:
<canvas id="hiddenCanvas" width="88" height="200"></canvas>
Set the CSS positioning absolute and x = -(width) to hide;

Related

Use of absolutePan() to move background image and elements around on canvas with touch event

I am using next code to force content of canvas to follow finger touch movement on screen (drag and drop).
document.addEventListener('touchmove', function(e){
if(e.targetTouches.length ==1) {
var canavasMovex=e.targetTouches[0].pageX-canvasLeftofset-canvas.width/2;
var canavasMovey=e.targetTouches[0].pageY-canvasTopofset-canvas.height/2;
document.getElementById("a").innerHTML= canavasMovex;
document.getElementById("b").innerHTML= canavasMovey;
var delta = new fabric.Point(-canavasMovex,-canavasMovey);
canvas.absolutePan(delta);
canvas.renderAll();
}
}, false);
Note: "-canavasMovex,-canavasMovey" values have the negative sign (I had to use it because otherwise the canvas was moving opposite to finger moment(I think you can ignore this fact and that it has noting to do with offset, which I am trying to solve asking for help here))
I have displacement of position proportional to the distance of touch point from the centre of canvas. If I start the movement pressing from the centre of screen, then displacement is not so notable. But if I start from top corner (for example) I get centre of canvas there in that touch point. From attached drawing I was trying with taking of consideration of "d" parameter (distance of touching point to the centre of canvas), to find right value for the equation for the canvas.absolutePan() function, but no success. Can you help me with this case,pleas? I was trying to use some solutions from browsing the internet to find the way to move canvas content. But then I had lost some of functions of my app because "new" libraries were not interacting correctly with my existing fabric.min.js library. Image info(external rectangle-web page,internal rectangle-canvas): a-canvas offset,b-distance to centre of canvas,c- touch point,d-distance from touch point to centre of canvas. So on the end I am trying to get to accomplish this task in this way.
Malfunction:
Bigger than distance "d" is on starting of dragging , bigger is then the offset. (actuary if I start dragging from canvas corner, canvas content transfers its centre there)
The solution is to get the finger position, calculate the difference to previous position, add difference to the previous position and then finally do the panning.
var canvasMovexoold = 0;
var canvasMoveyoold = 0;
document.addEventListener('touchstart', function(e){
canvasMovexoold = canvasLeftofset+canvas.width/2-e.targetTouches[0].pageX;
canvasMoveyoold =canvasTopofset+canvas.height/2- e.targetTouches[0].pageY;
}, false);
document.addEventListener('touchmove', function(e){
if(e.targetTouches.length ==1) {
var canvasMovexo += canvasLeftofset+canvas.width/2-e.targetTouches[0].pageX-canvasMovexoold ;
var canvasMoveyo +=canvasTopofset+canvas.height/2- e.targetTouches[0].pageY-canvasMoveyoold ;
var delta = new fabric.Point(canvasMovexo,canvasMoveyo);
canvas.absolutePan(delta);
canvasMovexoold = canvasLeftofset+canvas.width/2-e.targetTouches[0].pageX;
canvasMoveyoold =canvasTopofset+canvas.height/2- e.targetTouches[0].pageY;
}
}, false);

Canvas viewport [duplicate]

This question already has answers here:
HTML5 Canvas camera/viewport - how to actually do it?
(7 answers)
Closed 7 years ago.
Creating an Agar clone, and it pretty much works now, but I want to improve it by making it so if the player leaves the screen, the screen follows them, which is done with a viewport. Only problem is I have no idea how to do it. I tried ctx.translate() but that just resulted in some weird stuff. (granted, I'm using a regular background, and I've heard an actual image is required for it to work, but I don't know how to do that either, so...) Here's my jsFiddle. Initializing canvas-related variables:
var canvas = document.createElement("canvas");
canvas.width = innerWidth;
canvas.height = innerHeight;
canvas.style.display = "none";
document.body.appendChild(canvas);
var ctx = canvas.getContext("2d");
You use translate to set the origin. Where on the canvas, or off, the coordinates X = 0, y = 0 are.
Your character is wandering about a game world which is many times bigger than the canvas. You also have many other items and players each with their own coordinates in the game world..
One way to render this is... For each item in the game, find the distance you are from the origin and subtract that from each item as you draw them. Then draw your own character at the center of the screen. But that is slow, and requires all that extra math for each item/player.
So let the hardware do it with the transform. The transform holds the scales (x,y) the rotation (ang), and the translation (origin offset). Every time an object is drawn on the canvas the transformation matrices is applied to it, you can't avoid it, it has to happen. So if you dont use it to your advantage its just wasted.
How to use it.
Your playfield is top left -10,000,-10,000 pixels to bottom right 10,000, 10,000 pixels. Every one has a location on the map between those numbers, including your player, which may be at 5,000, 6,000 pixels. If you ctx.drawImage(playerImage,5000,6000) the image will not appear, its way of the bottom right of the canvas, but you want him in the center of the screen.
To do this move the origin (currently at the upper left corner of the canvas (0,0)) so you at 5000,6000 appear in the center.
var playerX = 5000; // your position
var playerY = 6000;
var screenWidth = 1920; // size of the screen
var screenHeight = 1080;
var originX = 0; // where the origin is
ver originY = 0
To center yourself move the origin so that it subtracts you position.
originX -= playerX;
originY -= playerY;
now you are at the top left corner of the canvas, but you want the center. So move it back a bit by half the screen size.
originX += screenWidth/2;
originY += screenHeight/2;
Combine it all into one. Assume the origin is alway 0,0 and you get
originX = -(playerX-screenWidth/2);
originY = -(playerY-screenHeight/2);
Now you have the numbers that you can put into the translate.
But its much better if you put it straight into the transformation matrix, and you don't really need the origin variables.
Thus in one step.
ctx.setTransform(1,0,0,1,-(playerX-screenWidth/2),-(playerY-screenHeight/2));
Now when you draw your player at
ctx.drawImage(payerImage,5000,6000);
You appear at the center of the canvas. The bad guy at 5500,6200 draw with
ctx.drawImage(enemyImage,5500,6200);
will appear on the canvas as well 500 pixels right and 200 down from you.
Now you don't have to mess around with anyones coordinates and just render them where they are, you have moved everything in the game with only one line of code. It has not slowed your game down at all, because the transform is always applied, you just made sure its doing what you want.
Set the transform near the start of every frame, after you have updated your position and everything will follow your character.

CreateJS - scaling the canvas does not scale the mouse coordinates

I am working on a big project where exercises in Canvas are created through JSON-data and CreateJS. The purpose of having it in HTML 5 is to not have to use a separate app for your phone, you can always use the website.
Everything works fine, however in mobile the Canvas is rescaled to full screen. This is done through checking the screen size, and if it's small enough to be mobile the canvas is scaled through this code:
// browser viewport size
var w = window.innerWidth;
var h = window.innerHeight;
// stage dimensions
var ow = canvasWidth;
var oh = canvasHeight;
// keep aspect ratio
var scale = Math.min(w / ow, h / oh);
stage.scaleX = scale;
stage.scaleY = scale;
// adjust canvas size
stage.canvas.width = ow * scale;
stage.canvas.height = oh * scale;
This works great for most of the exercises, like quizzes and such, where all you have to do is click on a button. However we also have some drag and drop-exercises, and an exercise where you can color a drawing. These of course rely on the mouse coordinates to work properly. The problem is, when the canvas is scaled the mouse coordinates are not. So when you drag an item or try to draw, there is an offset happening. So your drawing appears way left of your click, and when picking up a draggable object it doesn't quite follow your click correctly.
Had I made the code from the beginning I'm fairly sure how I would have recalculated the coordinates, but since they are calculated by CreateJS I don't really know how I should go about this.
This was reported as a problem by someone about a year ago here, where this solution was suggested:
I was able to work around this by adding a top-level container and attaching my Bitmaps to that and scaling it.
The whole exercise is inside a container which I have tried to scale but to no avail. I have also tried sending the scale as a parameter to the parts of the exercise created (for example the menu, background images and such) and not scale it all together, and it seems to work okay since then I can exclude the drawing layer. But since it is a large project and many different exercises and parts to be scaled it would take quite some time to implement, and I'm not sure it's a viable solution.
Is there a good and easy way to rescale the mouse coordinates along with the canvas size in CreateJS? I have found pure Javascript examples here on SO, but nothing for CreateJS in particular.
Continued searching and finally stumbled upon this, which I hadn't seen before:
EaselJS - dragging children of scaled parent. It was exactly what I was looking for. I needed to change the coordinates I drew with this:
var coords = e.target.globalToLocal(e.stageX, e.stageY);
Then I could use the coords.x and coords.y instead of directly using e.stageX and e.stageY like before.

Parallax effect with zoom and rotating

I am currently experimenting with parallax effect that i am planning to implement to my HTML5-canvas game engine.
The effect itself is fairly easy to achieve, but when you add zooming and rotating, things get a little more complicated, at least for me. My goal is to achieve something like this:Youtube video.
As you can see, you can zoom in and out "to the center", and also rotate around it and get the parallax effect.
In my engine i want to have multiple canvases that are going to be my parallax layers, and i am going to translate them.
I came up with something like this:
var parallax = {
target: {
x: Mouse.x,
y: Mouse.y
},
offset: {
x: -ctx.width / 2,
y: -ctx.height / 2
},
factor: {
x: 1,
y: 1
}
}
var angle = 0;
var zoomX = 1;
var zoomY = 1;
var loop = function(){
ctx.canvas.width = ctx.canvas.width; //Clear the canvas.
ctx.translate(parallax.target.x * parallax.factor.x, parallax.target.y * parallax.factor.y);
ctx.rotate(angle);
ctx.scale(zoomX, zoomY);
ctx.translate((-parallax.target.x - parallax.offset.x) * parallax.factor.x, (-parallax.target.y - parallax.offset.y) * parallax.factor.y);
Draw(); //Function that draws all the objects on the screen.
}
This is a very small and simplified part of my script, but i hope that's enough to get what i am doing. The object "parallax" contains the target position, the offset(the distance from the target), and the factor that is determining how fast the canvas is moving away relatively to the target. ctx is the canvas that is moving in the opposite direction of the target.(In this example i am using only one layer.) I am using the mouse as the "target", but i could also use the player, or some other object with x and y property. The target is also the point around which i rotate and scale the canvas.
This method works completely fine as long as the factor is equal to 1. If it is something else, the whole thing suddenly stops working correctly, and when i try to zoom, it zooms to the top-left corner, not the target. I also noticed that if i zoom out too much, the canvas is not moving in the opposite way of the target, but in the same direction.
So my question is: What is the correct way of implementing parallax with zooming and rotating?
P.S. It is important to me that i am using canvases as the layers.
To prepare for the next animation frame, you must undo any previous transforms in the reverse order they were executed:
context.translate(x,y);
context.scale(sx,sy);
context.rotate(r);
// draw stuff
context.rotate(-r);
context.scale(-sx,-sy);
context.translate(-x,-y);
Alternatively, you can use context.save / context.restore to undo the previous transforms.
Adjust your parallax values for the current frame,
Save the un-transformed context state using context.save(),
Do your transforms (translate, scale, rotate, etc),
Draw you objects as if they were in non-transformed space with [0,0] at your translate point,
Restore your context to it's untransformed state using context.restore()/
Either way will correctly give you a default-oriented canvas to use for your next animation frame.
The exact parallax effects you apply are up to your own design, but using these methods will make the canvas return to a normal default state for you to design with.

How redraw on canvas resize without blurring?

When the canvas element is resized (via the style changing) I also want to scale the canvas' drawn image as well. I cannot just change the height/width as this causes the canvas to clear itself, so I do:
Create a temporary canvas element
Draw the current canvas' image onto that temporary canvas
Resize the current canvas
Draw the temp canvas' image back to the current canvas but scaled to the new size
This results in some blurring - very noticeable after many resizes (example: when dragging to resize). How would I do this without any blurring?
EDIT: Turning off image smoothing (context.webkitImageSmoothingEnabled = false;) does not fix the problem, it simply makes it redraw it more and more jagged until the image looks nothing like the original after a number of resizes.
Called on resize event:
var tmpCanvas = null;
//Make a temporary canvas
tmpCanvas = document.createElement( "canvas" );
//Set its size to be equal
tmpCanvas.height = originalCanvas.height;
tmpCanvas.width = originalCanvas.width;
//Draw our current canvas onto it
tmpCanvas.getContext( "2d" ).drawImage( originalCanvas, 0, 0 );
//Set new dimensions
originalCanvas.width = originalCanvas.offsetWidth;
originalCanvas.height = originalCanvas.offsetHeight;
var originalContext = originalCanvas.getContext( "2d" );
//Set background and colors
originalContext.fillStyle = "#ffffff";
originalContext.strokeStyle = "#000000";
//Set paintbrush
originalContext.lineWidth = 4;
originalContext.lineCap = "round";
//Fill background as white
originalContext.fillRect( 0, 0, originalCanvas.width, originalCanvas.height );
//We have a saved signature
if ( SignatureCanvas.hasSignature === true )
{
//Draw it back but scaled (results in blurred image)
originalContext.drawImage( tmpCanvas, 0, 0, tmpCanvas.width, tmpCanvas.height, 0, 0, originalCanvas.width, originalCanvas.height );
/**
* This results in a blurred image as well
//Draw it back but scaled
originalContext.scale( originalCanvas.width / tmpCanvas.width, originalCanvas.height / tmpCanvas.height );
originalContext.drawImage( tmpCanvas, 0, 0, tmpCanvas.width, tmpCanvas.height, 0, 0, tmpCanvas.width, tmpCanvas.height );
*/
}
Is there a way to get the strokes and "scale" all those points and redraw?
Instead of taking the rendered image from the original canvas, actually redraw the image. By that, I mean execute the same logic you executed against the original canvas, but with the points involved scaled to the new size.
If you can, think about using SVG instead. It scales well by its nature.
Edit: Another option I've thought of is to simply use a gigantic canvas to start with. Sizing down tends to look better than sizing up, especially with smoothing on.
Edit II: The original answer was irrelevant, though the comment I had made is relevant, and am now promoting it and editing it to be an answer...and the answer I had given was not all that great anyway **.
Of course if you scale up raster graphics, that is, from an image with a smaller pixel dimensions of pixels, create an image with higher pixel dimensions, you are going to get blurred images. By scaling up, you're making a low resolution picture high resolution, but without the high resolution details.
There's absolutely no way around that blurriness unless you make multiple additional assumptions about your raster image like the only gray you'd see is at an image edge, or corners can only occur at apparent inflection points where the angle between the tangents of the joined curves must be 100 degrees or less. Essentially, you'd have to give additional information so that your higher resolution image can have detail "filled in". It's not all that terribly different from reverse engineering an SVG from a raster.
So, you appear to want to emulate is scaling vector graphics, in which the only solution is to save the commands, draw a SVG, or draw to a bigger canvas like Stuart Branham suggested.
** I had originally proposed that invoking drawImage would distort the pixels even if it were not scaled, and that it would be better to work with the actual pixel data. If that's true, I can't find proof, but that's irrelevant, as he wanted his image scaled up, without blurring...which is impossible, as I just mentioned.

Categories