Canvas HIT testing: dealing with element boundaries and anti-aliasing - javascript

The setup
As part of a web vector editing tool written in Javascript, I'm implementing hit testing using a hit canvas strategy similar to that of Concrete.js.
For most of it, it works pretty well: I'm drawing my shapes twice (once on the display canvas, and once on a hit canvas).
When querying the canvas, I check the hovered pixel of the hit canvas and extract interaction information (i.e. which object id is stored there).
The problem
This works well inside shapes, but is painfully flawed at the boundaries where anti-aliasing makes the stored data invalid (it gets mixed with whatever background data was there before).
Are there good strategies for dealing with this data boundary problem?
Without disabling anti-aliasing for canvas methods, then we're bound to have some boundary regions across overlapping regions that will store merged data from multiple regions.
The simple scenario
In a binary scenario (some foreground vs the background), then this can be mitigated as we can assume the background to have no value, and any value becomes some foreground.
The real scenario
However, in a general scenario with multiple shapes overlapping each other on top of the background, is there any reasonable strategy for error detection? (or error correction, but I assume that's harder)
If I can tell that the data is invalid (i.e. it consists of perturbed data due to anti-aliasing), then I can use a different strategy for those few pixels at the boundaries. But I feel that it's impossible to tell whether the data I'm extracting is valid in the general scenario where we can have many overlapping shapes.
Of course, one solution is to NOT use a hit canvas. But I was wondering whether people had found a solution using hit canvases since they seem great for dealing with complex geometries.
Anti-aliasing
The ideal solution would be to disable anti-aliasing, which I don't think is possible for canvas methods [*].
[*] I know we can disable filtering when rendering images (e.g. that question) such as with imageSmoothingEnabled=false or rescaling with image-rendering: pixelated, but those don't solve the problem of anti-aliasing when drawing shapes / paths.

Related

Is there any way to find the color of a pixel in JavaScript WITHOUT using getImageData()?

I'm making a small online multiplayer game with JavaScript and the native Canvas API (WebGL). I want to find the color of pixels on the screen as a form of collision detection, I figure it'd save resources to not have to process every shape every frame, but rather to simply check if the color of a pixel at a certain position is such that it is contacting a shape. (I hope that makes sense)
I ran some tests and I have an average frame delay of about 4-5 milliseconds without collision detection, and then when I make a single call to my canvas context's .getImageData() method, suddenly that frame delay shoots up to 19-20 milliseconds...
As far as I can find online getImageData() is the only means of checking the color of a given pixel, but I have to think there's some other way that doesn't introduce such a huge amount of lag.
I tried running getImageData() on a small section of the screen vs larger sections, and a 1x1 pixel request introduces 10ms latency, where a 600x600 pixel request is about 15ms... So the issue isn't the amount/size of the request, but rather just the request itself is extremely slow, so there's no potential for optimization here, I NEED another way.
Also, caching the image data is also not an option. I need to poll these pixels every single frame, I can't cache it (because the player and the object it needs to collide with are all constantly moving, and they're being controlled over the internet so there's also no way of predicting where they'll be at any given time... I NEED to poll every frame with no exceptions)
To be clear, I'm not asking how to write collisions or how to make pixel-perfect collision detection systems... I'm asking ONLY how to get the color of a pixel on the canvas without having to use .toImageData() because .toImageData() is far too slow for my use case.
Standard collision detection may end up being a better option. Pixel-perfect checking works well for perfect collision detection with complex objects but can get expensive with lots of pixels.
What I would probably recommend instead is to use standard collision detection with simple shapes. If done right, it should have good performance, even for a more complex game.
If you really do want to use pixel-perfect collision, you'll want to check the rectangular bounding boxes of the two objects first to ensure they aren't far away from each other. If their bounding boxes intersect, you could then use bitmasks of your sprites to quickly check each pixel for overlap. This question has a bit more info. It's not JS, but the concept would be the same.

Render objects based on 1 specific object on the canvas

I'm presently implementing a few features into a vector editor based off of fabric.js and have hit a bump in the road with how best to handle this scenario:
In order to resize/scale the canvas smaller and larger via scaling grippies/controls, I've begun the task of implementing a "fake canvas" that stays in the middle of the real canvas (the real canvas is resized to fill it's DOM parent at all times. This presents several issues, the biggest being, dealing with objects on the canvas, ensuring their coordinate are always relative to my "fake canvas" (which is just a Rect with some restrictions applied, e.g. locking movement, rotation, etc).
So I'm really just looking for strategies/suggestions at how to go about doing this. For instance, I know I'd be dealing with newly added objects, moving objects and preventing them to be moved outside of the boundaries of my fake canvas. Among other considerations.
Thanks for the input and suggestions. I'm not looking for code, I'm more just looking for suggestions of how best to handle all the canvas related things, transferred to a fake canvas. I see that the Shutterstock Editor does some of this too, so I guess it doesn't seem like it's out of the realm of possibilities.

Hiding crucial data from an SVG

I have a SVG generated map for the game I am developing. I have no problems with the game being open-source and it uses open web technologies such as HTML and SVG. No problems there.
But at the same time I want the players not to be able to see or reverse engineer a map of the whole world (to retain true exploration). For now I generate map using a seed that is secret and not version controlled. So even though the algorithm is known curious players can use open-sourced code to generate "game-like worlds" but not that exact one. This solves the "global" problem.
But since SVG is rendered on a page as a single Voronoi diagram all the data (I don't mind the coordinates of points) would be extractable. Data like resources, land types, biomes, climate etc. could be fetched from SVG to gain an upper hand in finding good locations for settlements.
Any idea how to prevent that? Players have limited vision so I thought about either:
not rendering the whole Voronoi diagram at all (just the visible part), but that could be potentially tricky to do (maybe, haven't looked into it yet),
inserting the resource/land tile data into SVG graph only to visible locations
I can see the benefits of both approaches and if done correctly it could even boost the performance (not rendering the whole thing/rendering with less data) and lead to bigger worlds without impacting performance.
Any other ideas/programming/architectural approaches to help with the issue?
(I am using Vue.js, d3.js, svg-pan-zoom and Laravel backend just in case it helps.)
The ideas that you gave are perfect, but for implementing them, you need to make hard work, and spend much time.
I have a suggestion. Is will work for most of the users. Maybe some users will "hack" it. But I believe it will work for 95% of the times.
You can create a very big rectangle, from the top left point 0,0 until the right bottom point. The rectangle will be white, and it will be over all other shapes.
This way if someone will download the SVG, we will see nothing. Just a big white rectangle.
In you game HTML, you can add a CSS selector, to hide this rectangle.
If you following this method, most of the users (who don't have a photo editing software) will not be able to see the map.
Users who knows how to inspect elements in HTML may see the map. But I believe that most of them who will see a white box, will not believe that there is something behind.
I think that this is a simple temporary approach that you can do, before doing other more defensive ways.

How much slower is drawing to a transparent canvas and checking if point is transparent than checking if point is in a complex polygon?

This is more of a generic question to be honest, just wondering if anyone has done any sort of research on the subject.
Basically I am adding event support to a small game engine I am creating for my own personal use. I would like pixel perfect hover over 2d object event support and am just thinking of the best way of doing it. Realistically it would be faster for me personally just to invoke a draw of my objects onto a transparent canvas and checking if the mouse x y is over a transparent pixel or not since I dont have to make a set of points defining the outside of an object. This would also allow me to have holes in my object and it would still correctly know if I hovered over or not.
What I am wondering is using methods shown here: How can I determine whether a 2D Point is within a Polygon?
How much slower would my method be to the methods shown there?
Im currently still learning so its not easy for me to implement all of this and just test it myself since it would probably take me ages to get to work correctly and test the speeds.
Side note: I would still have a basic bounding box to save it from redrawing and testing every single time.
Checking if a point is in a polygon will 99.999999% of the time be vastly, vastly faster.
To be slower the polygon would need to be extremely complex.
To do the other method you need to use getImageData, and getting image data on the canvas is very slow.
Point in polygon algorithms do properly account for holes. Make sure you have one that obeys the non-zero winding number rule, because that is what canvas uses (as opposed to the even odd rule) and you may want compatibility with paths constructed in the canvas (either now or later).

What's the fastest way to draw to an HTML 5 canvas?

I'm investigating the possibility of producing a game using only HTML's canvas as the display media. To take an example task I need to do, I need to construct the game environment from a number of isometric tiles. Of course, working in 2D means they by necessity come in rectangular packages so there's a large overlap between tiles.
I'm old enough that the natural solution to this problem is to call BitBltMasked. Oh wait, no, an HTML canvas doesn't have something as simple and as pleasing as BitBlt. It seems that the only way to dump pixel data in to a canvas is either with drawImage() which has no useful drawing modes that ignore the alpha channel or to use ImageData objects that have the image data in an array.. to which every. access. is. bounds. checked. and. therefore. dog. slow.
OK, that's more of a rant than a question (things the W3C like tend to provoke that from me), but what I really want to know is how to draw fast to a canvas? I'm finding it very difficult to ditch the feeling that doing 100s of drawImages() a second where every draw respects the alpha channel is inherently sinful and likely to make my application perform like arse in many browsers. On the other hand, the only way to implement BitBlt proper relies heavily on a browser using a hotspot-like execution technique to make it run fast.
Is there any way to draw fast across every possible implementation, or do I just have to forget about performance?
This is a really interesting problem, and there's a few interesting things you can do to solve it.
First, you should know that drawImage can accept a Canvas, not just an image. The "sub-Canvas"es don't even need to be in the DOM. This means that you can do some compositing on one canvas, then draw it to another. This opens a whole world of optimization opportunities, especially in the context of isometric tiles.
Let's say you have an area that's 50 tiles long by 50 tiles wide (I'll say meters for the sake of my own sanity). You might divide the area into 10x10m chunks. Each chunk is represented by its own Canvas. To draw the full scene, you'd simply draw each of the chunks' Canvas objects to the main canvas that's shown to the user. If only four chunks (a 20x20m area), you would only perform four drawImage operations.
Of course, each of those individual chunks will need to render its own Canvas. On game ticks where nothing happens in the chunk, you simply don't do anything: the Canvas will remain unchanged and will be drawn as you'd expect. When something does change, you can do one of a few things depending on your game:
If your tiles extend into the third dimension (i.e.: you have a Z-axis), you can draw each "layer" of the chunk into its own Canvas and only update the layers that need to be updated. For example, if each chunk contains ten layers of depth, you'd have ten Canvas objects. If something on layer 6 was updated, you would only need to re-paint layer 6's Canvas (probably one drawImage per square meter, which would be 100), then perform one drawImage operation per layer in the chunk (ten) to re-draw the chunk's Canvas. Decreasing or increasing the chunk size may increase or decrease performance depending on the number of update you make to the environment in your game. Further optimizations can be made to eliminate drawImage calls for obscured tiles and the like.
If you don't have a third dimension, you can simply perform one drawImage per square meter of a chunk. If two chunks are updated, that's only 200 drawImage calls per tick (plus one call per chunk visible on the screen). If your game involves very few updates, decreasing the chunk size will decrease the number of calls even further.
You can perform updates to the chunks in their own game loop. If you're using requestAnimationFrame (as you should be), you only need to paint the chunk Canvas objects to the screen. Independently, you can perform game logic in a setTimeout loop or the like. Then, each chunk could be updated in its own tick between frames without affecting performance. This can also be done in a web worker using getImageData and putImageData to send the rendered chunk back to the main thread whenever it needs to be updated, though making this work seamlessly will take a good deal of effort.
The other option that you have is to use a library like pixi.js to render the scene using WebGL. Even for 2D, it will increase performance by decreasing the amount of work that the CPU needs to do and shifting that over to the GPU. I'd highly recommend checking it out.
I know that GameJS has blit operations, and I certainly assume any other html5 game libraries do as well (gameQuery, LimeJS, etc etc). I don't know if these packages have addressed the specific array-bounds-checking concern that you had, but in practice their samples seem to work plenty fast on all platforms.
You should not make assumptions about what speedups make sense. For example, the GameJS developer reports that he was going to implement dirty rectangle tracking but it turned out that modern browsers do this automatically---link.
For this reason and others, I suggest to get something working before thinking about the speed. Also, make use of drawing libraries, as the authors have presumably spent some time optimizing performance.
I have no personal knowledge about this, but you can look into the appMobi "direct canvas" HTML element which is allegedly a much faster version of normal canvas, link. I'm confused about whether this works in all browsers or just webkit browsers or just appMobi's own special browser.
Again, you should not make assumptions about what speedups make sense without a very deep knowledge of web browser internal processes. That webpage about "direct canvas" mentions a bunch of things that slow down canvas-drawing: "Reflowing text, mapping hot spots, creating indexes for reference links, on and on." Alpha-blending and array-bounds-checking are not mentioned as prominent causes of slowness!
Unfortunately, there's no way around the alpha composition overhead. Clipping may be one solution, but I doubt there would be much, if any, performance gain. Not to mention how complicated such a route would be to implement on irregular shapes.
When you have to draw the entire display, you're going to have to deal with the performance hit. Although afterwards, you have a whole screen's worth of pre-calculated alpha imagery and you can draw this image data at an offset in one drawImage call. Then, you would only have to individually draw the new tiles that are scrolled into view.
But still, the browser is having to redraw each pixel at a different location in the canvas. Which is quite expensive. It would be nice if there was a method for just scrolling pixels, but no luck there either.
One idea that comes to mind is that you could implement multiple canvases, translating each individual canvas instead of redrawing the pixels. This would allow the browser to decide how to redraw those pixels, in a more native way, at least in theory anyway. Then you could render the newly visible tiles on a new, or used/cached, canvas element. Positioning it to match up with the last screen render.
But that's just my two blits... I mean bits... duh, I mean cents :]

Categories