I'm animating a sprite on a pixel grid. I have a few options, with pros and cons for each. I have a fair amount of javascript experience (six years), but none with this kind of thing. The problem is I don't know how expensive each option will be.
The sprite needs to render quite fast, and be inexpensive enough to have at least five running at the same time while running collision detection.
Ideally, I would like to use a grid of elements inside of a wrapper, rendering colour and alpha channels to each element from a multidimensional array.
The major pro here is that I can run pixel-by-pixel collision detection and click past the transparent parts of the sprite. With any image-based sprite, the onClick event will fire even if I click on a transparent pixel (I'll have to do a lot of work to pass clicks through transparent pixels, and it might be quite expensive).
The next option is to use css sprites. css-tricks.com/css-sprites/
This would be easy peasy, but as mentioned previously, onClicks won't pass through the transparent pixels. I can probably force it, but again, it may be expensive, and take a lot of time to impliment.
Another option is animated gifs, but they are huge, limited in the colour department, and hard to control animation-wise. I'd rather not go there.
And then there's the html5 canvas element, which I don't know very much about and would like to stay away from if at all possible. I don't know how any of my code would even work in relation to the canvas element and I doubt it would do what I want in the long-run.
So which is the best for performance? Would the first (and most preferable) be a viable option? Or have I missed something out?
With today's browsers you will be fine on desktop computers for building a sprite out of positioned pixel sub-elements (as long as they aren't too complicated or large), and just to be safe I'd limit yourself to about 10 active sprites. With Mobile things might get a bit slow and clunky, but considering you seem to be designing a game that requires precision "onclicks" I doubt that this will be a problem.
Your most flexible bet is to use HTML5 Canvas, as you have already worked out, but it will involve quite a bit more JavaScript coding. But this sytem will allow you to apply a number of effects to your sprites and will allow you to use pixel perfect detection by using getImageData (which allows you to read the exact pixel colour at any pixel offset).
getPixel from HTML Canvas?
If you wanted to avoid the techinical problems and challenges of having a full screen canvas system (which can be tricky), you can actually create as many smaller Canvas elements and move them around as your sprites (with the ease of HTML Elements).. Then all you have to do is design the code that draws your animation frames, and also tells if the mouse has hit or not hit the sprite using the aforementioned method (along with a click handler and some code to calculate where the user has clicked relative to your canvas elements position). Obviously, it would be best to do this in a generalised way so your code can be applied to all your sprites :)
To draw your images on the canvas you can use a spritesheet as you were mentioning in your question, and use the rather flexible drawImage() method which supports a slicing mode. This just needs to be tied up to a setInterval or requestAnimationFrame style game loop.
https://developer.mozilla.org/en-US/docs/Web/Guide/HTML/Canvas_tutorial/Using_images
http://www.playmycode.com/blog/2011/08/building-a-game-mainloop-in-javascript/
UPDATE - for those who wish to be very optimal
If you wish to take a more optimal route - which is a little bit more involved again - you can do the following. This method benefits if you have many sprites that are exactly the same with only a few (20 or 30) frames of animation:
Power your sprites by normal DIVs with a background image sprite sheet that you shift the background position of. This is the most optimal you can be, save having static images as sprites, because the browser does all the work.
for each sprite type draw your spritesheet on a hidden canvas element that is big enough to incorporate your whole spritesheet.
When a user clicks on one of your DIV sprites, take the background position as coordinates, invert them, and you should then know where on your canvas element (looked up by sprite's type) the pixel data resides.
Use the getPixelData method on your hidden canvas to work out if the user has clicked on the sprite or not.
The above means you only have one canvas element in use - per sprite type, the browser handles all the graphics for you and you get pixel perfect collisions with an onclick.
Hope the above makes sense?
How about splitting your image spirit into 30x30 cells and only have elements where the cell is opaque and leave a gap where the cell is transparent so that clicks fall through. You lose a bit of accuracy in where the cells can be clicked though.
Related
I'm making a small online multiplayer game with JavaScript and the native Canvas API (WebGL). I want to find the color of pixels on the screen as a form of collision detection, I figure it'd save resources to not have to process every shape every frame, but rather to simply check if the color of a pixel at a certain position is such that it is contacting a shape. (I hope that makes sense)
I ran some tests and I have an average frame delay of about 4-5 milliseconds without collision detection, and then when I make a single call to my canvas context's .getImageData() method, suddenly that frame delay shoots up to 19-20 milliseconds...
As far as I can find online getImageData() is the only means of checking the color of a given pixel, but I have to think there's some other way that doesn't introduce such a huge amount of lag.
I tried running getImageData() on a small section of the screen vs larger sections, and a 1x1 pixel request introduces 10ms latency, where a 600x600 pixel request is about 15ms... So the issue isn't the amount/size of the request, but rather just the request itself is extremely slow, so there's no potential for optimization here, I NEED another way.
Also, caching the image data is also not an option. I need to poll these pixels every single frame, I can't cache it (because the player and the object it needs to collide with are all constantly moving, and they're being controlled over the internet so there's also no way of predicting where they'll be at any given time... I NEED to poll every frame with no exceptions)
To be clear, I'm not asking how to write collisions or how to make pixel-perfect collision detection systems... I'm asking ONLY how to get the color of a pixel on the canvas without having to use .toImageData() because .toImageData() is far too slow for my use case.
Standard collision detection may end up being a better option. Pixel-perfect checking works well for perfect collision detection with complex objects but can get expensive with lots of pixels.
What I would probably recommend instead is to use standard collision detection with simple shapes. If done right, it should have good performance, even for a more complex game.
If you really do want to use pixel-perfect collision, you'll want to check the rectangular bounding boxes of the two objects first to ensure they aren't far away from each other. If their bounding boxes intersect, you could then use bitmasks of your sprites to quickly check each pixel for overlap. This question has a bit more info. It's not JS, but the concept would be the same.
I am trying to change the background of video to transparent. I have red and use couple of solution but they did not work quite will with video I use from youtube.
The solution I use are
1) https://github.com/m90/seeThru
2) https://jakearchibald.com/scratch/alphavid/
The both the solution is working fine with their demo video but not with the video I use.
If someone can explain about how is it working. So might be I can able to fix the issue.
Here is the result which I am getting.
There are several methods that are used. The best effect is the use of a precomputed mask (done during post production) that is added to the bottom half of the video. The video is then rendered to a canvas and the two half's combined to create the transparent frame by getting the pixel data and moving the a colour channel to the alpha channel.
The process is very CPU (for Javascript) intensive and only good for low resolution video. Plus you need to create the animated mask and double the size of the video. If you view the second example in the standard player you will see the other half.
The Second method used by the first example is to compute the alpha on the fly. This is even more CPU intensive but very simple to do. Again if you have low resolution video and a fast device it is practical. You are then faced with the problem of setting the thresholds for transparency, because videos use lossy compression you will have trouble with edges and the threshold colour.
Your best bet is to use a WebGL solution (if you don't want to do it in post production) and do the masking on the GPU where you can have a more complex algorithm and some chronological filtering as well. Though it will depend on the video quality, the type of background (single colour or static background). You could also find a asm.js solution that will work better. I remember seeing one some time ago, I will provide the link if I can find it.
Unfortunately JavaScript is not up to the job of high quality matte effects in realtime for the time being. It is a shame as the 2D API would only need a single additional global composite operation, "chroma-alpha" (move the mean source RGB to the destination alpha) that would open up so many addition canvas effects (which god do I pray to for that to happen?). For now you have to move every pixel in javascript
I'm investigating the possibility of producing a game using only HTML's canvas as the display media. To take an example task I need to do, I need to construct the game environment from a number of isometric tiles. Of course, working in 2D means they by necessity come in rectangular packages so there's a large overlap between tiles.
I'm old enough that the natural solution to this problem is to call BitBltMasked. Oh wait, no, an HTML canvas doesn't have something as simple and as pleasing as BitBlt. It seems that the only way to dump pixel data in to a canvas is either with drawImage() which has no useful drawing modes that ignore the alpha channel or to use ImageData objects that have the image data in an array.. to which every. access. is. bounds. checked. and. therefore. dog. slow.
OK, that's more of a rant than a question (things the W3C like tend to provoke that from me), but what I really want to know is how to draw fast to a canvas? I'm finding it very difficult to ditch the feeling that doing 100s of drawImages() a second where every draw respects the alpha channel is inherently sinful and likely to make my application perform like arse in many browsers. On the other hand, the only way to implement BitBlt proper relies heavily on a browser using a hotspot-like execution technique to make it run fast.
Is there any way to draw fast across every possible implementation, or do I just have to forget about performance?
This is a really interesting problem, and there's a few interesting things you can do to solve it.
First, you should know that drawImage can accept a Canvas, not just an image. The "sub-Canvas"es don't even need to be in the DOM. This means that you can do some compositing on one canvas, then draw it to another. This opens a whole world of optimization opportunities, especially in the context of isometric tiles.
Let's say you have an area that's 50 tiles long by 50 tiles wide (I'll say meters for the sake of my own sanity). You might divide the area into 10x10m chunks. Each chunk is represented by its own Canvas. To draw the full scene, you'd simply draw each of the chunks' Canvas objects to the main canvas that's shown to the user. If only four chunks (a 20x20m area), you would only perform four drawImage operations.
Of course, each of those individual chunks will need to render its own Canvas. On game ticks where nothing happens in the chunk, you simply don't do anything: the Canvas will remain unchanged and will be drawn as you'd expect. When something does change, you can do one of a few things depending on your game:
If your tiles extend into the third dimension (i.e.: you have a Z-axis), you can draw each "layer" of the chunk into its own Canvas and only update the layers that need to be updated. For example, if each chunk contains ten layers of depth, you'd have ten Canvas objects. If something on layer 6 was updated, you would only need to re-paint layer 6's Canvas (probably one drawImage per square meter, which would be 100), then perform one drawImage operation per layer in the chunk (ten) to re-draw the chunk's Canvas. Decreasing or increasing the chunk size may increase or decrease performance depending on the number of update you make to the environment in your game. Further optimizations can be made to eliminate drawImage calls for obscured tiles and the like.
If you don't have a third dimension, you can simply perform one drawImage per square meter of a chunk. If two chunks are updated, that's only 200 drawImage calls per tick (plus one call per chunk visible on the screen). If your game involves very few updates, decreasing the chunk size will decrease the number of calls even further.
You can perform updates to the chunks in their own game loop. If you're using requestAnimationFrame (as you should be), you only need to paint the chunk Canvas objects to the screen. Independently, you can perform game logic in a setTimeout loop or the like. Then, each chunk could be updated in its own tick between frames without affecting performance. This can also be done in a web worker using getImageData and putImageData to send the rendered chunk back to the main thread whenever it needs to be updated, though making this work seamlessly will take a good deal of effort.
The other option that you have is to use a library like pixi.js to render the scene using WebGL. Even for 2D, it will increase performance by decreasing the amount of work that the CPU needs to do and shifting that over to the GPU. I'd highly recommend checking it out.
I know that GameJS has blit operations, and I certainly assume any other html5 game libraries do as well (gameQuery, LimeJS, etc etc). I don't know if these packages have addressed the specific array-bounds-checking concern that you had, but in practice their samples seem to work plenty fast on all platforms.
You should not make assumptions about what speedups make sense. For example, the GameJS developer reports that he was going to implement dirty rectangle tracking but it turned out that modern browsers do this automatically---link.
For this reason and others, I suggest to get something working before thinking about the speed. Also, make use of drawing libraries, as the authors have presumably spent some time optimizing performance.
I have no personal knowledge about this, but you can look into the appMobi "direct canvas" HTML element which is allegedly a much faster version of normal canvas, link. I'm confused about whether this works in all browsers or just webkit browsers or just appMobi's own special browser.
Again, you should not make assumptions about what speedups make sense without a very deep knowledge of web browser internal processes. That webpage about "direct canvas" mentions a bunch of things that slow down canvas-drawing: "Reflowing text, mapping hot spots, creating indexes for reference links, on and on." Alpha-blending and array-bounds-checking are not mentioned as prominent causes of slowness!
Unfortunately, there's no way around the alpha composition overhead. Clipping may be one solution, but I doubt there would be much, if any, performance gain. Not to mention how complicated such a route would be to implement on irregular shapes.
When you have to draw the entire display, you're going to have to deal with the performance hit. Although afterwards, you have a whole screen's worth of pre-calculated alpha imagery and you can draw this image data at an offset in one drawImage call. Then, you would only have to individually draw the new tiles that are scrolled into view.
But still, the browser is having to redraw each pixel at a different location in the canvas. Which is quite expensive. It would be nice if there was a method for just scrolling pixels, but no luck there either.
One idea that comes to mind is that you could implement multiple canvases, translating each individual canvas instead of redrawing the pixels. This would allow the browser to decide how to redraw those pixels, in a more native way, at least in theory anyway. Then you could render the newly visible tiles on a new, or used/cached, canvas element. Positioning it to match up with the last screen render.
But that's just my two blits... I mean bits... duh, I mean cents :]
Intent on creating a canvas-based game in javascript I stand before a choice:
Should I performance-wise keep all the stuff happening in the screen in one canvas (all the moving characters, sprites) and redraw it at constant rate of, say, 60 FPS or should I break the scene into several smaller canvases thus removing the need of redundant redrawing of the scene? I could even create separate canvas elements for the characters' limbs and then do most of the animation by simply manipulating the CSS of the given canvas element (rotation, positioning, opacity).
To me the latter sounds way more plausible and easier to implement, but is it also faster? Also, shouldn't I perhaps use SVG, keep the characters and sprites as elements inside of it and manipulate their XML and CSS properties directly?
So what do you think is the most fitting solution to a scene with severals sprites and characters:
One canvas object redrawn manually (and wastefully) at FPS rate
Several canvas elements, redrawn manually in a more reasonable fashion
Structured vector graphics document like SVG / VML manipulated via DOM
I am mainly concerned about the performance differences, but the legibility of the logical code behind is also of interest (I, having already worked with canvas before, am for example fairly sure that the redrawing function for the entire canvas would be one hard-to-maintain beast of a script).
DOM manipulations are slow in comparison to GPU-accelerated canvas operations, so I would stay away from SVG and VML.
As far as structuring your canvas code goes, it certainly doesn't make sense (especially for performance reasons) to clear and re-draw the entire canvas because the player moved or performed an action. Based on your description here, I'm guessing that your game will be 2D. These types of games lend themselves extremely well to layering unless you're doing something rather complex like Paper Mario. You should be looking at the issue from an object-oriented viewpoint and encapsulating your drawing procedures and objects together as appropriate.
For instance, create a player object that maintains a small canvas representing the character. All the logic needed to maintain the character's state is kept within the object and any changes made to it need not worry about other components of the game's visual representation. Likewise, do the same for the background, user interface, and anything else you can abstract into a layer (within reason). For example, if you're doing a parallax game, you might have a foreground, background, character, and user interface layer.
Ultimately you will need to maintain the state of the different components in your game individually. Player animations, background clouds moving, trees swaying, etc. will all be managed by appropriate objects. Since you will already have that type of code structure setup, it makes sense to just maintain major components on separate canvas elements and composite them together as needed for better performance. If the character moves in the bottom left corner of a scene with a static background, you don't need to re-draw the top right corner (or 95% of the scene, for that matter). If you're considering full-screen capabilities, this is definitely a performance concern.
There's a rule in programming, that you should never try and optimize something before you have a speed problem with it. You'll spend more time trying to figure out how to code something in the best way than to actually code, and will never finish anything.
Draw them all on your canvas at a fixed rate. That's how it's done. If you start creating a canvas for each limb and element and manipulate them using CSS, you're wasting the potential of canvas. Might as well just use images. That's how they did it before canvas. That's the problem canvas was made to solve.
If you ever encounter speed issues, then you can start hammering at them. Check out these slides for some tips (related video). This guy's blog also has some handy tips on canvas performance.
The 2nd part of the question is, which javascript library is better/easier to manipulate images with? I won't be actually drawing any shapes or anything. Other info: I'll be using jQuery and don't need to support all browsers, just webkit.
Edit:
More information: the current design is to layout/draw several rows/columns of images in a grid-like layout, with the image in the center being in "focus" (a little larger, with a border or something and some text next to it). The tricky thing is that we want the whole canvas of images to appear to slide/glide over to bring another random image into focus. So obviously the number of images in this grid needs to exceed what is visible in the viewport so that when the transition occurs there are always images occupying the canvas. Other than moving the images around, I won't be blurring them or otherwise modifying them. Eventually we will add user interactions like clicking/touching on a visible image to bring it to focus manually.
Let me know if this is not clear or still confusing.
I ran across scripty2 which seems like an alternative to using canvas/SVG for my purposes. I also started farting around with EaselJS last night, and it seems like this might work, but I'm wondering if it'll end up being more work/complex than just using standard HTML/CSS and a tool like Scripty2 to aid with animations and click/touch events. Just looking for any suggestions. Thanks!
The answer depends on your manipulation and animation.
If it's just translations, CSS wins for speed compared to canvas. I haven't tested, but I feel confident it easily beats SVG for the same sort of thing.
If you're going to be doing non-affine transformations or otherwise messing with the images (e.g. blurring them) you clearly want Canvas.
If you need event handlers per object, you clearly want a retained-mode drawing system like SVG or HTML+CSS. I haven't done enough CSS3 transforms to say how they compare in terms of speed to SVG, but they clearly do not have the robust transformation DOM of SVG.
This is a rather subjective question (or suite of questions) and you haven't yet given sufficient information for a clear answer to be possible.