We are creating an app that lets a user capture a number of images and it will try to create a 3D model of the target object. In order to help the users capture useful images we give them some guidance while they move their phone from one capture to the next.
We have a nice prototype working by means of navigator.mediaDevices.getUserMedia() that captures video, displays it in a <video> element, and has an overlay that shows how to move the phone. When they are ready they press a button and we grab the current frame of the streamed video.
We were quite happy with this until we realized that very often the captured image would not have enough quality; mainly they tend to be a bit blurred because the user may not hold the device totally still. This causes the math behind creating the 3D model to fail.
I am now tasked with attempting to improve this but I think I don't have many options. Here is what I have been investigating and their drawbacks:
JavaScript's ImageCapture API. This seems to be exactly what we need: a way to actually take a picture instead of grabbing a frame from a video. While the API has still an experimental status, it seems pretty stable and Chrome has it implemented since version 59. The problem is that Safari (our main target) does not have it implemented and it seems they won't ever do. I can't really find information on what their plan is though but as of today, this is not an option.
Use the input element of type file with the attribute capture. While this lets me capture images with the native camera, I cannot give the user any guide as far as I know.
Create a whole mobile app. This requires another year of work and requesting our existing users to install an app, which may not be possible. Also leaves Android devices out which we'd prefer not to.
While typing this I thought of perhaps using the video instead of capturing the images, but not sure this would help in any way.
Instead of a different approach to the way of capturing the image, I could try to only grab the image if I can confirm that the device is as close as still as possible (using a threshold value). Perhaps I could use the gyroscope for this (we are using it to check they have moved the device to a place and angle we consider useful for the process). The drawback of this is that I am not sure it would really mitigate our problem... how still is still enough? is it possible for the person to be that still for a second?
So my question here is, can anyone think of another alternative to those I descrived? or perhaps improve one of the enumerated ones?
BTW does anyone know what are Apple's plans for the ImageCapture API?
I am working on a HTML canvas app that displays graphics to the screen. The graphics aren't important for the website, they just make things look pretty.
My problem is the app needs the cpu of the device to run at a certain speed or the frame rate becomes unacceptable.
Any modern phone/laptop can easily run the app, but of course not everyone has updated tech.
At that point I'd like to drop support for the device and stop rendering the animations because it will just do more harm than good.
This idea is pretty standard on the internet. For example if you want an image background for your site, but you don't want mobile phones to load the same large image as desktops then you just use some css queries to only serve the image to desktops.
This is how we can get new features pushed out while keeping backward compatibility.
However when it comes to detecting performance this isn't as easy of a task as it sounds. There's no way to get cpu specs with Javascript, and even if I could there's no way of telling what else the user is running on their machine.
This leaves me with 2 options, either run a small performance test before I start the canvas app. Or start the app and try to run a few frames and stop it if the frame rate is too low.
The problem is both of these options are pretty sketchy because the device may just have a "speed hiccup" at the start of the app, and so I shut down the animations for nothing.
Also if a user has a device that sits right on the border of the threshold sometimes the animations will load, and sometimes they won't which would probably be confusing.
Is there any "standard" on the internet to deal with this sort of problem? Would it be best to leave a footnote at the bottom of the site window when animations are turned off?
Or is it just something you have to accept when pushing the boundaries and dealing with performance?
Is there any way to have two or more (preferably three) html5 < video > tags playing simultaneously and to be in perfect sync.
If I have let's say three tiles of one video and I want them to appear in browser as one big video. They need to be perfectly synchronized. Without even smallest visual/vertical hint that they are tiled.
Unfortunately I cannot use MediaController because it is not supported well enough.
I've tried some workouts, including canvases, but I still get visual differentiation. Has anyone had any similar problem/solution?
Disclaimer: I'm not a video guy, but here are some thoughts anyway.
If they need to be absolutely perfect...you are fighting several problems at once:
A device might not be powerful enough to acquire, synchronize and render 3 streams at once.
Even if #1 is solved, a device is never totally dedicated to your task. For example, it might pause for garbage collection between processing stream#1 and stream#2--resulting in dropped/unsynchronized frames.
So to give yourself the best chance at perfection, you should first merge your 3 videos into 1 vertical video in the studio (or using studio software).
Then you can use the extended clipping properties of canvas context.drawImage to break each single frame into 2-3 separate frames.
Additionally, buffer a few frames you acquire on the stream (this goes without saying!).
Use requestAnimationFrame (RAF) to control the drawing. RAF does a fairly good job of drawing frames when system resources are available and delaying frames when system resources are lacking.
Your result won't be perfect, but they will be synchronized. You will always have to make the decision whether to drop or delay frames when system resources are unavailable, but at least the frames you do present will be synchronized.
As far as I know it's currently impossible to play HTML5 video frame-by-frame, or seek to a frame accurate time-code. The nearest seek seems to be precise to roughly 1-second.
But you can still get pretty close using the some of the media frameworks:
Popcorn.js library made for synchronizing video with content.
mediagroup.js another library used to add support for mediagroup attributes on HTML5 media elements
The only feature that allowed that is named mediaGroup and it was removed from Chrome(apparently for not being popular enough). It's still present in WebKit. Relevant discussion here and here.
I think you can implement you own "mediagroup"-like tag using wasm though without DOM support it may be tricky.
Coming from the backbone side of web development we are trying to find a solution for a request to add visual and sound effects to our task management web application.
For starters - we are looking for a way to create an exploding stars effect like you see in games.
Can this be done with HTML5 canvas?
Should we use flash?
Any ideas how to start?
By now, just about anything Flash can do visually can be done by the HTML5 canvas on modern browsers.
For a 'star burst' visual effect, it sounds like a simple matter of creating a random array of objects that move away in random pre-set directions every time the canvas updates.
Example: http://jsfiddle.net/amDAW/ (click on the canvas to create a starburst)
As for sounds, this isn't handled in the canvas, but rather either the Audio tag or the fairly new WebAudio API. If you go with the former (more browser support), your biggest concern will be with resource preloading, but there are some helper libraries that can abstract this away (shameless advertising: https://github.com/jsweeneydev/ResourceLoader).
With apple browsers not supporting flash or silverlight, there is a real incentive to avoid flash / silverlight to avoid losing that audience when building a web site. That being said there is certain functionality that it seems like you can only really do in flash / silverlight
for example alot of simple games where you can move things on the screen like this site all seem to be built in flash. also, a lot of drag and drop functionality where you can drag one object onto another like these game sites.
After lots of searching I can't find any that are not either flash or silverlight based.
In particular i am looking for drag and drop support of one element onto another
my question is if you need this type of functionality is javascript / html 5 able to do this type of stuff (so you can support iphone / ipad) or are you out of luck.
is there any resource that highlight examples or suggestions of trying to do this type of interactive functionality and how / if you can do this type of stuff without silverlight / flash. also, if anyone has any good examples of existing site who are doing that today that would be great as well.
This is going to be a long discussion about the ability of html5 to compete with flash.
In my opinion jquery is not any close to performance flash or silverlight animation give.
if the comparison is in terms of drag-and-drop, menu dropdowns, fadeIn.fadeOut - jQuery is competitive.
If i will see the jQuery cartoons with lot of layers and objects moving simultaneously - i will probably agree that jQuery has competitive performance.
the things are compare to see the difference:
magnifying glass over the raster picture
smoke/water/fire emulation
compound 3D objects like fractals with deep branches
when HTML5 will have it - then i will agree that it has competitive performance. All that i see today is picture slideshows and couple of games that work on html5.
You can check Easel.js by Grant Skinner, used in Pirates Love Daisies.
Also, other frameworks are:
enchant
limeJS
akihabara
JQuery UI has that for long time, works in all modern browsers, not just HTML5
JQuery UI Demos
You should checkout canvasdemos.com. It has a lot of good examples of what can be done. You can take a look at the source code behind these - some even help you in that regard. e.g. the pool game
Other good examples include the doom like "game" (you can walk around in 3D dungeons).
The Frog Log game was the winner in the 10KB coding challange
Also this was the first result for a search of html5 animation demo in google. It has links to 48 demos. Some of them are really cool. Unfortunately the code for a lot of these have been minified, but they still might give you a few ideas about what you can achieve.
So it's fine for making simple dressup type games. However, if you are looking to make anything that's CPU intensive, you should look into some performance benchmarks like this. HTML5/Canvas based animation is still quite far behind flash in terms of performance. Getting consistent behaviour across the various browsers will also be an issue.
HTML5 and related technologies (WebSockets, WebGL, web storage, File API, media capture, etc) are quickly moving towards parity with (and in some cases exceeding) what can be done in Flash/Silverlight.
The HTML5 Rocks slides are a reasonable starting point to see what is possible (you need an HTML5 capable browser). In particular, the Canvas example demonstrates image manipulation (drag, rotate, resize) which is the core functionality needed to implement dress-up games.
Flash was designed for animation. The tweens were meant to be used for animating drawings. Because it was marketed to every Tom, Dick and Harry, people started using it to animate hideous menus and flying content text. And Adobe complied to this new use, building an abode of total chaos.
Flash is still the best animation engine for the web, it should never have catered to full flash websites.
Many HTML5 fans out there, but it needs to be said: Canvas is a decade behind Flash. But for everything other than animation, Flash is an abomination.
In old browsers, you can emulate drag'n'drop of elemnts from the DOM but in new browsers, you can also drag'n'drop files (like images) and there are events for it: https://developer.mozilla.org/en/DragDrop/Drag_and_Drop
For flash-like animations, it's been possible with JavaScript for a long time but doing some bug ones was really hard and often slowing down the page.
Now, there is canvas and WebGL that allow you to do it in a more convinient way (for the complex ones).
And with canvas, WebGL, CSS animations (if you use the tric to make the browser think it's 3D), you get CPU accelerated so it's way faster.
There is also requestAnimationFrame that allows to optimise the reflows and therefore the script.
The best example I know on what can be done is the Quake II port to the Web :
Video: http://www.youtube.com/watch?v=XhMN0wlITLk&feature=player_embedded
Project's site: http://code.google.com/p/quake2-gwt-port/
as said in soe of the answers above, I'm pretty sure jquery's draggable/droppable plugin will do the job for you if you just need a basic drag and drop dress me up type of game. Basic premise is this:
in the default example, your avatar to wear the clothes would be a div with a png/gif.jpg background image of a girl/boy whatever(instead of an orange drop here background)
the clothes will be the "drag me to my target" objects in the example. you can create them as divs or even image tags that have the draggable class in them so you can drag them around and drop them in the orange boxes/avatars.
you can save the data using ajax, which is also covered in the examples there(or other tutorials in the net, it's easy)
???
PROFIT
Just try it out and see for yourself. If you need any help you can just ask here again, but I do think that the jquery UI answers are valid answers to your problem.
I don't have an example site to show you, but I'm pretty sure given some images and stuff I can whip out something...if I had the time lol.
You can definitely do these "flash-like" things now in HTML5 web browsers. Check out the examples at http://www.chromeexperiments.com/
In fact HTML5/CSS3/JS can do anything flash can do. But there are some drawback :
It is not yet mature. Lot of bugs, lot of difference in implementation depending of browsers and many people simply don't have yet browsers that support it.
Adobe has a really nice set of tools that help making complex flash applications. This doesn't exist (yet ?) at the same level for HTML5.
On a side note, neither flash, neither HTML5 will really shine on mobiles phone. People prefer native applications anyway. You might need to provide a web version, but you'll need too a app version (one per big phone player).
We can speculate how HTML5 will rule the word in the next few years, but as of now, impact are limited outside of nice looking demo that consume 100% of CPU (really, really bad for mobile device).
For drag and drop support, anybody can do that - in HTML4 and in any browser with a few lines of javascript - inside one page, or can think to do it on between 2 browser windows of the same website. Doing drag & drop between browser and any native desktop application is another thing.
For a great example of what you can do with HTML5 in terms of drag and drop, I suggest you take a look at this article and in particular the short demo at the beginning. The article also highlights a few other goodies that come with HTML5, such as localStorage and the HTML5 Canvas.
For a more detailed tutorial on the HTML5 drag-n-drop API specifically (really it is a Javascript API), take a look at this other article. It is dated from December 2009 but still valid.
Lastly, this video gives you a good insight on some of the cool visual effects that you'll be able to do with HTML5 (SVG, Canvas, CSS3, WebGL, ...). More of a marketing video I'll admit but a good illustration of some of the more powerful HTML5 features (at least from a visual stand point) and of what we'll start seeing in our browsers in a not so distant future...
Disclaimer: I don't work for Mozilla. I just happen to have researched this topic in the past and found that the material produced by Mozilla, and in particular the demos from Paul Rouget, to be the most instructive.
jQuery UI is an amazing library...
Drag: http://jqueryui.com/demos/draggable/#default
Drop: http://jqueryui.com/demos/droppable/
I belive that the droppable examples will answer your question of "In particular i am looking for drag and drop support of one element onto another"