I'm integrating a *.webm Video with alpha transparency. At the moment, the transparency is only supported in Chrome and Opera. (Demo: http://simpl.info/videoalpha/) Firefox for example plays the video as it supports the WebM format, but instead of the transparency, there's a black background.
My plan is to display the video poster image instead of the video, if the browser does not support alpha transparency. So the video should only play, if the browser supports WebM alpha transparency. I know how to detect the browser or the rendering engine and therefore play the video (see code below) - but is there a "feature detection" way?
var supportsAlphaVideo = /Chrome/.test(navigator.userAgent) && /Google Inc/.test(navigator.vendor) || (/OPR/.test (navigator.userAgent));
if (supportsAlphaVideo) {
document.querySelector(".js-video").play();
}
See also http://updates.html5rocks.com/2013/07/Alpha-transparency-in-Chrome-video
Here's a working solution to test for alpha support in WebM.
I basically combined Capture first frame of an embedded video and check_webp_feature
The video used to test with is base64-encoded into the source. It's actually a tiny VP9 WebM video encoded using:
ffmpeg -i alpha.png -c:v libvpx-vp9 alpha.webm
If you want to test for VP8 alpha support instead, just encode your own and remove the -vp9. alpha.png is a 64x64 pixel 100% transparent PNG image.
var supportsWebMAlpha = function(callback)
{
var vid = document.createElement('video');
vid.autoplay = false;
vid.loop = false;
vid.style.display = "none";
vid.addEventListener("loadeddata", function()
{
document.body.removeChild(vid);
// Create a canvas element, this is what user sees.
var canvas = document.createElement("canvas");
//If we don't support the canvas, we definitely don't support webm alpha video.
if (!(canvas.getContext && canvas.getContext('2d')))
{
callback(false);
return;
}
// Get the drawing context for canvas.
var ctx = canvas.getContext("2d");
// Draw the current frame of video onto canvas.
ctx.drawImage(vid, 0, 0);
if (ctx.getImageData(0, 0, 1, 1).data[3] === 0)
{
callback(true);
}
else
{
callback(false);
}
}, false);
vid.addEventListener("error", function()
{
document.body.removeChild(vid);
callback(false);
});
vid.addEventListener("stalled", function()
{
document.body.removeChild(vid);
callback(false);
});
//Just in case
vid.addEventListener("abort", function()
{
document.body.removeChild(vid);
callback(false);
});
var source = document.createElement("source");
source.src="data:video/webm;base64,GkXfowEAAAAAAAAfQoaBAUL3gQFC8oEEQvOBCEKChHdlYm1Ch4ECQoWBAhhTgGcBAAAAAAACBRFNm3RALE27i1OrhBVJqWZTrIHlTbuMU6uEFlSua1OsggEjTbuMU6uEHFO7a1OsggHo7AEAAAAAAACqAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAVSalmAQAAAAAAADIq17GDD0JATYCNTGF2ZjU3LjU3LjEwMFdBjUxhdmY1Ny41Ny4xMDBEiYhARAAAAAAAABZUrmsBAAAAAAAARq4BAAAAAAAAPdeBAXPFgQGcgQAitZyDdW5khoVWX1ZQOYOBASPjg4QCYloA4AEAAAAAAAARsIFAuoFAmoECU8CBAVSygQQfQ7Z1AQAAAAAAAGfngQCgAQAAAAAAAFuhooEAAACCSYNCAAPwA/YAOCQcGFQAADBgAABnP///NXgndmB1oQEAAAAAAAAtpgEAAAAAAAAk7oEBpZ+CSYNCAAPwA/YAOCQcGFQAADBgAABnP///Vttk7swAHFO7awEAAAAAAAARu4+zgQC3iveBAfGCAXXwgQM=";
source.addEventListener("error", function()
{
document.body.removeChild(vid);
callback(false);
});
vid.appendChild(source);
//This is required for IE
document.body.appendChild(vid);
};
supportsWebMAlpha(function(result)
{
if (result)
{
alert('Supports WebM Alpha');
}
else
{
alert('Doesn\'t support WebM Alpha');
}
});
There are no properties exposed giving any information about the video and its channels.
The only way to do this is either:
Knowing in advance, incorporate that knowledge with the data and serve it to the browser when video is requested as meta-data
Use a canvas to analyze the image data
Load the file as binary data, then parse the webm format manually to extract this information. Do-able but very inconvenient as the complete file must be downloaded, and of course a parser must be made.
If you don't know in advance, or have no way to supply the metadata, then canvas is your best option.
Canvas
You can use a canvas to test for actual transparency, however, this do have CORS requirements (video must be on the same server, or the external server need to accept cross-origin usage).
Additionally you have to actually start loading the video which of course can have an impact on bandwidth as well as performance. You probably want to do this with a dynamically created video and canvas tag.
From there, it is fairly straight forward.
Create a small canvas
Draw a frame into it (one that is expected to have an alpha channel)
Extract the pixels (CORS requirements here)
Loop through the buffer using a Uint32Array view and check for alpha channel for values < 255 (pixel & 0xff000000 !== 0xff000000).
This is fairly fast to do, you can use a frame size of half or even smaller.
Related
I am currently using webcam (not native camera) on a web page to take a photo on users' mobile phone. Like this:
var video: HTMLVideoElement;
...
var context = canvas.getContext('2d');
context.drawImage(video, 0, 0, width, height);
var jpegData = canvas.toDataURL('image/jpeg', compression);
In such a way, I can now successfully generate a JPEG image data from web camera, and display it on the web page.
However, I found that the EXIF data is missing.
according to this:
Canvas.drawImage() will ignore all EXIF metadata in images,
including the Orientation. This behavior is especially troublesome
on iOS devices. You should detect the Orientation yourself and use
rotate() to make it right.
I would love the JPEG image contain the EXIF GPS data. Is there a simple way to include camera EXIF data during the process?
Thanks!
Tested on Pixel 3 - it works. Please note - sometimes it does not work with some desktop web-cameras. you will need exif-js to get the EXIF object from example.
const stream = await navigator.mediaDevices.getUserMedia({ video : true });
const track = stream.getVideoTracks()[0];
let imageCapture = new ImageCapture(track);
imageCapture.takePhoto().then((blob) => {
const newFile = new File([blob], "MyJPEG.jpg", { type: "image/jpeg" });
EXIF.getData(newFile, function () {
const make = EXIF.getAllTags(newFile);
console.log("All data", make);
});
});
unfortunately there's no way to extract exif from canvas.
Although, if you have access to jpeg, you can extract exif from that. For that I'd recommend exifr instead of widely popular exif-js because exif-js has been unmaintained for two years and still has breaking bugs in it (n is undefined).
With exifr you can either parse everything
exifr.parse('./myimage.jpg').then(output => {
console.log('Camera:', output.Make, output.Model))
})
or just a few tags
let output = await exifr.parse(file, ['ISO', 'Orientation', 'LensModel'])
First of, according to what I found so far, there is no way to include exif data during canvas context drawing.
Second, there is a way to work around, which is to extract the exif data from the original JPEG file, then after canvas context drawing, put the extracted exif data back into the newly drawn JPEG file.
It's messy and a little hacky, but for now this is the work around.
Thanks!
I am attempting to download an entire canvas image using canvas.toDataURL(). The image is a map rendered on the canvas (Open Layers 3).
In Firefox I can use the following to download the map on the click of a link:
var exportPNGElement = document.getElementById('export_image_button');
if ('download' in exportPNGElement) {
map.once('postcompose', function(event) {
var canvas = event.context.canvas;
exportPNGElement.href = canvas.toDataURL('image/png');
});
map.renderSync();
} else {
alert("Sorry, something went wrong during our attempt to create the image");
}
However in Chrome and Opera I'm hitting a size limit on the link. I have to physically make the window smaller for the download to work.
There are size limit differences between browsers, Chrome is particularly limiting. A similar post here (over 2 years old now) suggests an extensive server side workaround:
canvas.toDataURL() for large canvas
Is there a client side work around for this at all?
Check out toBlob https://developer.mozilla.org/en-US/docs/Web/API/HTMLCanvasElement/toBlob
canvas.toBlob(function(blob) {
exportPNGElement.href = URL.createObjectURL(blob);
});
browser support is not as awesome as toDataURL though. But Chrome and Firefox have it, so it solves your biggest issue. The mdn link above also has a polyfill based on toDataURL, so you get the best possible support.
Just in case you didn't know, you can also dramatically reduce the size using jpeg compression
exportPNGElement.href = canvas.toDataURL('image/jpeg', 0.7);
I'm not yet very familiar with HTML5 but have been looking for a project to delve into it.
Would the following functionality be possible using HTML5 and camera access?
Stage1: live camera replay with adjustable delay (aka delayed mirror)
Stage2: selecting parts of the previously recorded live stream and have replay options available (continuous loop, slow motion, drawing into the picture etc.)
Ideally this should run on an Android tablet.
This is meant as an application to provide immediate visual feedback for coaches and athletes.
Thanks for any feedback, it is much appreciated! :)
Tom
There are actually a few js libs that can record a webcam feed. Check out RecordRTC. Here is some example code that might work (I haven't tested).
navigator.getUserMedia = navigator.getUserMedia ||
navigator.webkitGetUserMedia || navigator.mozGetUserMedia;
function gotVideoStream(localMediaStream) {
var video = document.querySelector("video");
var recordRTC = RecordRTC(mediaStream);
recordRTC.startRecording();
recordRTC.stopRecording(function(videoURL) {
var playbackVideo = document.getElemenById('playback-vid');
playbackVideo.src = videoURL; // set src for playback
playbackVideo.playbackRate = .5; // slow down playback
});
// set src for live preview
video.src = window.URL.createObjectURL(localMediaStream);
video.play();
}
function errorCallback(error){
console.log("navigator.getUserMedia error: ", error);
}
// get things rolling
navigator.getUserMedia({video: true}, gotVideoStream, error);
If that doesn't work, Google the subject for more resources.
The MDN tutorial on taking pictures with a webcam provides most of the pieces you need to implement this in a simple way.
Request a video media stream and connect it to a video element.
Draw the video element to a canvas.
Copy the canvas either to a data URL or raw image data.
After a delay show it on another canvas or in an img element.
Here is an example I wrote implementing a delayed mirror.
This is fine for a few seconds of video. For example, I can practice dance moves with it. Recording and playing back longer streams, you might run into memory problems.
I need to take HTML5 canvas output as video or swf png sequence.
I found the following link on stackoverflow for capturing images.
Capture HTML Canvas as gif/jpg/png/pdf?
But can anyone suggest if we want the output to be video or swf of png sequence?
EDIT:
Ok now I understood how to capture the canvas data to store on server, I tried it and it is working fine if I use only shapes, rectangle or some other graphic, but not if I draw external images on canvas element.
Can anyone tell me how to capture canvas data completely whether we use graphic or external images for drawing on canvas?
I used the following code:
var cnv = document.getElementById("myCanvas");
var ctx = cnv.getContext("2d");
if(ctx)
{
var img = new Image();
ctx.fillStyle = "rgba(255,0,0,0.5)";
ctx.fillRect(0,0,300,300);
ctx.fill();
img.onload = function()
{
ctx.drawImage(img, 0,0);
}
img.src = "my external image path";
var data = cnv.toDataURL("image/png");
}
after taking the canvas data into my "data" variable I created a new canvas and draw the captured data on that, the red rectangle drawn on the second canvas but that external image doesn't.
Thanks in advance.
I would suggest:
Use setInterval to capture the contents of your Canvas as a PNG data URL.
function PNGSequence( canvas ){
this.canvas = canvas;
this.sequence = [];
};
PNGSequence.prototype.capture = function( fps ){
var cap = this;
this.sequence.length=0;
this.timer = setInterval(function(){
cap.sequence.push( cap.canvas.toDataURL() );
},1000/fps);
};
PNGSequence.prototype.stop = function(){
if (this.timer) clearInterval(this.timer);
delete this.timer;
return this.sequence;
};
var myCanvas = document.getElementById('my-canvas-id');
var recorder = new PNGSequence( myCanvas );
recorder.capture(15);
// Record 5 seconds
setTimeout(function(){
var thePNGDataURLs = recorder.stop();
}, 5000 );
Send all these PNG DataURLs to your server. It'll be a very large pile of data.
Using whatever server-side language you like (PHP, Ruby, Python) strip the headers from the data URLs so that you are left with just the base64 encoded PNGs
Using whatever server-side language you like, convert the base64 data to binary and write out temporary files.
Using whatever 3rd party library you like on the server, convert the sequence of PNG files to a video.
Edit: Regarding your comment of external images, you cannot create a data URL from a canvas that is not origin-clean. As soon as you use drawImage() with an external image, your canvas is tainted. From that link:
All canvas elements must start with their origin-clean set to true.
The flag must be set to false if any of the following actions occur:
[...]
The element's 2D context's drawImage() method is called with an HTMLImageElement or an HTMLVideoElement whose origin is not the same as that of the Document object that owns the canvas element.
[...]
Whenever the toDataURL() method of a canvas element whose origin-clean flag is set to false is called, the method must raise a SECURITY_ERR exception.
Whenever the getImageData() method of the 2D context of a canvas element whose origin-clean flag is set to false is called with otherwise correct arguments, the method must raise a SECURITY_ERR exception.
To start out, you want to capture the pixel data from the canvas on a regular interval (using JavaScript timers probably). You can do this by calling context.getImageData on the canvas's context. That will create a series of images that you can turn into a video stream.
http://www.whatwg.org/specs/web-apps/current-work/multipage/the-canvas-element.html#pixel-manipulation
I want to capture the first frame of an embedded video as an image without using any server side scripting. Probably with javascript, is it possible?
Actually, pretty sure you can, using HTML5. Take a look at this link: HTML5 Video Destruction.
It's copying the video frame into another canvas every 33ms. You could play around with this and see if you can capture the first frame when the video starts running.
I can look into this further if you like (it fascinates me)
EDIT: oh my GOD THIS IS COOL. I just came up with a solution. Go to sambro.is-super-awesome.com/videofirstframe/
You need to open this in Google Chrome. Firefox doesn't support mp4 (I think).
First time I've ever done anything with HTML5, I CANNOT wait until this is in the majority of browsers :)
EDIT EDIT: Okay I uploaded the .ogg version of this video too, and setup my web server to handle the video type correctly, Firefox should work in this little example too.
EDIT EDIT EDIT: Nitpickers wanting source up here, well here it is:
// Create a video element.
var vid = document.createElement("video");
// We don't want it to start playing yet.
vid.autoplay = false;
vid.loop = false;
// No need for user to see the video itself.
vid.style.display = "none";
// This will fire when there's some data loaded for the video, should be at least 1 frame here.
vid.addEventListener("loadeddata", function()
{
// Let's wait another 100ms just in case?
setTimeout(function()
{
// Create a canvas element, this is what user sees.
var canvas = document.createElement("canvas");
// Set it to same dimensions as video.
canvas.width = vid.videoWidth;
canvas.height = vid.videoHeight;
// Put it on page.
document.getElementById("done").innerHTML = "";
document.getElementById("done").appendChild(canvas);
// Get the drawing context for canvas.
var ctx = canvas.getContext("2d");
// Draw the current frame of video onto canvas.
ctx.drawImage(vid, 0, 0);
// Done!
});
}, false);
// Have to include .ogv for firefox. I don't think this is working atm because my webserver isn't serving up
// videos properly.
if(BrowserDetect.browser == "Firefox")
{
var source = document.createElement("source");
source.src = "BigBuckBunny_640x360.ogv";
source.type = "video/ogg";
vid.appendChild(source);
}
else
{
var source = document.createElement("source");
source.src = "BigBuckBunny_640x360.mp4";
source.type = "video/mp4";
vid.appendChild(source);
}
// Add video to document to start loading process now.
document.body.appendChild(vid);