We Keep Coding

JavaScript - How to take a picture in webcam with EXIF data?

I am currently using webcam (not native camera) on a web page to take a photo on users' mobile phone. Like this:
var video: HTMLVideoElement;
...
var context = canvas.getContext('2d');
context.drawImage(video, 0, 0, width, height);
var jpegData = canvas.toDataURL('image/jpeg', compression);
In such a way, I can now successfully generate a JPEG image data from web camera, and display it on the web page.
However, I found that the EXIF data is missing.
according to this:
Canvas.drawImage() will ignore all EXIF metadata in images,
including the Orientation. This behavior is especially troublesome
on iOS devices. You should detect the Orientation yourself and use
rotate() to make it right.
I would love the JPEG image contain the EXIF GPS data. Is there a simple way to include camera EXIF data during the process?
Thanks!

Tested on Pixel 3 - it works. Please note - sometimes it does not work with some desktop web-cameras. you will need exif-js to get the EXIF object from example.
const stream = await navigator.mediaDevices.getUserMedia({ video : true });
const track = stream.getVideoTracks()[0];
let imageCapture = new ImageCapture(track);
imageCapture.takePhoto().then((blob) => {
const newFile = new File([blob], "MyJPEG.jpg", { type: "image/jpeg" });
EXIF.getData(newFile, function () {
const make = EXIF.getAllTags(newFile);
console.log("All data", make);
});
});

unfortunately there's no way to extract exif from canvas.
Although, if you have access to jpeg, you can extract exif from that. For that I'd recommend exifr instead of widely popular exif-js because exif-js has been unmaintained for two years and still has breaking bugs in it (n is undefined).
With exifr you can either parse everything
exifr.parse('./myimage.jpg').then(output => {
console.log('Camera:', output.Make, output.Model))
})
or just a few tags
let output = await exifr.parse(file, ['ISO', 'Orientation', 'LensModel'])

First of, according to what I found so far, there is no way to include exif data during canvas context drawing.
Second, there is a way to work around, which is to extract the exif data from the original JPEG file, then after canvas context drawing, put the extracted exif data back into the newly drawn JPEG file.
It's messy and a little hacky, but for now this is the work around.
Thanks!

Get pixel data back from programmatically generated image

I generate array of pixels in client side JavaScript code and convert it to a blob. Then I pass URL of the blob as image.src and revoke it at image.onload callback. I don't keep any references to the data, generated by the previous steps, so this data may be freed by GC.
There are many images generated this way, and it works fine. But sometimes user may want to save the generated image by clicking on a Save button near the image. I don't want to generate image again, because generation is slow, and the image is already generated and is visible on screen. So I want to get my pixels back from the image. I tried to create canvas again, draw image on it and then call toBlob, but browser treats this image as cross origin and throws an exception: "Failed to execute 'toBlob' on 'HTMLCanvasElement': tainted canvases may not be exported". Similar errors I get with canvas.toDataUrl and canvasContext.getImageData.
Is there a workaround for this problem?
I also tried to create canvases instead of images, but when I create the second canvas, the content of the first one clears.
Added
This error occurs only in the Chrome and other WebKit browsers. Firefox and MS Edge work fine. And when I commented out line of code that revoked blob url, this error disappeared - I can draw the image on the canvas and get its pixel data without CORS issues. But it is pointless to do so, because I already have blob that is not deleted.
But my page may generate many images - it depends on its user and is unlimited. Size of images is also unlimited - it may be useful to generate even 4096x4096 images. So I want to reduce memory consumption of my page as much as possible. And all these images should be downloadable. Generation of most images uses previously generated images, so to regenerate last image in a chain, I must to regenerate all images.
So I need a workaround only for Chrome browser.
Added 2
I tried to reproduce this problem in JS Fiddle, but couldn't. However locally my code doesn't work - I developed my app locally and I haven't tried running it on server. Create test.html file on your computer and open it in browser (locally, without server):
<html>
<body>
<pre id="log"></pre>
</body>
<script>
var log = document.getElementById("log");
var canvas = document.createElement("canvas");
canvas.width = canvas.height = 256;
var ctx = canvas.getContext("2d");
canvas.toBlob(function(blob) {
var img = new Image();
var blobUrl = URL.createObjectURL(blob);
img.src = blobUrl;
img.onload = function()
{
URL.revokeObjectURL(blobUrl);
ctx.drawImage(img, 0, 0);
try { canvas.toBlob(function(blob) { log.textContent += 'success\n'; }); }
catch(e) {log.textContent += e.message + '\n';}
};
});
</script>
</html>
It will print Failed to execute 'toBlob' on 'HTMLCanvasElement': Tainted canvases may not be exported..
So, I think, my workaround is to detect that page is run on combination of WebKit browser and file:/// protocol. And I can to defer revoking blob URL until page unload only for this combination.

That indeed sounds like a bug in chrome's implementation, you may want to report it.
What seems to happen is that chrome only checks if the image has been loaded through a clean origin when it is drawn for the first time on the canvas.
Since at this time, you already did revoke the blobURI, it can't map this URI to a clean origin (file:// protocol is quite strict on chrome).
But there seems to be a simple workaround:
By drawing this image on a [the reviver]* canvas before revoking the blobURI, chrome will mark the image as clean, and will remember it.
*[edit] Actually it seems it needs to be the same canvas...
So you can simply create your export canvas before-hand, and draw each images on it (to save memory you can even set it to 1x1px when you don't use it for the export) before you do revoke the image's src:
// somewhere accessible to the export logic
var reviver = document.createElement('canvas').getContext('2d');
// in your canvas to img logic
canvas.toBlob(function(blob){
var img = new Image();
img.onload = function(){
reviver.canvas.width = reviver.canvas.height = 1; // save memory
reviver.drawImage(this, 0,0); // mark our image as origin clean
URL.revokeObjectURL(this.src);
}
img.src = URL.createObjectURL(blob);
probablySaveInAnArray(img);
};
// and when you want to save your images to disk
function reviveBlob(img){
reviver.canvas.width = img.width;
reviver.canvas.height = img.height;
reviver.drawImage(img,0,0);
reviver.canvas.toBlob(...
}
But note that this method will create a new Blob, not retrieve the previous one, which has probably been Collected by the GarbageCollector at this time. But it is unfortunately your only way since you did revoke the blobURI...

html2canvas , save render canvas as a gif instead of a png?

I'm using html2canvas to save a snapshot from the webcam as an image.
However, it save only in png, I'm trying to save it as a gif, but can not find out how to do this
So far this is my function:
renderCanvasImage: function(){
setTimeout(function () {
// Add image with Quote to Canvas (hidden).
html2canvas($('.snap'), {
onrendered: function (canvas) {
document.body.appendChild(canvas).id = 'hidden';
var canvas = document.getElementById('hidden');
var image = new Image();
//Create a new Image with url
image.src = canvas.toDataURL("image/.png");
// Look at URI only and assign to localStorage
imageURI = image.src;
localStorage.setItem('image', imageURI);
//****TODO better removal*/
$('#cameraContainer, .wrapperInfo').hide();
$('#result a, #result img').fadeOut(100).remove();
$(image).appendTo('#result');
$('#result').fadeIn(200);
//Send Data to DB
tibo.setData();
//PopUp Message
tibo.popupMsg();
}
});
}, 1000);
},
I tried to replace the following:
image.src = canvas.toDataURL("image/.png");
By jpg of gif, but it doesn't change anything.... any tips to make this work will be amazing !!
Thanks a lot !!

You said in the comments above that you've got it working, however I still feel the need to tell you that the supported mime types of toDataUrl depend on the browser.
You can test it here https://jsfiddle.net/v91y0zqr/
Here's a visual example with even more mime types: http://kangax.github.io/jstests/toDataUrl_mime_type_test/
All browsers I've tested (Firefox, Chrome, Opera, IE) did support image/png and image/jpeg
Additionally, Chrome could export image/webp
Additionally, Firefox could export image/bmp
Results may differ for you.
So while in theory canvas.toDataURL("image/gif"); should create a GIF image, the browser may still decide to create a PNG (it's the default fallback).
You can read more about toDataUrl here: https://developer.mozilla.org/en-US/docs/Web/API/HTMLCanvasElement/toDataURL

How to (feature) detect if browser supports WebM alpha transparency?

I'm integrating a *.webm Video with alpha transparency. At the moment, the transparency is only supported in Chrome and Opera. (Demo: http://simpl.info/videoalpha/) Firefox for example plays the video as it supports the WebM format, but instead of the transparency, there's a black background.
My plan is to display the video poster image instead of the video, if the browser does not support alpha transparency. So the video should only play, if the browser supports WebM alpha transparency. I know how to detect the browser or the rendering engine and therefore play the video (see code below) - but is there a "feature detection" way?
var supportsAlphaVideo = /Chrome/.test(navigator.userAgent) && /Google Inc/.test(navigator.vendor) || (/OPR/.test (navigator.userAgent));
if (supportsAlphaVideo) {
document.querySelector(".js-video").play();
}
See also http://updates.html5rocks.com/2013/07/Alpha-transparency-in-Chrome-video

Here's a working solution to test for alpha support in WebM.
I basically combined Capture first frame of an embedded video and check_webp_feature
The video used to test with is base64-encoded into the source. It's actually a tiny VP9 WebM video encoded using:
ffmpeg -i alpha.png -c:v libvpx-vp9 alpha.webm
If you want to test for VP8 alpha support instead, just encode your own and remove the -vp9. alpha.png is a 64x64 pixel 100% transparent PNG image.
var supportsWebMAlpha = function(callback)
{
var vid = document.createElement('video');
vid.autoplay = false;
vid.loop = false;
vid.style.display = "none";
vid.addEventListener("loadeddata", function()
{
document.body.removeChild(vid);
// Create a canvas element, this is what user sees.
var canvas = document.createElement("canvas");
//If we don't support the canvas, we definitely don't support webm alpha video.
if (!(canvas.getContext && canvas.getContext('2d')))
{
callback(false);
return;
}
// Get the drawing context for canvas.
var ctx = canvas.getContext("2d");
// Draw the current frame of video onto canvas.
ctx.drawImage(vid, 0, 0);
if (ctx.getImageData(0, 0, 1, 1).data[3] === 0)
{
callback(true);
}
else
{
callback(false);
}
}, false);
vid.addEventListener("error", function()
{
document.body.removeChild(vid);
callback(false);
});
vid.addEventListener("stalled", function()
{
document.body.removeChild(vid);
callback(false);
});
//Just in case
vid.addEventListener("abort", function()
{
document.body.removeChild(vid);
callback(false);
});
var source = document.createElement("source");
source.src="data:video/webm;base64,GkXfowEAAAAAAAAfQoaBAUL3gQFC8oEEQvOBCEKChHdlYm1Ch4ECQoWBAhhTgGcBAAAAAAACBRFNm3RALE27i1OrhBVJqWZTrIHlTbuMU6uEFlSua1OsggEjTbuMU6uEHFO7a1OsggHo7AEAAAAAAACqAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAVSalmAQAAAAAAADIq17GDD0JATYCNTGF2ZjU3LjU3LjEwMFdBjUxhdmY1Ny41Ny4xMDBEiYhARAAAAAAAABZUrmsBAAAAAAAARq4BAAAAAAAAPdeBAXPFgQGcgQAitZyDdW5khoVWX1ZQOYOBASPjg4QCYloA4AEAAAAAAAARsIFAuoFAmoECU8CBAVSygQQfQ7Z1AQAAAAAAAGfngQCgAQAAAAAAAFuhooEAAACCSYNCAAPwA/YAOCQcGFQAADBgAABnP///NXgndmB1oQEAAAAAAAAtpgEAAAAAAAAk7oEBpZ+CSYNCAAPwA/YAOCQcGFQAADBgAABnP///Vttk7swAHFO7awEAAAAAAAARu4+zgQC3iveBAfGCAXXwgQM=";
source.addEventListener("error", function()
{
document.body.removeChild(vid);
callback(false);
});
vid.appendChild(source);
//This is required for IE
document.body.appendChild(vid);
};
supportsWebMAlpha(function(result)
{
if (result)
{
alert('Supports WebM Alpha');
}
else
{
alert('Doesn\'t support WebM Alpha');
}
});

There are no properties exposed giving any information about the video and its channels.
The only way to do this is either:
Knowing in advance, incorporate that knowledge with the data and serve it to the browser when video is requested as meta-data
Use a canvas to analyze the image data
Load the file as binary data, then parse the webm format manually to extract this information. Do-able but very inconvenient as the complete file must be downloaded, and of course a parser must be made.
If you don't know in advance, or have no way to supply the metadata, then canvas is your best option.
Canvas
You can use a canvas to test for actual transparency, however, this do have CORS requirements (video must be on the same server, or the external server need to accept cross-origin usage).
Additionally you have to actually start loading the video which of course can have an impact on bandwidth as well as performance. You probably want to do this with a dynamically created video and canvas tag.
From there, it is fairly straight forward.
Create a small canvas
Draw a frame into it (one that is expected to have an alpha channel)
Extract the pixels (CORS requirements here)
Loop through the buffer using a Uint32Array view and check for alpha channel for values < 255 (pixel & 0xff000000 !== 0xff000000).
This is fairly fast to do, you can use a frame size of half or even smaller.

Canvas.toDataUrl sometimes truncated in Chrome/Firefox

In the HTML5 version of my LibGDX game, sometimes canvas.toDataUrl("image/png") returns a truncated string yielding a black image.
CanvasElement canvas = ((GwtApplication)Gdx.app).getCanvasElement();
String dataUrl = canvas.toDataUrl("image/png");
Window.open(dataUrl, "_blank", "");
The odd part is that sometimes it works. When it does work I get a ~100KiB image as expected, and the new window opens with an address bar just saying "data:". I can send this to a webservice and translate from Base64 into the bytes of a proper PNG and OSX preview shows it just fine too.
When it doesn't work the new window shows a black image of the correct dimensions, and an address bar with Base64-encoded data in (starting data:image/png;base64,iVBORw0KGgoAAAAN...), but ending in an elipsis that appears to be rendered by the browser UI rather than three periods in the actual data string. The data in this case is ~31KiB. When I try transcoding this via my webservice, I get the same black rectangle.
I see this happen in both Chome and Firefox.
Any ideas? The code to get the canvas contents is very simple, so I can't see how I can be doing that wrong. I'm thinking either a bug in the browsers, or some kind of timing issue with LibGDX and rendering?

This was caused by LibGDX not preserving the drawing buffer. The LibGDX guys very kindly fixed this in a nightly build that is now available.
Updating to the latest build of 1.0-SNAPSHOT and setting the below flag now works reliably:
#Override
public GwtApplicationConfiguration getConfig () {
if(config == null)
{
config = new GwtApplicationConfiguration(1280, 960);
config.preserveDrawingBuffer = true;
}
return config;
}