HTML5 Canvas captureStream - Problem with hi-res images [duplicate] - javascript

I've looked at a sample for Streaming from canvas to video element so I can see that the principle works but i can't get it to play/display a static image in the video.
Here is my code so far with an image borrowed from stackoverflow. How can I change my code to display the canvas as a video?
const canvas = document.getElementById('viewport');
const context = canvas.getContext('2d');
const video = document.getElementById('videoPlayBack');
make_base();
function make_base() {
base_image = new Image();
base_image.onload = function () {
context.drawImage(base_image, 0, 0);
}
base_image.src = "https://cdn.sstatic.net/Img/ico-binoculars.svg?v=d4dbaac4eec9";
}
const stream = canvas.captureStream(25);
video.srcObject = stream;
<canvas id="viewport"></canvas>
<video id="videoPlayBack" playsinline autoplay muted></video>

You are drawing a cross-origin image, this will thus taint the canvas and mute the CanvasCaptureMediaStreamTrack, resulting in no data being passed.
Keep your canvas untainted if you wish to stream it:
const canvas = document.getElementById('viewport');
const context = canvas.getContext('2d');
const video = document.getElementById('videoPlayBack');
make_base();
function make_base() {
base_image = new Image();
base_image.onload = function () {
context.drawImage(base_image, 0, 0, 400, 300);
}
base_image.crossOrigin = "anonymous";
base_image.src = "https://upload.wikimedia.org/wikipedia/commons/4/47/PNG_transparency_demonstration_1.png";
}
const stream = canvas.captureStream(25);
video.srcObject = stream;
<canvas id="viewport" width="400" height="300"></canvas>
<video id="videoPlayBack" controls playsinline autoplay muted ></video>

You don't get to save from a canvas if you draw "not your images" on it, because of the canvas security model (the moment you draw any image to it that isn't same origin, and doesn't explicitly have a CORS header that okays your use, you no longer have access to the pixels - not through ImageData, not through toDataURL, etc. etc).
To prevent the canvas from flagging itself as "tainted", see https://developer.mozilla.org/en-US/docs/Web/HTML/CORS_enabled_image for what your options might be.

Related

Video capture of canvas element by MediaStream Recording API is not working

I am trying to record and download video of canvas element using official MediaStream Recording API
<!DOCTYPE html>
<html>
<body>
<h1>Lets test mediaRecorder</h1>
<canvas id="myCanvas" width="200" height="100" style="border:1px solid #d3d3d3;">
Your browser does not support the HTML canvas tag.
</canvas>
<script>
var c = document.getElementById("myCanvas");
var ctx = c.getContext("2d");
ctx.font = "30px Arial";
ctx.fillText("Hello World", 10, 50);
const stream = c.captureStream(25);
var recordedChunks = [];
console.log(stream);
var options = { mimeType: "video/webm; codecs=vp9" };
mediaRecorder = new MediaRecorder(stream, options);
mediaRecorder.ondataavailable = handleDataAvailable;
mediaRecorder.start();
function handleDataAvailable(event) {
console.log("data-available");
if (event.data.size > 0) {
recordedChunks.push(event.data);
console.log(recordedChunks);
download();
} else {
// ...
}
}
function download() {
var blob = new Blob(recordedChunks, {
type: "video/webm"
});
var url = URL.createObjectURL(blob);
var a = document.createElement("a");
document.body.appendChild(a);
a.style = "display: none";
a.href = url;
a.download = "test.webm";
a.click();
window.URL.revokeObjectURL(url);
}
// demo: to download after 10 sec
setTimeout(event => {
console.log("stopping");
mediaRecorder.stop();
}, 10000);
</script>
</body>
</html>
code is working and I am able to download test.webm but I guess that does not have any data as I am not seeing any content while playing this file in VLC Media Player
What I am missing to make it working?
You are facing a few bugs here.
First one is a bit excusable, Chrome doesn't generate seekable webm files. This is because of how media files are built and how the MediaRecorder API works. For them to be able to add this information they'd have to keep the chunk where the metadata is in order to add this information when the recording is done.
I'm not too sure what Firefox does differently here, but VLC prefers their file.
An other Chrome bug, a lot less excusable, is that they don't pass a new frame to the MediaRecorder until we draw on the source canvas again.
So since in your case you are not drawing anything after you started the MediaRecorder, you'll get nothing in the output...
To workaround that, simply drawing a frame right before we stop the recorder should have been enough, except that there is nothing letting us know exactly when the browser will push that frame to the recorder...
So the only working workaround here is to draw on the canvas continuously while we record it. The good thing is that it doesn't need to be painting anything new: we can trick the browser in thinking something new was painted by drawing a transparent rectangle.
A final note, while Chrome does support exporting the canvas with transparency, not all browsers can and even when supported most players have a default black background. So be sure to draw yourself a background in an other color when you record it.
All that said, here is a fixed demo:
var c = document.getElementById("myCanvas");
var ctx = c.getContext("2d");
// draw a white background
ctx.fillStyle = "white";
ctx.fillRect(0, 0, c.width, c.height);
ctx.fillStyle = "black";
ctx.font = "30px Arial";
ctx.fillText("Hello World", 10, 50);
const stream = c.captureStream(25);
var recordedChunks = [];
var options = {};
mediaRecorder = new MediaRecorder(stream, options);
mediaRecorder.ondataavailable = handleDataAvailable;
mediaRecorder.start();
// Chrome requires we draw on the canvas while recording
mediaRecorder.onstart = animationLoop;
function animationLoop() {
// draw nothing, but still draw
ctx.globalAlpha = 0;
ctx.fillRect(0, 0, 1, 1);
// while we're recording
if (mediaRecorder.state !== "inactive") {
requestAnimationFrame(animationLoop);
}
}
// wait for the stop event to export the final video
// the dataavailable can fire before
mediaRecorder.onstop = (evt) => download();
function handleDataAvailable(event) {
recordedChunks.push(event.data);
}
function download() {
var blob = new Blob(recordedChunks, {
type: "video/webm"
});
var url = URL.createObjectURL(blob);
// exporting to a video element for that demo
// the downloaded video will still not work in some programs
// For this one would need to fix the markers using something like ffmpeg.
var video = document.getElementById('video');
video.src = url;
// hack to make the video seekable in the browser
// see https://stackoverflow.com/questions/38443084/
video.onloadedmetadata = (evt) => {
video.currentTime = 10e6;
video.addEventListener("seeked", () => video.currentTime = 0, {
once: true
})
}
}
setTimeout(() => {
console.clear();
mediaRecorder.stop();
}, 10000);
console.log("please wait while recording (10s)");
<h1>Lets test mediaRecorder</h1>
<canvas id="myCanvas" width="200" height="100" style="border:1px solid #d3d3d3;">
Your browser does not support the HTML canvas tag.
</canvas>
<video controls id="video"></video>

set canvas data to an input

I want to take a picture from the user and send it to the server without ajax.
I searched and found this code that takes the photo from video and draws it on canvas. now I want to set that image to an input on form but I don't know how.
this is HTML code:
<video id="player" controls autoplay></video>
<button id="capture" onclick="$('#loader').hide()">Capture</button>
<canvas id="canvas" width=320 height=240></canvas>
<input id="image-input">
and this is javascript code :
const player = document.getElementById('player');
const canvas = document.getElementById('canvas');
const context = canvas.getContext('2d');
const captureButton = document.getElementById('capture');
const constraints = {
video: true,
};
captureButton.addEventListener('click', () => {
// Draw the video frame to the canvas.
context.drawImage(player, 0, 0, canvas.width, canvas.height);
player.srcObject.getVideoTracks().forEach(track => track.stop());
var Pic = document.getElementById("canvas").toDataURL("image/png");
Pic = Pic.replace(/^data:image\/(png|jpg);base64,/, "")
});
// Attach the video stream to the video element and autoplay.
navigator.mediaDevices.getUserMedia(constraints)
.then((stream) => {
player.srcObject = stream;
});
now I want to set this image as the image of input.
how can I do that?

How to use canvas for parsing imageData from video without appending canvas to the page?

I have Webcam class, that uses for capturing data from webcam. This class streames data to video tag and also draw it on canvas., Also I have QRScanner class, that uses for parsing QR code from imageData of canvas.
class Webcam {
constructor(videoTag, canvasTag) {
// using for real-time video capture
this.videoTag = videoTag;
// qr scanner parses imageData from this element
this.canvasTag = canvasTag;
// waiting for qr code here
this.captureArea = {
x: 100,
y: 60,
width: 120,
height: 120
}
// getting access to webcam
navigator.mediaDevices
.getUserMedia({
video: true
})
.then(stream => this.videoTag.srcObject = stream)
.catch(console.log);
}
capture() {
setInterval(() => {
let ctx = this.canvasTag.getContext('2d');
ctx.drawImage(this.videoTag, 0, 0, 320, 240);
// drawing capture area
ctx.strokeStyle = 'red';
// this arguments also should be passed into qr scanner
ctx.strokeRect(
this.captureArea.x,
this.captureArea.y,
this.captureArea.width,
this.captureArea.height
);
}, 100);
}
}
class QRScanner {
constructor(canvas, router, captureArea) {
this.canvas = canvas;
this.router = router;
this.captureArea = captureArea;
this.qrCode = null;
}
scan() {
setInterval(() => {
let imageData = this.canvas
.getContext('2d')
.getImageData(
this.captureArea.x,
this.captureArea.y,
this.captureArea.width,
this.captureArea.height
).data;
// parsing qr code from canvas
this.qrCode = jsQR(imageData, this.captureArea.width, this.captureArea.height);
if (this.qrCode) {
router.redirect(Router.pages.result, this.qrCode);
let resultPage = document.querySelector('#result .qr-code-data');
resultPage.innerHTML = this.qrCode.data;
}
}, 100);
}
}
<div id="scan">
<video id="video" width="320" height="240" autoplay title="Real-time video stream from webcam"></video>
<canvas id="preview" width="320" height="240" title="Capture area for QR code"></canvas>
</div>
This works as expected, but when I removes canvas element from page and trying to use temporary canvas (using document.createElement('canvas') without appending to the page) - this solution do not work. Does it possible to using temporary canvas for getting video imageData without appending this canvas to the page?
P.S. I'm using https://github.com/cozmo/jsQR
Canvas elements have a default width height when they aren't explicitly set. Since you never set those for you created canvas it is going to default to 300 x 150, at least for Chrome might be different for other browsers implementations.
var canvas = document.createElement("canvas");
console.log(canvas.width,canvas.height)
And since this default size is different than the size that you are trying to draw the image/video to there is going to be some cropping going on and therefore effecting your QR library from properly reading the image.
Just set the width and height to what you need
var canvas = document.createElement('canvas')
canvas.width = 320;
canvas.height = 240;

Play streaming video from video tag to canvas

i'm working on a website with video streaming, the video is actually in a video tag but i need to draw on it so i'm trying to send the streaming from the video tag to canvas with no luck, already read severeal similar questions over here with not luck either, any suggestion? this is the current way i'm testing with no luck:
var VIDEOELEMENT = document.getElementById("camchat"); //Video Tag
var canvas2 = document.getElementById('myCanvas'); //Canvas Tag
var ctx = canvas2.getContext('2d');
VIDEOELEMENT.addEventListener('play', () => {
console.log("INITIATING VIDEO PLAY");
function step() {
ctx.drawImage(VIDEOELEMENT, 0, 0, canvas2.width, canvas2.height)
requestAnimationFrame(step)
}
requestAnimationFrame(step);
})

javascript- unable to convert canvas to image data

guys!
I'm trying to make a demo using which a user will be able to capture an image from the web camera enabled through his browser and this image will be saved in the backend by passing it to a PHP service using AJAX call.
I'm able to capture as canvas but not being to convert it into image data.
I'm getting canvas is undefined error
Setup is pretty simple.
HTML
<video id="videoElement" autoplay></video>
<button id="snap">Snap Photo</button>
<canvas id="canvas" style="display:none" ></canvas>
JAVASCRIPT
// Grab elements, create settings, etc.
var video = document.getElementById('videoElement');
// Get access to the camera!
if(navigator.mediaDevices && navigator.mediaDevices.getUserMedia) {
// Not adding `{ audio: true }` since we only want video now
navigator.mediaDevices.getUserMedia({ video: true }).then(function(stream) {
video.src = window.URL.createObjectURL(stream);
video.play();
});
}
// Elements for taking the snapshot
var canvas = document.getElementById('canvas');
var context = canvas.getContext('2d');
var video = document.getElementById('videoElement');
// Trigger photo take
document.getElementById("snap").addEventListener("click", function() {
var cann= context.drawImage(video, 0, 0, 640, 480); //draw canvas
//then convert canvas to image so i can obtain dataurl
//and pass it to another function using AJAX
var img= convertCanvasToImage(cann);
console.log(img);
});
// Converts canvas to an image
function convertCanvasToImage(canvas)
{
var image = new Image();
image.src = canvas.toDataURL("image/png");
return image;
}
I'm pretty new to this field. I have one more question. Will this work on latest browsers?
Can anyone help me figure out the mistake I made?
FIDDLE
Reference:
https://davidwalsh.name/browser-camera
https://davidwalsh.name/convert-canvas-image
Just call this
var img= convertCanvasToImage();
function convertCanvasToImage()
{
var image = new Image();
image.src = canvas.toDataURL("image/png");
return image;
}
No need to pass canvas as you defined canvas above. But if you have convertCanvasToImage function in another file then you can use
var img= convertCanvasToImage(canvas);
This may help.
function demoFromHTML() {
html2canvas(document.getElementById("talltweets"), {
onrendered: function(canvas) {
var imageData = canvas.toDataURL('image/png',1.0);
}
});
}
Pass canvas object instead of cann
var img= convertCanvasToImage(canvas);
If you can try to avoid dataURL and use blob instead - it will save you more memory and load images faster
$canvas.toBlob(function(blob){
var url = URL.createObjectURL(blob)
var img = new Image
img.onload = function() {
URL.revokeObjecURL(this.src)
}
img.src = url
console.log(blob)
console.log(url)
})
<canvas id="$canvas">
there is polyfills you can use to support canvas#toBlob

Categories