I'm using Electron Desktop Capturer https://github.com/electron/electron/blob/master/docs/api/desktop-capturer.md to capture a screenshot of this stream at regular intervals. I'm using this code but for some reason I get an error:
function takeScr(stream) {
const video = localStream.getVideoTracks()[0];
console.log(video)
const canvas = document.createElement('canvas');
canvas.getContext("2d").drawImage(video, 0, 0, 300, 300, 0, 0, 300, 300);
}
At the moment I'm simply pressing a button to activate this take screenshot function after the stream has started playing. The console log shows the video track no problem with output:
MediaStreamTrack {kind: "video", id: "7a19a94f-6077-4e3d-b534-03d138b3f300", label: "Screen", enabled: true, muted: false, …}
but the canvas.getContext function throws the error:
Uncaught TypeError: Failed to execute 'drawImage' on 'CanvasRenderingContext2D': The provided value is not of type '(CSSImageValue or HTMLImageElement or SVGImageElement or HTMLVideoElement or HTMLCanvasElement or ImageBitmap or OffscreenCanvas)'
There are many questions on here about this error but none seem to solve my problem and none are regarding video streams. Some solutions were that the image was not loaded when trying to draw to canvas but since I'm pressing a button several seconds after the stream has started I'm sure it must be loaded?
Maybe I'm doing this the wrong way and there is a better way to take screenshots of the video source from Desktop Capturer?
I found an example in the following question regarding taking snapshots from a video.
You could do something like this:
document.getElementById("snap").addEventListener("click", function() {
snap();
});
// Get handles on the video and canvas elements
var video = document.querySelector('video');
var canvas = document.querySelector('canvas');
// Get a handle on the 2d context of the canvas element
var context = canvas.getContext('2d');
// Define some vars required later
var w, h, ratio;
// Add a listener to wait for the 'loadedmetadata' state so the video's dimensions can be read
video.addEventListener('loadedmetadata', function() {
// Calculate the ratio of the video's width to height
ratio = video.videoWidth / video.videoHeight;
// Define the required width as 100 pixels smaller than the actual video's width
w = video.videoWidth - 100;
// Calculate the height based on the video's width and the ratio
h = parseInt(w / ratio, 10);
// Set the canvas width and height to the values just calculated
canvas.width = w;
canvas.height = h;
}, false);
// Takes a snapshot of the video
function snap() {
// Define the size of the rectangle that will be filled (basically the entire element)
context.fillRect(0, 0, w, h);
// Grab the image from the video
context.drawImage(video, 0, 0, w, h);
}
<video width="400" controls>
<source src="https://www.w3schools.com/html/mov_bbb.mp4" type="video/mp4">
<source src="https://www.w3schools.com/html/mov_bbb.ogg" type="video/ogg">
Your browser does not support HTML5 video.
</video>
<canvas width="364" height="204"></canvas>
<button id="snap">Take screenshot</button>
JSFiddle
Related
I have a video player (video.js) in my application, and a function which takes a snapshot of the video.
The function create a canvas which draws the snapshot and convert the canvas into a DataURL.
But this doesn't work in Safari.
I'm getting a black screen on Safari.
the video tag looks like:
<video crossorigin="anonymous" playsinline="playsinline" id="player_2140_html5_api" class="vjs-tech" data-setup="{ "inactivityTimeout": 0 }" tabindex="-1" autoplay="autoplay" loop="loop"></video>
This is the function which creates the snapshot:
var canvas = document.createElement('canvas');
canvas.width = 640;
canvas.height = 480;
var ctx = canvas.getContext('2d');
var video = document.querySelector('video');
ctx.drawImage(video, 0, 0, canvas.width, canvas.height);
this.snapshotVideo = canvas.toDataURL('image/jpeg');
The generated dataUrl in Chrome is an image.
The generated dataUrl in Safari browser is a black rectangle.
In Safari the output is black instead of an image of the taken snapshot via the video, how can I solve this?
I resolved this problem on iOS...
I used setTimeout function:
setTimeout(() => {
ctx.drawImage(video, 0, 0, canvas.width, canvas.height);
canvas.toBlob(() => {
this.snapshotVideo = canvas.toDataURL('image/jpeg');
});
},100);
Same here.
The problem seen first time at Safari 15. Before it works well with your code snipped.
See last comments here:
https://bugs.webkit.org/show_bug.cgi?id=181663
or here:
https://bugs.webkit.org/show_bug.cgi?id=229611
I have the same issue. Calling toDataURL on safari 15 up (mac, iPad or iPhone) seems to occasionally return "" blank string. Previous versions of safari work fine. I've checked that it is not exceeding the browser canvas size limits which would return "data:," but that is not causing the issue and is returning a blank string.
I got the same problem on Safari iOS 15.5 & 15.6 for handling image drawing.
After calling drawImage, the toDataURL() always return blank image.
I changed some code to avoid using canvas in the drawImage and use image object as the first parameter.
var scrollTop = 100;
var oc2 = document.createElement('canvas');
if (oc2.getContext) {
var octx2 = oc2.getContext('2d');
oc2.width = 200;
oc2.height = 300;
var img1 = new Image();
img1.onload = function() {
octx2.drawImage(img1, 0, scrollTop, 200, 300, 0, 0, 200, 300);
var bg_data = oc2.toDataURL('image/png');
if (bg_data) {
drawing_board.setImg(bg_data);
drawing_board.initHistory();
}
};
img1.src = oc.toDataURL('image/png');
}
I have Webcam class, that uses for capturing data from webcam. This class streames data to video tag and also draw it on canvas., Also I have QRScanner class, that uses for parsing QR code from imageData of canvas.
class Webcam {
constructor(videoTag, canvasTag) {
// using for real-time video capture
this.videoTag = videoTag;
// qr scanner parses imageData from this element
this.canvasTag = canvasTag;
// waiting for qr code here
this.captureArea = {
x: 100,
y: 60,
width: 120,
height: 120
}
// getting access to webcam
navigator.mediaDevices
.getUserMedia({
video: true
})
.then(stream => this.videoTag.srcObject = stream)
.catch(console.log);
}
capture() {
setInterval(() => {
let ctx = this.canvasTag.getContext('2d');
ctx.drawImage(this.videoTag, 0, 0, 320, 240);
// drawing capture area
ctx.strokeStyle = 'red';
// this arguments also should be passed into qr scanner
ctx.strokeRect(
this.captureArea.x,
this.captureArea.y,
this.captureArea.width,
this.captureArea.height
);
}, 100);
}
}
class QRScanner {
constructor(canvas, router, captureArea) {
this.canvas = canvas;
this.router = router;
this.captureArea = captureArea;
this.qrCode = null;
}
scan() {
setInterval(() => {
let imageData = this.canvas
.getContext('2d')
.getImageData(
this.captureArea.x,
this.captureArea.y,
this.captureArea.width,
this.captureArea.height
).data;
// parsing qr code from canvas
this.qrCode = jsQR(imageData, this.captureArea.width, this.captureArea.height);
if (this.qrCode) {
router.redirect(Router.pages.result, this.qrCode);
let resultPage = document.querySelector('#result .qr-code-data');
resultPage.innerHTML = this.qrCode.data;
}
}, 100);
}
}
<div id="scan">
<video id="video" width="320" height="240" autoplay title="Real-time video stream from webcam"></video>
<canvas id="preview" width="320" height="240" title="Capture area for QR code"></canvas>
</div>
This works as expected, but when I removes canvas element from page and trying to use temporary canvas (using document.createElement('canvas') without appending to the page) - this solution do not work. Does it possible to using temporary canvas for getting video imageData without appending this canvas to the page?
P.S. I'm using https://github.com/cozmo/jsQR
Canvas elements have a default width height when they aren't explicitly set. Since you never set those for you created canvas it is going to default to 300 x 150, at least for Chrome might be different for other browsers implementations.
var canvas = document.createElement("canvas");
console.log(canvas.width,canvas.height)
And since this default size is different than the size that you are trying to draw the image/video to there is going to be some cropping going on and therefore effecting your QR library from properly reading the image.
Just set the width and height to what you need
var canvas = document.createElement('canvas')
canvas.width = 320;
canvas.height = 240;
Context
I'm trying to capture a snapshot through the client's video camera and save it as an image. I have a JSFiddle demo.
Problem
I'm not succeeding in getting the whole video area from the Video element to the Canvas with Context2D's drawImage. Althought the Video is 400 by 300
<video width="400" height="300" ...
and the canvas is the same 400 by 300
<canvas width="400" height="300" ...
and when I draw I specify for both source and target 400 by 300...
ctx.drawImage(video, 0, 0, 400, 300, 0, 0, 400, 300);
This is a sample of what is happening on my client.
The red framed element is the video and the green one is the canvas.
I suspect this is because the video stream is not 400 by 300 (not even 4:3 aspect ratio), but I couldn't find yet a solution for this or at least some indication in the HTML specs that this behavior is because of the video stream.
Does anyone have some more experience in this area and can point me in the right direction?
Sorry for the bad quality of the code in the fiddle.
When you set the HTMLElement's widthand height attributes, you're only setting its display size, not the size of the frame inside it.
Canvas2dContext.drawImage(video, x, y) will take the frame size, (which can be set by the mandatory of getUserMedia btw).
So you need either to set your canvas' width and height to the video.videoWidthand video.videoHeight properties to get the full quality image grabbed by the webcam, ot to call ctx.drawImage(video, 0,0, 400, 300) to rescale it on the canvas as it is scaled by CSS, or setting directly the mandatory during the getUserMedia call (latests specs are navigator.mediaDevices.getUserMedia(video: { width: 300, height:300 });) but unfortunately, for this latest, you can't be sure the browser will honor it, except by checking the video.videoHeight/Width ;-)
var ctx = rendering.getContext('2d');
btn.onclick = function() {
switch (select.value) {
case 'css rescale':
rendering.width = 400;
rendering.height = 300;
ctx.drawImage(video, 0, 0, 400, 300);
break;
case 'full quality':
rendering.width = video.videoWidth;
rendering.height = video.videoHeight;
ctx.drawImage(video, 0, 0);
break;
}
}
var initVideo = function(stream) {
video.src = URL.createObjectURL(stream);
video.play();
};
navigator.mediaDevices.getUserMedia({
video: true
})
.then(function(s) {
initVideo(s)
})
.catch(function(err) {
if (err.message.indexOf('secure origins') > -1) {
document.body.innerHTML = '<h3><a target="_blank" href="https://jsfiddle.net/x64ta5r3/">jsfiddle link for chrome and its https policy...</a></h3>(right click -> open Link...';
}
});
<button id="btn">take a snapshot</button>
<select id="select">
<option value="css rescale">css rescale</option>
<option value="full quality">full quality</option>
</select>
<video width="400" height="300" id="video"></video>
<canvas id="rendering"></canvas>
jsfiddle link for chrome and its https policy...
I am working on an image generator using HTML5 canvas and jQuery/JS. What I want to accomplish is the following.
The user can upload 2 or max 3 images (type should be png or jpg) to the canvas. The generated images should always be 1080x1920. If the hart uploads only 2 images, the images are 1080x960. If 3 images are uploaded, the size of each image should be 1080x640.
After they upload 2 or 3 images, the user can click on the download button to get the merged image, with a format of 1080x1920px.
It should make use of html canvas to get this done.
I came up with this:
HTML:
<canvas id="canvas">
Sorry, canvas not supported
</canvas><!-- /canvas.offers -->
<input id="fileInput" type="file" />
Generate
jQuery:
var canvas = document.getElementById('canvas');
var ctx = canvas.getContext('2d');
canvas.height = 400;
canvas.width = 800;
var img1 = loadImage('http://www.shsu.edu/dotAsset/0e829093-971c-4037-9c1b-864a7be1dbe8.png', main);
var img2 = loadImage('https://upload.wikimedia.org/wikipedia/commons/thumb/c/c5/Ikea_logo.svg/266px-Ikea_logo.svg.png', main);
var minImages = 2;
var imagesLoaded = 0;
function main() {
imagesLoaded += 1;
if(imagesLoaded >= minImages) {
ctx.clearRect(0,0,canvas.width,canvas.height);
ctx.save();
ctx.drawImage(img1, 0, 0);
// ctx.translate(canvas.height/2,canvas.width/2); // move to the center of the canvas
// ctx.rotate(270*Math.PI/180); // rotate the canvas to the specified degrees
// ctx.drawImage(img2,0,canvas.height/2);
ctx.translate(-canvas.height/2,canvas.width/2); // move to the center of the canvas
ctx.rotate(90*Math.PI/180); // rotate the canvas to the specified degrees
ctx.drawImage(img2,-img2.width/2,-img2.width/2);
ctx.restore(); // restore the unrotated context
}
}
function loadImage(src, onload) {
var img = new Image();
img.onload = onload;
img.src = src;
console.log(img);
return img;
}
Above code will create the canvas and place both images (that are now hard-coded in JS) to the created canvas. It will rotate 90 degrees, but it will not position to the right corner. Also the second image should be position beside the first one.
How can I do the rotation and positioning of each image side by side?
Working Fiddle: https://jsfiddle.net/8ww1x4eq/2/
Have a look at the updated jsFiddle, is that what you wanted?
Have a look here regarding image rotation
Updated jsFiddle, drawing multiple images.
Notice:
The save script was just a lazy way to make sure I've got the
external scripts loaded before I save the merged_image...
There is no synchornisation in the sample script, notice that addToCanvas
was called on image loaded event, there could be a race condition
here (but I doubt it, since the image is loaded to memory on
client-side)
function addToCanvas(img) {
// resize canvas to fit the image
// height should be the max width of the images added, since we rotate -90 degree
// width is just a sum of all images' height
canvas.height = max(lastHeight, img.width);
canvas.width = lastWidth + img.height;
ctx.clearRect(0, 0, canvas.width, canvas.height);
if (lastImage) {
ctx.drawImage(lastImage, 0, canvas.height - lastImage.height);
}
ctx.rotate(270 * Math.PI / 180); // rotate the canvas to the specified degrees
ctx.drawImage(img, -canvas.height, lastWidth);
lastImage = new Image();
lastImage.src = canvas.toDataURL();
lastWidth += img.height;
lastHeight = canvas.height;
imagesLoaded += 1;
}
PS: I've added some script to download the merged image, but it would fail. The error message was: "Uncaught SecurityError: Failed to execute 'toDataURL' on 'HTMLCanvasElement': Tainted canvases may not be exported."
I've done a quick Google search and it seemed to be related to Cross-origin resources. I assumed that it wouldn't be an issue with FileReader. I haven't had time to test that so please test it (and please let me know :) It works with FileReader!
You can use toDataURL. But in this way user must do something like Save image as...
var img = canvas.toDataURL("image/png");
And then set for example img result src:
$("#result").attr("src",img);
Canvas is already an Image.
The canvas and img are interchangeable so there is no need to add the risky step of canvas.toDataURL which can fail depending on the image source domain. Just treat the canvas as if it were and img and put it in the DOM. Converting to a jpg does not save space (actually a resource hungry operation) as the an img needs to be decoded before it can be displayed.
var canvas = document.getElementById('canvas');
var ctx = canvas.getContext('2d');
canvas.height = 400;
canvas.width = 800;
document.body.appendChild(canvas); // add to the end of the document
// or add it to a containing element
var container = document.getElementById("containerID"); // or use JQuery
if(container !== null){
container.appendChild(canvas);
}
I want to know if it's possible to play music with the HTML5 audio tag at a specific time without loading all the data beforehand: so the music can start directly and not wait for a full download before playing (like Youtube does when you jump in the video file).
If I'm not mistaken, I think it does this out of the box (at least in Firefox).
The Mozilla Developer Network offers a pretty good tutorial that should help you in the right direction. If you're interested in skipping around programatically, the tutorial shows how this can be done using Javascript.
They also offer a really cool visualization of how the <audio> element buffers data here. The red bars beneath the audio player show which blocks of data have been downloaded. If you skip around, it'll become evident that there's gaps of unloaded audio data between segments of the song that you didn't listen to yet (shown in grey).
The JSBin doesn't work in Chrome, but it will work in Firefox. Here's the code from the link:
<p>
<audio id="my-audio" controls>
<source src="http://jPlayer.org/audio/mp3/Miaow-07-Bubble.mp3" type="audio/mpeg">
<source src="http://jPlayer.org/audio/ogg/Miaow-07-Bubble.ogg" type="audio/ogg">
</audio>
</p>
<p>
<canvas id="my-canvas" width="300" height="20">
</canvas>
</p>
<script>
window.onload = function(){
var myAudio = document.getElementById('my-audio');
var myCanvas = document.getElementById('my-canvas');
var context = myCanvas.getContext('2d');
context.fillStyle = 'lightgray';
context.fillRect(0, 0, myCanvas.width, 20);
context.fillStyle = 'red';
context.strokeStyle = 'white';
var inc = myCanvas.width / myAudio.duration;
// display TimeRanges
myAudio.addEventListener('seeked', function() {
for (i = 0; i < myAudio.buffered.length; i++) {
var startX = myAudio.buffered.start(i) * inc;
var endX = myAudio.buffered.end(i) * inc;
context.fillRect(startX, 0, endX, 20);
context.rect(startX, 0, endX, 20);
context.stroke();
}
});
}
</script>