I'm creating a PDF output tool using jsPDF but need to add multiple pages, each holding a canvas image of a video frame.
I am stuck on the logic as to the best way to achieve this as I can't reconcile how to queue the operations and wait on events to achieve the best result.
To start I have a video loaded into a video tag and can get or set its seek point simply with:
video.currentTime
I also have an array of video seconds like the following:
var vidSecs = [1,9,13,25,63];
What I need to do is loop through this array, seek in the video to the seconds defined in the array, create a canvas at these seconds and then add each canvas to a PDF page.
I have a create canvas from video frame function as follows:
function capture_frame(video_ctrl, width, height){
if(width == null){
width = video_ctrl.videoWidth;
}
if(height == null){
height = video_ctrl.videoHeight;
}
canvas = document.createElement('canvas');
canvas.width = width;
canvas.height = height;
var ctx = canvas.getContext('2d');
ctx.drawImage(video_ctrl, 0, 0, width, height);
return canvas;
}
This function works fine in conjunction with the following to add an image to the PDF:
function addPdfImage(pdfObj, videoObj){
pdfObj.addPage();
pdfObj.text("Image at time point X:", 10, 20);
var vidImage = capture_frame(videoObj, null, null);
var dataURLWidth = 0;
var dataURLHeight = 0;
if(videoObj.videoWidth > pdfObj.internal.pageSize.width){
dataURLWidth = pdfObj.internal.pageSize.width;
dataURLHeight = (pdfObj.internal.pageSize.width/videoObj.videoWidth) * videoObj.videoHeight;
}else{
dataURLWidth = videoObj.videoWidth;
dataURLHeight = videoObj.videoHeight;
}
pdfObj.addImage(vidImage.toDataURL('image/jpg'), 'JPEG', 10, 50, dataURLWidth, dataURLHeight);
}
My logic confusion is how best to call these bits of code while looping through the vidSecs array as the problem is that setting the video.currentTime needs the loop to wait for the video.onseeked event to fire before code to capture the frame and add it to the PDF can be run.
I've tried the following but only get the last image as the loop has completed before the onseeked event fires and calls the frame capture code.
for(var i = 0; i < vidSecs.length; i++){
video.currentTime = vidSecs[i];
video.onseeked = function() {
addPdfImage(jsPDF_Variable, video);
};
}
Any thoughts much appreciated.
This is not a real answer but a comment, since I develop alike application and got no solution.
I am trying to extract viddeo frames from webcam live video stream and save as canvas/context, updated every 1 - 5 sec.
How to loop HTML5 webcam video + snap photo with delay and photo refresh?
I have created 2 canvases to be populated by setTimeout (5000) event and on test run I don't get 5 sec delay between canvas/contextes, sometimes, 2 5 sec. delayed contextes get populated with image at the same time.
So I am trying to implement
Draw HTML5 Video onto Canvas - Google Chrome Crash, Aw Snap
var toggle = true;
function loop() {
toggle = !toggle;
if (toggle) {
if (!v.paused) requestAnimationFrame(loop);
return;
}
/// draw video frame every 1/30 frame
ctx.drawImage(v, 0, 0);
/// loop if video is playing
if (!v.paused) requestAnimationFrame(loop);
}
to replace setInterval/setTimeout to get video and video frames properly synced
"
Use requestAnimationFrame (rAF) instead. The 20ms makes no sense. Most video runs at 30 FPS in the US (NTSC system) and at 25 FPS in Europe (PAL system). That would be 33.3ms and 40ms respectively.
"
I am afraid HTML5 provided no quality support for synced real time live video processing via canvas/ context, since HTML5 offers no proper timing since was intended to be event/s controlled and not run as real time run app code ( C, C++ ...).
My 100+ queries via search engine resulted in not a single HTML5 app I intend to develop.
What worked for me was Snap Photo from webcam video input, Click button event controlled.
If I am wrong, please correct me.
Two approaches:
create a new video element for every seek event, code provided by Chris West
reuse the video element via async/await, code provided by Adrian Wong
Related
I'm trying to extract frames from a video, and it's working fine. But when i try to extract videos longer than 1 minutes i got
RangeError: Failed to execute 'getImageData' on 'CanvasRenderingContext2D': Out of memory at ImageData creation
i've extracted 1100 frames just fine, that's around 1 minute video with 24fps. Is there any limit when extracting frames from a video at once?
here's how i extracted those frames
this.video.getFrames(this.file, this.totalFrame, VideoToFramesMethod.totalFrames).then((frames) => {
frames.forEach((frame, i) => {
let img = document.createElement('img');
var canvas = document.createElement('canvas');
canvas.width = frame.width;
canvas.height = frame.height;
canvas.getContext('2d').putImageData(frame, 0, 0);
img.src = canvas.toDataURL('image/jpeg');
this.listOfImages.push({
id: i,
name: 'Frame ' + i,
src: img.src
});
});
this.addCheckboxes();
this.loading = false;
this.playVideo = true;
this.poster = this.listOfImages[0];
});
EDIT
What i need from images that have been extracted is, later the user can draw a points on to the images to determine some object, i use fabricjs for this, but i don't manipulate the images just add new image (which is fabricjs canvas) on top of that image. you can try it here https://bory-cam.web.app
And after that, my algorithm can track the movement of the object inside the points. Also i need to able to show each frame. My first is works, but it's too expensive to memory. So i tried this solution but it really slow. I wanna try this solution using webcodecs but i can't make it work with angular
I'm trying to extract frames from a video on angular. I found several post here on SO, especially this post https://stackoverflow.com/a/32708998, i've tried to implement first solution but i can't get any frame i still don't know why. So i research about it again and i found this resource videotoframes he already has a type version, after some workaround i finally can extract the frames from video like this and put it to canvas.
But the thing is, i need to specify how many frames would i wanted from this. here's what i did
loadImages() {
this.loading = true;
this.video.getFrames('/assets/ForBiggerBlazes.mp4', 375, VideoToFramesMethod.totalFrames).then((frames) => {
console.log(frames);
frames.forEach((frame) => {
var canvas = document.createElement('canvas');
canvas.width = frame.width;
canvas.height = frame.height;
canvas.getContext('2d').putImageData(frame, 0, 0);
document.getElementsByTagName('div')[0].appendChild(canvas);
});
});
}
number 375 is an estimate how many frame my video is, is there any way to get how many frames a video have on upload?
The average HTML frame rate is 24 FPS, so if the video is 30 seconds, the total frame number is 30 x 24
Get the time of the video
const videoDOM=document.getElementById("video");
videoDom.duration
this will give u NaN, so to avoid that, we need to wait for the duration to change then call videoDom.duration
videoDom.ondurationchange = () => {
this.vidDuration = vid.duration;
};
that way we can get an actual duration
I am creating a game where I have to use same gif at multiple places. I used this method to create and display a gif.
But when I try to use it multiple times in different time interval. All the frames remain sync instead of starting from start next time: I modified above code to this one:
var open, frame, o1;
function preload() {
load = loadImage("doorOpen.gif");
}
function setup() {
createCanvas(0, 0);
background(0);
}
function draw() {
open = createImg("doorOpen.gif");
open.position(0, 0);
open.size(100,100);
setTimeout(function(){
o1 = createImg("doorOpen.gif")
o1.position(0,150);
o1.size(100,100);
}, 1000);
}
What I want is that when I add second GIF, it plays from start. But instead of that; it starts from the position of first GIF.
Good news, it's possible.
You are creating new html elements with the gif as the image source. Your browser tries to be smart about it and trying to save on resources, loads the gif once and then displays it over and over.
You can trick the browser to think it's a different image by adding '?' and a random number to the url. It will still load the same image but it will be unique. I've shown how to do this below.
On another note; when calling createImg() in the draw function like you did above, it adds 2 image elements EVERY frame, and will result in thousands of images in the DOM and crash your browser. I moved it to the setup function.
var open, o1;
function setup() {
createCanvas(0, 0);
background(0);
open = createImg("doorOpen.gif");
open.position(0, 0);
open.size(100, 100);
setTimeout(function() {
o1 = createImg("doorOpen.gif?" + random());
o1.position(0, 150);
o1.size(100, 100);
}, 800);
}
I've built this HTML5 video player that I am loading into a canvas to manipulate and back onto a canvas to display it. The video starts out quite slow and the frame rate only gets worse each time it is played. All I am currently manipulating in the video now is the color value when the video is paused, but will eventually be using real time manipulation throughout videos that will be posted in the future.
I used the below tutorial to learn this trick https://www.youtube.com/watch?v=zjQzP3mOXdc
Here is the relevant code, but there may possibly be interference coming from elsewhere so feel free to check the source code at the link at the bottom
var v = document.getElementById('video');
var color = "#DA7AC1";
var processes={
timerCallback:function() {
if (this.v2.paused || this.v2.ended) {
return;
}
this.ctxIn.drawImage(this.v2,0,0,this.width,this.height);
this.pixelScan();
var self=this;
setTimeout(function() {
self.timerCallback();
}, 0);
},
doLoad:function(){
this.v2=document.getElementById("video");
this.cIn=document.getElementById("cIn");
this.ctxIn=this.cIn.getContext("2d");
this.cOut=document.getElementById("cOut");
this.ctxOut=this.cOut.getContext("2d");
var self=this;
this.v2.addEventListener("playing", function() {
self.width=self.v2.videoWidth;
self.height=self.v2.videoHeight;
cIn.width=self.v2.videoWidth;
cIn.height=self.v2.videoHeight;
cOut.width=self.v2.videoWidth;
cOut.height=self.v2.videoHeight;
self.timerCallback();
}, false);
},
pixelScan: function() {
var frame = this.ctxIn.getImageData(0,0,this.width,this.height);
for(var i=0; i<frame.data.length;i+=4) {
var grayscale=frame.data[i]*.3+frame.data[i+1]*.59+frame.data[i+2]*.11;
frame.data[i]=grayscale;
frame.data[i+1]=grayscale;
frame.data[i+2]=grayscale;
}
this.ctxOut.putImageData(frame,0,0);
return;
}
}
http://coreytegeler.com/ethan/
Any and all help would be greatly appreciated! Thanks!
Reason 1
Try to adjust your timer avoiding 0 as timeout value:
setTimeout(function() {
self.timerCallback();
}, 34);
34ms is plenty as video frame rate is typically never more than 30 FPS (NTSC) or 25 FPS (PAL), ie 1000 / 30. If you use 0 you risk stacking up your calls which means the browser will be busy trying to empty the event queue.
If you use anything lower than 33-34ms you end up having the same frame processed twice or more which of course is unnecessary (your video is actually 29.97 FPS/NTSC so you might want to consider keeping 34ms).
Reason 2
The video resolution is also full HD (1920x1080) which is a bit too much for canvas and JS to process in real-time (for a typcial consumer computer). Try to reduce the video size so a normal spec'ed computer will be able to process the data.
Reason 3 (in part)
You don't need two on-screen canvases or even an on-screen video. Try to create these tags dynamically and not inserting them into the DOM. Use a single canvas on-screen and draw the result to that (you can putImageData from one canvas to another).
Reason 4 (in part)
Ideally, replace setTimeout with a requestAnimationFrame approach as this improves the synchronization and efficiency considerably. You can implement a toggle to reduce the FPS to for example 30 as you don't need to process each frame twice (ref. 30 FPS video frame rate).
Update
To create these elements dynamically (ref reason 3) you can do something like this:
var canvas = document.createElement('canvas'),
video = document.createElement('video'),
ctx = canvas.getContext('2d');
video.preload = 'auto';
video.addEventListener('canplay', start, false);
if (video.canPlayType('video/mp4')) {
video.src = 'videoUrl.mp4';
} else if ...etc.
Then when the video has loaded enough data (on metadata or canplay) you set the off-screen (and on-screen) canvas element to the size of the video:
canvas.width = video.videoWidth;
canvas.height = video.videoHeight;
Then when playing process its buffer and copy to the on-screen canvas you defined before.
You don't have have an off-screen canvas - I merely mention this as you in your original code used and in and out canvas IIRC. You can simply use a single on-screen canvas and the off-screen video and draw to the video frame to the canvas, process it and put back the processed data. Should work fine too in this case.
I ran a profile in chrome and it points to line 46 as taking up the most CPU.
setTimeout(function() {
self.timerCallback();
}, 0);
Perhaps increasing the timeout will stop it from lagging.
I had the same issues and tried a number of fixes. I was using Premier Elements which didn't export to mp4 and using HandBrake to convert the format. I also Tried FFMpeg to do the conversion, but neither worked.
What I did was switch to Kdenlive as my video editor, it exported directly to MP4, and that video worked perfectly.
So, if you are have this slow render issue, it is probably an issues with the video encoding. Easiest fix is to get a high quality video editor like Premier Pro, Final Cut, or Kdenlive. Kdenlive is free but it has a huge learning curve and poor public documentation.
We're developing arcade (a lot of action and speed) browser 2d-game using canvas.
Sometimes our testing players report us that there is a delay: player still moving 5-10 pixels away after keyup.
I've digged this issue, you can see yourself delay http://jsfiddle.net/C4ev3/7/ (try keydown/up any key as fast as you can). My results is from 70 to 120ms. And i think that's a lot. (FYI, our network latency is 10-20ms).
Any ideas how to reduce this delay?
upd i've noticed that on good hardware this delay is under 30-40ms. But i'm testing on core2duo, winxp, chrome 19 - it's not a P4 with IE6 :)
Hi one thing you could do is instead of using an anonymous function try using defined functions,
http://jsfiddle.net/C4ev3/10/ - for me this reported at 50-100 MS
However i would not recommend jQuery for Canvas Applications it's very big for the very little you using, you should try using native Javascript
http://jsfiddle.net/C4ev3/11/ - for me this reported 30-70 MS
Javascript Threading
One thing i noticed in the comments Javascript is not Multi-Threaded Well Urm-Arr,
it sort of is setInterval is Async not Sync, however affecting the window is a single thread E.G if you have a Class that has some number is it using a setInteval will use another thread and not have a problem altering the math however in the Task then requires a Draw on the page it will enter the bottom of the JS handle Que,
Certain parts of Javascript are on a different thread how ever any thing changing the page has to run on the Main Thread same as any Windows application if your thread want to change the Form your have to invoke the main thread to do it for you
however it is not multi-threaded like any thing else you cant just handle or abort at a given Wim like windows,
Other ASync Tasks include AJAX has the option to be both Async and Sync
Updated to show my comment about FPS limiting:
Please bear with me. This is linking to a project that is allready built to show the example:
so my Game is Completely OOP
var elem = document.getElementById('myCanvas');
var context = elem.getContext("2d");
context.fillStyle = '#888';
context.lineWidth = 4;
// Draw some rectangles.
context.fillRect(0, 0, 800, 600);
context.fillStyle = '#f00';
var ball = new Ball();
var leftPadel = new Padel(10, 60, 40, 120);
var rightPadel = new Padel(750, 520, 40, 120);
pong = new Pong();
pong.draw();
setTimeout("ball.move()", pong.redrawTime());
Inside my pong class is where all the main workings of the game goes but here are the FPS bit you need to see
this.fps = 30;
this.maxFPS = 60;
this.redrawTime = function(){
return (1000 / this.fps)
}
this.lastDraw = (new Date)*1 - 1;
Then as you can see my Interval is on ball.move this calls the main pong class again on redraw at the End of the redraw i have the FPS checking and limiting code
this.fps = ((now=new Date) - this.lastDraw);
if(this.fps > this.maxFPS){
this.fps = this.maxFPS;
}
this.lastDraw = (new Date)*1 - 1;
if(this.reporting = true){
console.clear();
console.log("FPS: "+this.fps.toFixed(1))
}
setTimeout("ball.move()", pong.redrawTime());
This then forces you to get the Best Possible FPS without queuing the Main Thread
Try this:
e.stopPropagation()
Stops the bubbling of an event to parent elements, preventing any
parent handlers from being notified of the event.
e.preventDefault()
Prevents the browser from executing the default action. Use the method
isDefaultPrevented to know whether this method was ever called (on
that event object).
My min. results in Google chrome: 7ms