We Keep Coding

Queue getting multiple video frames

I'm creating a PDF output tool using jsPDF but need to add multiple pages, each holding a canvas image of a video frame.
I am stuck on the logic as to the best way to achieve this as I can't reconcile how to queue the operations and wait on events to achieve the best result.
To start I have a video loaded into a video tag and can get or set its seek point simply with:
video.currentTime
I also have an array of video seconds like the following:
var vidSecs = [1,9,13,25,63];
What I need to do is loop through this array, seek in the video to the seconds defined in the array, create a canvas at these seconds and then add each canvas to a PDF page.
I have a create canvas from video frame function as follows:
function capture_frame(video_ctrl, width, height){
if(width == null){
width = video_ctrl.videoWidth;
}
if(height == null){
height = video_ctrl.videoHeight;
}
canvas = document.createElement('canvas');
canvas.width = width;
canvas.height = height;
var ctx = canvas.getContext('2d');
ctx.drawImage(video_ctrl, 0, 0, width, height);
return canvas;
}
This function works fine in conjunction with the following to add an image to the PDF:
function addPdfImage(pdfObj, videoObj){
pdfObj.addPage();
pdfObj.text("Image at time point X:", 10, 20);
var vidImage = capture_frame(videoObj, null, null);
var dataURLWidth = 0;
var dataURLHeight = 0;
if(videoObj.videoWidth > pdfObj.internal.pageSize.width){
dataURLWidth = pdfObj.internal.pageSize.width;
dataURLHeight = (pdfObj.internal.pageSize.width/videoObj.videoWidth) * videoObj.videoHeight;
}else{
dataURLWidth = videoObj.videoWidth;
dataURLHeight = videoObj.videoHeight;
}
pdfObj.addImage(vidImage.toDataURL('image/jpg'), 'JPEG', 10, 50, dataURLWidth, dataURLHeight);
}
My logic confusion is how best to call these bits of code while looping through the vidSecs array as the problem is that setting the video.currentTime needs the loop to wait for the video.onseeked event to fire before code to capture the frame and add it to the PDF can be run.
I've tried the following but only get the last image as the loop has completed before the onseeked event fires and calls the frame capture code.
for(var i = 0; i < vidSecs.length; i++){
video.currentTime = vidSecs[i];
video.onseeked = function() {
addPdfImage(jsPDF_Variable, video);
};
}
Any thoughts much appreciated.

This is not a real answer but a comment, since I develop alike application and got no solution.
I am trying to extract viddeo frames from webcam live video stream and save as canvas/context, updated every 1 - 5 sec.
How to loop HTML5 webcam video + snap photo with delay and photo refresh?
I have created 2 canvases to be populated by setTimeout (5000) event and on test run I don't get 5 sec delay between canvas/contextes, sometimes, 2 5 sec. delayed contextes get populated with image at the same time.
So I am trying to implement
Draw HTML5 Video onto Canvas - Google Chrome Crash, Aw Snap
var toggle = true;
function loop() {
toggle = !toggle;
if (toggle) {
if (!v.paused) requestAnimationFrame(loop);
return;
}
/// draw video frame every 1/30 frame
ctx.drawImage(v, 0, 0);
/// loop if video is playing
if (!v.paused) requestAnimationFrame(loop);
}
to replace setInterval/setTimeout to get video and video frames properly synced
"
Use requestAnimationFrame (rAF) instead. The 20ms makes no sense. Most video runs at 30 FPS in the US (NTSC system) and at 25 FPS in Europe (PAL system). That would be 33.3ms and 40ms respectively.
"
I am afraid HTML5 provided no quality support for synced real time live video processing via canvas/ context, since HTML5 offers no proper timing since was intended to be event/s controlled and not run as real time run app code ( C, C++ ...).
My 100+ queries via search engine resulted in not a single HTML5 app I intend to develop.
What worked for me was Snap Photo from webcam video input, Click button event controlled.
If I am wrong, please correct me.

Two approaches:
create a new video element for every seek event, code provided by Chris West
reuse the video element via async/await, code provided by Adrian Wong

html 5 canvas: using draw image in game makes loop very slow

i am making a game based on draw images and clear it every some part of second.
i started with:
var peng = new Image();
and then:
peng.onload = function() {
context.drawImage(peng, pengXPosition, pengYPosition, pengWidth, pengHight);
};
and in the loop:
var i=0;
function pengMoveRight(){ i++;if(i==1){peng.src = 'images/1.png';}else if(i==2)
{peng.src = 'images/2.png';} else if(i==3){peng.src = 'images/3.png';}else if(i==4){
peng.src = 'images/4.png';}else if(i==5){peng.src = 'images/5.png';}else if(i==6){
peng.src = 'images/6.png';i-=6;}}
when i run it it works well on IE but on chrome and mozilla it`s too slow and the character is about to disappear .. i used setinterval(); once and window.requestAnimationFrame(); once and both of them cause the same problem.
what should i do to make it smooth move?
here is the full script http://mark-tec.com/custom%20game/

Instead of changing the source, try to create several Image objects instead. That way, the drawImage call can always use a pre-loaded image.

You need to preload all the images or use the sprite method (all images packed into a single sprite) in order to avoid the initial delay caused by the image loading only when it's needed.
However, after that initial problem, your example should run fine once all the images are cached.

How to reset its position back to the origin when it reach the end of frame?

This one of the phaser examples, what Im trying to do is to load the image again when it reach the end o the frame. Can somebody explain how to do this
var game = new Phaser.Game(1500, 200, Phaser.AUTO, 'phaser-example', { preload: preload, create: create });
function preload() {
// You can fill the preloader with as many assets as your game requires
// Here we are loading an image. The first parameter is the unique
// string by which we'll identify the image later in our code.
// The second parameter is the URL of the image (relative)
game.load.image('Car', 'car.jpg');
}
function create() {
// This creates a simple sprite that is using our loaded image and
// displays it on-screen
// and assign it to a variable
var image = game.add.sprite(0, 0, 'Car');
game.physics.enable(image, Phaser.Physics.ARCADE);
image.body.velocity.x=750;
if (image>game.width)
{
game.load.image('Car', 'car.jpg');
var image = game.add.sprite(0, 0, 'Car');
}
}

I assume the above is just pseudo code because there are lots of little errors. But there are a number of issues with this approach.
If you know what the new image needs to be in advance, then preload it up front and then use loadTexture to apply it:
function preload() {
game.load.image('Car', 'car.jpg');
game.load.image('Pumpkin', 'pumpkin.png');
}
...
function update() {
if (car.x > game.width) {
car.loadTexture('Pumpkin');
}
}
Also some other things to watch out for:
1) Only change the texture if it's not already set (in the example above it will run loadTexture over and over, unless you reset the car.x coordinate)
2) If you can't preload the image up front, then you can load it during play. Look at the "Loader Events" example code for details.
3) You need to check the sprite position in update and not create (as it won't have moved at all by that point).

what is best to use canvas or img for frequent updates?

I have a javascript timer.
It refreshes the img src on a 200ms interval.
I have taken a look at the canvas object. I am unsure whether it is recommended to use the canvas instead of the img element?
I am running tests on both and cannot see any differences in performance.
This is my code for using the timer/img:
This is my code:
var timer4x4
var cache4x4 = new Image();
var alias = 'test';
var lastUpdate = 0;
function setImageSrc4x4(src) {
live4x4.src = src;
timer4x4 = window.setTimeout(swapImages4x4, 200);
}
function swapImages4x4() {
cache4x4.onload = function () {
setImageSrc4x4(cache4x4.src);
};
cache4x4.onerror = function () {
setImageSrc4x4("http://127.0.0.1/images/ERROR.jpg");
};
cache4x4.src = null;
cache4x4.src = 'http://127.0.0.1/Cloud/LiveXP.ashx?id=' + createGuid() + '&Alias=' + alias + '&ReSync=' + reSync;
reSync = 0;
}
*nb will add canvas code in a bit
I am streaming images from my client desktop PC to my web server. I am trying to display as many images (FPS) as possible. The image is a container for 4 smaller images. Stitched up on the client and sent to the server.
I have Googled and it says if doing pixel manipulation and aniumation use canvas.
But, I am just doing animation.
Thanks

The canvas element was designed to draw / edit / interact with images in it. If all you do is display the image, then you don't need that and a simple img is the semantically correct choice (with the added bonus of being compatible on more devices).
In both cases, the performance will be similar (if not the same) because the only thing to happen is that the image is downloaded.

While performance-wise you won't notice much of a difference, since you still cannot fully rely on HTML5 support yet, it is probably best to go with the img-solution for now.

Check if Images are loaded before gameloop

So far my program is working the way I want it to. This works fine:
// Player object
var player = {
x: 10,
y: 10,
draw: function () {
ctx.drawImage(playerImg, 0, 0);
...
Should I check if playerImg is loaded first, even though it works correctly so far?
Also, what is the best way to check. I was thinking about putting all the images in an array. Then check with the onLoad function. If they are all loaded then I will start the game loop. Is this a good idea?
Thanks

How image loading works
You need to check if the image is loaded as image loading is asynchronous. You may experience that your code works sometimes without. This is mainly because your image exists in the cache and the browser is able to load it fast enough before the drawImage is called, or the image exists on local disk.
However, new users will need to download the data first and you don't want first-time users to experience errors such as images not showing because they are not finished loading.
As it works asynchronous your code will continue to execute while the image loading takes place in the background. This may cause your code to execute before the image has finished loading. So handling image loading is important
Handling multiple images
You can load all your images first (or those you need to start with) and you can define them using array:
var imageURLs = [url1, url2, url3, ...],
images = [],
count = imageURLs.length;
Then iterate and create the image elements:
for(var i = 0; i < count; i++) {
/// create a new image element
var img = new Image();
/// element is valid so we can push that to stack
images.push(img);
/// set handler and url
img.onload = onloadHandler;
img.src = imageURLs[i];
/// if image is cached IE (surprise!) may not trigger onload
if (img.complete) onloadHandler().bind(img);
}
and in the callback function do the inventory count:
function onloadHandler() {
/// optionally: "this" contains current image just loaded
count--;
if (count === 0) callbackDone();
}
Your callback is the code you want to execute next. Your images will be in the array images in the same order as the imageURLs.
For production you should also incorporate an onerror handler in case something goes wrong.