I want to play a video in WebGL but MovieTextures re not supported. i've tried using Jonas' Simple MoTextures Sample for WebGL and it works great, but only with certain videos. I've deduced that it only works with videos that have a 2:1 aspect ratio. Such as the example one (640x320) or one I did that was 2000x1000.
If I try to use any other video I always have the same warning (hundreds of times):
WARNING: texture bound to texture unit 0 is not renderable. It maybe
non-power-of-2 and have incompatible texture filtering.
I am completely stuck at this point. The code is 100% the one in the sample.
BTW, in my 2000x1000 video I also get that warning, but only 5 times, and the video plays perfectly.
The code in JSlib for create and update functions:
WebGLMovieTextureCreate: function(url)
{
var str = Pointer_stringify(url);
var video = document.createElement('video');
video.style.display = 'none';
video.src = str;
return videoInstances.push(video) - 1;
},
WebGLMovieTextureUpdate: function(video, tex)
{
if (videoInstances[video].paused)
return;
GLctx.bindTexture(GLctx.TEXTURE_2D, GL.textures[tex]);
GLctx.pixelStorei(GLctx.UNPACK_FLIP_Y_WEBGL, true);
GLctx.texImage2D(GLctx.TEXTURE_2D, 0, GLctx.RGBA, GLctx.RGBA, GLctx.UNSIGNED_BYTE, videoInstances[video]);
GLctx.pixelStorei(GLctx.UNPACK_FLIP_Y_WEBGL, false);
},
And the C# code:
public WebGLMovieTexture (string url)
{
m_Instance = WebGLMovieTextureCreate(url);
m_Texture = new Texture2D(0, 0, TextureFormat.ARGB32, false);
m_Texture.wrapMode = TextureWrapMode.Clamp;
}
public void Update()
{
var width = WebGLMovieTextureWidth(m_Instance);
var height = WebGLMovieTextureHeight(m_Instance);
if (width != m_Texture.width || height != m_Texture.height)
{
m_Texture.Resize(width, height, TextureFormat.ARGB32, false);
m_Texture.Apply();
}
WebGLMovieTextureUpdate(m_Instance, m_Texture.GetNativeTextureID());
}
Related
I'm using CefSharp 55.0.0 WinForms.
In order to take screenshots in CefSharp, I scroll the webpage using window.scroll within JavaScript and take an image of the current viewport. Once that has completed, it is then stitched back together again. This works fine for monitors that have their DPI setting set to 100%. However, when the monitor has a DPI greater than 100% screenshots do not work as expected and miss content.
Image 1 - 100%
Image 2 - 150%
Compare Image 1 to Image 2. While they both (practically) have the same width and height, Image 2 is missing a large portion of the content, compared to a perfect Image 1.
When DPI settings are above 100%, how can I correctly scroll and capture a screenshot that ensures I obtain everything it would if settings were at 100%?
Other details
The application has the correct DPI aware settings in theapp.manifest file and Cef.EnableHighDPISupport(); has been called in the Main method within Program.cs.
Screenshot Code (abridged)
int scrollHeight = GetDocHeight(); //some javascript that calcs the height of the document
int viewportHeight = ClientRectangle.Size.Height;
int viewportWidth = ClientRectangle.Size.Width;
int count = 0;
int pageLeft = scrollHeight;
bool atBottom = false;
while (!atBottom)
{
if (pageLeft > viewportHeight)
{
await GetBrowser().MainFrame.EvaluateScriptAsync("(function() { window.scroll(0," + (count * viewportHeight) + "); })();"); //I think the issue lies here
count++;
await PutTaskDelay();
using (Bitmap image = GetCurrentViewScreenshot())
{
//just a class that saves the partial images to a disk cache
cache.AddImage(count, image);
}
}
else
{
await GetBrowser().MainFrame.EvaluateScriptAsync("(function() { window.scrollBy(0," + pageLeft + "); })();");
atBottom = true;
count++;
await PutTaskDelay();
Rectangle cropRect = new Rectangle(new Point(0, viewportHeight - pageLeft), new Size(viewportWidth, pageLeft));
using (Bitmap src = GetCurrentViewScreenshot())
using (Bitmap target = new Bitmap(cropRect.Width, cropRect.Height))
using (Graphics g = Graphics.FromImage(target))
{
g.DrawImage(src, new Rectangle(0, 0, target.Width, target.Height), cropRect, GraphicsUnit.Pixel);
cache.AddImage(count, target);
}
}
pageLeft = pageLeft - viewportHeight;
}
Current View Screenshot Method
private Bitmap GetCurrentViewScreenshot()
{
int width, height;
width = ClientRectangle.Width;
height = ClientRectangle.Height;
using (Bitmap image = new Bitmap(width, height))
{
using (Graphics graphics = Graphics.FromImage(image))
{
Point p, upperLeftDestination;
Point upperLeftSource = new Point(0, 0);
p = new Point(0, 0);
upperLeftSource = PointToScreen(p);
upperLeftDestination = new Point(0, 0);
Size blockRegionSize = ClientRectangle.Size;
graphics.CopyFromScreen(upperLeftSource, upperLeftDestination, blockRegionSize);
}
return new Bitmap(image);
}
}
While not a perfect solution, I found a workaround that involves setting the force-device-scale-factor flag to 1 in CefSettings. It's not perfect because the browser isn't scaled on high DPI displays, potentially making text hard to read for users. However, it does fix my more pressing issue of missing data in screenshots.
CefSettings settings = new CefSettings();
settings.CefCommandLineArgs.Add("force-device-scale-factor", "1");
Cef.Initialize(settings);
This question is still open for better suggestions, if there are any. :-)
I have been practicing using sprites for a game I am going to make and have watched and read a few tutorials, I thought I was close to getting my sprite to appear so I could finally start my game but while practicing I cant get it to work, I have dont 2 seperate tutorials where I can get the sprite and the background to appear by themselfs but cannot get them to work together, I have been using EaselJS too. some of the sprite animation code has been copied from tutorials too.
<!DOCTYPE HTML>
<html>
<head>
<meta charset="UTF-8">
<title>sprite prac<title>
<!-- EaselJS library -->
<script src="lib/easel.js"></script>
<script>
// Initialize on start up so game runs smoothly
function init() {
canvas = document.getElementById("canvas");
stage = new Stage(canvas);
bg = new Image();
bg.src = "img/grassbg.jpg";
bg.onload = setBG;
stage.addChild(background);
imgMonsterARun = new Image();
imgMonsterARun.onload = handleImageLoad;
imgMonsterARun.onerror = handleImageError;
imgMonsterARun.src = "img/MonsterARun.png";
stage.update();
}
function handleImageLoad(e) {
startGame();
}
// Simple function for setting up the background
function setBG(event){
var bgrnd = new Bitmap(bg);
stage.addChild(bgrnd);
stage.update();
}
function startGame() {
// create a new stage and point it at our canvas:
stage = new createjs.Stage(canvas);
// grab canvas width and height for later calculations:
screen_width = canvas.width;
screen_height = canvas.height;
// create spritesheet and assign the associated data.
var spriteSheet = new createjs.SpriteSheet({
// image to use
images: [imgMonsterARun],
// width, height & registration point of each sprite
frames: {width: 64, height: 64, regX: 32, regY: 32},
animations: {
walk: [0, 9, "walk"]
}
});
// create a BitmapAnimation instance to display and play back the sprite sheet:
bmpAnimation = new createjs.BitmapAnimation(spriteSheet);
// start playing the first sequence:
bmpAnimation.gotoAndPlay("walk"); //animate
// set up a shadow. Note that shadows are ridiculously expensive. You could display hundreds
// of animated rats if you disabled the shadow.
bmpAnimation.shadow = new createjs.Shadow("#454", 0, 5, 4);
bmpAnimation.name = "monster1";
bmpAnimation.direction = 90;
bmpAnimation.vX = 4;
bmpAnimation.x = 16;
bmpAnimation.y = 32;
// have each monster start at a specific frame
bmpAnimation.currentFrame = 0;
stage.addChild(bmpAnimation);
// we want to do some work before we update the canvas,
// otherwise we could use Ticker.addListener(stage);
createjs.Ticker.addListener(window);
createjs.Ticker.useRAF = true;
createjs.Ticker.setFPS(60);
}
//called if there is an error loading the image (usually due to a 404)
function handleImageError(e) {
console.log("Error Loading Image : " + e.target.src);
}
function tick() {
// Hit testing the screen width, otherwise our sprite would disappear
if (bmpAnimation.x >= screen_width - 16) {
// We've reached the right side of our screen
// We need to walk left now to go back to our initial position
bmpAnimation.direction = -90;
}
if (bmpAnimation.x < 16) {
// We've reached the left side of our screen
// We need to walk right now
bmpAnimation.direction = 90;
}
// Moving the sprite based on the direction & the speed
if (bmpAnimation.direction == 90) {
bmpAnimation.x += bmpAnimation.vX;
}
else {
bmpAnimation.x -= bmpAnimation.vX;
}
// update the stage:
stage.update();
}
</script>
</head>
<body onload="init();">
<canvas id="canvas" width="500" height="500" style="border: thin black solid;" ></canvas>
</body>
</html>
There are a few places where you are using some really old APIs, which may or may not be supported depending on your version of EaselJS. Where did you get the easel.js script you reference?
Assuming you have a version of EaselJS that matches the APIs you are using, there are a few issues:
You add background to the stage. There is no background, so you are probably getting an error when you add it. You already add bgrnd in the setBackground method, which should be fine. If you get an error here, then this could be your main issue.
You don't need to update the stage any time you add something, just when you want the stage to "refresh". In your code, you update after setting the background, and again immediately at the end of your init(). These will fire one after the other.
Are you getting errors in your console? That would be a good place to start debugging. I would also recommend posting code if you can to show an actual demo if you continue to have issues, which will help identify what is happening.
If you have a newer version of EaselJS:
BitmapAnimation is now Sprite, and doesn't support direction. To flip Sprites, use scaleX=-1
Ticker no longer uses addListener. Instead it uses the EventDispatcher. createjs.Ticker.addEventListener("tick", tickFunction);
You can get new versions of the CreateJS libraries at http://code.createjs.com, and you can get updated examples and code on the website and GitHub.
I'm making a game using CreateJS. On desktop my FPS is good but when I try to play this game on mobile (for example : iPhone 4) the FPS drops seriously.
I'm trying to figure out why but
Some code
My Canvas
<canvas id="gameCanvas"></canvas>
Setup
this.canvas = "gameCanvas";
this.stage = new createjs.Stage(this.canvas);
var context = this.stage.canvas.getContext("2d");
context.imageSmoothingEnabled = false;
createjs.Ticker.setFPS(30);
this.gameLoopBind = this.gameLoop.bind(this);
createjs.Ticker.addEventListener('tick', this.gameLoopBind);
GameLoop
// some extra code
this.stage.update();
When I comment the code 'this.stage.update()' my FPS on mobile/tablet is good...
I've no idea what I'm doing wrong...
EXTRA CODE
Play the game here => f.cowb.eu/mora/chick-ins
Gameloop Function
Game.prototype.gameLoop = function (e) {
if (this.running) {
this.timer++;
this.timer2++;
if (this.timer2 > 30) {
if (this.lastSnack + this.timeBewteen < this.stopwatch.seconds) {
var height = (this.topSnack) ? 150 : 300;
this.lastSnack = this.stopwatch.seconds;
new Snack(this, this.timer, height);
this.topSnack = this.topSnack ? false : true;
}
if (this.timer > (this.lastPostive + 300)) {
this.lastPostive = this.timer;
publisher.publish('showMessage',
this.positiveImages[Math.floor(Math.random() * this.positiveImages.length)],
common.lang,
'right');
}
this.timer2 = 0;
}
}
this.stage.update();
};
New Snack
You can find the code for creating a new snack here => http://jsfiddle.net/9ofpqq3z/
Here we create a new snack and animate it.
From looking at the architecture of the game, I would suggest that you draw the game elements on separate canvases as opposed to the way you have it right now where everything is on one canvas.
Place the TV parts that don't get redrawn often (or at all) on one canvas, and then the moving elements on a separate canvas.
That should help bring the frame rate up.
Also, you can take a look at the following set of slides on how to improve performance on mobile devices when using the canvas:
http://www.slideshare.net/DavidGoemans/html5-performance-optimization
I am attempting to create a thumbnail preview from a video file (mp4,3gp) from a form input type='file'. Many have said that this can be done server side only. I find this hard to believe since I just recently came across this Fiddle using HTML5 Canvas and Javascript.
Thumbnail Fiddle
The only problem is this requires the video to be present and the user to click play before they click a button to capture the thumbnail. I am wondering if there is a way to get the same results without the player being present and user clicking the button. For example: User click on file upload and selects video file and then thumbnail is generated. Any help/thoughts are welcome!
Canvas.drawImage must be based on html content.
source
here is a simplier jsfiddle
//and code
function capture(){
var canvas = document.getElementById('canvas');
var video = document.getElementById('video');
canvas.getContext('2d').drawImage(video, 0, 0, video.videoWidth, video.videoHeight);
}
The advantage of this solution is that you can select the thumbnail you want based on the time of the video.
Recently needed this so I wrote a function, to take in a video file and a desired timestamp, and return an image blob at that time of the video.
Sample Usage:
try {
// get the frame at 1.5 seconds of the video file
const cover = await getVideoCover(file, 1.5);
// print out the result image blob
console.log(cover);
} catch (ex) {
console.log("ERROR: ", ex);
}
Function:
function getVideoCover(file, seekTo = 0.0) {
console.log("getting video cover for file: ", file);
return new Promise((resolve, reject) => {
// load the file to a video player
const videoPlayer = document.createElement('video');
videoPlayer.setAttribute('src', URL.createObjectURL(file));
videoPlayer.load();
videoPlayer.addEventListener('error', (ex) => {
reject("error when loading video file", ex);
});
// load metadata of the video to get video duration and dimensions
videoPlayer.addEventListener('loadedmetadata', () => {
// seek to user defined timestamp (in seconds) if possible
if (videoPlayer.duration < seekTo) {
reject("video is too short.");
return;
}
// delay seeking or else 'seeked' event won't fire on Safari
setTimeout(() => {
videoPlayer.currentTime = seekTo;
}, 200);
// extract video thumbnail once seeking is complete
videoPlayer.addEventListener('seeked', () => {
console.log('video is now paused at %ss.', seekTo);
// define a canvas to have the same dimension as the video
const canvas = document.createElement("canvas");
canvas.width = videoPlayer.videoWidth;
canvas.height = videoPlayer.videoHeight;
// draw the video frame to canvas
const ctx = canvas.getContext("2d");
ctx.drawImage(videoPlayer, 0, 0, canvas.width, canvas.height);
// return the canvas image as a blob
ctx.canvas.toBlob(
blob => {
resolve(blob);
},
"image/jpeg",
0.75 /* quality */
);
});
});
});
}
Recently needed this and did quite some testing and boiling it down to the bare minimum, see https://codepen.io/aertmann/pen/mAVaPx
There are some limitations where it works, but fairly good browser support currently: Chrome, Firefox, Safari, Opera, IE10, IE11, Android (Chrome), iOS Safari (10+).
video.preload = 'metadata';
video.src = url;
// Load video in Safari / IE11
video.muted = true;
video.playsInline = true;
video.play();
You can use this function that I've written. You just need to pass the video file to it as an argument. It will return the dataURL of the thumbnail(i.e image preview) of that video. You can modify the return type according to your need.
const generateVideoThumbnail = (file: File) => {
return new Promise((resolve) => {
const canvas = document.createElement("canvas");
const video = document.createElement("video");
// this is important
video.autoplay = true;
video.muted = true;
video.src = URL.createObjectURL(file);
video.onloadeddata = () => {
let ctx = canvas.getContext("2d");
canvas.width = video.videoWidth;
canvas.height = video.videoHeight;
ctx.drawImage(video, 0, 0, video.videoWidth, video.videoHeight);
video.pause();
return resolve(canvas.toDataURL("image/png"));
};
});
};
Please keep in mind that this is a async function. So make sure to use it accordingly.
For instance:
const handleFileUpload = async (e) => {
const thumbnail = await generateVideoThumbnail(e.target.files[0]);
console.log(thumbnail)
}
The easiest way to display a thumbnail is using the <video> tag itself.
<video src="http://www.w3schools.com/html/mov_bbb.mp4"></video>
Use #t in the URL, if you want the thumbnail of x seconds.
E.g.:
<video src="http://www.w3schools.com/html/mov_bbb.mp4#t=5"></video>
Make sure that it does not include any attributes like autoplay or controls and it should not have a source tag as a child element.
With a little bit of JavaScript, you may also be able to play the video, when the thumbnail has been clicked.
document.querySelector('video').addEventListener('click', (e) => {
if (!e.target.controls) { // Proceed, if there are no controls
e.target.src = e.target.src.replace(/#t=\d+/g, ''); // Remove the time, which is set in the URL
e.target.play(); // Play the video
e.target.controls = true; // Enable controls
}
});
<video src="http://www.w3schools.com/html/mov_bbb.mp4#t=5"></video>
With jQuery Lib you can use my code here. $video is a Video element.This function will return a string
function createPoster($video) {
//here you can set anytime you want
$video.currentTime = 5;
var canvas = document.createElement("canvas");
canvas.width = 350;
canvas.height = 200;
canvas.getContext("2d").drawImage($video, 0, 0, canvas.width, canvas.height);
return canvas.toDataURL("image/jpeg");;
}
Example usage:
$video.setAttribute("poster", createPoster($video));
I recently stumbled on the same issue and here is how I got around it.
firstly it will be easier if you have the video as an HTML element, so you either have it in the HTML like this
<video src="http://www.w3schools.com/html/mov_bbb.mp4"></video>
or you take from the input and create an HTML element with it.
The trick is to set the start time in the video tag to the part you want to seek and have as your thumbnail, you can do this by adding #t=1.5 to the end of the video source.
<video src="http://www.w3schools.com/html/mov_bbb.mp4#t=1.5"></video>
where 1.5 is the time you want to seek and get a thumbnail of.
This, however, makes the video start playing from that section of the video so to avoid that we add an event listener on the video's play button(s) and have the video start from the beginning by setting video.currentTime = 0
const video = document.querySelector('video');
video.addEventListener('click', (e)=> {
video.currentTime = 0 ;
video.play();
})
I'm having an issue while using canvas in a background page to create data URLs for desktop notifications' images.
I want to use the "image" notifications which require a 3:2 ratio to display properly. The images I want to use (from hulu.com) are a different ratio, so I decided to use the canvas element to create the corresponding data URL off of these images so that the ratio is correct. It kind of works in theory, but…
…I'm having issues if I'm creating more than one canvas/notification in the background page. One image is created properly, but the rest comes out empty.
Confusingly, opening the same background page in a new tab (i.e. exact same code) makes everything works just fine: all the notifications are created with the images loaded from hulu.com. Also, just changing the dimensions from 360x240 to 300x200 makes it work. Finally, though they're similar computers with the same Chrome version (34.0.1847.116), it works without modification at work while it doesn't on my own laptop.
I made a test extension available at the bottom of this post. Basically, it only has a generated background page. The code for that page is this:
var images = ["http://ib2.huluim.com/video/60376901?size=290x160&img=1",
"http://ib2.huluim.com/video/60366793?size=290x160&img=1",
"http://ib4.huluim.com/video/60372951?size=290x160&img=1",
"http://ib1.huluim.com/video/60365336?size=290x160&img=1",
"http://ib3.huluim.com/video/60376290?size=290x160&img=1",
"http://ib4.huluim.com/video/60377231?size=290x160&img=1",
"http://ib4.huluim.com/video/60312203?size=290x160&img=1",
"http://ib1.huluim.com/video/60376972?size=290x160&img=1",
"http://ib4.huluim.com/video/60376971?size=290x160&img=1",
"http://ib1.huluim.com/video/60376616?size=290x160&img=1"];
for (var i = 0; i < 10; i++) {
getDataURL(i);
}
/*
* Gets the data URL for an image URL
*/
function getDataURL(i) {
var img = new Image();
img.onload = function() {
var canvas = document.createElement('canvas');
canvas.width = 360;
canvas.height = 240;
var ctx = canvas.getContext('2d');
ctx.drawImage(this, 0, 0);
ctx.fillStyle = "rgb(200,0,0)";
ctx.fillRect (10, 10, 55, 50);
var dataURL = canvas.toDataURL('image/png');
chrome.notifications.create('', {
type: 'image',
iconUrl: 'logo_128x128.png',
title: String(i),
message: 'message',
imageUrl: dataURL
}, function(id) {});
}
//img.src = chrome.extension.getURL('logo_128x128.png');;
img.src = images[i];
}
The commented out line for img.src = ... is a test where it loads a local file instead of a remote one. In that case, all the images are created.
The red rectangle added to the canvas is to show that it's not just the remote image that is an issue: the whole resulting canvas is empty, without any red rectangle.
If you download and add the test extension below, you should get 10 notifications but only one with an image.
Then, to open the background page in a new tab, you can inspect the background page, type this in the console:
chrome.extension.getURL('_generated_background_page.html')
and right-click the URL, and click "Open in a new Tab" (or window). Once open you should get 10 notifications that look fine.
Any idea of what is going on? I haven't been able to find any kind of limitations for background pages relevant to that. Any help would be appreciated, because this has been driving me crazy!
Files available here: https://www.dropbox.com/s/ejbh6wq0qixb7a8/canvastest.zip
edit: based on #GameAlchemist's comment, I also tried the following: same getDataURL method, but the loop wrapped inside an onload for the logo:
function loop() {
for (var i = 0; i < 10; i++) {
getDataURL(i);
}
}
var logo = new Image();
logo.onload = function () {
loop();
}
logo.src = chrome.extension.getURL('logo_128x128.png');
Remember that the create() method is asynchronous and you should use a callback with. The callback can invoke next image fetching.
I would suggest doing this in two steps:
Load all the images first
Process the image queue
The reason is that you can utilize the asynchronous image loading better this way instead of chaining the callbacks which would force you to load one and one image.
For example:
Image loader
var urls = ["http://ib2.huluim.com/video/60376901?size=290x160&img=1",
"http://ib2.huluim.com/video/60366793?size=290x160&img=1",
"http://ib4.huluim.com/video/60372951?size=290x160&img=1",
"http://ib1.huluim.com/video/60365336?size=290x160&img=1",
"http://ib3.huluim.com/video/60376290?size=290x160&img=1",
"http://ib4.huluim.com/video/60377231?size=290x160&img=1",
"http://ib4.huluim.com/video/60312203?size=290x160&img=1",
"http://ib1.huluim.com/video/60376972?size=290x160&img=1",
"http://ib4.huluim.com/video/60376971?size=290x160&img=1",
"http://ib1.huluim.com/video/60376616?size=290x160&img=1"];
var images = [], // store image objects
count = urls.length; // for loader
for (var i = 0; i < urls.length; i++) {
var img = new Image; // create image
img.onload = loader; // share loader handler
img.src = urls[i]; // start loading
images.push(img); // push image object in array
}
function loader() {
count--;
if (count === 0) process(); // all loaded, start processing
}
//TODO need error handling here as well
Fiddle with concept code for loader
Processing
Now the processing can be isolated from the loading:
function process() {
// share a single canvas (use clearRect() later if needed)
var canvas = document.createElement('canvas'),
ctx = canvas.getContext('2d'),
current = 0;
canvas.width = 360;
canvas.height = 240;
createImage(); // invoke processing for first image
function createImage() {
ctx.drawImage(images[current], 0, 0); // draw current image
ctx.fillStyle = "rgb(200,0,0)";
ctx.fillRect (10, 10, 55, 50);
chrome.notifications.create('', {
type : 'image',
iconUrl : 'logo_128x128.png',
title : String(i),
message : 'message',
imageUrl: canvas.toDataURL() // png is default
},
function(id) { // use callback
current++; // next in queue
if (current < images.length) {
createImage(); // call again if more images
}
else {
done(); // we're done -> continue to done()
}
});
}
}
Disclaimer: I don't have a test environment to test Chrome extensions so typos/errors may be present.
Hope this helps!