So I was trying to extract the pixels from my PIXI.js canvas, and I inspected them, and while the canvas on the screen had an image on it, the pixels extracted were completely black...
So then I investagated, and here is what I found (i need help)
Background info:
setup:
this.canvas = $('#canvas');
this.canvasDOM = this.canvas[0];
this.renderer = PIXI.autoDetectRenderer(0, 0, {
view: this.canvasDOM,
resolution: 1
});
Getting Image:
chooseImage(src) {
const self = this;
const image = new Image();
image.src = src;
image.onload = function() {
const base = new PIXI.BaseTexture(image);
const texture = new PIXI.Texture(base);
const sprite = new PIXI.Sprite(texture);
sprite.position.set(0, 0);
self.currentImage = sprite;
self.render();
};
}
rendering image:
render() {
if (this.currentImage) {
this.updateDimensions();
const stage = new PIXI.Container();
stage.addChild(this.currentImage);
stage.filters = this.gatherFilters();
const { width, height } = this.currentImage;
stage.filterArea = new PIXI.Rectangle(0, 0, width, height);
this.renderer.render(stage);
}
}
ok, so you can ignore the code which is before the rendering function, but in particular, the third last line of the rendering function is:
this.renderer.render(stage);
Now, this works perfectly fine! the image pops up and is a bright jellyfish on the canvas, but there is one small issue:
If i add this to the end of the render() function:
render() {
if (this.currentImage) {
this.updateDimensions();
const stage = new PIXI.Container();
stage.addChild(this.currentImage);
stage.filters = this.gatherFilters();
const { width, height } = this.currentImage;
stage.filterArea = new PIXI.Rectangle(0, 0, width, height);
this.renderer.render(stage);
const actualCanvas = this.renderer.extract.canvas; // the actual canvas on the screen
console.log(actualCanvas.toDataURL());
}
}
It returns a long string, and if replace
console.log(actualCanvas.toDataURL());
with: (the download lib)
download(actualCanvas.toDataURL('image/png'), 'image.png', 'image/png');
It downloads the jellyfish image!
now, in another function, I was going to have the image downloaded as either a png or jpg (and the array of possible download types was in another class and decided dynamically, so I couldn't simply cache the pixels after rendering)
to show u my issue, I will change the above code to:
setTimeout(() => download(actualCanvas.toDataURL('image/png'), 'image.png', 'image/png'), 1 /* One second delay */);
now the image is blank; black., unlike before when it worked
the only thing that changed was the delay I added.
I further inspected this, and i realised that if i did
setTimeout(() => console.log(this.renderer.extract.pixels()), 1); // extract the pixels after some delay for testing
then they were all black, but if i removed the delay, it worked again
again, in my case, I can't simply cache the image for my own reasons (don't ask)
So i was wandering if there was a workaround so that after some delay or later on the pixels when I extract them are not just black and work like it does without the delay.
Any help is greatly appreciated!
It seems some kind of browser protection to me. Maybe non-event-driven code protection accessing some resources or maybe non-https webapps protection (did you tried your app thru https even without certificate?), or maybe it can be even some kind of crossdomain protection (I doubt about it, but always present in the to-check list). Maybe you can deploy a fiddle or similar to test. Hope it helps!
Related
I have some image preview functionality as part of a file uploader that shows the images prior to upload. Within this process the image previews use a htmlImageElement.decode() method that returns a promise so various frontend validations etc can happen on the images. This decode() method is run within a function that is called in a forEach() loop in relation to the files from a file <input> element.
The context
Even though I am limiting the number of files per upload to 10, because large files are allowed, if a user attaches 10 (large) files the image previewer is laggy, both in terms of image render, and also when any images are deleted from the previewer.
The Question
Is there anyway to reduce the file size of the image preview, without affecting the file size of the image to be uploaded?
You can add width and height parameters to the new Image() constructor, i.e. new Image(300,300), but these only affect the display size, not the file size, and if you alter the naturalHeight and naturalWidth properties, this changes the size of the file itself that is being uploaded, whereas what I want is just the preview file size to be smaller?
// this function is invoked in a forEach loop as part of a wider code block related to the individual files from a file <input> element
function showFiles(file) {
let previewImage = new Image();
// Set <img> src attribute
previewImage.src = URL.createObjectURL(file);
// get the original width and height values of the thumbnail using the decode() method
previewImage.decode().then((response) => {
// get image dimensions for validations
let w = previewImage.naturalWidth;
let h = previewImage.naturalHeight;
let imgs = document.querySelectorAll('img') // redeclare under new var name inside promise
}).catch((encodingError) => {
// Do something with the error.
});
} // end of showfiles(file)
A canvas can be used to create a resized clone of the original image, then use the canvas blob as source for preview:
// this function is invoked in a forEach loop as part of a wider code block related to the individual files from a file <input> element
function showFiles(file) {
let previewImage = new Image();
// Set <img> src attribute
previewImage.src = URL.createObjectURL(file);
// get the original width and height values of the thumbnail using the decode() method
previewImage.decode().then(() => {
// get image dimensions for validations
let w = previewImage.naturalWidth;
let h = previewImage.naturalHeight;
const W = w * 0.3, H = h * 0.3;
const canvas = document.createElement('canvas');
const ctx = canvas.getContext('2d');
canvas.width = W, canvas.height = H;
ctx.drawImage(previewImage, 0, 0, W, H);
canvas.toBlob((blob) => {
previewImage.src = URL.createObjectURL(blob);
document.body.append(previewImage);
});
}).catch((encodingError) => {
// Do something with the error.
});
} // end of showfiles(file)
images.addEventListener('change', (e) => {
[...e.target.files].forEach(showFiles);
});
<input id="images" type="file" multiple>
I am modifying the background position of a background image that a user just uploaded (and so is a raw Data URI) using CSS but the re-rendering starts to lag if the image is >1mb.
The issue is not present for smaller images.
Is there anyway of dealing with this short of trying to optimize renders? I already am not re-rendering unless the background position changes by at least 1% (i.e. the change was 1% or greater between re-renders).
The issue is not present with the same image when I host it and then load the image from a URL.
You can see this issue in a small CodeSandBox that I made. Using the image in the default URL works well, but taking the file and uploading it makes it very laggy.
Could I somehow cache the data? It would seem like it is lagging because the raw data is not cached whereas the image is being cached.
I fiddled with the logic a bit and changed styles.backgroundPositionY to direct style manipulation by ref element. I don't see any big improvements. Uploaded image was indeed lagging while moving it where the default one was nimble.
const imageDivRef = useRef(null);
const onUpload = e => {
new Promise((resolve, reject) => {
const reader = new FileReader();
reader.onload = onLoadEvent => {
const dataUri = onLoadEvent.target.result;
let image = new Image() // try to cache that
image.src = dataUri;
image.onload = () => {
resolve(dataUri);
}
};
reader.readAsDataURL(e.target.files[0])
}).then(dataUri => {
updateBackgroundImage(dataUri)
})
};
const onPan = e => {
const y = position.y;
const delta = (e.deltaY / CONTAINER_HEIGHT) * -100;
requestAnimationFrame(() => {
imageDivRef.current.style.backgroundPositionY = `${Math.min(
Math.max(y + delta, 0),
100)}%`
})
};
const onPanEnd = e => {
const y = position.y;
const delta = (e.deltaY / CONTAINER_HEIGHT) * -100;
requestAnimationFrame(() => {
imageDivRef.current.style.backgroundPositionY = `${Math.min(
Math.max(y + delta, 0),
100)}%`
})
};
There is also one crazy idea. Ditch the backgroundPositionY, make wrapper div with overflow hidden and fixed size, put inside it img element, and manipulate it with transform: translateY(unit) property. I wonder if that can be quicker...
Next js has a image optimization future not a basic solution but...
I am trying to debug a problem I'm having with OpenCV.js. I am trying to create a simple circle-finding function, but my video feed is being displayed in my canvas. I've boiled it down to the smallest set that shows the issue.
What makes no sense is that I create a new, empty matrix and display it, and I see my video feed in it.
I start with the typical way of detecting circles: Stream the video into matrix scrMat, convert srcMat into a grayscale grayMat, and then call HoughCircles to detect circles from grayMat into circlesMat.
Then, independently, I create a new displayMat and display it.
I see the output below, where the right-handside is displayMat.
Somehow displayMat is being filled. The effect goes away if I comment out the HoughCircles line.
How is this happening?
const cv = require('opencv.js'); // v1.2.1
const video = document.getElementById('video');
const width = 300;
const height = 225;
const FPS = 30;
let stream;
let srcMat = new cv.Mat(height, width, cv.CV_8UC4);
let grayMat = new cv.Mat(height, width, cv.CV_8UC1);
let circlesMat = new cv.Mat();
const cap = new cv.VideoCapture(video);
export default function capture() {
navigator.mediaDevices.getUserMedia({ video: true, audio: false })
.then(_stream => {
stream = _stream;
video.srcObject = stream;
video.play();
setTimeout(processVideo, 0)
})
.catch(err => console.log(`An error occurred: ${err}`));
function processVideo () {
const begin = Date.now();
// these next three lines shouldn't affect displayMat
cap.read(srcMat);
cv.cvtColor(srcMat, grayMat, cv.COLOR_RGBA2GRAY);
// if this line is commented out, the effect goes away
cv.HoughCircles(grayMat, circlesMat, cv.HOUGH_GRADIENT, 1, 45, 75, 40, 0, 0);
// this ought to simply create a new matrix and draw it
let displayMat = new cv.Mat(height, width, cv.CV_8UC1);
cv.imshow('canvasOutput', displayMat);
const delay = 1000/FPS - (Date.now() - begin);
setTimeout(processVideo, delay);
}
}
Most probably displayMat is created in memory place where some image processing was done with HoughCircles() or something. That memory was released and became available for allocating new objects in it, but neither its freeing nor new Mat creation did not clear that memory block.
So just clean the displayMat first, as it is constructed on place of some "garbage" that left from previous operations, or use cv.Mat.zeros() to construct displayMat (zeros() fills the whole new matrix buffer with zeros).
let displayMat = cv.Mat.zeros(height, width, cv.CV_8UC1);
I'm very new to javascript and EaselJS and I've searched the documentation as well as the questions here and I've learned that my initial problem with my mouse events not working with my bitmaps was because I'm trying to use bitmaps from a filepath on my computer.
I understand now that I need to use img.crossOrigin to allow easeljs to work with these images, but even though I'd added this, I still can't get this to work.
Here's what I have:
function init(){
var canvas = document.getElementById('game'); //assign canvas to variable
var stage = new createjs.Stage(canvas);
var term = new Image();
term.src = 'terminal.png';
term.crossOrigin = "Anonymous";
var oswin = new Image();
oswin.src = 'frame.png';
oswin.crossOrigin = "Anonymous";
var terminal = new createjs.Bitmap(term);
terminal.x = 28;
terminal.y = 36;
terminal.name = "terminal";
stage.addChild(terminal);
var oswindow = new createjs.Bitmap(oswin);
stage.addChild(oswindow);
oswindow.x = 196;
oswindow.y = 5;
oswindow.name = "oswindow";
//hide this bitmap
oswindow.visible = false;
//onclick to unhide the bitmap
terminal.on("click", handleClick);
stage.update();
}
function handleClick(){
oswindow.visible = true;
stage.update();
}
I've added in the crossOrigin and unfortunately, this still doesn't work and I've no idea why because I don't understand cross domain very well despite reading on it. (I'm a beginning programmer)
I work entirely offline right now, just trying to learn, so a way to fix this would be really nice.
I have a web application that generates images on the fly and then renders (segments of) them on the client using canvas - the general code looks something like this (simplified)...
var width = ...
var height = ...
var img = new Image();
img.onload = function(){ // Set onload before setting image source ;)
var canvas = document.createElement("canvas");
canvas.setAttribute('width',width);
canvas.setAttribute('height',height);
var context = canvas.getContext("2d");
context.drawImage(this,0,0,width,height,0,0,width,height);
}
img.src = ...
In firefox and chrome, this works fine. In IE9 (and sometimes IE10), this sometimes results in a blank image. As a workaround, I put in...
var width = ...
var height = ...
var img = new Image();
img.onload = function(){
var canvas = document.createElement("canvas");
canvas.setAttribute('width',width);
canvas.setAttribute('height',height);
var context = canvas.getContext("2d");
window.setTimeout(function(){
context.drawImage(img,0,0,width,height,0,0,width,height);
}, 10); // 10 millisecond delay here seems to give IE time to make sure the image is actually ready
}
img.src = ...
Obviously, putting in random delays is making me nervous - I would much rather figure out why this happens. Is there another event type / method / property on images that may be used? Could there perhaps be something in the http headers sent down with the images / the encoding / transfer method that is causing this?
Since drawing images on the canvas seems to be something that trips up new users a lot, searches mostly yield instances where the developer set the source before the onload method. The closest other question / reply I was able to find was this : https://stackoverflow.com/a/15186350/470062
Edit:
I have updated the examples to include width and height definitions at the start
Edit 29/Apr/2013:
The 10 millisecond delay is not always sufficient - I have changed the code to check the canvas pixels to determine if anything has been drawn - This feels like such an ugly hack though - I would love to know if there is a better way :
... snip - this is at the end of the image load callback..
if((navigator.appVersion.indexOf("MSIE")) >= 0){
function checkAndRedraw(){
context.clearRect(0,0,width,height);
context.drawImage(img, 0, 0, width, height);
var imageData = context.getImageData(0, 0, width, height),
data = imageData.data,
len = data.length;
for (var i = 0; i < len; i += 4){
if(data[i] || data[i+1] || data[i+2]){
scheduleRedraw = null; // prevent memory leak
return; // A pixel contained some non empty value so something was drawn - done.
}
}
scheduleRedraw();
}
var redrawHackTimeout;
function scheduleRedraw(){
window.clearTimeout(redrawHackTimeout);
redrawHackTimeout = window.setTimeout(checkAndRedraw, 500);
}
scheduleRedraw();
}
//END IE9 Hack
I have the same issue in IE9, what I had to do is this too:
theImage = new Image();
theImage.onload = function() {
that = this;
setTimeout(function() {
that.onload2();
}, 100);
};
theImage.onload2 = function(){
#your old load function code
}
theImage.src = 'http://....';
I don't understand why ie9 triggers load image function when it's not actually loaded or not "usable" yet in a canvas. it makes me pull my hair out.
var img = new Image();
img.src = ...
img.onload = function(){ // Set onload before after setting image source ;)
var canvas = document.createElement("canvas");
canvas.setAttribute('width',width);
canvas.setAttribute('height',height);
var context = canvas.getContext("2d");
context.drawImage(this,0,0,width,height,0,0,width,height);
}
You must set the src, then wait for the image to load.