Get pixel data back from programmatically generated image - javascript

I generate array of pixels in client side JavaScript code and convert it to a blob. Then I pass URL of the blob as image.src and revoke it at image.onload callback. I don't keep any references to the data, generated by the previous steps, so this data may be freed by GC.
There are many images generated this way, and it works fine. But sometimes user may want to save the generated image by clicking on a Save button near the image. I don't want to generate image again, because generation is slow, and the image is already generated and is visible on screen. So I want to get my pixels back from the image. I tried to create canvas again, draw image on it and then call toBlob, but browser treats this image as cross origin and throws an exception: "Failed to execute 'toBlob' on 'HTMLCanvasElement': tainted canvases may not be exported". Similar errors I get with canvas.toDataUrl and canvasContext.getImageData.
Is there a workaround for this problem?
I also tried to create canvases instead of images, but when I create the second canvas, the content of the first one clears.
Added
This error occurs only in the Chrome and other WebKit browsers. Firefox and MS Edge work fine. And when I commented out line of code that revoked blob url, this error disappeared - I can draw the image on the canvas and get its pixel data without CORS issues. But it is pointless to do so, because I already have blob that is not deleted.
But my page may generate many images - it depends on its user and is unlimited. Size of images is also unlimited - it may be useful to generate even 4096x4096 images. So I want to reduce memory consumption of my page as much as possible. And all these images should be downloadable. Generation of most images uses previously generated images, so to regenerate last image in a chain, I must to regenerate all images.
So I need a workaround only for Chrome browser.
Added 2
I tried to reproduce this problem in JS Fiddle, but couldn't. However locally my code doesn't work - I developed my app locally and I haven't tried running it on server. Create test.html file on your computer and open it in browser (locally, without server):
<html>
<body>
<pre id="log"></pre>
</body>
<script>
var log = document.getElementById("log");
var canvas = document.createElement("canvas");
canvas.width = canvas.height = 256;
var ctx = canvas.getContext("2d");
canvas.toBlob(function(blob) {
var img = new Image();
var blobUrl = URL.createObjectURL(blob);
img.src = blobUrl;
img.onload = function()
{
URL.revokeObjectURL(blobUrl);
ctx.drawImage(img, 0, 0);
try { canvas.toBlob(function(blob) { log.textContent += 'success\n'; }); }
catch(e) {log.textContent += e.message + '\n';}
};
});
</script>
</html>
It will print Failed to execute 'toBlob' on 'HTMLCanvasElement': Tainted canvases may not be exported..
So, I think, my workaround is to detect that page is run on combination of WebKit browser and file:/// protocol. And I can to defer revoking blob URL until page unload only for this combination.

That indeed sounds like a bug in chrome's implementation, you may want to report it.
What seems to happen is that chrome only checks if the image has been loaded through a clean origin when it is drawn for the first time on the canvas.
Since at this time, you already did revoke the blobURI, it can't map this URI to a clean origin (file:// protocol is quite strict on chrome).
But there seems to be a simple workaround:
By drawing this image on a [the reviver]* canvas before revoking the blobURI, chrome will mark the image as clean, and will remember it.
*[edit] Actually it seems it needs to be the same canvas...
So you can simply create your export canvas before-hand, and draw each images on it (to save memory you can even set it to 1x1px when you don't use it for the export) before you do revoke the image's src:
// somewhere accessible to the export logic
var reviver = document.createElement('canvas').getContext('2d');
// in your canvas to img logic
canvas.toBlob(function(blob){
var img = new Image();
img.onload = function(){
reviver.canvas.width = reviver.canvas.height = 1; // save memory
reviver.drawImage(this, 0,0); // mark our image as origin clean
URL.revokeObjectURL(this.src);
}
img.src = URL.createObjectURL(blob);
probablySaveInAnArray(img);
};
// and when you want to save your images to disk
function reviveBlob(img){
reviver.canvas.width = img.width;
reviver.canvas.height = img.height;
reviver.drawImage(img,0,0);
reviver.canvas.toBlob(...
}
But note that this method will create a new Blob, not retrieve the previous one, which has probably been Collected by the GarbageCollector at this time. But it is unfortunately your only way since you did revoke the blobURI...

Related

Retrieving image data URi from HTTP response

The Problem: Imagine I have a td element structured as such:
<td td="" colspan="2">
<br>
<img src="ImageServlet" alt="random image" border="0" id="simpleRandomImg">
</td>
of course, with other HTML around it. The image contained within the servlet doesn't have a useful src indicated in the HTML (clearly) - however, when I open up the Network tab in Chrome, I can preview the loaded image itself and copy the image as a data URI - which gives me a stable URL like:
data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAMgAAAAyCAIAAACWMwO2AAAhQklEQVR42u2dB1Sb5/nonTRt2pPkJGmSNm180zhNTkabuPGKMeAwxBaIJYEYWmhLSEIggRhaaAIaIEASEggQew+zzPLEdjyy07T/9t9129t129u9z7mP9LnfXxZCBuLeS/71c77zIRQCGj89z+993+d7vS/GH7GxsXH+wGAwCf5ITExMSkpKTk5OSUlJTU1N80d6etcetcetc...
so, clearly there exists a stable reference to the loaded image. And clearly my browser 'knows' what that reference is, because I can retrieve the link to it as a data URI - but there's no reference to that data URI in the actual HTTP response. This probably seems a lot less mystical to someone who understands JavaScript, but that someone is not myself - so could someone explain what's going on here, and if there is some way to gather the data image URI from the HTTP response?
Attemped Solutions:
Did a little digging around in the HTTP response and located this bit of JavaScript which, apparently, handles the changing of images:
function changeImage() {
// makes a new image load
var obj=document.getElementById('simpleRandomImg');
if (obj != null) {
// append a unique index to force browser to reload
obj.src='ImageServlet?'+(cnt++);
}
However, nothing I see there gives any indication as to the actual URI location of the image. As before, if I open the Google Chrome network tab and attempt to retrieve the data image URI from the individual response, it works and gives me a valid URI - so, clearly the browser is receiving it. How can I access it?
e: to be clear, I do not control the website in question, so I can't 'fix' it by just changing the internal javascript - I'm viewing the site and am interested in whether or not it's possible to retrieve the loaded images short of screenshotting the page itself.
Something like this should work. Canvas API has a function called toDataURI
function getDataUri(url, callback) {
var image = new Image();
image.onload = function () {
var canvas = document.createElement('canvas');
canvas.width = this.naturalWidth; // or 'width' if you want a special/scaled size
canvas.height = this.naturalHeight; // or 'height' if you want a special/scaled size
canvas.getContext('2d').drawImage(this, 0, 0);
// Get raw image data
callback(canvas.toDataURL('image/jpg').replace(/^data:image\/(png|jpg);base64,/, ''));
// ... or get as Data URI
callback(canvas.toDataURL('image/jpg'));
};
image.src = url;
}
// Usage
// beware of server cors settings..
getDataUri('image url here', function(dataUri) {
console.log(dataUri)
// Do whatever you'd like with the Data URI!
});
You can set the src of the image to your URI by using setAttribute, like so:
obj.setAttribute('src', uriString)

save css computed background image as a variable or object for later use

Is it possible to get the image data from a computed style and save it in a variable or object for later use ?
For example in HTML web page I load the background from an image with CSS:
document.body.style.background = "url(...path/image.php) #FFF";
Now I need to save the image which was loaded in document.body.style.background
because i will use it later. I would like to save the image in localStorage as DATA URI and access it even after i restart the browser.
I cannot use AJAX or send request to server. I have to copy/save somehow from webpage which was loaded. But I'm not sure if this is possible and if yes , how
I cannot use the image URL because the url generates random images everytime, it something like http://url.php ... and not a standard image url
Maybe using HTML5 Canvas is possible to copy from document.body ?
the webpage is populated with other child elements
Create a canvas > object - load your generated image into the canvas, then use canvas.toDataURL to grab the base64 representation of the image. You can then store that base64 string as a variable and reload it whenever you want.
Detailed example in this SO answer: How to convert image into base64 string using javascript
There is no way to take a real screen-shot of any element through canvas. (at least not in WebAPI, in some browsers, extensions are allowed to do so).
You will have to create a new Image object, set it's src to the one of the background-image, and then only be able to draw it on the canvas.
But if the server answers with a no-cache, then the Image will just load an other image than the one set as background.
So you will have to make things in the other way :
First load the image through an Image Object.
Then draw this image to a canvas and get its dataURI version by calling canvas.toDataURL().
Finally, set your element's background to this dataURI you just extracted.
// first load your image from the server once, into an Image object
var img = new Image();
// wait for the image has loaded
img.onload = function(){
var canvas = document.createElement('canvas');
var ctx = canvas.getContext('2d');
canvas.width = this.width;
canvas.height = this.height;
ctx.drawImage(this, 0,0);
// since I'm receiving photos, jpeg is better
var dataURL = canvas.toDataURL('image/jpeg');
// if we could use local storage in snippets...
//localStorage.setItem('background-image', dataURL);
// since we can't
saved = dataURL;
document.body.style.backgroundImage = 'url('+dataURL+')';
}
// I'm loading it from an external server
img.crossOrigin = 'anonymous';
img.src = 'http://lorempixel.com/200/200/?'+Math.random();
// since we can't use local storage in sand-boxed iframe
// we will use a global in this snippet
var saved;
button.onclick = function(){
// if we could use localStorage
// var saved = localStorage.getItem('background-image');
document.getElementById('result').style.backgroundImage = 'url('+saved+')';
};
div{width: 200px; height: 200px; border: 1px solid; background-color: white;}
body{width: 100vw; height: 100vh; margin: 0;}
<button id="button">set the div's background to the saved one</button>
<div id="result"></div>
Note that localStorage is limited so depending on your graphics, you may want to use the image/jpeg compression type of the toDataURL() method.
Also, you'll be limited by cross-origin policies, if the php page is not on the same domain, make sure it is correctly configured to accept cross-origin requests and set you Image object's crossOrigin property to "anonymous".
Ps : You could also avoid the canvas part by using an XMLHttpRequest and a FileReader(), wich will involve different cross-origin restrictions.

fabric.js - toDataURL shows blank page when there is image on canvas

I'm using fabric.js for my canvas application, toDataURL method works properly except when there is a image on canvas. When i add an image to canvas and call toDataURL it shows me a blank page.
//When i call it from chrome console
canvas.toDataURL();
//It returns a working data url with no problem.
"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAPAAAAGkCAYAAAAPPajHAAAgAElEQ…fpmwgogX1TrjoqP0FACewngtZh+iYCSmDflKuOyk8Q+H+CKCqUW0spTgAAAABJRU5ErkJggg=="
//When i execute same code in a .js file it returns a data url which shows a blank image.
"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAPAAAAGkCAYAAAAPPajHAAAKC0lEQ…sBAmEBAw6XJzoBA/YDBMICBhwuT3QCBuwHCIQFDDhcnugEHt/IAaW9dzALAAAAAElFTkSuQmCC"
It's interesting that it's working on chrome dev console but not works in .js file even if it's same code. I noticed that working data url finishes with '==' but other one not. However i don't know what this means.
You didn't give much to analyze but I'll go from there on my gut feeling.
The image you are using is probably violating Cross-Origin Resource Sharing (CORS) requirements. When this happens the canvas will return a blank image when you try to get the pixel data either by using canvas.toDataURL() or context.getImageData().
It basically happens when the image is not located on the same domain as the page or is loaded from the file:// protocol.
You can try to add the following to the image object before setting the source:
image.crossOrigin = 'anonymous';
image.src = '...';
From tag:
<img crossOrigin='anonymous' src='...' alt='' />
This will request permission from the server to use the image cross-origin. If the server allows you should be fine.
If not you will either have to copy the image to your own server so it loads from the same domain as your page, or create a proxy page that loads the image externally and serves it to your page from the proxy page (it sounds complicated but really isn't).
If the image does not show up at all on the canvas you are probably not using load callback which you need since image loading is asynchronous. If so just add:
image.onload = function() {
/// now you can draw the image to canvas
}
image.crossOrigin = 'anonymous';
image.src = '...';
The problem is solved. The point i missed is, i was calling toDataURL function before canvas is loaded, that's why it was showing me a blank page.
canvas.loadFromJSON(json,function(){
var dataURL = canvas.toDataURL();
});
This solved my problem when i gave toDataURL() function as a callback to loadFromJSON function.
But after a while i had a different issue about CORS when i tried to upload my images from s3 bucket and i solved this problem as upward solution.
I was facing the same issues when I was trying to generate images from the canvas using Fabricsjs and generate PDF from images using JSPDF so below is my case I also spend hours on this may this can help someone to save time.
Load the canvas from JSON that i.e
canvas.loadFromJSON(json, canvas.renderAll.bind(canvas), function(obj, object) {
//fabric.log(obj, object);
});
Canvas was drawing images all fine then I was generating the images from that canvas it was a case of multiple images in a single canvas and I was generating PDF based on each page of the canvas.
Instead of this
canvas.toDataURL('image/png', 0.1)
I used this and it starts returning me the propper images dataurl
var imgData = document.getElementById('canvas_0').toDataURL();
Below are snippets
$("#pdfSelector .canvas-container").each(function(index, value){ // canvas selector
html2canvas($("#canvas_"+index), { // html2canvas used to get images
onrendered: function(canvas) { // on successfully render of images
//var imgData = canvas.toDataURL('image/png', 0.1);
var imgData = document.getElementById('canvas_0').toDataURL();
const imgProps= doc.getImageProperties(imgData);
const pdfWidth = doc.internal.pageSize.getWidth();
const pdfHeight = (imgProps.height * pdfWidth) / imgProps.width;
doc.addImage(imgData, 'PNG', 0, 0, pdfWidth, pdfHeight, 'page_'+index, 'FAST');
}
});
});

Security Error with canvas.toDataURL() and drawImage()

<!doctype html>
<canvas id="canvas" height="200" width="200"></canvas>
<div id="new"></div>
<script>
var div = document.getElementById("new");
var canvas = document.getElementById("canvas");
var ctx = canvas.getContext("2d");
var img = new Image();
img.src = 'http://www.w3.org/html/logo/downloads/HTML5_Logo_512.png';
//img.src = 'local.png';
img.onload = function(){
//draws the image on the canvas (works)
ctx.drawImage(img,0,0,200,200);
//creates an image from the canvas (works only for local.png)
var sourceStr = canvas.toDataURL();
//creates the img-tag and adds it to the div-container
var newImage = document.createElement("img");
newImage.src = sourceStr;
div.appendChild(newImage);
}
</script>
This script creates a canvas with the html5-logo. From this canvas I want to create an image, using the "toDataURL()"-method. Here I get a security error.
Firefox says - Error: uncaught exception: [Exception... "Security error" code: "1000" nsresult: "0x805303e8 (NS_ERROR_DOM_SECURITY_ERR)"
Chrome says - Uncaught Error: SECURITY_ERR: DOM Exception 18
If I use the script with an image on the server it works fine. So it think this time it is really a feature and not a bug.
Does anyone has an idea how I can create an image using canvas with out of an image form an other server?
BTW: the error occurs if you run the site as a local file without a webserver.
You are right, this is a security feature, not a bug.
If reading the Image (for instance with toDataURL or getImageData) would work, you could also read https://mail.google.com/mail/ from the context of your visitor get his emails or whatever.
Therefore, canvas elements have a origin-clean flag, which is set when external images are written to the canvas. In that case, you can no longer read from it.
You can read more about this topic here.
i believe that the error is an extension to the same origin policy, basically it isnt allowing you to manipulate a resource from another server. although insecure you could build into the server a method of retrieving external resources: myserver.com/?external=url/path/to/img... that would work in theory.
Ya thats a feature. As the image is on another server. Check this
Why does canvas.toDataURL() throw a security exception?
You can catch these exceptions. But this will be headache to do for other browsers too and also not right for security.
So better solution is download that image on local.And give the image src path to that.

capturing html5 canvas output as video or swf or png sequence?

I need to take HTML5 canvas output as video or swf png sequence.
I found the following link on stackoverflow for capturing images.
Capture HTML Canvas as gif/jpg/png/pdf?
But can anyone suggest if we want the output to be video or swf of png sequence?
EDIT:
Ok now I understood how to capture the canvas data to store on server, I tried it and it is working fine if I use only shapes, rectangle or some other graphic, but not if I draw external images on canvas element.
Can anyone tell me how to capture canvas data completely whether we use graphic or external images for drawing on canvas?
I used the following code:
var cnv = document.getElementById("myCanvas");
var ctx = cnv.getContext("2d");
if(ctx)
{
var img = new Image();
ctx.fillStyle = "rgba(255,0,0,0.5)";
ctx.fillRect(0,0,300,300);
ctx.fill();
img.onload = function()
{
ctx.drawImage(img, 0,0);
}
img.src = "my external image path";
var data = cnv.toDataURL("image/png");
}
after taking the canvas data into my "data" variable I created a new canvas and draw the captured data on that, the red rectangle drawn on the second canvas but that external image doesn't.
Thanks in advance.
I would suggest:
Use setInterval to capture the contents of your Canvas as a PNG data URL.
function PNGSequence( canvas ){
this.canvas = canvas;
this.sequence = [];
};
PNGSequence.prototype.capture = function( fps ){
var cap = this;
this.sequence.length=0;
this.timer = setInterval(function(){
cap.sequence.push( cap.canvas.toDataURL() );
},1000/fps);
};
PNGSequence.prototype.stop = function(){
if (this.timer) clearInterval(this.timer);
delete this.timer;
return this.sequence;
};
var myCanvas = document.getElementById('my-canvas-id');
var recorder = new PNGSequence( myCanvas );
recorder.capture(15);
// Record 5 seconds
setTimeout(function(){
var thePNGDataURLs = recorder.stop();
}, 5000 );
Send all these PNG DataURLs to your server. It'll be a very large pile of data.
Using whatever server-side language you like (PHP, Ruby, Python) strip the headers from the data URLs so that you are left with just the base64 encoded PNGs
Using whatever server-side language you like, convert the base64 data to binary and write out temporary files.
Using whatever 3rd party library you like on the server, convert the sequence of PNG files to a video.
Edit: Regarding your comment of external images, you cannot create a data URL from a canvas that is not origin-clean. As soon as you use drawImage() with an external image, your canvas is tainted. From that link:
All canvas elements must start with their origin-clean set to true.
The flag must be set to false if any of the following actions occur:
[...]
The element's 2D context's drawImage() method is called with an HTMLImageElement or an HTMLVideoElement whose origin is not the same as that of the Document object that owns the canvas element.
[...]
Whenever the toDataURL() method of a canvas element whose origin-clean flag is set to false is called, the method must raise a SECURITY_ERR exception.
Whenever the getImageData() method of the 2D context of a canvas element whose origin-clean flag is set to false is called with otherwise correct arguments, the method must raise a SECURITY_ERR exception.
To start out, you want to capture the pixel data from the canvas on a regular interval (using JavaScript timers probably). You can do this by calling context.getImageData on the canvas's context. That will create a series of images that you can turn into a video stream.
http://www.whatwg.org/specs/web-apps/current-work/multipage/the-canvas-element.html#pixel-manipulation

Categories