How to display base64 image in Javascript - javascript

I am trying to stream Raspberry Pi camera, and for security reasons I'm tunneling my server through a secured server even though the client is on the same LAN. However, this resulted in horrible latency. I'm trying now to stream the video over the LAN instead of through the server. I know that browsers only allow HTTP connections over HTTPS servers only on passive elements, such as .
My server is saving the image as a local jpeg file encoded as base64:
camera = cv2.VideoCapture(0)
grabbed, frame = camera.read()
frame = cv2.resize(frame, (320, 240))
buffer = cv2.imencode('.jpg', frame)[1]
buffer = base64.b64encode(buffer)
buffer.decode('utf-8')
with open(ROOT + '/static/image.jpeg', mode='wb+') as image:
image.write(buffer)
my client has an image tag, and a simple script to request the file saved:
<img id='img'>
<script>
setInterval(function() {
var myImageElement = document.getElementById('img');
myImageElement.src = 'data:image/jpg;base64,http://10.0.0.35:8000/static/image.jpg?rand=' + Math.random();
}, 500);
</script>
The result is a constant stream of console errors such as:
data:image/jpg;charset=utf-8;base64,http://10.0.0.35:8000/static/image.jpg?rand=0.7520646586573345:1 GET data:image/jpg;charset=utf-8;base64,http://10.0.0.35:8000/static/image.jpg?rand=0.7520646586573345 net::ERR_INVALID_URL
I have checked that the image is accessible (entered http://10.0.0.35:8000/static/image.jpg in my broswer, and haven't gotten any errors)
I have also checked that the file is a base64 jpeg using an online tool.
I have looked at this question but nothing there worked,
I can't understand why am I getting this error, and how to solve it.
Can someone please direct me in the right direction?

To solve this issue, I used 2 different solutions:
[A]: I used mjpg-streamer to stream my video to LAN
mjpg-streamer is opening a local server to stream or capture images from camera on port 8080. so I've changed the src attribute of the image to:
[B]: myImageElement.src = http://10.0.0.35:8080
And that's it.

Related

Cross Origin Error when using getImageData [duplicate]

My code is working very well on my localhost but it is not working on the site.
I got this error from the console, for this line .getImageData(x,y,1,1).data:
Uncaught SecurityError: Failed to execute 'getImageData' on 'CanvasRenderingContext2D': The canvas has been tainted by cross-origin data.
part of my code:
jQuery.Event.prototype.rgb=function(){
var x = this.offsetX || (this.pageX - $(this.target).offset().left),y = this.offsetY || (this.pageY - $(this.target).offset().top);
if (this.target.nodeName!=="CANVAS")return null;
return this.target.getContext('2d').getImageData(x,y,1,1).data;
}
Note: my image url (src) is from a subdomain url
As others have said you are "tainting" the canvas by loading from a cross origins domain.
https://developer.mozilla.org/en-US/docs/HTML/CORS_Enabled_Image
However, you may be able to prevent this by simply setting:
img.crossOrigin = "Anonymous";
This only works if the remote server sets the following header appropriately:
Access-Control-Allow-Origin "*"
The Dropbox file chooser when using the "direct link" option is a great example of this. I use it on oddprints.com to hoover up images from the remote dropbox image url, into my canvas, and then submit the image data back into my server. All in javascript
I found that I had to use .setAttribute('crossOrigin', '') and had to append a timestamp to the URL's query string to avoid a 304 response lacking the Access-Control-Allow-Origin header.
This gives me
var url = 'http://lorempixel.com/g/400/200/';
var imgObj = new Image();
imgObj.src = url + '?' + new Date().getTime();
imgObj.setAttribute('crossOrigin', '');
You won't be able to draw images directly from another server into a canvas and then use getImageData. It's a security issue and the canvas will be considered "tainted".
Would it work for you to save a copy of the image to your server using PHP and then just load the new image? For example, you could send the URL to the PHP script and save it to your server, then return the new filename to your javascript like this:
<?php //The name of this file in this example is imgdata.php
$url=$_GET['url'];
// prevent hackers from uploading PHP scripts and pwning your system
if(!#is_array(getimagesize($url))){
echo "path/to/placeholderImage.png";
exit("wrong file type.");
}
$img = file_get_contents($url);
$fn = substr(strrchr($url, "/"), 1);
file_put_contents($fn,$img);
echo $fn;
?>
You'd use the PHP script with some ajax javascript like this:
xi=new XMLHttpRequest();
xi.open("GET","imgdata.php?url="+yourImageURL,true);
xi.send();
xi.onreadystatechange=function() {
if(xi.readyState==4 && xi.status==200) {
img=new Image;
img.onload=function(){
ctx.drawImage(img, 0, 0, canvas.width, canvas.height);
}
img.src=xi.responseText;
}
}
If you use getImageData on the canvas after that, it will work fine.
Alternatively, if you don't want to save the whole image, you could pass x & y coordinates to your PHP script and calculate the pixel's rgba value on that side. I think there are good libraries for doing that kind of image processing in PHP.
If you want to use this approach, let me know if you need help implementing it.
edit-1: peeps pointed out that the php script is exposed and allows the internet to potentially use it maliciously. there are a million ways to handle this, one of the simplest being some sort of URL obfuscation... i reckon secure php practices deserves a separate google ;P
edit-2: by popular demand, I've added a check to ensure it is an image and not a php script (from: PHP check if file is an image).
I was seeing this error on Chrome while I was testing my code locally. I switched to Firefox and I am not seeing the error any more. Maybe switching to another browser is a quick fix.
If you are using the solution given in the first answer, then make sure you add img.crossOrigin = "Anonymous"; just after you declare the img variable (for eg. var img = new Image();).
When working on local, add a server.
I had a similar issue when working on local. Your URL is going to be the path to the local file, for example, file:///Users/PeterP/Desktop/folder/index.html.
Please note that I am on a Mac.
I got around this by installing an HTTP server globally. I used https://www.npmjs.com/package/http-server
Steps
Global install: npm install http-server -g
Run server: http-server ~/Desktop/folder/
These steps assume that you have node installed, otherwise you won't get very far running npm commands.
My problem was so messed up I just base64 encoded the image to ensure there couldn't be any CORS issues
Your problem is that you load an external image, meaning from another domain. This causes a security error when you try to access any data of your canvas context.
You are "tainting" the canvas by loading from a cross origins domain. Check out this MDN article:
https://developer.mozilla.org/en-US/docs/HTML/CORS_Enabled_Image
Set the Image's crossOrigin attribute to Anonymous.
let image = new Image();
// image.src = ...;
image.crossOrigin = `Anonymous`;
As matt burns says in his answer, you may need to enable CORS on the server where the problem images are hosted.
If the server is Apache, this can be done by adding the following snippet (from here) to either your VirtualHost config or an .htaccess file:
<IfModule mod_setenvif.c>
<IfModule mod_headers.c>
<FilesMatch "\.(cur|gif|ico|jpe?g|png|svgz?|webp)$">
SetEnvIf Origin ":" IS_CORS
Header set Access-Control-Allow-Origin "*" env=IS_CORS
</FilesMatch>
</IfModule>
</IfModule>
...if adding it to a VirtualHost, you'll probably need to reload Apache's config too (eg. sudo service apache2 reload if Apache's running on a Linux server)
you can convert the image to a data string as use the image source instead of the actual image source.
[https://www.base64-image.de/][1] to convert image to data string.
Convert and copy string data from the above website.
set image.src = <copied_data_string>.
workaround solution,
convert source image URL to Base64 data and assign to img
for example, use Axios
const getBase64 = async(url)=>{
try {
let image = await axios.get(url, { responseType: 'arraybuffer' });
let raw = Buffer.from(image.data).toString('base64');
return "data:" + image.headers["content-type"] + ";base64,"+raw;
} catch (error) {
console.log(error)
}
}
var image = new Image()
image.src=getBase64(url)
no cross-origin dependency from canvas
I meet the same problem today, and solve it by the code follows.
html code:
<div style='display: none'>
<img id='img' src='img/iak.png' width='600' height='400' />
</div>
<canvas id='iak'>broswer don't support canvas</canvas>
js code:
var canvas = document.getElementById('iak')
var iakImg = document.getElementById('img')
var ctx = canvas.getContext('2d')
var image = new Image()
image.src=iakImg.src
image.onload = function () {
ctx.drawImage(image,0,0)
var data = ctx.getImageData(0,0,600,400)
}
code like above, and there is no cross-domain problem.
I was having the same issue, and for me it worked by simply concatenating https:${image.getAttribute('src')}

Why does datatransfer between sockets takes lot of time?

I have implemented a web based client-server system. The goal is to request for an image file to server, through the socket.
Here is my code at client end. [embedded Javascript code]
<a id="downloadLnk" download="new.jpeg" style="color:red">Download as image</a>
var socket = io("ipaddress");
socket.on("image", function(info) {
if (info.image) {
var end1 = new Date().getTime();
document.getElementById("demo1").innerHTML = end1;
var img = new Image();
img.src = 'data:image/jpeg;base64,' + info.buffer;
}
function download() {
this.href = img.src;
};
downloadLnk.addEventListener('click', download, false);
});
And this is the code at server side: [node.js server, express module, fs module]
io.on('connection', function(socket){
var start1 = new Date().getTime();
console.log(start1);
fs.readFile(__dirname + '/aorta-high512.jpg', function(err, buf){
socket.emit('image', { image: true, buffer: buf.toString('base64') });
});
});
I am transferring a 512x512 resolution image of size 88KB and it is taking approximately one second. Similarly for a 259KB file it takes around 1.2s and 2MB file it takes 2.5s. I do not understand why it is taking so much time?
I checked the bandwidth avalable, internet speed of my network in speedtest.net. The download speed is 95.97Mbps and upload speed is 23.30Mbps.
Could you please let me know, why the transfer time of data is so slow? Is there any other method to transfer data in a faster way? I definitely know that 96Mbps is the bandwidth available but still to test I downloaded a 100Mb pdf file from internet it took approximately 12-14s. Looking at this I atleast expect faster transfer of data at the rate of atleast 2-3 Mbps.
Socket.IO supports sending/receiving binary data, so taking advantage of that will allow you to avoid expensive encoding of data.
Secondly, when generating/using a data URL in browsers you have to be careful about the URL length. Many browsers impose various limits on the maximum size of such data URLs. One possible workaround to this (not including serving the image directly via HTTP GET) could include having the server split the image into a smaller set of images, which you then use with stacked img tags to give the appearance of a single image.

How to fix getImageData() error The canvas has been tainted by cross-origin data?

My code is working very well on my localhost but it is not working on the site.
I got this error from the console, for this line .getImageData(x,y,1,1).data:
Uncaught SecurityError: Failed to execute 'getImageData' on 'CanvasRenderingContext2D': The canvas has been tainted by cross-origin data.
part of my code:
jQuery.Event.prototype.rgb=function(){
var x = this.offsetX || (this.pageX - $(this.target).offset().left),y = this.offsetY || (this.pageY - $(this.target).offset().top);
if (this.target.nodeName!=="CANVAS")return null;
return this.target.getContext('2d').getImageData(x,y,1,1).data;
}
Note: my image url (src) is from a subdomain url
As others have said you are "tainting" the canvas by loading from a cross origins domain.
https://developer.mozilla.org/en-US/docs/HTML/CORS_Enabled_Image
However, you may be able to prevent this by simply setting:
img.crossOrigin = "Anonymous";
This only works if the remote server sets the following header appropriately:
Access-Control-Allow-Origin "*"
The Dropbox file chooser when using the "direct link" option is a great example of this. I use it on oddprints.com to hoover up images from the remote dropbox image url, into my canvas, and then submit the image data back into my server. All in javascript
I found that I had to use .setAttribute('crossOrigin', '') and had to append a timestamp to the URL's query string to avoid a 304 response lacking the Access-Control-Allow-Origin header.
This gives me
var url = 'http://lorempixel.com/g/400/200/';
var imgObj = new Image();
imgObj.src = url + '?' + new Date().getTime();
imgObj.setAttribute('crossOrigin', '');
You won't be able to draw images directly from another server into a canvas and then use getImageData. It's a security issue and the canvas will be considered "tainted".
Would it work for you to save a copy of the image to your server using PHP and then just load the new image? For example, you could send the URL to the PHP script and save it to your server, then return the new filename to your javascript like this:
<?php //The name of this file in this example is imgdata.php
$url=$_GET['url'];
// prevent hackers from uploading PHP scripts and pwning your system
if(!#is_array(getimagesize($url))){
echo "path/to/placeholderImage.png";
exit("wrong file type.");
}
$img = file_get_contents($url);
$fn = substr(strrchr($url, "/"), 1);
file_put_contents($fn,$img);
echo $fn;
?>
You'd use the PHP script with some ajax javascript like this:
xi=new XMLHttpRequest();
xi.open("GET","imgdata.php?url="+yourImageURL,true);
xi.send();
xi.onreadystatechange=function() {
if(xi.readyState==4 && xi.status==200) {
img=new Image;
img.onload=function(){
ctx.drawImage(img, 0, 0, canvas.width, canvas.height);
}
img.src=xi.responseText;
}
}
If you use getImageData on the canvas after that, it will work fine.
Alternatively, if you don't want to save the whole image, you could pass x & y coordinates to your PHP script and calculate the pixel's rgba value on that side. I think there are good libraries for doing that kind of image processing in PHP.
If you want to use this approach, let me know if you need help implementing it.
edit-1: peeps pointed out that the php script is exposed and allows the internet to potentially use it maliciously. there are a million ways to handle this, one of the simplest being some sort of URL obfuscation... i reckon secure php practices deserves a separate google ;P
edit-2: by popular demand, I've added a check to ensure it is an image and not a php script (from: PHP check if file is an image).
I was seeing this error on Chrome while I was testing my code locally. I switched to Firefox and I am not seeing the error any more. Maybe switching to another browser is a quick fix.
If you are using the solution given in the first answer, then make sure you add img.crossOrigin = "Anonymous"; just after you declare the img variable (for eg. var img = new Image();).
When working on local, add a server.
I had a similar issue when working on local. Your URL is going to be the path to the local file, for example, file:///Users/PeterP/Desktop/folder/index.html.
Please note that I am on a Mac.
I got around this by installing an HTTP server globally. I used https://www.npmjs.com/package/http-server
Steps
Global install: npm install http-server -g
Run server: http-server ~/Desktop/folder/
These steps assume that you have node installed, otherwise you won't get very far running npm commands.
My problem was so messed up I just base64 encoded the image to ensure there couldn't be any CORS issues
Your problem is that you load an external image, meaning from another domain. This causes a security error when you try to access any data of your canvas context.
You are "tainting" the canvas by loading from a cross origins domain. Check out this MDN article:
https://developer.mozilla.org/en-US/docs/HTML/CORS_Enabled_Image
Set the Image's crossOrigin attribute to Anonymous.
let image = new Image();
// image.src = ...;
image.crossOrigin = `Anonymous`;
As matt burns says in his answer, you may need to enable CORS on the server where the problem images are hosted.
If the server is Apache, this can be done by adding the following snippet (from here) to either your VirtualHost config or an .htaccess file:
<IfModule mod_setenvif.c>
<IfModule mod_headers.c>
<FilesMatch "\.(cur|gif|ico|jpe?g|png|svgz?|webp)$">
SetEnvIf Origin ":" IS_CORS
Header set Access-Control-Allow-Origin "*" env=IS_CORS
</FilesMatch>
</IfModule>
</IfModule>
...if adding it to a VirtualHost, you'll probably need to reload Apache's config too (eg. sudo service apache2 reload if Apache's running on a Linux server)
you can convert the image to a data string as use the image source instead of the actual image source.
[https://www.base64-image.de/][1] to convert image to data string.
Convert and copy string data from the above website.
set image.src = <copied_data_string>.
workaround solution,
convert source image URL to Base64 data and assign to img
for example, use Axios
const getBase64 = async(url)=>{
try {
let image = await axios.get(url, { responseType: 'arraybuffer' });
let raw = Buffer.from(image.data).toString('base64');
return "data:" + image.headers["content-type"] + ";base64,"+raw;
} catch (error) {
console.log(error)
}
}
var image = new Image()
image.src=getBase64(url)
no cross-origin dependency from canvas
I meet the same problem today, and solve it by the code follows.
html code:
<div style='display: none'>
<img id='img' src='img/iak.png' width='600' height='400' />
</div>
<canvas id='iak'>broswer don't support canvas</canvas>
js code:
var canvas = document.getElementById('iak')
var iakImg = document.getElementById('img')
var ctx = canvas.getContext('2d')
var image = new Image()
image.src=iakImg.src
image.onload = function () {
ctx.drawImage(image,0,0)
var data = ctx.getImageData(0,0,600,400)
}
code like above, and there is no cross-domain problem.
I was having the same issue, and for me it worked by simply concatenating https:${image.getAttribute('src')}

Rendering MJpeg stream in html5

I'm trying to render MJpeg stream in HTML5 using the img tag.
When I'm running the following, everything works great, meaning, the video starts to play until the video ends:
<img src="http://[some ip]:[port]/mjpg">
My question is how can I get the stream frame by frame.
For each frame, I want to get it, do something (ajax call to the server) and then display the frame as an image.
Thanks.
You can do this without repeatedly making Http requests. Only one will suffice. You can use the fetch api to create a ReadableStream, access it's Reader and keep reading from the stream.
Once you have the reader keep reading chunks from the stream recursively. Look for the SOI ( 0xFF 0xD8) in the byte stream which signals the end of the header and the beginning of the JPEG frame. The header will contain the length of the JPEG in bytes to be read. Read that many bytes from the chunk and any successive chunks and store it into a Uint8Array. Once you've
successfully read the frame convert it into a blob, create a UrlObject out of it and assign it to the src property of your img object.
Keep doing this till the connection is closed.
Shameless plug. Here's a link to a working sample on github.
If the camera exposes raw JPEG images (not .MJPEG extension) you'll have to reaload it manually (if the extension is .MJPEG the browser will do everything, just put the correct src). If you have .MJPEG and want to use the raw .JPEG check your camera documentation. Most cameras expose both the .MJPEG and raw .JPEG streams (just on different URLs).
Unfortunately you won't be able to easily get the image through ajax, but you could change the src of the image periodically.
You can use Date.getTime() and add it to the querystring to force the browser to reload the image, and repeat each time the image loads.
If you use jQuery the code will look something like this:
camera.html
<!DOCTYPE html>
<html>
<head>
<title>ipCam</title>
</head>
<body>
<h1>ipCam</h1>
<img id="motionjpeg" src="http://user:pass#127.0.0.1:8080/" />
<script src="motionjpeg.js"></script>
<script>
//Using jQuery for simplicity
$(document).ready(function() {
motionjpeg("#motionjpeg"); // Use the function on the image
});
</script>
</body>
</html>
motionjpeg.js
function motionjpeg(id) {
var image = $(id), src;
if (!image.length) return;
src = image.attr("src");
if (src.indexOf("?") < 0) {
image.attr("src", src + "?"); // must have querystring
}
image.on("load", function() {
// this cause the load event to be called "recursively"
this.src = this.src.replace(/\?[^\n]*$/, "?") +
(new Date()).getTime(); // 'this' refers to the image
});
}
Note that my example will play the MotionJPEG on page load, but won't allow play/pause/stop functionalities
If you stream source can't return frames on another address (http://[some ip]:[port]/frame/XXX) then you can use MJPEG stream parser on the server. For example, Paparazzo.js parse stream and return single jpeg. Actually it returns only last frame without saving previous, but it can be changed.
Problem can't be solved only in browser with js without some plugins and server.

WebSocket JavaScript: Sending complex objects

I am using WebSockets as the connection between a Node.js server and my client JS code.
I want to send a number of different media types (Text, Audio, Video, Images) through the socket.
This is not difficult of course. message.data instanceof Blob separates text from media files. The problem is, that I want to include several additional attributes to those media files.
F.e.:
Dimension of an image
Name of the image
. . .
Now I could send one message containing these informations in text form and follow it up with another message containing the blob.
I would very much prefer though, to be able to build an object:
imageObject = {
xDimension : '50px',
yDimension : '50px',
name : 'PinkFlowers.jpg'
imageData : fs.readFileSync(".resources/images/PinkFlowers.jpg")
}
And send this object as it is via socket.send(imageObject).
So far so good, this actually works, but how do I collect the object and make its fields accessible in the client again?
I have been tampering with it for a while now and I would be grateful for any ideas.
Best regards,
Sticks
Well I did get it to work using base64.
On the server side I am running this piece of code:
var imageObject = newMessageObject('img', 'flower.png');
imageObject.image = new Buffer(fs.readFileSync('./resources/images/flower.png'), 'binary').toString('base64');
imageObject.datatype = 'png';
connection.send(JSON.stringify(imageObject));
The new Buffer() is necessary to ensure a valid utf encoding. If used without, Chrome(dont know about Firefox and others) throws an error, that invalid utf8 encoding was detected and shuts down the execution after JSON.parse(message).
Note: newMessageObject is just an object construction method with two fields, type and name which I use.
On the client side its really straight forward:
websocketConnection.onmessage = function(evt) {
var message = JSON.parse(evt.data);
... // Some app specific stuff
var image = new Image();
image.onload = function() {
canvas.getContext("2d").drawImage(image, 0, 0);
}
image.src = "data:image/" + message.datatype + ";base64," + message.image;
}
This draws the image on the canvas.
I am not convinced, that this is practicable for audio or video files, but for images it does the job.
I will probably fall back to simply sending an obfuscated URL instead of audio/video data and read the files directly from the server. I dont like the security implications though.

Categories