How to use HTML5 and transferControlToProxy? - javascript

I work on an JS application which use canvas to manipulate a picture (i.e and convert to png/base64 with .toBlob() and .toDataURL().
I would use .transferControlToProxy() to let a worker do the job and get a smooth GUI.
But it seems to be unsupported... as they said on Mozilla devs
Some of you have other information ?
Maybe a workaround ?

Whatwg.org has a javascript sample of using canvas.transferControlToProxy() at https://developers.whatwg.org/the-canvas-element.html#dom-canvas-transfercontroltoproxy, but it does not seem to work in any browser, even not in bleeding edge versions (Chrome Canary or Opera Next).
Even turning "Enable experimental canvas features" at chrome://flags has no effect in Chrome Canary.
Test live: http://jsbin.com/bocoti/5/edit?html,output
It says: "TypeError: canvas.transferControlToProxy is not a function".
This would be a very fine addition. Think of drawing everything on canvas in a worker and make a blob/arraybuffer/dataurl of canvas and transfer this to main thread using Transferable objects. Nowadays if you want to draw something on canvas using canvas functions (fill(), drawImage() etc.), you have to do it in main thread...
<!DOCTYPE html><html><head><meta charset="utf-8" /></head><body>
<div id="log"></div>
<canvas style="border:1px solid red"></canvas>
<script id="worker1" type="javascript/worker">
self.onmessage = function(e) {
var context = new CanvasRenderingContext2D();
e.data.setContext(context); // event.data is the CanvasProxy object
setInterval(function () {
context.clearRect(0, 0, context.width, context.height);
context.fillText(new Date(), 0, 100);
context.commit();
}, 1000);
}
</script>
<script>
var blob = new Blob([document.querySelector('#worker1').textContent]);
var worker = new Worker(window.URL.createObjectURL(blob));
worker.onmessage = function(e) {
//document.querySelector("#log").innerHTML = "Received: " + e.data;
}
var canvas = document.getElementsByTagName('canvas')[0];
try { var proxy = canvas.transferControlToProxy();
worker.postMessage(proxy, [proxy]);}
catch(e) { document.querySelector("#log").innerHTML = e; }
</script>
<br>
From: https://developers.whatwg.org/the-canvas-element.html#the-canvas-element
</body></html>

Related

OpenCV.JS Uncaught ReferenceError: cv is not defined

I am working on a computer vision project using OpneCV.JS and springboot using Thymeleaf as my HTML5 template engine. I have been following the OpenCJ.Js tutorial here. I am suppose to get two output, one that will display the VideoInput and the other one the canvas that will display the Canvaoutput where the face tracking will take place.
However, the Video display and its working as expected. However, the Canvas display did not show. When I inspect my code in the chrome browser, I realize that I am getting an Uncaught Reference error which says CV is not defined.
Can somebody assist to tell me if their is anything I am doing wrong in my Code.
Below is my code
<!DOCTYPE html>
<html xmlns:th="http://www.thymeleaf.org" xmlns:layout="http://www.ultraq.net.nz/web/thymeleaf/layout" layout:decorate="layout">
<head>
<script async type="text/javascript" th:src="#{/opencv/opencv.js}"></script>
<title>Self-Service Portal</title>
</head>
<body>
<h2>Trying OpenCV Javascript Computer Vision</h2>
<p id="status">Loading with OpenCV.js...</p>
<video id="video" autoplay="true" play width="300" height="225"></video> <br/>
<canvas id="canvasOutput" autoplay="true" width="300" height="225"></canvas>
<!--div>
<div class="inputoutput">
<img id="imageSrc" alt="No Image" />
<div class="caption">ImageScr<input type="file" id="fileInput" name="file" /></div>
</div>
<div class="inputoutput">
<canvas id="canvasOutput" ></canvas>
<div class="caption">canvasOutput</div>
</div>
</div-->
<script type="text/javascript">
navigator.mediaDevices.getUserMedia({
video: true,
audio: false
})
.then(function(stream) {
video.srcObject = stream;
video.play();
})
.catch(function(err) {
console.log("An error occured While accessing media! " + err);
});
let video = document.getElementById('video');
let src = new cv.Mat(video.height, video.width, cv.CV_8UC4);
let dst = new cv.Mat(video.height, video.width, cv.CV_8UC4);
let gray = new cv.Mat();
let cap = new cv.VideoCapture(video);
let faces = new cv.RectVector();
let classifier = new cv.CascadeClassifier();
//load pre-trained classifiers
classifier.load('haarcascade_frontalface_default.xml');
const FPS = 30;
function processVideo() {
try {
if (!streaming) {
// clean and stop.
src.delete();
dst.delete();
gray.delete();
faces.delete();
classifier.delete();
return;
}
let begin = Date.now();
// start processing.
cap.read(src);
src.copyTo(dst);
cv.cvtColor(dst, gray, cv.COLOR_RGBA2GRAY, 0);
// detect faces.
classifier.detectMultiScale(gray, faces, 1.1, 3, 0);
// draw faces.
for (let i = 0; i < faces.size(); ++i) {
let face = faces.get(i);
let point1 = new cv.Point(face.x, face.y);
let point2 = new cv.Point(face.x + face.width, face.y + face.height);
cv.rectangle(dst, point1, point2, [255, 0, 0, 255]);
}
cv.imshow('canvasOutput', dst);
// schedule the next one.
let delay = 1000 / FPS - (Date.now() - begin);
setTimeout(processVideo, delay);
}
catch (err) {
utils.printError(err);
}
};
//schedule the first one.
setTimeout(processVideo, 0);
</script>
<!--script async src="/opencv/opencv.js" onload="onOpenCvReady;" type="text/javascript"></script-->
</body>
</html>
The cv is not defined error can be fixed by assigning cv to the window cv object like, let cv = window.cv
Turning off async would not be ideal because the OpenCV JS library is large and it would affect the time to initial load. Maybe assign a state variable that changes when it finishes loading and run a check on this variable and update the UI accordingly
the code you find in the examples on the opencv page usually focuses on the functionality of the library and sometimes they do not have all the necessary functions to work. The examples on the opencv page usually use a utils.js file that contains the additional functions to work and this is not so obvious.
Most likely your problem will be solved with the answer of this forum.
Additionally I have created a codesandbox that contains all the necessary functions for the video examples to work. Normally they will work for you if you replace the code under the comment
/ ** Your example code here * /
by your facial recognition code.
The async on
<script async type="text/javascript" th:src="#{/opencv/opencv.js}"></script>
...means it won't hold up parsing and processing of the following markup while waiting for the file to load; details. So your inline script later in the file can run before that file is loaded and its JavaScript is run.
Remove async and move that script tag to the bottom of the document, just before your inline script that uses what it defines.

Turning HTML content in to a canvas element [duplicate]

It would be incredibly useful to be able to temporarily convert a regular element into a canvas. For example, say I have a styled div that I want to flip. I want to dynamically create a canvas, "render" the HTMLElement into the canvas, hide the original element and animate the canvas.
Can it be done?
There is a library that try to do what you say.
See this examples and get the code
http://hertzen.com/experiments/jsfeedback/
http://html2canvas.hertzen.com/
Reads the DOM, from the html and render it to a canvas, fail on some, but in general works.
Take a look at this tutorial on MDN: https://developer.mozilla.org/en/HTML/Canvas/Drawing_DOM_objects_into_a_canvas (archived)
Its key trick was:
var canvas = document.getElementById('canvas');
var ctx = canvas.getContext('2d');
var data = '<svg xmlns="http://www.w3.org/2000/svg" width="200" height="200">' +
'<foreignObject width="100%" height="100%">' +
'<div xmlns="http://www.w3.org/1999/xhtml" style="font-size:40px">' +
'<em>I</em> like ' +
'<span style="color:white; text-shadow:0 0 2px blue;">' +
'cheese</span>' +
'</div>' +
'</foreignObject>' +
'</svg>';
var DOMURL = window.URL || window.webkitURL || window;
var img = new Image();
var svg = new Blob([data], {type: 'image/svg+xml;charset=utf-8'});
var url = DOMURL.createObjectURL(svg);
img.onload = function () {
ctx.drawImage(img, 0, 0);
DOMURL.revokeObjectURL(url);
}
img.src = url;
That is, it used a temporary SVG image to include the HTML content as a "foreign element", then renders said SVG image into a canvas element. There are significant restrictions on what you can include in an SVG image in this way, however. (See the "Security" section for details — basically it's a lot more limited than an iframe or AJAX due to privacy and cross-domain concerns.)
Sorry, the browser won't render HTML into a canvas.
It would be a potential security risk if you could, as HTML can include content (in particular images and iframes) from third-party sites. If canvas could turn HTML content into an image and then you read the image data, you could potentially extract privileged content from other sites.
To get a canvas from HTML, you'd have to basically write your own HTML renderer from scratch using drawImage and fillText, which is a potentially huge task. There's one such attempt here but it's a bit dodgy and a long way from complete. (It even attempts to parse the HTML/CSS from scratch, which I think is crazy! It'd be easier to start from a real DOM node with styles applied, and read the styling using getComputedStyle and relative positions of parts of it using offsetTop et al.)
You can use dom-to-image library (I'm the maintainer).
Here's how you could approach your problem:
var parent = document.getElementById('my-node-parent');
var node = document.getElementById('my-node');
var canvas = document.createElement('canvas');
canvas.width = node.scrollWidth;
canvas.height = node.scrollHeight;
domtoimage.toPng(node).then(function (pngDataUrl) {
var img = new Image();
img.onload = function () {
var context = canvas.getContext('2d');
context.translate(canvas.width, 0);
context.scale(-1, 1);
context.drawImage(img, 0, 0);
parent.removeChild(node);
parent.appendChild(canvas);
};
img.src = pngDataUrl;
});
And here is jsfiddle
Building on top of the Mozdev post that natevw references I've started a small project to render HTML to canvas in Firefox, Chrome & Safari. So for example you can simply do:
rasterizeHTML.drawHTML('<span class="color: green">This is HTML</span>'
+ '<img src="local_img.png"/>', canvas);
Source code and a more extensive example is here.
No such thing, sorry.
Though the spec states:
A future version of the 2D context API may provide a way to render fragments of documents, rendered using CSS, straight to the canvas.
Which may be as close as you'll get.
A lot of people want a ctx.drawArbitraryHTML/Element kind of deal but there's nothing built in like that.
The only exception is Mozilla's exclusive drawWindow, which draws a snapshot of the contents of a DOM window into the canvas. This feature is only available for code running with Chrome ("local only") privileges. It is not allowed in normal HTML pages. So you can use it for writing FireFox extensions like this one does but that's it.
You could spare yourself the transformations, you could use CSS3 Transitions to flip <div>'s and <ol>'s and any HTML tag you want. Here are some demos with source code explain to see and learn: http://www.webdesignerwall.com/trends/47-amazing-css3-animation-demos/
the next code can be used in 2 modes, mode 1 save the html code to a image, mode 2 save the html code to a canvas.
this code work with the library: https://github.com/tsayen/dom-to-image
*the "id_div" is the id of the element html that you want to transform.
**the "canvas_out" is the id of the div that will contain the canvas
so try this code.
:
function Guardardiv(id_div){
var mode = 2 // default 1 (save to image), mode 2 = save to canvas
console.log("Process start");
var node = document.getElementById(id_div);
// get the div that will contain the canvas
var canvas_out = document.getElementById('canvas_out');
var canvas = document.createElement('canvas');
canvas.width = node.scrollWidth;
canvas.height = node.scrollHeight;
domtoimage.toPng(node).then(function (pngDataUrl) {
var img = new Image();
img.onload = function () {
var context = canvas.getContext('2d');
context.drawImage(img, 0, 0);
};
if (mode == 1){ // save to image
downloadURI(pngDataUrl, "salida.png");
}else if (mode == 2){ // save to canvas
img.src = pngDataUrl;
canvas_out.appendChild(img);
}
console.log("Process finish");
});
}
so, if you want to save to image just add this function:
function downloadURI(uri, name) {
var link = document.createElement("a");
link.download = name;
link.href = uri;
document.body.appendChild(link);
link.click();
}
Example of use:
<html>
<head>
</script src="/dom-to-image.js"></script>
</head>
<body>
<div id="container">
All content that want to transform
</div>
<button onclick="Guardardiv('container');">Convert<button>
<!-- if use mode 2 -->
<div id="canvas_out"></div>
</html>
Comment if that work.
Comenten si les sirvio :)
The easiest solution to animate the DOM elements is using CSS transitions/animations but I think you already know that and you try to use canvas to do stuff CSS doesn't let you to do. What about CSS custom filters? you can transform your elements in any imaginable way if you know how to write shaders. Some other link and don't forget to check the CSS filter lab.
Note: As you can probably imagine browser support is bad.
function convert() {
dom = document.getElementById('divname');
var script,
$this = this,
options = this.options,
runH2c = function(){
try {
var canvas = window.html2canvas([ document.getElementById('divname') ], {
onrendered: function( canvas ) {
window.open(canvas.toDataURL());
}
});
} catch( e ) {
$this.h2cDone = true;
log("Error in html2canvas: " + e.message);
}
};
if ( window.html2canvas === undefined && script === undefined ) {
} else {.
// html2canvas already loaded, just run it then
runH2c();
}
}

Javascript function from a JS file doesn't get called from the canvas

I'm trying to learn about JavaScript and the html5 canvas, however, it's proving a little confusing and I don't understand why it doesn't seem to work...
I am working on creating a simple map that has some of the capabilities of google maps(drag and drop/zoom in/out/etc). In order to do this, I chose html5 canvas and easeljs for the drag and drop functions.
I have a javascript file (path.js) which contains 2 functions:
pathConstructor() - example function from the easeljs tutorial
drawMap() - copy of the first function slightly modified (and probably wrong right now)
Everything worked fine when I called pathConstructor() from the canvas, however, after I replaced it with drawMap(), everything stopped working. It won't even work if I replace drawMap() with pathContrcutor() right now.
I put some alerts before and after calling the function from the canvas and inside pathConstructor(). The before alert goes off but the others don't so for some reason the function never gets called...
If I use the pathConstructor code as inline code in the canvas then it works just fine, however, I would like to avoid that since I believe it's bad programming. I want it to be neat and each script to have its own file.
Anyone know why this is happening?
HTML
<!Doctype html>
<html>
<head>
<script src="http://code.createjs.com/easeljs-0.7.0.min.js"></script>
<script src="path.js" type="text/javascript"></script>
</head>
<body>
<canvas id="canvas" width="1300px" height="800px"style="border:1px dotted black;">
<script>pathConstructor();</script>
</canvas>
</body>
</html>
Javascript
var stage;
function pathConstructor() {
alert('inside pathConstructor');
stage = new createjs.Stage('canvas');
// this lets our drag continue to track the mouse even when it leaves the canvas:
// play with commenting this out to see the difference.
stage.mouseMoveOutside = true;
var circle = new createjs.Shape();
circle.graphics.beginFill("red").drawCircle(0, 0, 50);
var label = new createjs.Text("drag me", "bold 14px Arial", "#FFFFFF");
label.textAlign = "center";
label.y = -7;
var dragger = new createjs.Container();
dragger.x = dragger.y = 100;
dragger.addChild(circle, label);
stage.addChild(dragger);
dragger.on("pressmove", function(evt) {
// currentTarget will be the container that the event listener was added to:
evt.currentTarget.x = evt.stageX;
evt.currentTarget.y = evt.stageY;
// make sure to redraw the stage to show the change:
stage.update();
});
stage.update();
}
function drawMap() {
stage = new createjs.Stage('canvas');
var bitMap = new createjs.Bitmap('middle-earth-map.jpg');
stage.mouseMoveOutside = true;
var dragger = new createjs.Container();
dragger.x = dragger.y = 0;
dragger.addChild(bitMap);
stage.addChild(dragger);
dragger.on('pressmove', function(evt2)) {
evt2.currentTarget.x = evt2.stageX;
evt2.currentTarget.y = evt2.stageY;
stage.update();
});
stage.update();
}
For me it's working fine, you just have to remove that extra ")" in dragger.on('pressmove', function(evt2)) {;
dragger.on('pressmove', function(evt2)) {
evt2.currentTarget.x = evt2.stageX;
evt2.currentTarget.y = evt2.stageY;
stage.update();
});

how to save html5 canvas to server

Im loading a few images to my canvas and then after they load I want to click a button that saves that canvas image to my server. I can see the script works fine until it gets to the 'toDataURL' part and my function stops executing. What am I doing wrong? Here is my code:
<!DOCTYPE HTML>
<html>
<head>
<style>
body {
margin: 0px;
padding: 0px;
}
</style>
</head>
<body>
<canvas id="myCanvas" width="578"
height="200"></canvas>
<div>
<button onClick="saveCards();">Save</button>
</div>
<script>
function loadImages(sources, callback)
{
var images = {};
var loadedImages = 0;
var numImages = 0;
// get num of sources
for(var src in sources) {
numImages++;
}
for(var src in sources) {
images[src] = new Image();
images[src].onload = function() {
if(++loadedImages >= numImages)
{
callback(images);
}
};
images[src].src = sources[src];
}
}
var canvas =
document.getElementById('myCanvas');
var context = canvas.getContext('2d');
var sources = {
great:
'images/great.jpg',
star:
'images/1Star.jpg', good:
'images/good.jpg'
};
loadImages(sources, function(images) {
context.drawImage(images.great,
0, 0, 80, 120);
context.drawImage(images.star, 80,
0, 80, 120);
context.drawImage(images.good, 160, 0, 80,
120);
});
</script>
<script type="text/javascript">
function saveCards()
{
var canvas=
document.getElementById("myCanvas");
alert("stops");
var theString= canvas.toDataURL();
var postData= "CanvasData="+theString;
var ajax= new XMLHttpRequest();
ajax.open("POST", 'saveCards.php', true);
ajax.setRequestHeader('Content-Type',
'canvas/upload');
ajax.onreadystatechange=function()
{
if(ajax.readyState == 4)
{
alert("image was saved");
}else{
alert("image was not saved");
}
}
ajax.send(postData);
}
</script>
</body>
</html>
Thank you for any help is it because the images are not loaded before toDataUrl is called? If so can you please help me fix it.
This is the php script:
<?php
if(isset($GLOBALS['HTTP_RAW_POST_DATA']));
{
$rawImage=$GLOBALS['HTTP_RAW_POST_DATA'];
$removeHeaders=
substr($rawImage,strpos($rawImage, ",")+1);
$decode=base64_decode($removeHeaders);
$fopen= fopen('images/image.png', 'wb');
fwrite($fopen, $decode);
fclose($fopen);
}
?>
I am getting a security error though.
In the specification for the canvas element it states:
Information leakage can occur if scripts from one origin can access
information (e.g. read pixels) from images from another origin (one
that isn't the same).
To mitigate this, bitmaps used with canvas elements are defined to
have a flag indicating whether they are origin-clean. All bitmaps
start with their origin-clean set to true. The flag is set to false
when cross-origin images or fonts are used.
The toDataURL(), toDataURLHD(), toBlob(), getImageData(), and
getImageDataHD() methods check the flag and will throw a SecurityError
exception rather than leak cross-origin data.
The flag can be reset in certain situations; for example, when a
CanvasRenderingContext2D is bound to a new canvas, the bitmap is
cleared and its flag reset.
Since you are loading images from a different server into a canvas element, the work-around to be able to use toDataURL() is to "copy" the canvas into a new canvas element to reset the origin-clean flag to "true".
You can see an example of this here
I figured what I did wrong, but not really sure why it works now. In my actual code that I am using instead of images/image.png I was using the full url https://www.mywebsite.com/images/image.png For some reason when I just write the shortened images/image.png . It works fine. Thank you for all the help debugging and for your alternative solutions.

How to make webkit free canvas context memory when canvas is destroyed

The code below seems to leak memory at a rather alarming rate on webkit (mobile safari & konqueror). I realize the test case could be rewritten to reuse the canvas instead of creating a new one, but I'm wondering why the below doesn't also work. Any insight would be appreciated.
<html>
<head>
<script>
function draw() {
var holder = document.getElementById("holder");
holder.innerHTML = "<canvas id=cnv height=250 width=250>";
var ctx = document.getElementById("cnv").getContext("2d");
ctx.beginPath();
ctx.moveTo(50,50);
ctx.lineTo(Math.random()*100,Math.random()*100);
ctx.stroke();
}
function start() {
setInterval(draw, 100);
}
</script>
</head>
<body onload="start()">
<div id="holder"></div>
</body>
</html>
This issue happens on Webkit even when an image SRC is modified, so I don't get surprised if this happens when handling canvas too.
There is a bug filled on Chrome (that made them filling a bug to Webkit) and we hope this will be fixed soon, as it is making many Chrome extension unusable.
references
http://code.google.com/p/chromium/issues/detail?id=36142
https://bugs.webkit.org/show_bug.cgi?id=23372
Anyway, the suggestions above should mitigate this.
Don't:
Create the <canvas> element with .innerHTML
Create the <canvas> in each Interval
Do:
use var cv = document.createElement('canvas'); cv.setAttribute('height', '250'); // ...
cache the reference of cv at some init point and re-use that!!
<script>
var holder = document.getElementById("holder"),
var cv = document.createElement('canvas');
cv.setAttribute('id', 'cnv');
cv.setAttribute('height', '250');
cv.setAttribute('width', '250');
holder.appendChild(cv);
function draw() {
var ctx = cv.getContext("2d");
ctx.beginPath();
ctx.moveTo(50,50);
ctx.lineTo(Math.random()*100,Math.random()*100);
ctx.stroke();
}
function start() {
setInterval(draw, 100);
}
</script>

Categories