pass canvas content from client to admin using web socket - javascript

The application aim to transfer drawings from client canvas to the admin canvas using web sockets the most smoothly possible. At every mousemove event, the client canvas is compressed to png and sent to server through websocket to finally arrive in the admin window where the image is drawn into the admin canvas.
Actually, the code lag a little in the admin window. It seems the bottleneck is ctx.drawImage() but I am not sure.
I wonder if there is a way to first find the bottleneck and then a way to optimise the transfer with debounce or web workers or other.
Client side:
<canvas id="clientCanvas" style="position: absolute; top: 0px; left: 0px;" width="1563" height="528">
function onMouseMove(e) {
var canvas = document.getElementById('clientCanvas');
var ctx = canvas.getContext("2d");
var imageData = canvas.toDataURL("image/png");
socket.emit('SS_onMouseMove', {imageData: imageData});
};
admin side:
<canvas id="adminCanvas" style="position: absolute; top: 0px; left: 0px;" width="1563" height="528">
var canvas = document.getElementById("adminCanvas");
var ctx = canvas.getContext("2d");
var image = new Image();
image.onload = function() {
ctx.clearRect(0, 0, canvas.width, canvas.height);
ctx.drawImage(image, 0, 0);
};
socket.on("SS_onMouseMove", function(response) { onClientMouseMove(response); });
function onClientMouseMove(response) {
image.src = response.imageData;
}

The problem is you are passing the entire image multiple times a second which is both a heavy load on the client generating the image and the connection which is passing around 2 mb (image as dataUrl) 30 to 40 times a second.
The solution would be to:
When the connection is made send the original image once
When mouse events occur on the client, send only those mouse events to the admin and use the same functionality used on the client to render the changes from the mouse event on the admin.
This will decrease the load on the connection dramatically and will require less processing on the client since it wont have to generate a png -> dataUrl multiple times a second

Related

First stream freezes after seccond connection opens?

I'm constructing an application that allows multiple devices to stream their webcam, through the browser, to another device. Each streaming device takes a DataURL image from the webcam and sends it over WebSocket to a server, which will then distribute the array of images to viewers. The viewer will parse the message and display each image on its own individual <canvas>.
It works fine when one user is streaming, but when a second device begins to stream, the first one freezes. It appears the last image in the array is the only one to actually be rendered.
I've been reviewing this code for several hours and can't understand what I'm missing here.
var socket = new WebSocket("wss://my_server:8443");
socket.onmessage = function(msg) {
//parse the incoming message into a usable array
msg = JSON.parse(msg.data);
var div = document.getElementById("streams");
//for every incoming stream
for (var i in msg) {
//create a new Image, then render it onto a canvas element.
var img = new Image();
img.onload = function() {
//if there isn't a canvas element to render it onto already, create one (for when a stream is created)
if (document.getElementById(i) == null) {
var c = document.createElement("canvas");
c.width = img.width;
c.height = img.height;
c.setAttribute("id", i);
div.appendChild(c);
}
//draw the image onto the canvas
var canvas = document.getElementById(i);
var ctx = canvas.getContext("2d");
ctx.clearRect(0, 0, canvas.width, canvas.height);
ctx.drawImage(img, 0, 0);
}
//set the Data URL as the image's SRC
img.src = msg[i];
}
}
I originally just added <img> elements to the <div>, but it ended up taking too long to load, creating short flashes in between each frame.
What's wrong with my code? It seems to freeze all active streams except for the one most recently started (last in the array).

getDataImage to a node server

I'm trying to run an html5 canvas script on a node server. The ideas is to have a client page show a canvas with some simple animation/graphics and then send this image data (info about each pixel) to the server.
function draw() {
var canvas = document.getElementById("canvas");
var ctx = canvas.getContext("2d");
var socket = io.connect();
canvas.addEventListener('mousemove', function(evt)
{
ctx.beginPath();
ctx.moveTo(0, 0);
ctx.lineTo(evt.clientX, evt.clientY);
ctx.stroke();
pixels = ctx.getImageData(0, 0, 400, 400);
socket.emit('message', pixels);
});
}
However, socket.emit causes the script (the animation in the canvas) to run too slow. The script is supposed to display a line from the origin (top left) to current mouse position. The lines are appearing but irregularly - a bunch of lines every 2 or 3 seconds. If I comment out the socket.emit, the sketch runs smoothly. This means that ctx.getImageData is not the one causing the delay.
Is there a way to send this information to the server without slowing down the script?
The pixel information sent to the server will later be used by a second client to populate its canvas. I'm trying to build something like a set of synchronized sketch - draw in one canvas and see the results in many. I'm really new to developing for the web so I'm not sure if this can even be done in real time.

How to auto update canvas screen via ajax every X seconds?

I have a task to create social network
I have a home page that displays all drawings created by users via canvas. Users can draw on their own drawing and can add contributors which have righ to draw on their drawing also. The thing is they can both draw in the same time somehow.
The thing i want to do is, that when one user draws the canvas image, it is updating for other user who is watching that same drawing without refreshing the page.
I want to note that program works well, the only problem is that you have to refresh the page to see changes of drawing
Here is my client side code where i add previously drawed picture to empty canvas before new drawing
canvas = document.getElementById('can');
contain = document.getElementById('contain');
ctx = canvas.getContext("2d");
var imageObj = new Image();
imageObj.src = ''+drugi+'/'+prvi+'.png';
imageObj.onload = function() {
ctx.drawImage(this, 0, 0);
};
Here is the request to ajax to save drawing after every start of the draw() function (which is everytime you move your mouse while drawing on canvas)
ctx.beginPath();
ctx.moveTo(prevX, prevY);
ctx.lineTo(currX, currY);
ctx.strokeStyle = x;
ctx.lineWidth = y;
ctx.stroke();
ctx.closePath();
var dataURL = canvas.toDataURL("image/png");
var ajax = new XMLHttpRequest();
ajax.open("POST",'saveimg.php',false);
ajax.setRequestHeader('Content-Type', 'application/upload');
ajax.send(dataURL );
This is server side code
<?php
session_start();
$crtez = $_SESSION['tmpdrawing'];
$kategorija = $_SESSION['tmpkat'];
if (isset($GLOBALS["HTTP_RAW_POST_DATA"]))
{
$imageData=$GLOBALS['HTTP_RAW_POST_DATA'];
$filteredData=substr($imageData, strpos($imageData, ",")+1);
$unencodedData=base64_decode($filteredData);
$fp = fopen($kategorija . '/' . $crtez . '.png', 'wb' );
fwrite( $fp, $unencodedData);
fclose( $fp );
?>
Now when we have a code, actual thing i want is the first code, which changes canvas appearance to be launched for example in every 3 seconds, so the other users currently on that drawing can see the picture updating without refreshing page.
If you need more code in client side i will post.
I want to say again that the program works perfectyl, the only thing i need to finish it is auto updating canvas via ajax
First you should wrap javascript code into function and than just call that function when something is changed on the server side. As I know, there are two concepts for communitaion between clients and server:
Server Pooling - using javascript, client sends a HTTP request every X seconds and do something when server send some updates, for example update canvas. This is an "old school" and performance-poor pattern, maybe you should avoid it. If you like The Simpsons, this is it: Are we there yet Take a look: Server Pooling Example
Web Socket Communication - full duplex communication between server and clients. When something happened on the server side, clients are informed almost immediately. I had a lot of problems to implement web socket communication with PHP and Apache Server last year when I worked with that kind of project and don't know if anything is better right now. But, after reading some books for realtime applications development, I made it with Pusher, that's an commercial service but also have free plans.

How to take a screenshot of video that is hosted on Amazon S3?

I am implementing a video tagging system. We use Zencoder to convert the video uploaded by the user and VideoJS to play the video. The user should be able to tag and adding a comment while he is watching the video . Everything works except, I would like to take a small snapshot (54px height) of the video at the time that the user is tagging. I have this Javascript:
function getSnapshot(htmlPlayerId){
var video = document.getElementById(htmlPlayerId);
var canvas = document.createElement('canvas');
var ratio = 54 / video.videoHeight;
canvas.width = ratio * video.videoWidth;
canvas.height = 54;
var ctx = canvas.getContext('2d');
ctx.drawImage(video, 0, 0, video.videoWidth, video.videoHeight, 0, 0, canvas.width, canvas.height);
return canvas.toDataURL('image/jpeg');
}
Unfortunately I receive a Security Error because the file is hosted on Amazon S3. I know that it is a Access-Control-Allow-Origin problem.
I already saw these Why does canvas.toDataURL() throw a security exception? and https://forums.aws.amazon.com/thread.jspa?messageID=329118 and associated threads.
I wonder if there is a work around to be able to take the snapshot.
I've run into the same problem recently, my solution was to host a small html file in S3 with the video and some javascript, and then to load that page in a iframe. When you want to take a screenshoot, you cant use postMessage to ask the iframe to take the photo for you, then take the screenshoot in the iframe and pass the data:// URL back to the parent page using postMessage again.
also try window.open(canvas.toDataURL("image/png"));

SecurityException 1000, even though using same domain

I'm facing a troublesome Javascript/Firefox problem.
Relevant code is listed below.
What happens basically is the following:
1. document.ready fires and initiates an AJAX request (to document.domain:8484/getTrack.php or whatever)
2. AJAX response is received. This response contains the url (same domain) of the location of the image. So, sourceImage.onload is set, then sourceImage.src is set
3. sourceImage.onload fires. The idea is now to keep a resized image in memory that perfectly fits the canvas it's going to be drawn on. I want to keep this resized image in memory because I'm going to write (parts of) it to my canvas a lot of times, and resizing every time should be a lot slower.
var SourceImage = new Image();
var preparedImageData;
sourceImage.onload = function() {
var canvas = document.createElement('canvas');
canvas.width = 100; canvas.height = 100;
var ctx = canvas.getContext("2d");
// resize image
ctx.drawImage(sourceImage, 0, 0, sourceImage.width, sourceImage.height, 0, 0, canvas.width, canvas.height);
// save as imagedata
try {
try {
preparedImageData = ctx.getImageData(0, 0, canvas.width, canvas.height);
}
catch (e) {
netscape.security.PrivilegeManager.enablePrivilege("UniversalBrowserRead");
preparedImageData = ctx.getImageData(0, 0, canvas.width, canvas.height);
}
}
catch (e) {
throw new Error("unable to access image data: " + e)
}
}
The first getImageData call throws and the enablePrivilege call also throws inmediately. The errror text is "A script from "http://127.0.0.1" was denied UniversalBrowserRead privileges.". I've checked and it appears these messages should only appear when trying to access getImageData on an image from another domain, which isn't the case though (right?). have no strict security policy in place (everything default), Firefox 4.0. Same code works fine on Chrome.
By 'same origin' ref the Same Origin Policy, the protocol, hostname AND port needs to be identical. I'm guessing you are using different ports here?
What I think happens is that your call to netscape.security.PrivilegeManager.enablePrivilege fails due to the script not being signed - have you tried removing this code?
The context.getImageData and the PrivilegeManager.enablePrivilege calls fail as soon as I set document.domain = document.domain which is done for cooperation with iframes hosted on a different subdomain. As a workaround I proxied domain.tld/subdomain/ to subdomain.domain.tld/ and obtained the desired result.

Categories