We Keep Coding

Pass canvas from client (browser) to server (nodejs)

I have a canvas in my browser that displays a feed from my webcam. What I want to do, is send the canvas data to my nodejs server, manipulate it, and send it back.
I can do it sending the canvas data via socket.io like so:
socket.emit('canvas_data', canvas.toDataURL());
And then rebuilding it on the nodejs server:
let img = new Image();
img.src = data; // this is the canvas_data from the first step
const canvas = createCanvas(640,480);
const ctx = canvas.getContext('2d');
ctx.drawImage(img,0,0,640,480);
However this seems really wasteful as I'm taking an already rendered canvas, converting it to base64, sending it, and then rebuilding it on the other side.
The whole point of this is to use tfjs on the server side:
let converted = tfjs.browser.fromPixels(canvas);
If I just send the canvas from the first step:
socket.emit('canvas_data', canvas);
And then run tfjs:
let converted = tfjs.browser.fromPixels(data);
I get the following error:
Error: pixels passed to tf.browser.fromPixels() must be either an HTMLVideoElement, HTMLImageElement, HTMLCanvasElement, ImageData in browser, or OffscreenCanvas, ImageData in webworker or {data: Uint32Array, width: number, height: number}, but was object
Is there a more efficient way to accomplish this?

using toDataURL is always going to be slow as browser needs to
encode all data before sending it.
your second example is better, just on the node side you need to create tensor from Buffer that you receive on socket (that would be fastest way), no need to use higher-level functions such as fromPixels
take a look at https://github.com/vladmandic/anime/blob/main/sockets/anime.ts for client-side code and https://github.com/vladmandic/anime/blob/main/sockets/server.ts for server-side code
note you also may need to account for channel-depth (does your model work with rgba or rgb) and/or model specific any pre-processing normalization, that's handled in https://github.com/vladmandic/anime/blob/main/sockets/inference.ts

Saving file with JavaScript File API results wrong encoding

I have a problem (or may be two) with saving files using HTML5 File API.
A files comes from the server as a byte array and I need to save it. I tried several ways described on SO:
creating blob and opening it in a new tab
creating a hidden anchor tag with "data:" in href attribute
using FileSaver.js
All approaches allow to save the file but with breaking it by changing the encoding to UTF-8, while the file (in current test case) is in ANSI. And it seems that I have to problems: at the server side and at the client side.
Server side:
Server side is ASP.NET Web API 2 app, which controller sends the file using HttpResponseMessage with StreamContent. The ContentType is correct and corresponds with actual file type.
But as can be seen on the screenshot below server's answer (data.length) is less then actual file size calculated at upload (file.size). Also here could be seen that HTML5 File object has yet another size (f.size).
If I add CharSet with value "ANSI" to server's response message's ContentType property, file data will be the same as it was uploaded, but on saving result file still has wrong size and become broken:
Client side:
I tried to set charset using the JS File options, but it didn't help. As could be found here and here Eli Grey, the author of FileUplaod.js says that
The encoding/charset in the type is just metadata for the browser, not an encoding directive.
which means, if I understood it right, that it is impossible to change the encoding of the file.
Issue result: at the end I can successfully download broken files which are unable to open.
So I have two questions:
How can I save file "as is" using File API. At present time I cannot use simple way with direct link and 'download' attribute because of serverside check for access_token in request header. May be this is the "bottle neck" of the problem?
How can I avoid setting CharSet at server side and also send byte array "as is"? While this problem could be hacked in some way I guess it's more critical. For example, while "ANSI" charset solves the problem with the current file, WinMerge shows that it's encoding is Cyrillic 'Windows-1251' and also can any other.
P.S. the issue is related to all file types (extensions) except *.txt.
Update
Server side code:
public HttpResponseMessage DownloadAttachment(Guid fileId)
{
var stream = GetFileStream(fileId);
var message = new HttpResponseMessage(HttpStatusCode.OK);
message.Content = new StreamContent(stream);
message.Content.Headers.ContentLength = file.Size;
message.Content.Headers.ContentType = new MediaTypeHeaderValue(file.ContentType)
{
// without this charset files sent with bigger size
// than they are as shown on image 1
CharSet = "ANSI"
};
message.Content.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment")
{
FileName = file.FileName + file.Extension,
Size = file.Size
};
return message;
}
Client side code (TypeScript):
/*
* Handler for click event on download <a> tag
*/
private downloadFile(file: Models.File) {
var self = this;
this.$service.downloadAttachment(this.entityId, file.fileId).then(
// on success
function (data, status, headers, config) {
var fileName = file.fileName + file.extension;
var clientFile = new File([data], fileName);
// here's the issue ---^
saveAs(clientFile, fileName);
},
// on fail
function (error) {
self.alertError(error);
});
}
My code is almost the same as in answers on related questions on SO: instead of setting direct link in 'a' tag, I handle click on it and download file content via XHR (in my case using Angularjs $http service). Getting the file content I create a Blob object (in my case I use File class that derives from Blob) and then try to save it using FileSaver.js. I also tried approach with encoded URL to Blob in href attribute, but it only opens a new tab with a file broken the same way. I found that the problem is in Blob class - calling it's constructor with 'normal' file data I get an instance with 'wrong' size as could be seen on first two screenshots. So, as I understand, my problem not in the way I try to save my file, but in the way I create it - File API

check uploaded file format on client side

I am creating a web portal where end user will upload a csv file and I will do some manipulation on that file on the server side (python). There is some latency and lag on the server side so I dont want to send the message from server to client regarding the bad format of uploaded file. Is there any way to do heavy lifting on client side may be using js or jquery to check if the uploaded file is "comma" separated or not etc etc?
I know we can do "accept=.csv" in the html so that file extension has csv format but how to do with contents to be sure.

Accessing local files from Javascript is only possible by using the File API (https://developer.mozilla.org/en-US/docs/Using_files_from_web_applications) - by using this you might be able to check the content whether it matches your expectations or not.

Here's some bits of code I used to display a preview image clientside when a file is selected. You should be able to use this as a starting point to do something else with the file data. Determining whether its csv is up to you.
Obvious caveat:
You still have to check server side. Anyone can modify your clientside javascript to pretend a bad file is good.
Another caveat:
I'm pretty sure that you can have escaped comma characters in a valid csv file. I think the escape character might be different across some implementations too...
// Fired when the user chooses a file in the OS dialog box
// They will have clicked <input id="fileId" type="file">
document.getElementById('fileId').onchange = function (evt) {
if(!evt.target.files || evt.target.files.length === 0){
console.log('No files selected');
return;
}
var uploadTitle = evt2.target.files[0].name;
var uploadSize = evt2.target.files[0].size;
var uploadType = evt2.target.files[0].type;
// To manipulate the file you set a callback for the whole contents:
var FR = new FileReader();
// I've only used this readAsDataURL which will encode the file like data:image/gif;base64,R0lGODl...
// I'm sure there's a similar call for plaintext
FR.readAsDataURL($('#file')[0].files[0]);
FR.onload = function(evt2){
var evtData = {
filesEvent: evt,
}
var uploadData = evt2.result
console.log(uploadTitle, uploadSize, uploadType, uploadData);
}
}

HTML syntax unknown to display a javascript binary string containing a jpeg image

I can't get through this problem though it must be only a very small syntax problem, as you will see: In fact, I'm searching for just a little piece of syntax, unless what I intend to do would be impossible (But I can see no reason why it should be impossible...).
I have written a function to encode an image into Base64 on server side, because I want to store numerous images into an array:
So, with Base64 I can download images as ordinary strings that I can organize in an array, then put them into an object just when I have chosen the right image and the right moment, without having to refer to the server again, so that the user doesn't have to wait.
Then I do something like this:
First phase:
function download64(imageUrl) //->string
{ // ask the server to send the 'imageUrl' as a base64 string
var tx = DoTheJob(); // ...connect through ajax and download the image converted in base64 as a string in var 'tx'
return tx
}
At this stage, I'm holding the image in the 'tx' Base64-string on client side.
Somewhat later I want to display my image in the div called "cadre", so I do the following:
Second phase:
I just have to call the "display64" function to set my image into the "cadre" div-object on the screen:
display64("cadre",tx);
using this function:
function display64(destinationDiv,imgText64) //->void
{ // display 'imgText64' into destinationDiv
var oImg = "<img alt='' src='data:image/jpg;base64," + imgText64 + "'>";
var x = document.getElementById(destinationDiv);
x.innerHTML = oImg;
}
Now the image is displayed. Unfortunately, this works well only with Firefox, because Internet Explorer 8 can't read Base64 images above 32Kb! And in my entreprise, we use IE 8 only!
Then I dropped my base64 encoder and decided to fetch the image as a binary string, which I could manage though I initially had a problem with nul character.
Now, I'm here with my binary string containing the exact copy of the source JPG file (including zeros that I have encoded on server side then restored on client side). So, what I need now is the simple function 'displayBin', but I can't find the syntax on the web:
function displayBin(destinationDiv,imgTextBin) //->void
{ // display 'imgTextBin' into destinationDiv
var oImg = "<img alt='' src='??????? + imgTextBin + "'>"; // What's the syntax here, please?
var x = document.getElementById(destinationDiv);
x.innerHTML = oImg;
}
Can anyone help ? Thanks a lot.

If I understand your question your are wanting to store images so you do not have to connect back with the server. One way of doing this is to preload the images like var img=new Image; img.src='path'; You could load all of the images into an array and you could then add them to the page as necessary. There is a lot out there on image preloading since that seems what your trying to accomplish.

HTML5 FileApi + FileReader - Feed <object> with SWF

I want to use the HTML5 FileApi to read a SWF to an OBJECT (or EMBED, if it's better to do?).
My current code crashes on Chrome/Iron (the only stable browser which also supports the xmlhttprequest v2 FormData). I got it to read image data into a on-the-fly created IMG. But the object one crashes the current tab in the browser.
else if (file.type == "application/x-shockwave-flash") {
var show = document.createElement("object");
show.type = "application/x-shockwave-flash"
show.style.width = "100%";
show.style.height = "100%";
show.id = "thumb";
document.getElementById("thumbnails").appendChild(show);
var reader = new FileReader();
reader.onload = (function (aImg) {
return function (e) { aImg.data = e.target.result; };
})(show);
reader.readAsDataURL(file);
Do I really read to the object.data part? How is it done right? Anybody know? Or is this incomplete and I have to wait for better implementation?

A few things I'd recommend trying (in order of increasing complexity):
base64 encode the data with btoa and set it using a data: URI,
instead of creating the object using createElement, construct the <object> tag with all attributes as an HTML string (including the base64 advice above), then inject it into a DOM element with innerHTML,
create a reflector web service where you POST the swf content, it gives you a URL, then pass the URL off to the object,
similar to the previous, create a reflector web service where you POST the swf content, targeting a full-screen IFRAM as the target, have the service spits back an HTML doc including an <object> pointing back to the server.
The later of these options is more intense, and requires round-trips from the server that you'd probably want to avoid - just some more options you might want to consider.
ActionScript 3 has a Loader which may be useful as well. I don't know if it supports data: URI's, but if it does, you could write a boot loader SWF which runs the contents of the local swf file directly.