I have a .Net (VB/C#) backend on one server and a javascript/jquery front-end on another. I am attempting to pass file data from the backend to the front-end by reading the file in .Net as a byte array, converting it to a base64 string and passing that to the js front-end along with a variable indicating the file type.
Once the data is in the js front-end I attempt to convert the base64 string to a file blob in order to read it with readAsDataURL() so that it can be displayed in an iframe or downloaded by the client.
For testing I am using a pdf file. The process of passing the data works but the file data is not recognized by the pdf viewer. The iframe loads the viewer but I get a message about the file not being recognized as a pdf file.
I have done a tone of searching and have gotten a lot of questions answered by searching stackoverflow but this eludes me.
Any suggestions would be much appreciated. Thank you.
Update:
Here is my VB.Net code:
Sub OnReadFileToViewer(ByVal filePath As String)
Dim okToView As Boolean = False
Dim fileBLOB As String = String.Empty
Dim fileEncoded As String = String.Empty
If (file.Exists(filePath)) Then
Try
Using fRead As FileStream = New FileStream(filePath, FileMode.Open, FileAccess.Read)
Dim bytes() As Byte = New Byte((CType(fRead.Length, Integer)) - 1) {}
Dim numBytesToRead As Integer = CType(fRead.Length, Integer)
Dim numBytesRead As Integer = 0
While (numBytesToRead > 0)
' Read may return anything from 0 to numBytesToRead.
Dim n As Integer = fRead.Read(bytes, numBytesRead, _
numBytesToRead)
' Break when the end of the file is reached.
If (n = 0) Then
Exit While
End If
numBytesRead = (numBytesRead + n)
numBytesToRead = (numBytesToRead - n)
End While
numBytesToRead = bytes.Length
fileEncoded = System.Convert.ToBase64String(bytes)
End Using
okToView = True
Catch ex As Exception
'
End Try
End If
If (okToView) Then
' Code to send data to js fron-end
End If
End Sub
And my js front-end for converting the base64 string to BLOB and the iframe preview:
function PreviewFile(file) {
var fileBlob = b64toBlob(file,'application/pdf','');
var reader = new FileReader();
var fileType = e.target.getAttribute("type");
alert(fileType);
reader.onload = function (e) {
if (fileType)
$("#fileViewerDiv").append('<iframe id="fileVieweriFrame" class="fileViewer"></iframe>'); // + e.target.result + '">');
$("#fileVieweriFrame").attr('src', file); //e.target.result);
$("#fileVieweriFrame").dialog();
}
reader.readAsDataURL(fileBlob);
}
function b64toBlob(b64Data, contentType, sliceSize) {
contentType = contentType || '';
sliceSize = sliceSize || 512;
var byteCharacters = atob(b64Data);
var byteArrays = [];
for (var offset = 0; offset < byteCharacters.length; offset += sliceSize) {
var slice = byteCharacters.slice(offset, offset + sliceSize);
var byteNumbers = new Array(slice.length);
for (var i = 0; i < slice.length; i++) {
byteNumbers[i] = slice.charCodeAt(i);
}
var byteArray = new Uint8Array(byteNumbers);
byteArrays.push(byteArray);
}
var blob = new Blob(byteArrays, {type: contentType});
return blob;
}
The alert(fileType) does display the correct file type - application/pdf.
I have to admit that I lifted the b64toBlob() function from a blog somewhere but I have forgotten which one and have lost the URL to the site. My apologies to the original author. If anyone recognizes the code and knows the author/site please let me know
Edit
b64toBlob() by Jeremy Banks - Based on this SO answer
Related
Im receiving some file chunks in bytes format from my server and im collecting them into one variable in my frontend to download it later. And I can't change my server conception (receiving a file sliced into chunks).
My problem is that if the file is heavy (from 500MB), my variable length is starting to be very very big and im having an error :
RangeError: Invalid string length
This is because my variable has reach the limit of character (536 800 000).
Here's how im adding my data to my variable :
this.socket.on('new_file', (data: string) => {
this.receivedFile += data;
}
My download part :
public download(): void {
const byteCharacters = this.receivedFile;
const byteArrays = [];
const sliceSize=512
for (let offset = 0; offset < byteCharacters.length; offset +=
sliceSize) {
const slice = byteCharacters.slice(offset, offset + sliceSize);
const byteNumbers = new Array(slice.length);
for (let i = 0; i < slice.length; i++) {
byteNumbers[i] = slice.charCodeAt(i);
}
const byteArray = new Uint8Array(byteNumbers);
byteArrays.push(byteArray);
}
const blob = new Blob(byteArrays, {type: this.fileInfos.type});
saveAs(blob, this.fileInfos.name);
}
Which approach can I do ? Or does it exist some variable type in Javascript to accept more char ? Thanks
Don't collect the chunks into a huge string. Instead, just convert each chunk into a bytearray immediately (that you'll need later anyway) and collect these:
this.socket.on('new_file', (data: string) => {
const bytes = new Uint8Array(data.length);
for (let i = 0; i < data.length; i++) {
bytes[i] = data.charCodeAt(i);
}
this.byteArrays.push(bytes);
}
then
public download(): void {
const blob = new Blob(this.byteArrays, {type: this.fileInfos.type});
saveAs(blob, this.fileInfos.name);
}
I don't think you need to make 512-byte-sized slices.
I am trying to create a file object from a base64 image which was cut from another image. I am able to do it but the resulting file size is almost thrice the actual size. Below is the function that I am using:
convertDataURItoFile(dataURI, fileName) {
// convert base64 to raw binary data held in a string
// doesn't handle URLEncoded DataURIs - see SO answer #6850276 for code that does this
var byteString = atob(dataURI.split(',')[1]);
// separate out the mime component
var mimeString = dataURI.split(',')[0].split(':')[1].split(';')[0]
// write the bytes of the string to an ArrayBuffer
var ab = new ArrayBuffer(byteString.length);
var ia = new Uint8Array(ab);
for (var i = 0; i < byteString.length; i++) {
ia[i] = byteString.charCodeAt(i);
}
// write the ArrayBuffer to a blob, and you're done
var blob: any = new Blob([ia], { type: mimeString });
//A Blob() is almost a File() - it's just missing the two properties below which we will add
blob.lastModifiedDate = new Date();
blob.name = fileName;
//Cast to a File() type
return <File>blob;
}
Any idea on why the file size increasing so drastically? How can I compress it? Thanks in advance.
I am trying to create a file object from a base64 image which was cut
from another image. I am able to do it but the resulting file size is
almost thrice the actual size.
Cannot reproduce resulting .size of Blob being thrice the size of the content of input data URI. Do you mean the data URI .length can be thrice the size of Blob .size?
function convertDataURItoFile(dataURI, fileName) {
// convert base64 to raw binary data held in a string
// doesn't handle URLEncoded DataURIs - see SO answer #6850276 for code that does this
var byteString = atob(dataURI.split(',')[1]);
// separate out the mime component
var mimeString = dataURI.split(',')[0].split(':')[1].split(';')[0]
// write the bytes of the string to an ArrayBuffer
var ab = new ArrayBuffer(byteString.length);
var ia = new Uint8Array(ab);
for (var i = 0; i < byteString.length; i++) {
ia[i] = byteString.charCodeAt(i);
}
// write the ArrayBuffer to a blob, and you're done
var blob = new Blob([ia], { type: mimeString });
//A Blob() is almost a File() - it's just missing the two properties below which we will add
blob.lastModifiedDate = new Date();
blob.name = fileName;
//Cast to a File() type
console.log(`input data size: ${datauriLength} Blob.size: ${blob.size}`);
return blob;
}
const [datauri, filename] = ["data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAASoAAADCCAYAAAD+Wo90AAAABHNCSVQICAgIfAhkiAAAApVJREFUeJzt1DEBACAMwLCBf88ggZMeiYJeXTPnDEDY/h0A8GJUQJ5RAXlGBeQZFZBnVECeUQF5RgXkGRWQZ1RAnlEBeUYF5BkVkGdUQJ5RAXlGBeQZFZBnVECeUQF5RgXkGRWQZ1RAnlEBeUYF5BkVkGdUQJ5RAXlGBeQZFZBnVECeUQF5RgXkGRWQZ1RAnlEBeUYF5BkVkGdUQJ5RAXlGBeQZFZBnVECeUQF5RgXkGRWQZ1RAnlEBeUYF5BkVkGdUQJ5RAXlGBeQZFZBnVECeUQF5RgXkGRWQZ1RAnlEBeUYF5BkVkGdUQJ5RAXlGBeQZFZBnVECeUQF5RgXkGRWQZ1RAnlEBeUYF5BkVkGdUQJ5RAXlGBeQZFZBnVECeUQF5RgXkGRWQZ1RAnlEBeUYF5BkVkGdUQJ5RAXlGBeQZFZBnVECeUQF5RgXkGRWQZ1RAnlEBeUYF5BkVkGdUQJ5RAXlGBeQZFZBnVECeUQF5RgXkGRWQZ1RAnlEBeUYF5BkVkGdUQJ5RAXlGBeQZFZBnVECeUQF5RgXkGRWQZ1RAnlEBeUYF5BkVkGdUQJ5RAXlGBeQZFZBnVECeUQF5RgXkGRWQZ1RAnlEBeUYF5BkVkGdUQJ5RAXlGBeQZFZBnVECeUQF5RgXkGRWQZ1RAnlEBeUYF5BkVkGdUQJ5RAXlGBeQZFZBnVECeUQF5RgXkGRWQZ1RAnlEBeUYF5BkVkGdUQJ5RAXlGBeQZFZBnVECeUQF5RgXkGRWQZ1RAnlEBeUYF5BkVkGdUQJ5RAXlGBeQZFZBnVECeUQF5RgXkGRWQZ1RAnlEBeUYF5BkVkGdUQJ5RAXlGBeQZFZBnVECeUQF5RgXkGRWQZ1RAnlEBeUYF5BkVkHcBRH8DgsmlTc8AAAAASUVORK5CYII=", "filename.png"];
const datauriLength = datauri.length;
const reader = new FileReader;
reader.onload = () => {
console.log(`data URI: ${reader.result}`)
document.querySelector("iframe").src = reader.result;
};
reader.readAsDataURL(convertDataURItoFile(datauri, filename));
<iframe></iframe>
If .name and .lastModifiedDate need to be added to Blob, you can substitute using File constructor for Blob, which expects file name parameter to be set at second parameter to File constructor, and optionally expected .lastModidied and or .lastModifiedDate parameters at third parameter to constructor.
function convertDataURItoFile(dataURI, fileName) {
// convert base64 to raw binary data held in a string
// doesn't handle URLEncoded DataURIs - see SO answer #6850276 for code that does this
var byteString = atob(dataURI.split(',')[1]);
// separate out the mime component
var mimeString = dataURI.split(',')[0].split(':')[1].split(';')[0]
// write the bytes of the string to an ArrayBuffer
var ab = new ArrayBuffer(byteString.length);
var ia = new Uint8Array(ab);
for (var i = 0; i < byteString.length; i++) {
ia[i] = byteString.charCodeAt(i);
}
// write the ArrayBuffer to a blob, and you're done
// use `File` constructor here
var blob = new File([ia], fileName, { type: mimeString, lastModifiedDate: new Date() });
//A Blob() is almost a File() - it's just missing the two properties below which we will add
// blob.lastModifiedDate = new Date();
// blob.name = fileName;
//Cast to a File() type
console.log(`input data size: ${datauriLength} Blob.size: ${blob.size}`);
return blob;
}
const [datauri, filename] = ["data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAASoAAADCCAYAAAD+Wo90AAAABHNCSVQICAgIfAhkiAAAApVJREFUeJzt1DEBACAMwLCBf88ggZMeiYJeXTPnDEDY/h0A8GJUQJ5RAXlGBeQZFZBnVECeUQF5RgXkGRWQZ1RAnlEBeUYF5BkVkGdUQJ5RAXlGBeQZFZBnVECeUQF5RgXkGRWQZ1RAnlEBeUYF5BkVkGdUQJ5RAXlGBeQZFZBnVECeUQF5RgXkGRWQZ1RAnlEBeUYF5BkVkGdUQJ5RAXlGBeQZFZBnVECeUQF5RgXkGRWQZ1RAnlEBeUYF5BkVkGdUQJ5RAXlGBeQZFZBnVECeUQF5RgXkGRWQZ1RAnlEBeUYF5BkVkGdUQJ5RAXlGBeQZFZBnVECeUQF5RgXkGRWQZ1RAnlEBeUYF5BkVkGdUQJ5RAXlGBeQZFZBnVECeUQF5RgXkGRWQZ1RAnlEBeUYF5BkVkGdUQJ5RAXlGBeQZFZBnVECeUQF5RgXkGRWQZ1RAnlEBeUYF5BkVkGdUQJ5RAXlGBeQZFZBnVECeUQF5RgXkGRWQZ1RAnlEBeUYF5BkVkGdUQJ5RAXlGBeQZFZBnVECeUQF5RgXkGRWQZ1RAnlEBeUYF5BkVkGdUQJ5RAXlGBeQZFZBnVECeUQF5RgXkGRWQZ1RAnlEBeUYF5BkVkGdUQJ5RAXlGBeQZFZBnVECeUQF5RgXkGRWQZ1RAnlEBeUYF5BkVkGdUQJ5RAXlGBeQZFZBnVECeUQF5RgXkGRWQZ1RAnlEBeUYF5BkVkGdUQJ5RAXlGBeQZFZBnVECeUQF5RgXkGRWQZ1RAnlEBeUYF5BkVkGdUQJ5RAXlGBeQZFZBnVECeUQF5RgXkGRWQZ1RAnlEBeUYF5BkVkGdUQJ5RAXlGBeQZFZBnVECeUQF5RgXkGRWQZ1RAnlEBeUYF5BkVkHcBRH8DgsmlTc8AAAAASUVORK5CYII=", "filename.png"];
const datauriLength = datauri.length;
const reader = new FileReader;
reader.onload = () => {
console.log(`data URI: ${reader.result}`)
document.querySelector("iframe").src = reader.result;
};
reader.readAsDataURL(convertDataURItoFile(datauri, filename));
<iframe></iframe>
You can also utilize fetch() to create and get Blob representation of data URI, see Answer by #Endless at Creating a Blob from a base64 string in JavaScript
I have an Angular service with a function for writing files away. The function can work on either an Ionic or Electron platform. For Ionic, it uses $cordovaFile for file actions and for Electron it uses the node fs library.
The function is as follows:
writeFile(filename: string, dirname: string, data: string | Blob, replace?: boolean): ngCordova.IFilePromise<ProgressEvent> {
if (this.isElectron) {
let promiseObj = this.$q.defer();
if (replace) {
try {
fs.unlinkSync('./' + dirname + '/' + filename);
}
catch (err) {
//err
}
}
fs.writeFile('./' + dirname + '/' + filename, data, 'binary', () => {
promiseObj.resolve(true);
});
return promiseObj.promise;
}
else {
return this.$cordovaFile.writeFile(cordova.file.dataDirectory + dirname, filename, data, replace);
}
};
When the Ionic platform is used, the function works fine and the downloaded files are written away correctly. However, when the Electron platform is used, all the downloaded files contain is the string [object Blob].
What is the correct way to write Blobs to files using fs?
MORE INFO
The data originally comes down as base64 in a JSON message but we then do this with it
let fileBlob = this.stringUtilityService.b64ToBlob(dataObj.Data[i].FileContents, 'image/png');
this.fileSystemService.writeFile(dataObj.Data[i].FileName, 'icons', fileBlob);
EXTRA MORE INFO
Here is the b64ToBlob() function, although as far as i can tell this function works fine and correctly returns a blob which the Ionic app correctly saves away and can display.
b64ToBlob(b64Data: string, contentType: string): any {
let sliceSize = 512;
let byteCharacters = atob(b64Data);
let byteArrays = [];
for (let offset = 0; offset < byteCharacters.length; offset += sliceSize) {
let slice = byteCharacters.slice(offset, offset + sliceSize);
let byteNumbers = new Array(slice.length);
for (let i = 0; i < slice.length; i++) {
byteNumbers[i] = slice.charCodeAt(i);
}
let byteArray = new Uint8Array(byteNumbers);
byteArrays.push(byteArray);
}
let blob = new Blob(byteArrays, { type: contentType });
return blob;
}
Rewriting b64ToBlob function to the one like the code below would work fine.
You need to take out a heading signature string like data:image/gif;base64, if the base64 encoded string has it.
b64ToBlob(b64Data: string): any {
return Uint8Array.from(atob(b64Data), (c) => c.charCodeAt(0));
}
I'm trying to read a file in order to perform certain actions on the binary data before sending it to a server.
At a certain point I'm trying to convert the data returned by FileReader.readAsArrayBuffer() to an Uint16Array(). However upon doing so the code allocating the array fails with: 'Error: invalid arguments'. I need the data to be a hex string representing the entire binary.
This is the code I'm using:
function HexToHexString(ByteBuffer)
{
//Similar constructs like: 'var Array = new Uint16Array(ByteBuffer);' also fail
var View = new DataView(ByteBuffer);
var Array = new Uint16Array((ByteBuffer.byteLength / 2)); // <- this line fails
for(var i = 0; i < Array.length; i++)
{
Array[i] = View.getUint16(i*2);
}
return String.fromCharCode.apply(null, Array);
}
function OnReadFileCompletion(FileReadEvent)
{
if(FileReadEvent.target.readyState == FileReader.DONE)
{
// Debug code, will be replaced:
document.getElementById('byte_content').textContent = HexToHexString(FileReadEvent.target.result);
//FileReadEvent.target.result;
}
}
function ReadFile(File, ResultFunction)
{
var Reader = new FileReader();
Reader.onloadend = ResultFunction;
Reader.readAsArrayBuffer(File.slice(0, File.size - 1));
}
File Is a file object, ResultFunction is OnReadFileCompletion(), ByteBuffer is an '[object ArrayBuffer]'.
When I output the size of the ArrayBuffer it matches the size of the file (82kb). I'm on firefox 32 with no plugins installed.
I'm not a javascript programmer, does anyone know what I'm doing wrong?
Edit1:
It appears to have something to-do with the size of the file I'm trying to read, using a 1kb text file appears to work while a 82kb binary file does not.
Edit2
I spoke too soon, perhaps it has something to do with file types. An image file of 200kb works, while an executable of 82 does not.
It appears that javascript does not allow executable files to be accesed this way, does anybody know of any way where I could possibly access the data in hex form?
try using .length instead of byteLength
I've hacked together code that works for me, I don't know why this works or what I did wrong the other time. But it works.
function ApplyPadding(Number, PaddingLength)
{
var s = Number + "";
while (s.length < PaddingLength)
s = "0" + s;
return s;
}
function HexToHexString(ByteBuffers)
{
var AnArray = new Uint8Array(ByteBuffers);
var Result = "";
for(var i = 0; i < AnArray.length; i++)
{
if(i%2==0)
Result += ApplyPadding(AnArray[i].toString(16), 2);
}
return Result;
}
function HexStringToHex(aString)
{
var Buffer = new ArrayBuffer(aString.length*2); // 2 bytes for each char
var BufferView = new Uint16Array(Buffer);
for (var i = 0;i < aString.length; i++)
{
BufferView[i] = aString.charCodeAt(i);
}
return Buffer;
}
function OnReadFileCompletion(FileReadEvent)
{
if(FileReadEvent.target.readyState == FileReader.DONE)
{
//document.getElementById('byte_content').textContent =FileReadEvent.target.result;
var DataOfFile = HexStringToHex(FileReadEvent.target.result);
var FinalData = HexToHexString(DataOfFile);
document.getElementById('byte_content').textContent = FinalData;
//FileReadEvent.target.result;
}
}
function ReadFile(File, ResultFunction)
{
var Reader = new FileReader();
Reader.onloadend = ResultFunction;
Reader.readAsBinaryString(File.slice(0, File.size - 1));
}
I am attempting to allow the user to save images that have been rendered to the canvas. There may be several images and I want the user to be able to download all the images at once as a single tar file. I am trying to write the code to generate this tar file. I have most of this working, but when the tar file is downloaded to my computer at the end I discover that some of the binary data that composes the png files has been corrupted. Here is the relevant code:
var canvBuffer = $('<canvas>').attr('width', canvasWidth).attr('height', canvasHeight);
var ctxBuffer = canvBuffer[0].getContext('2d');
imgData.data.set(renderedFrames[0]);
ctxBuffer.putImageData(imgData,0,0);
var strURI = canvBuffer[0].toDataURL();
var byteString = atob(decodeURIComponent(strURI.substring(strURI.indexOf(',')+1)));
toTar([byteString]);
function toTar(files /* array of blobs to convert */){
var tar = '';
for (var i = 0, f = false, chkSumString, totalChkSum, out; i < files.length; i++) {
f = files[i];
chkSumString = '';
var content = f;
var name = 'p1.png'.padRight('\0', 100);
var mode = '0000664'.padRight('\0', 8);
var uid = (1000).toString(8).padLeft('0', 7).padRight('\0',8);
var gid = (1000).toString(8).padLeft('0', 7).padRight('\0',8);
var size = (f.length).toString(8).padLeft('0', 11).padRight('\0',12);
var mtime = '12123623701'.padRight('\0', 12); // modification time
var chksum = ' '; // enter all spaces to calculate chksum
var typeflag = '0';
var linkname = ''.padRight('\0',100);
var ustar = 'ustar \0';
var uname = 'chris'.padRight('\0', 32);
var gname = 'chris'.padRight('\0', 32);
// Construct header with spaces filling in for chksum value
chkSumString = (name + mode + uid + gid + size + mtime + chksum + typeflag + linkname + ustar + uname + gname).padRight('\0', 512);
// Calculate chksum for header
totalChkSum = 0;
for (var i = 0, ch; i < chkSumString.length; i++){
ch = chkSumString.charCodeAt(i);
totalChkSum += ch;
}
// reconstruct header plus content with chksum inserted
chksum = (totalChkSum).toString(8).padLeft('0', 6) + '\0 ';
out = (name + mode + uid + gid + size + mtime + chksum + typeflag + linkname + ustar + uname + gname).padRight('\0', 512);
out += content.padRight('\0', (512 + Math.floor(content.length/512) * 512)); // pad out to a multiple of 512
out += ''.padRight('\0', 1024); // two 512 blocks to terminate the file
tar += out;
}
var b = new Blob([tar], {'type': 'application/tar'});
window.location.href = window.URL.createObjectURL(b);
}
I am putting a previously rendered frame onto a non-rendered canvas and using the canvases toDataURL() method to an encoded png version of the frame with Base64 encoding. Next I use atob to convert this to a byte string so it can be appended to the contents of the tar file I am generating.
When I view the file in a hex editor my tar header is correct, but the contents of the png are not quite right. The ASCII contents looks normal but binary data is scrambled.
Any help offered would be greatly appreciated. Thanks.
PS
I have attached links to related posts that I have looked at. They have been helpful, but I have not yet seen anything there that fully resolves my issues. Thanks.
Convert Data URI to File then append to FormData, and Data URI to Object URL with createObjectURL in chrome/ff
OK, I have resolved the issue. The problem was the Blob constructor at the end of toTar. passing it a string caused the blob to treat my data as UTF-8 instead of binary, I need to instead pass it arrayBuffer for an array of unsigned integers. Below is my solution
var byteArray = new Uint8Array(tar.length);
for (var b = 0; b < tar.length; b++) {
byteArray[b] = tar.charCodeAt(b);
}
var b = new Blob([byteArray.buffer], {'type': 'application/tar'});
window.location.href = window.URL.createObjectURL(b);
I should rewrite toTar to build the file in Uint8Array and remove the need to convert at the end, but this adequately answers my question and hopefully will help someone else. Thanks.