Im receiving some file chunks in bytes format from my server and im collecting them into one variable in my frontend to download it later. And I can't change my server conception (receiving a file sliced into chunks).
My problem is that if the file is heavy (from 500MB), my variable length is starting to be very very big and im having an error :
RangeError: Invalid string length
This is because my variable has reach the limit of character (536 800 000).
Here's how im adding my data to my variable :
this.socket.on('new_file', (data: string) => {
this.receivedFile += data;
}
My download part :
public download(): void {
const byteCharacters = this.receivedFile;
const byteArrays = [];
const sliceSize=512
for (let offset = 0; offset < byteCharacters.length; offset +=
sliceSize) {
const slice = byteCharacters.slice(offset, offset + sliceSize);
const byteNumbers = new Array(slice.length);
for (let i = 0; i < slice.length; i++) {
byteNumbers[i] = slice.charCodeAt(i);
}
const byteArray = new Uint8Array(byteNumbers);
byteArrays.push(byteArray);
}
const blob = new Blob(byteArrays, {type: this.fileInfos.type});
saveAs(blob, this.fileInfos.name);
}
Which approach can I do ? Or does it exist some variable type in Javascript to accept more char ? Thanks
Don't collect the chunks into a huge string. Instead, just convert each chunk into a bytearray immediately (that you'll need later anyway) and collect these:
this.socket.on('new_file', (data: string) => {
const bytes = new Uint8Array(data.length);
for (let i = 0; i < data.length; i++) {
bytes[i] = data.charCodeAt(i);
}
this.byteArrays.push(bytes);
}
then
public download(): void {
const blob = new Blob(this.byteArrays, {type: this.fileInfos.type});
saveAs(blob, this.fileInfos.name);
}
I don't think you need to make 512-byte-sized slices.
Related
I receive a raw data of an excel when I send my http request. I am using fileSaver package to save base64 files like pdf (turned into a blob) with it and it works alright. what I receive is as the image below. My question is how can I turn this into saveable blob without data corruption? note that when I save this sting as file in postman it saves fine.
this approach did not work and saves a corrupt excel that cannot be opened:
var blob = new Blob([s2ab(response)], {
type: "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"
});
FileSaver.saveAs(blob, action.payload.culture + ".xlsx");
Solved in this way:
before you do anything add arrayBuffer as response-type to your http req header.
firstly convert the binary of arraybuffer to base64:
export function arrayBufferToBase64(buffer: ArrayBufferLike) {
let binary = "";
let bytes = new Uint8Array(buffer);
let len = bytes.byteLength;
for (let i = 0; i < len; i++) {
binary += String.fromCharCode(bytes[i]);
}
return window.btoa(binary);
}
then decode this base64 to a blob:
export const base64toBlob = (b64Data: string, contentType = "", sliceSize = 512): Blob => {
const byteCharacters = atob(b64Data);
const byteArrays = [];
for (let offset = 0; offset < byteCharacters.length; offset += sliceSize) {
const slice = byteCharacters.slice(offset, offset + sliceSize);
const byteNumbers = new Array(slice.length);
for (let i = 0; i < slice.length; i++) {
byteNumbers[i] = slice.charCodeAt(i);
}
const byteArray = new Uint8Array(byteNumbers);
byteArrays.push(byteArray);
}
const blob = new Blob(byteArrays, { type: contentType });
return blob;
};
Now download it:
const blob = base64toBlob(arrayBufferToBase64(response.data));
FileSaver.saveAs(blob, `${orderType}_${formatedDates.from}_${formatedDates.to}.xlsx`);
This is JavaScript code which convert base64 to blob.
but I want to make this kind of code in objective-c.
I can't search in stack overflow.
function base64_to_blob(base64String) {
var byteString = window.atob(base64String);
var array = new Uint8Array(byteString.length);
for (var i = 0; I < byteString.length; I++) {
array[i] = byteString.charCodeAt(i);
}
var blob = new Blob([array], {
type: ‘image / jpeg’,
name: “image.jpg”
});
return blob;
}
Please Help me.
Time use this code my be helpful for you if.
-(NSData *) base64_to_blob(base64String) {
NSData * blobData = [[NSData alloc] initWithBase64EncodedString: base64String options:1];
return blobData;
}
I have an Angular service with a function for writing files away. The function can work on either an Ionic or Electron platform. For Ionic, it uses $cordovaFile for file actions and for Electron it uses the node fs library.
The function is as follows:
writeFile(filename: string, dirname: string, data: string | Blob, replace?: boolean): ngCordova.IFilePromise<ProgressEvent> {
if (this.isElectron) {
let promiseObj = this.$q.defer();
if (replace) {
try {
fs.unlinkSync('./' + dirname + '/' + filename);
}
catch (err) {
//err
}
}
fs.writeFile('./' + dirname + '/' + filename, data, 'binary', () => {
promiseObj.resolve(true);
});
return promiseObj.promise;
}
else {
return this.$cordovaFile.writeFile(cordova.file.dataDirectory + dirname, filename, data, replace);
}
};
When the Ionic platform is used, the function works fine and the downloaded files are written away correctly. However, when the Electron platform is used, all the downloaded files contain is the string [object Blob].
What is the correct way to write Blobs to files using fs?
MORE INFO
The data originally comes down as base64 in a JSON message but we then do this with it
let fileBlob = this.stringUtilityService.b64ToBlob(dataObj.Data[i].FileContents, 'image/png');
this.fileSystemService.writeFile(dataObj.Data[i].FileName, 'icons', fileBlob);
EXTRA MORE INFO
Here is the b64ToBlob() function, although as far as i can tell this function works fine and correctly returns a blob which the Ionic app correctly saves away and can display.
b64ToBlob(b64Data: string, contentType: string): any {
let sliceSize = 512;
let byteCharacters = atob(b64Data);
let byteArrays = [];
for (let offset = 0; offset < byteCharacters.length; offset += sliceSize) {
let slice = byteCharacters.slice(offset, offset + sliceSize);
let byteNumbers = new Array(slice.length);
for (let i = 0; i < slice.length; i++) {
byteNumbers[i] = slice.charCodeAt(i);
}
let byteArray = new Uint8Array(byteNumbers);
byteArrays.push(byteArray);
}
let blob = new Blob(byteArrays, { type: contentType });
return blob;
}
Rewriting b64ToBlob function to the one like the code below would work fine.
You need to take out a heading signature string like data:image/gif;base64, if the base64 encoded string has it.
b64ToBlob(b64Data: string): any {
return Uint8Array.from(atob(b64Data), (c) => c.charCodeAt(0));
}
I can convert a blob to string using FileReader, but I want to convert it back:
var reader = new window.FileReader();
reader.readAsDataURL(blob);
reader.onloadend = function() {
base64data = reader.result;
var blobToSend = base64data.substr(base64data.indexOf(',')+1);
rtcMultiConnection.send({"command":{
"recording":blobToSend,
"type":blob.type,
"size":blob.size
}});
}
This is sent with https://github.com/muaz-khan/RTCMultiConnection but the main question is how to reconstruct the blob after being sent. Sadly sending the blob as is didn't work.
source: Creating a Blob from a base64 string in JavaScript
This method correctly converts back base64 data to the original binary data.
For the sake of performance improvements, the data is processed in blocks of the size of sliceSize.
NOTE: source is in TypeScript
public static Base64ToBlob(b64Data, contentType = "", sliceSize = 512): Blob
{
const byteCharacters = atob(b64Data);
const byteArrays = [];
for (let offset = 0; offset < byteCharacters.length; offset += sliceSize)
{
const slice = byteCharacters.slice(offset, offset + sliceSize);
const byteNumbers = new Array(slice.length);
for (let i = 0; i < slice.length; i++)
{
byteNumbers[i] = slice.charCodeAt(i);
}
const byteArray = new Uint8Array(byteNumbers);
byteArrays.push(byteArray);
}
const blob = new Blob(byteArrays, { type: contentType });
return blob;
}
I have a .Net (VB/C#) backend on one server and a javascript/jquery front-end on another. I am attempting to pass file data from the backend to the front-end by reading the file in .Net as a byte array, converting it to a base64 string and passing that to the js front-end along with a variable indicating the file type.
Once the data is in the js front-end I attempt to convert the base64 string to a file blob in order to read it with readAsDataURL() so that it can be displayed in an iframe or downloaded by the client.
For testing I am using a pdf file. The process of passing the data works but the file data is not recognized by the pdf viewer. The iframe loads the viewer but I get a message about the file not being recognized as a pdf file.
I have done a tone of searching and have gotten a lot of questions answered by searching stackoverflow but this eludes me.
Any suggestions would be much appreciated. Thank you.
Update:
Here is my VB.Net code:
Sub OnReadFileToViewer(ByVal filePath As String)
Dim okToView As Boolean = False
Dim fileBLOB As String = String.Empty
Dim fileEncoded As String = String.Empty
If (file.Exists(filePath)) Then
Try
Using fRead As FileStream = New FileStream(filePath, FileMode.Open, FileAccess.Read)
Dim bytes() As Byte = New Byte((CType(fRead.Length, Integer)) - 1) {}
Dim numBytesToRead As Integer = CType(fRead.Length, Integer)
Dim numBytesRead As Integer = 0
While (numBytesToRead > 0)
' Read may return anything from 0 to numBytesToRead.
Dim n As Integer = fRead.Read(bytes, numBytesRead, _
numBytesToRead)
' Break when the end of the file is reached.
If (n = 0) Then
Exit While
End If
numBytesRead = (numBytesRead + n)
numBytesToRead = (numBytesToRead - n)
End While
numBytesToRead = bytes.Length
fileEncoded = System.Convert.ToBase64String(bytes)
End Using
okToView = True
Catch ex As Exception
'
End Try
End If
If (okToView) Then
' Code to send data to js fron-end
End If
End Sub
And my js front-end for converting the base64 string to BLOB and the iframe preview:
function PreviewFile(file) {
var fileBlob = b64toBlob(file,'application/pdf','');
var reader = new FileReader();
var fileType = e.target.getAttribute("type");
alert(fileType);
reader.onload = function (e) {
if (fileType)
$("#fileViewerDiv").append('<iframe id="fileVieweriFrame" class="fileViewer"></iframe>'); // + e.target.result + '">');
$("#fileVieweriFrame").attr('src', file); //e.target.result);
$("#fileVieweriFrame").dialog();
}
reader.readAsDataURL(fileBlob);
}
function b64toBlob(b64Data, contentType, sliceSize) {
contentType = contentType || '';
sliceSize = sliceSize || 512;
var byteCharacters = atob(b64Data);
var byteArrays = [];
for (var offset = 0; offset < byteCharacters.length; offset += sliceSize) {
var slice = byteCharacters.slice(offset, offset + sliceSize);
var byteNumbers = new Array(slice.length);
for (var i = 0; i < slice.length; i++) {
byteNumbers[i] = slice.charCodeAt(i);
}
var byteArray = new Uint8Array(byteNumbers);
byteArrays.push(byteArray);
}
var blob = new Blob(byteArrays, {type: contentType});
return blob;
}
The alert(fileType) does display the correct file type - application/pdf.
I have to admit that I lifted the b64toBlob() function from a blog somewhere but I have forgotten which one and have lost the URL to the site. My apologies to the original author. If anyone recognizes the code and knows the author/site please let me know
Edit
b64toBlob() by Jeremy Banks - Based on this SO answer