I have a web api that reads a file from azure and downloads it into a byte array. The client receives this byte array and downloads it as pdf. This does not work well with large files.
I am not able to figure out how can I send the bytes in chunks from web api to client.
Below is the web api code which just returns the byte array to client:
CloudBlockBlob blockBlob = container.GetBlockBlobReference(fileName);
blockBlob.FetchAttributes();
byte[] data = new byte[blockBlob.Properties.Length];
blockBlob.DownloadToByteArray(data, 0);
return report;
Client side code gets the data when ajax request completes, creates a hyperlink and set its download attribute which downloads the file:
var a = document.createElement("a");
a.href = 'data:application/pdf;base64,' + data.$value;;
a.setAttribute("download", filename);
The error occurred for a file of 1.86 MB.
The browser displays the message:
Something went wrong while displaying the web page.To continue, reload the webpage.
The issue is most likely your server running out of memory on these large files. Don't load the entire file into a variable only to then send it out as the response. This causes a double download, your server has to download it from azure storage and keep it in memory, then your client has to download it from the server. You can do a stream to stream copy instead so memory is not chewed up. Here is an example from your WebApi Controller.
public async Task<HttpResponseMessage> GetPdf()
{
//normally us a using statement for streams, but if you use one here, the stream will be closed before your client downloads it.
Stream stream;
try
{
//container setup earlier in code
var blockBlob = container.GetBlockBlobReference(fileName);
stream = await blockBlob.OpenReadAsync();
//Set your response as the stream content from Azure Storage
response.Content = new StreamContent(stream);
response.Content.Headers.ContentLength = stream.Length;
//This could change based on your file type
response.Content.Headers.ContentType = new MediaTypeHeaderValue("application/pdf");
}
catch (HttpException ex)
{
//A network error between your server and Azure storage
return this.Request.CreateErrorResponse((HttpStatusCode)ex.GetHttpCode(), ex.Message);
}
catch (StorageException ex)
{
//An Azure storage exception
return this.Request.CreateErrorResponse((HttpStatusCode)ex.RequestInformation.HttpStatusCode, "Error getting the requested file.");
}
catch (Exception ex)
{
//catch all exception...log this, but don't bleed the exception to the client
return this.Request.CreateErrorResponse(HttpStatusCode.BadRequest, "Bad Request");
}
finally
{
stream = null;
}
}
I have used (almost exactly) this code and have been able to download files well over 1GB in size.
Related
I'm practicing in programming with websocket, i'm building a java websocket server and want to send a binary data to websocket client, then client side can get that data through websocket and save it to a local file on disk, but i don't know what way using javascript to write an arraybuffer into a file.
My example code is shown below
var ws = new WebSocket(URL);
ws.binaryType = "arraybuffer";
ws.onmessage = function(evt) {
if (evt.data instanceOf ArrayBuffer) {
// binary message is riecieved from server and i want to save it as a local file
}
};
Thank you for any help
You can create a textual/string representation that can be saved to and read from a file:
// returns String
function getArrayBufferString(arrayBuffer) {
return new Uint8Array(arrayBuffer).toString()
}
// returns ArrayBuffer
function parseArrayBufferString(string) {
return new Uint8Array(string.split(',')).buffer
}
I'm trying to download huge CSV file from server which is being generated on the fly.
Im returning ResponseEntity<StreamingResponseBody> in async so as soon as i have part of my data i'm returning it.
this is my controller code:
StreamingResponseBody streamingResponseBody = out -> {
csvService.exportToCsvBySessionId(applicationId, sessionIdsInRange, out, tags);
}
return ResponseEntity.ok()
.headers(csvService.getHeaders(CSV_FILE_NAME))
.body(streamingResponseBody);
in the header i'm adding
produces: text\csv;
Content-Disposition: attachment; filename=%s.csv;
On the client side im using aurelia framework and sending the request using HttpClient (fetch)
public getFraudAlertsCsv() {
this.serverProxy.fetch(`/sessions/fraud/csv)
.then(response => {
logger.debug('waiting for response');
return response.blob());
.then((blob: Blob) => this.donwnloadCsv(blob, `Fraud_Alerts_${new Date()}.csv`))
.catch( (err)=> {
this.logger.error("Failed to get appSessionId sessions csv file", err);
});
}
even though i can see in the network analysis that my request is starting to get response (it size increases) there is no popup window asking to download the file, and the log doesn't print "waiting for response".
instead im getting the whole file being download what the entire response arrived (when server close the stream).
I want to show the progress of the download, how can i do it?
I think fetch doesn't support progress API yet, so you may want to use traditional XHR and use onprogress or progress event:
xhr.onprogress = function updateProgress(oEvent) {
if (oEvent.lengthComputable) {
var percentComplete = oEvent.loaded / oEvent.total * 100;
// ...
} else {
// Unable to compute progress information since the total size is unknown
}
}
Note: code example taken from MDN page https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest/Using_XMLHttpRequest#Monitoring_progress
I have two applications. One of them is an HTML based web application which works in IE9 mode and which receives scanned documents' images in base64 string format via XDomainRequest.
The other application is a Windows service written in C#. It listens requests on a port via HttpListener. Then it scans document and sends scanned image via HttpListenerResponse.
The scanned images will be processed by the web application on client side and then they will be uploaded to the server. So I have to do all job in javascript. There is not a server side solution (for now).
If I receive just one image it is not a big problem. But if the sender sends multiple images, base64 string becomes too large and it takes too much time.
I use below code to convert image to base64 string:
public static string ImageToBase64(Image image, System.Drawing.Imaging.ImageFormat format)
{
using (MemoryStream ms = new MemoryStream())
{
image.Save(ms, format);
byte[] imageBytes = ms.ToArray();
string base64String = Convert.ToBase64String(imageBytes);
return base64String;
}
}
And the below code is used to receive images:
if (xdrScanning.responseText != "") {
try {
json = JSON.parse(xdrScanning.responseText);
$('#scannedImage').attr('src', 'data:image/png;base64, ' + json[0].BASE64IMAGE);
} catch (e) {
xdrScanningError();
}
}
For all I know XDomainRequest receives string responses only.
So, what can I do to overcome that performance issue?
I am Creating a chat app between two users now I can do Simple text chat with different users using node.js and socket.io. Now problem arises here as I have to send image in chat application and after searching for whole long day I am not able to get perfect node.js in which I can send image in chat app. So I want to know is it possible to send image using node.js. Here is my simple node.js file for sending simple text message from one user to another.
socket.on('privateMessage', function(data) {
socket.get('name', function (err, name) {
if(!err) {
// get the user from list by its name to get its socket,
// then emit event privateMessage
// again here we want to make you clear
// that every single client connection has its own
// unique SOcket Object, we need to get this Socket object
// to communicate with every other client. The socket variable
// in this scope is the client who wants to send the private
// message but the socket of the receiver is not know.
// Get it from the saved list when connectMe handlers gets called
// by each user.
onLine[data.to].emit('newPrivateMessage',{from:name, msg:data.msg, type:'Private Msg'})
}
});
});
You can use the Base64 version of your image and send it like this:
onLine[data.to].emit('newPrivateMessage',{from:name, img:data.img.toString('base64'), type:'Private Msg'})
.. and then on the client side receive it and create an image
socket.on("newPrivateMessage", function(data) {
if (data.img) {
var img = new Image();
img.src = 'data:image/jpeg;base64,' + data.img;
// Do whatever you want with your image.
}
});
UPDATE
The following is a snippet taken from the link I've commented below. As you can see it takes the image from the input, reads it and sends to the server. After that you can send the same data from the server to another client.
For the full example, please read the article.
JavaScript (client)
...
$('#imageFile').on('change', function(e) {
var file = e.originalEvent.target.files[0],
reader = new FileReader();
reader.onload = function(evt) {
var jsonObject = {
'imageData': evt.target.result
}
// send a custom socket message to server
socket.emit('user image', jsonObject);
};
reader.readAsDataURL(file);
});
...
HTML
...
Image file: <input type="file" id="imageFile" /><br/>
...
UPDATE 2
Here is one example I have found:
Java (client)
File file = new File("path/to/the/image");
try {
FileInputStream imageInFile = new FileInputStream(file);
byte imageData[] = new byte[(int) file.length()];
imageInFile.read(imageData);
// Converting Image byte array into Base64 String
String imageDataString = Base64.encodeBase64URLSafeString(imageData);
} catch (...) {
...
}
The above snippet shows how to read the file and encode the data into a base64 string. So then you can send it just like a string (I assume).
Here is the complete example: How to convert Image to String and String to Image in Java?
Also I have found encodeToString function of Base64.Encoder (java.util package), which you can use.
The easiest way I can think of is to simply Base64 encode the image and send it through the text pipe. You would need to distinguish text and image messages with header information (Maybe send a JSON object?).
I try to create code to upload files using XHR2 and web workers.
I thought I should use web workers , so if a file is big, web page will not freeze.
This is not working for two reasons, I never used web workers before, and I want to post to the server the file and vars at the same time, with the same xhr. When I say vars I mean the name of the file, and an int.
Heres is what I got
Client side
//create worker
var worker = new Worker('fileupload.js');
worker.onmessage = function(e) {
alert('worker says '+e.data);
}
//handle workers error
worker.onerror =werror;
function werror(e) {
console.log('ERROR: Line ', e.lineno, ' in ', e.filename, ': ', e.message);
}
//send stuff to the worker
worker.postMessage({
'files' : files, //img or video
'name' : nameofthepic, //text
'id':imageinsertid //number
});
Inside the worker (fileupload.js file)
onmessage = function (e) {var name=e.data.name; var id=e.data.id ; var file=e.data.files;
//create a var to catch the anser of the server
var datax;
var xhr = new XMLHttpRequest();
xhr.onload = function() {
if (xhr.status == 200) {datax=xhr.response;}
else { datax=525;}//actually, whatever, just give a value
};
xhr.open('POST', 'upload.php');
xhr.send(file,name,id);
//i also tried xhr.send('file=file&name=name&id=id'); and still nothing
//i also tried just the text/int xhr.send('name=name&id=id'); and still nothing
I am confused. I cannot send anything to the server. I get no feedback from the worker. I dont even know if the data are send to the fileupload.js. Server side does not INSERT.
Is that possible, sending files and text at the same time? What am I missing?
I need to pass text and int along with the file, so server side not only will upload the file, but also will INSERT to the database the int and the text, if the file is uploaded succesfully. This was easy just with formData and xhr, but, putting web workers in the middle, I cant get it right.
Also, can I use Transferable Objects to speed things up? Are Transferable Objects supported in all major browsers?
Thanks in advance