We have an REST API endpoint that will stream multiple GB of data as a response. Currently using xhr and responseType: blob, then In our web interface, we would like to stream that response do a file instead of storing the entire response in memory, and the trying to save it to a file. {oked around the fetch API. Still can't quite figure out how to get something like that to work. What are we missing?
The canonical HTTP way of doing this is to use a HTTP chunked response. That means that instead of using XHR one points the browser at the file URI and the server should respond with a response that indicates it supports chunking.
You're essentially trying to reinvent the wheel (of downloading files) with XHR and JavaScript.
Point the browser at the proper URI and it will be streamed directly to client's disk. Supports resuming broken downloads out-of-the-box!
Related
How a streaming (e.g. a sound or video) HTTP server should report failure in the DB from which it streams (when it already streams, 503 header cannot be generated).
The client should be able to differentiate if it received complete file or its beginning only.
I am especially interested how to return such errors in Rust frameworks.
The HTTP server library you are using should be taking care of this for you.
If you are writing your own server for some reason, you should use the chunked encoding method, which has a clear way to specify that the stream is over (a zero-lengthed chunk).
I am creating a website where I publish Video files and Pdf files. I need to hide the original URLs of those files while inspecting them. So I want to know how to create Blob URLs using JavaScript.
That's useless.
To create a blob: URL, you need a Blob, which is binary data accessible to the browser's memory.
And to have that binary data from the server to the browser's memory, you need to fetch it.
Browsers' dev tools come with a Network monitor, from where all the Network request can be monitored, and the one network request required to fetch the binary data to the browser's memory will be accessible there.
So your evil users willing to steal your files will just have to copy the request from that panel and save the response directly to their disk.
The only sensible way to protect data is through DRMs, for videos see the Encrypted Media Extension API.
This article states that:
When using cookies, you can trigger a file download and stream the contents easily. However, in the tokens world, where the request is done via XHR, you can't rely on that.
Is there something tricky about downloading a file using XHR from a URL that requires a JWK token for authorization?
I am writing a web application for my company. The company has a server that I have to use and I can not change anything on the server side.
When I send an HTTP GET request I get a stream of XML as response, i.e. the HTTP connection remains opened until it is closed manually. When I test the request in my browser I immediately get the response which is updated automatically every time the data changes (and because it is a stream, the loading animation of the browser keeps running).
How can I handle the response in javascript? Is there any way I can get notified when the data changes? I have done research for available frameworks to handle this but without any luck.
There is a SAX parser written in JavaScript which would be useful in an architecture such as:
Company Server
WebSocket stream
Sax.js
References
Sax-js
npm:websocket-stream
NodeJS Stream Playground
I know how to handle chunked downloads in javascript, using the XmlHttpRequest object. Is there any way to perform a chunked upload using javascript, opening a connection but only uploading blobs of data bit by bit?
I know chunked uploads should be possible with Http 1.1 servers, and have found a lot of references to making chunked uploads using various other platforms (C# java etc.) but have not found any references to doing so in the browser with javascript.
EDIT: The use case is to stream data up to the server, and not to upload a large file, kind of mirroring the use of a chunked response to stream data down to the client. This is as an alternative to making individual ajax requests, since the chunks of data that's going up from client to server are pretty frequent (< 0.5s interval).
As of today (November 2021), I believe support for UPLOADS using HTTP chunked data transfer is still largely missing in browsers.
If you look at the "Send ReadableStream in request body" column of the browser support matrix for the Request (Fetch API), you can see that it is currently "No" for all browsers except "Deno". You will however notice the "Experimental" flag beside the column. So it is available experimentally in some browsers such as Chrome. I wouldn't hold your breath about it becoming mainstream anytime soon though.
HTTP chunked data transfer encoding is not technically necessary for sending data a few chunks at a time, I believe regular HTTP data transfer also only sends data a few chunks at a time but the "chunking" is done at the TCP level instead (please correct me if I'm wrong here). Consequently, both protocols can be used to stream a file upload. WebSockets are of course another option as well. The main difference in which protocol to choose is based on whether or not you know the final length of the stream in advance or not.
If you need to stream upload data for which you DON'T know the length in advance (such as live video, video conference calls, remote desktop sessions, chats, etc.) then your best bet is perhaps the WebSocket API (or something built on top of it).
If you need to stream upload data for which you DO know the length in advance (files, images, videos, etc.) then I believe your best bet is probably a normal POST or PUT using the Fetch API or even the old XmlHttpRequest API.
You can use the FileReader API and the slice method.
with slice you can get block of data that you can upload, then you need to reassemble them server side.
here is a good intro on how to handle files in javascript
http://www.html5rocks.com/en/tutorials/file/dndfiles/
you can have a look at http://caniuse.com/#feat=filereader for browser support