I'm currently creating an ObjectURL from a Blob, derived from a growing array of incoming, live-streamed media chunks:
URL ⬅ Blob ⬅ some processing ⬅ dynamic array ⬅ growing list of MP4 chunks
Problem being, I require a single URL to work with this array, and cannot access a remote server to acquire it. Using Blobs works to an extent, but since they are immutable and each URL returned from the createObjectURL routine is unique – I'm at a loss as to how to do this. Essentially, as my array grows, it needs to be accessible from the programmatically-derived 'http://example.com/asdfasdf' address.
Is there another method to generate a more consistent, consumable outlet? I say 'consumable' as my testing criteria states that the should be compatible with the native HTML5 video element.
Related
Using the Fetch API I'm able to make a network request for a large asset of binary data (say more than 500 MB) and then convert the Response to either a Blob or an ArrayBuffer.
Afterwards, I can either do worker.postMessage and let the standard structured clone algorithm copy the Blob over to a Web Worker or transfer the ArrayBuffer over to the worker context (making effectively no longer available from the main thread).
At first, it would seem that it would be much preferable to fetch the data as an ArrayBuffer, since a Blob is not transferrable and thus, will need to be copied over. However, blobs are immutable and thus, it seems that the browser doesn't store it in the JS heap associated to the page, but rather in a dedicated blob storage space and thus, what's ended up being copied over to the worker context is just a reference.
I've prepared a demo to try out the difference between the two approaches: https://blobvsab.vercel.app/. I'm fetching 656 MB worth of binary data using both approaches.
Something interesting I've observed in my local tests, is that copying the Blob is even faster than transferring the ArrayBuffer:
Blob copy time from main thread to worker: 1.828125 ms
ArrayBuffer transfer time from main thread to worker: 3.393310546875 ms
This is a strong indicator that dealing with Blobs is actually pretty cheap. Since they're immutable, the browser seems to be smart enough to treat them as a reference rather than linking the overlying binary data to those references.
Here are the heap memory snapshots I've taken when fetching as a Blob:
The first two snapshots were taken after the resulting Blob of fetching was copied over the worker context using postMessage. Notice that neither of those heaps include the 656 MBs.
The latter two snapshots were taken after I've used a FileReader to actually access the underlying data, and as expected, the heap grew a lot.
Now, this is what happens with fetching directly as an ArrayBuffer:
Here, since the binary data was simply transferred over the worker thread, the heap of the main thread is small but the worker heap contains the entirety of the 656 MBs, even before reading this data.
Now, looking around at SO I see that What is the difference between an ArrayBuffer and a Blob? mentions a lot of underlying differences between the two structures, but I haven't found a good reference regarding if one should be worried about copying over a Blob between execution contexts vs. what would seem an inherent advantage of ArrayBuffer that they're transferrable. However, my experiments show that copying the Blob might actually be faster and thus I think preferable.
It seems to be up to each browser vendor how they're storing and handling Blobs. I've found this Chromium documentation describing that all Blobs are transferred from each renderer process (i.e. a page on a tab) to the browser process and that way Chrome can even offload the Blob to the secondary memory if needed.
Does anyone have some more insights regarding all of this? If I can choose to fetch some large binary data over the network and move that to a Web Worker should I prefer a Blob or a ArrayBuffer?
No, it's not expensive at all to postMessage a Blob.
The cloning steps of a Blob are
Their serialization steps, given value and serialized, are:
Set serialized.[[SnapshotState]] to value’s snapshot state.
Set serialized.[[ByteSequence]] to value’s underlying byte sequence.
Their deserialization step, given serialized and value, are:
Set value’s snapshot state to serialized.[[SnapshotState]].
Set value’s underlying byte sequence to serialized.[[ByteSequence]].
In other words, nothing is copied, both the snapshot state and the byte sequence are passed by reference, (even though the wrapping JS object is not).
However regarding your full project, I wouldn't advise using Blobs here for two reasons:
The fetch algorithm first fetches as an ArrayBuffer internally. Requesting a Blob adds an extra step there (which consumes memory).
You'll probably need to read that Blob from the Worker, adding yet an other step (which will also consume memory since here the data will actually get copied).
As said from the MDN Web Docs:
The Blob object represents a blob, which is a file-like object of immutable, raw data; they can be read as text or binary data, or converted into a ReadableStream so its methods can be used for processing the data.
I also know that a File object inherits some of the properties of a Blob and it can be used pretty much everywhere that Blob can. But, if File can be used in the same context as Blob, how should I choose between them? Are there some cases that one is preferable from another?
As said on the very page you linked:
The File interface is based on Blob, inheriting blob functionality and expanding it to support files on the user's system.
The File docs show that in addition to the blob properties, files also do have a lastModified date, a name, and possibly a path. It also states that
File objects are generally retrieved from a FileList object returned as a result of a user selecting files using the <input> element [or] from a drag and drop operation's DataTransfer object
Whenever you can actually choose which one to use, that means you are constructing them yourself. In contrast to the Blob constructor, the File constructor takes a non-optional name argument. So whenever you're constructing something that you'd give a file name to, use File.
Currently, my website is using a Mongo.Collection to hold data submitted from another site. Strings are sent over through HTTP methods and packed into the collection afterward. However, this collection now needs to support storing larger files, but still needs to hold the data already stored, so I've been looking into converting the collection into GridFS. Is there a way to attach the data onto empty files as metadata, or is the conversion more convoluted than this?
I have been working with the WebcamJS library to stream video from the camera in the browser, but I have run into a major performance bottleneck. Since I am using Internet Explorer 11 (and cannot switch to a different browser), this library reverts to a Flash fallback for accessing the camera.
The ActionScript callback that returns the image is prohibitively slow, due to its many steps. When it returns the image, it first encodes its byte array as a PNG or JPG, and then to a base 64 string. This string is then passed using ExternalInterface to JavaScript, which decodes the image through a data URI. Given that all I need is the byte array in JavaScript, these extra steps seem wasteful.
I have had to tackle a similar problem before, in C++/Python. Rather than repeatedly pass the array data back and forth between the two languages, I used Python to pass a NumPy array reference at the start of the program. Then, they could both access the same data from then on without any extra communication.
Now that you understand my situation, here is the question: is it possible to pass a JavaScript Array or ArrayBuffer by reference to ActionScript? In that case, I could have ActionScript modify the JavaScript array directly, rather than waste time converting, encoding, and decoding the image for each frame.
(WebcamJS: https://github.com/jhuckaby/webcamjs)
Just for completeness, SharedObjects in flash store data, serialised with the AMF protocol, on the file system (in a very specific, sandboxed and locked place) where Javascript has no way to access to read the data.
Have you tried to simply call the ExternalInterface method and pass an array of bytes as an argument? it would be passed by value, automatically converted from the Actionscript data structure to the Javascript one, but you'd skip all the encoding steps and it should be fast enough ...
I'm so stuck on this. I need to retrieve a picture e.g. http://ip:port/icon_contact.png using JavaScript from another server requiring basic authentication. The server can't give base64. Don't worry about x-domain restriction.
thanks in advance,
louenas
If I'm reading your question correctly — which is by no means certain — you want to retrieve the binary data of an image file providing basic authentication information directly (not via the user).
You should be able to do this with the XMLHttpRequest object (you can supply auth information in the open call), but to read binary data from the response I'm fairly sure you'll have to stray into brand-new and/or implementation-specific stuff. Here are links to the MSDN, MDC, and (fairly new) W3C docs. Microsoft's XMLHttpRequest has responseBody, Mozilla's (Firefox's) has mozResponseArrayBuffer, and I believe the W3C docs discuss binary data here.
To display the image having loaded it via the above, you could transform the binary data into a data URL (more correctly "data URI", but no one says that) string and assign the result to an img tag's src. You'd have to convert from whatever the browser-specific binary stuff was into the base64 encoding (for the data URL). (You probably don't have to write the conversion yourself, a quick search indicates that people have been tackling this problem and you can reuse [and possibly contribute back to] their efforts...)
The bad news is that IE only supports data URIs as of IE8, and it limits them to 32k, so you'd have to nifty slicing techniques like Google does for search preview.
Once you have the data:// string, the img tag part is easy. If you're not using a library:
var img, element;
img = document.createElement('img');
img.src = /* ... the data URI ... */
element = /* ... find the element you want to put the image in, via
document.getElementById or document.getElementsByTagName
or other DOM traversal ... */;
element.appendChild(img);