Most efficient way of turning an array of UInt8Array into an ArrayBuffer? - javascript

I am streaming a file from a gRPC backend to my javascript client. The stream sends a series of UInt8Array objects. I want to combine this series of UInt8Array into one single arrayBuffer (i.e. the original file's arrayBuffer). How can I do this?

Related

How to convert javascript Buffer to array of byte for c/c++ API?

I have a nodejs application and need to convert a Buffer object to array of byte as c/c++ API's payload. I tried "Uint8Array" and "Array.prototype.slice.call", but it seems they don't work as expect. The API could not decode them correctly.
Is there any way I can do this?
Thanks.
Problem solved. I made a mistake before. After retest, to convert Buffer to Uint8Array which could be passed to c/c++ API as array of byte.

Node.js node-mysql2 insert Buffer as Blob directly

What is the correct way to insert Buffer as BLOB just 1-to-1 without serializing to string/hex? Is it possible at all from Node.js?
At my backend I already have Buffers with binary data and want to store them with minimum overheads using prepared statements. I'm getting ER_TRUNCATED_WRONG_VALUE_FOR_FIELD errno:1366 Incorrect string value error. It looks like the library just assumes there is a valid utf8 string in a buffer.

preparing and sending binary data to javascript for webgl

I have a very long list of points (lat/long) in my C# program and I need to pass it to my webgl code. I have done it before using json but I thought I could reduce the bandwidth if I sent the data in binary format. Based on my research, on the client side I should use XMLHttpRequest with arraybuffer as response type. I just do not know what to do on the server side. I mean how to prepare the binary data in C# that could be interpreted as arraybuffer in javascript.
I am new to web programming, so if I am not clear on any part, please let me know.
I am not an expert, but I found this solution to work:
On the server side, convert the array of numbers to a byte array and then send the byte array to javascript as a binary file (Set the Mime type to "application/octet-stream")

How can i get the dataType when Array is castet into an ArrayBuffer?

I have an Array:
var dataArray = new Uint16Array(256);
then my array goes over nodeJs to an other pc. (nodejs binaryType = "arraybuffer")
How do I know on the other client that i have to cast to Uint16Array and not Uint8Array?
Note: the bit depth and array length is a dynamic value and can change while programm is running.
Thanks :)
When transferring data as ArrayBuffer its just binary buffer data, you need to know with what view you want to access the data with. TypedArrays are just a view on a binary ArrayBuffer.
There are several ways to handle this:
Use different routes to submit your data
Add a prefix byte to your data
And possibly the best solution, do not change your data type mid application.
imho a clean interface should only handle one type of data.

What is the difference between an ArrayBuffer and a Blob?

I'm reading http://www.html5rocks.com/en/tutorials/file/xhr2/ and trying to figure out the difference between an ArrayBuffer and a Blob.
Aren't both containers comprised of bits? Hence, couldn't both containers be viewed in many ways (as 32-bit chunks, 16-bit chunks, etc.)?
Summary
Unless you need the ability to write/edit (using an ArrayBuffer), then Blob format is probably best.
Detail
I came to this question from a different html5rocks page., and I found #Bart van Heukelom's comments to be helpful, so I wanted to elevate them to an answer here.
I also found helpful resources specific to ArrayBuffer and Blob objects. In summary: despite the emphasis on Blob being immutable/"raw data" Blob objects are easy to work with.
Resources that compare / contrast ArrayBuffer vs Blob:
Mutability
an ArrayBuffer can be changed (e.g. with a DataView)
a Blob is immutable
Source / Availability in Memory
Quoting Bart van Heukelom:
An ArrayBuffer is in the memory, available for manipulation.
A Blob can be on disk, in cache memory, and other places not readily available
Access Layer
ArrayBuffer will require some access layer like typed arrays
Blob can be passed directly into other functions like window.URL.createObjectURL, as seen in the example from OP's URL.
However, as Mörre points out you may still need File-related interfaces and API's like FileReader to work with a Blob.
Convert / Generate
You can generate Blob from ArrayBuffer and vice versa, which addresses the OP's "Aren't both containers comprised of bits?"
ArrayBuffer can be generated from a Blob using the FileReader's readAsArrayBuffer method , or the async method const arrayBuffer = await blob.arrayBuffer() (thanks to #Darren G)
Blob can be generated from an ArrayBuffer as #user3405291 points out new Blob([new Uint8Array(data)]);, shown in
this answer
Use in Other Libraries
jsZip; (new JSZip()).loadAsync(...) accepts both ArrayBuffer and Blob: String/Array of bytes/ArrayBuffer/Uint8Array/Buffer/Blob/Promise
How does protocol handle ArrayBuffer vs Blob
Websocket (aka WS / WSS)
Use the webSocket's binaryType property (could have values "arraybuffer" or "blob") to "control the type of binary data being received over the WebSocket connection."
XmlHttpRequest (aka XHR)
Use the xhr's responseType property to "to change the expected response type from the server" (valid values include "arraybuffer", "blob", and others like "document", "json", and "text")
the response property will contain the entity body according to responseType, as an ArrayBuffer, Blob, Document, JSON, or string.
Other helpful documentation:
ArrayBuffer
The ArrayBuffer object is used to represent a generic, fixed-length
raw binary data buffer. You cannot directly manipulate the contents of
an ArrayBuffer; instead, you create one of the typed array objects or
a DataView object which represents the buffer in a specific format,
and use that to read and write the contents of the buffer.
Blob
A Blob object represents a file-like object of immutable, raw data.
Blob represent data that isn't necessarily in a JavaScript-native
format. The File interface is based on Blob, inheriting blob
functionality and expanding it to support files on the user's system.
It's explained on the page.
ArrayBuffer
An ArrayBuffer is a generic fixed-length container for binary data. They are super handy if you need a generalized buffer of raw data, but the real power behind these guys is that you can create "views" of the underlying data using JavaScript typed arrays. In fact, multiple views can be created from a single ArrayBuffer source. For example, you could create an 8-bit integer array that shares the same ArrayBuffer as an existing 32-bit integer array from the same data. The underlying data remains the same, we just create different representations of it.
BLOB
If you want to work directly with a Blob and/or don't need to manipulate any of the file's bytes, use xhr.responseType='blob':
If you are dealing with something that is more similar to an immutable file that may be retrieved, stored, or served as a file over HTTP, a Blob has a useful feature: blob.type (Web API docs, Nodejs docs). This returns a MIME type (such as image/png) that you can you use for your Content-Type HTTP header when serving the blob.

Categories