I need to combine 2 parts of 2 existing arrayBuffers into a new one.
I am building a parser and the data comes in arraybuffers of random sizes, the data will spill over the end of one, into the beginning of the other. So I need to create a new output buffer and copy in a portion of the end of one buffer and a portion of the beginning of the other. The output will just be an Arraybuffer.
Starting out with this demo, I was going to make Uint8Arrays with some offsets then use set, the problem is certain combinations throw Invalid typed array length. I will not know the length of each array or offsets beforehand.
var buffer1 = new ArrayBuffer(8);
var buffer2 = new ArrayBuffer(8);
var buffer3 = new ArrayBuffer(8);
var uint8_1 = new Uint8Array(buffer1);
var uint8_2 = new Uint8Array(buffer2);
var uint8_3 = new Uint8Array(buffer3);
uint8_1.fill(1);
uint8_2.fill(2);
var uint8_1_slice = new Uint8Array(buffer1 , 0 , 3);
var uint8_2_slice = new Uint8Array(buffer2 , 4, 7);
For this demo need to get buffer3 to be 1,1,1,1,2,2,2,2.
Cannot Use Slice
I have seen some people only use array.length. It's fine as long as the array is only 1 byte per element. It's also fine if the other typed arrays are filled up but in this example a2 isn't. That is why it's better to use byteLength this is also how Blob constructor concatenate the parts.
// Concatenate a mix of typed arrays
function concatenate(...arrays) {
// Calculate byteSize from all arrays
let size = arrays.reduce((a,b) => a + b.byteLength, 0)
// Allcolate a new buffer
let result = new Uint8Array(size)
// Build the new array
let offset = 0
for (let arr of arrays) {
result.set(arr, offset)
offset += arr.byteLength
}
return result
}
// the total length of 1-3 = 5
// the total byteLength of 1-3 = 6
let a1 = Uint8Array.of(1, 2) // [1, 2]
let a2 = Uint16Array.of(3) // [3] just for the fun of it 16 takes up 2 bytes
let a3 = Uint8Array.of(4, 5) // [4, 5]
concatenate(a1, a2, a3) // [1, 2, 3, 0, 4, 5]
/********/
var blob = new Blob([a1, a2, a3])
var res = new Response(blob)
res.arrayBuffer().then(buffer => console.log(new Uint8Array(buffer)))
// [1, 2, 3, 0, 4, 5]
For this demo need to get buffer3 to be 1,1,1,1,2,2,2,2.
You can use for loop, set uint8_3 to uint8_1 value if variable n is less than uint8_1.byteLength / 2 else set uint8_3 to value at uint8_2 .
var len = 8;
var buffer1 = new ArrayBuffer(len);
var buffer2 = new ArrayBuffer(len);
var buffer3 = new ArrayBuffer(len);
var uint8_1 = new Uint8Array(buffer1);
var uint8_2 = new Uint8Array(buffer2);
var uint8_3 = new Uint8Array(buffer3);
uint8_1.fill(1);
uint8_2.fill(2);
// `len` : uint8_1.byteLength / 2 + uint8_2.byteLength / 2
for (var n = 0; n < len; n++) {
uint8_3[n] = n < len / 2 ? uint8_1[n] : uint8_2[n];
}
console.log(uint8_3);
Related
What's the best way to convert this array of comma separated values
[ 'com--test,LFutx9mQbTTyRo4A9Re5ksjdnfsI4cKN4q2,on',
'com--fxtrimester,SEzlksdfMpW3FxkSbzL7eo5MmqkPczCl2,on',
'com--fxtrimester,LFutx9mQbTTyRoldksfns4A9Re5I4cKN4q2,on' ]
Into three arrays?
[ 'com--test', [ LFutx9mQbTTyRo4A9Re5Ilsdf4cKN4q2', ['on',
'com--fxtrimester', SEzMpW3FxkSbzL7eo5MmlkdfqkPczCl2', 'on',
'com--fxtrimester' ] LFutksdfx9mQbTTyRo4A9Re5I4cKN4q2 ] 'on']
I was trying something like:
var indexToSplit = unique.indexOf(',');
var status = unique.slice(3, indexToSplit - 1);
var use = unique.slice(2, indexToSplit - 2);
var pros = unique.slice(0, indexToSplit - 3);
console.log(pros);
But I figured that is wrong ... any help is appreciated!
You will have to loop over array and use string.split to get seperate parts.
Once you have seperate parts, you can push them to necessary array;
var d = [ 'com--test,LFutx9mQbTTyRo4A9Re5ksjdnfsI4cKN4q2,on',
'com--fxtrimester,SEzlksdfMpW3FxkSbzL7eo5MmqkPczCl2,on',
'com--fxtrimester,LFutx9mQbTTyRoldksfns4A9Re5I4cKN4q2,on'];
var result = [[],[],[]];
var len = 3;
d.forEach(function(str, i){
var _tmp = str.split(',');
for (var i = 0; i<len; i++){
result[i].push(_tmp[i])
}
})
console.log(result)
A little generic way.
Loop over data and split each string using comma(,)
Loop over split values and check if necessary array exists.
If not, initialise array, but you cannot do p[i] = [] as this will push to first value. You will have to also initialise all previous values. For this, you can use new Array(length). By default, if length is greater than 0, all indexes will be initialise to undefined.
Now push to necessary array. Position will be maintained.
var d = ['com--test,LFutx9mQbTTyRo4A9Re5ksjdnfsI4cKN4q2,on',
'com--fxtrimester,SEzlksdfMpW3FxkSbzL7eo5MmqkPczCl2,on',
'com--fxtrimester,LFutx9mQbTTyRoldksfns4A9Re5I4cKN4q2,on,test'
];
var result = d.reduce(function(p, c, index) {
var _tmp = c.split(',');
for (var i = 0; i < _tmp.length; i++) {
// Check if position not defined.
if (p[i] === undefined)
// Initialize array and add default (undefined) to all elements before current element
p[i] = new Array(index);
p[i].push(_tmp[i])
}
return p;
}, [])
console.log(result)
With map this becomes:
for positions X out of 0, 1 and 2:
convert each item in the list into an array, and choose the Xth item
var start = [ 'com--test,LFutx9mQbTTyRo4A9Re5ksjdnfsI4cKN4q2,on',
'com--fxtrimester,SEzlksdfMpW3FxkSbzL7eo5MmqkPczCl2,on',
'com--fxtrimester,LFutx9mQbTTyRoldksfns4A9Re5I4cKN4q2,on' ]
var out = [0,1,2].map(i =>
start.map(x => x.split(',')[i]) )
console.log(out)
Since your question does not ask for a more general case, i am safely assuming it for 3 array. We can use forEach function on array below code can be one amongst the possible solutions
var arr1 = [];
var arr2 = [];
var arr3 = [];
var x = ['com--test,LFutx9mQbTTyRo4A9Re5ksjdnfsI4cKN4q2,on', 'com--fxtrimester,SEzlksdfMpW3FxkSbzL7eo5MmqkPczCl2,on', 'com--fxtrimester,LFutx9mQbTTyRoldksfns4A9Re5I4cKN4q2,on']
x.forEach(function(data) {
var dataArray = data.split(',');
arr1.push(dataArray[0]);
arr2.push(dataArray[1]);
arr3.push(dataArray[2]);
});
console.log(arr1)
console.log(arr2)
console.log(arr3)
Javascript ArrayBuffer or TypedArrays dont have any kind of appendByte(), appendBytes(), or appendBuffer() methods. So if I want to fill an ArrayBuffer one value at a time, how do I do it?
var firstVal = 0xAB; // 1 byte
var secondVal = 0x3D7F // 2 bytes
var anotherUint8Array = someArr;
var buffer = new ArrayBuffer(); // I don't know the length yet
var bufferArr = new UInt8Array(buffer);
// following methods do not exist. What are the alternatives for each??
bufferArr.appendByte(firstVal);
bufferArr.appendBytes(secondVal);
bufferArr.appendBuffer(anotherUint8Array);
You can create a new TypedArray with a new ArrayBuffer, but you can't change the size of an existing buffer
function concatTypedArrays(a, b) { // a, b TypedArray of same type
var c = new (a.constructor)(a.length + b.length);
c.set(a, 0);
c.set(b, a.length);
return c;
}
Now can do
var a = new Uint8Array(2),
b = new Uint8Array(3);
a[0] = 1; a[1] = 2;
b[0] = 3; b[1] = 4;
concatTypedArrays(a, b); // [1, 2, 3, 4, 0] Uint8Array length 5
If you want to use different types, go via Uint8Array as the smallest unit is a byte, i.e.
function concatBuffers(a, b) {
return concatTypedArrays(
new Uint8Array(a.buffer || a),
new Uint8Array(b.buffer || b)
).buffer;
}
This means .length will work as expected, you could now convert this to your typed array of choice (make sure it's a type that would accept the .byteLength of the buffer though)
From here, you could now implement any method you like for concatenating your data, e.g.
function concatBytes(ui8a, byte) {
var b = new Uint8Array(1);
b[0] = byte;
return concatTypedArrays(ui8a, b);
}
var u8 = new Uint8Array(0);
u8 = concatBytes(u8, 0x80); // [128]
Paul's answer allows you to concatenate one TypedArray to an existing TypedArray. In ES6, you can use the following function to concatenate multiple TypedArrays:
function concatenate(resultConstructor, ...arrays) {
let totalLength = 0;
for (const arr of arrays) {
totalLength += arr.length;
}
const result = new resultConstructor(totalLength);
let offset = 0;
for (const arr of arrays) {
result.set(arr, offset);
offset += arr.length;
}
return result;
}
const ta = concatenate(Uint8Array,
Uint8Array.of(1, 2), Uint8Array.of(3, 4));
console.log(ta); // Uint8Array [1, 2, 3, 4]
console.log(ta.buffer.byteLength); // 4
To append a new byte is:
const byte = 3;
concatenate(Uint8Array, Uint8Array.of(1, 2), Uint8Array.of(byte));
This method is found in ExploringJS.
I want to convert a string to double then store it into array of type double at the last, I did the following:
var num = "";
var sym = new Float64Array();
sym[sym.length] = parseFloat(num);
but when I prints the array I get undefined , so where is the wrong I did ?
Make necessary modfications. Refer Float64Array API
.You define length in Float64array as a parameter.
your length is coming 1.
var num = "65";
var x = new Float64Array(1);
x[x.length-1] = parseFloat(num);
console.log(x[0]);
Float64Arrays can't grow and shrink like ordinary JS arrays. You have to allocate all the elements when creating it:
var sym = new Float64Array(10);
You can then assign elements:
sym[0] = 123.456;
console.log(sym);
> [123.456, 0, 0, 0, 0, 0, 0, 0, 0, 0]
var arr = [];
var num = parseFloat("123.123")
arr.push(num);
console.log(arr);
I'm working with ArrayBuffer objects, and I would like to duplicate them. While this is rather easy with actual pointers and memcpy, I couldn't find any straightforward way to do it in Javascript.
Right now, this is how I copy my ArrayBuffers:
function copy(buffer)
{
var bytes = new Uint8Array(buffer);
var output = new ArrayBuffer(buffer.byteLength);
var outputBytes = new Uint8Array(output);
for (var i = 0; i < bytes.length; i++)
outputBytes[i] = bytes[i];
return output;
}
Is there a prettier way?
I prefer the following method
function copy(src) {
var dst = new ArrayBuffer(src.byteLength);
new Uint8Array(dst).set(new Uint8Array(src));
return dst;
}
It appears that simply passing in the source dataview performs a copy:
var a = new Uint8Array([2,3,4,5]);
var b = new Uint8Array(a);
a[0] = 6;
console.log(a); // [6, 3, 4, 5]
console.log(b); // [2, 3, 4, 5]
Tested in FF 33 and Chrome 36.
ArrayBuffer is supposed to support slice (http://www.khronos.org/registry/typedarray/specs/latest/) so you can try,
buffer.slice(0);
which works in Chrome 18 but not Firefox 10 or 11. As for Firefox, you need to copy it manually. You can monkey patch the slice() in Firefox because the Chrome slice() will outperform a manual copy. This would look something like,
if (!ArrayBuffer.prototype.slice)
ArrayBuffer.prototype.slice = function (start, end) {
var that = new Uint8Array(this);
if (end == undefined) end = that.length;
var result = new ArrayBuffer(end - start);
var resultArray = new Uint8Array(result);
for (var i = 0; i < resultArray.length; i++)
resultArray[i] = that[i + start];
return result;
}
Then you can call,
buffer.slice(0);
to copy the array in both Chrome and Firefox.
Hmmm... if it's the Uint8Array you want to slice (which logically, it should be), this may work.
if (!Uint8Array.prototype.slice && 'subarray' in Uint8Array.prototype)
Uint8Array.prototype.slice = Uint8Array.prototype.subarray;
Faster and slightly more complicated version of chuckj's answer. Should use ~8x less copy operations on large Typed Arrays. Basically we copy as much 8-byte chunks as possible and then copy the remaining 0-7 bytes. This is especially useful in current version of IE, since it doesn't have slice method implemented for ArrayBuffer.
if (!ArrayBuffer.prototype.slice)
ArrayBuffer.prototype.slice = function (start, end) {
if (end == undefined) end = that.length;
var length = end - start;
var lengthDouble = Math.floor(length / Float64Array.BYTES_PER_ELEMENT);
// ArrayBuffer that will be returned
var result = new ArrayBuffer(length);
var that = new Float64Array(this, start, lengthDouble)
var resultArray = new Float64Array(result, 0, lengthDouble);
for (var i = 0; i < resultArray.length; i++)
resultArray[i] = that[i];
// copying over the remaining bytes
that = new Uint8Array(this, start + lengthDouble * Float64Array.BYTES_PER_ELEMENT)
resultArray = new Uint8Array(result, lengthDouble * Float64Array.BYTES_PER_ELEMENT);
for (var i = 0; i < resultArray.length; i++)
resultArray[i] = that[i];
return result;
}
Wrap a Buffer around the ArrayBuffer. This is shared memory and no copy is made. Then create a new Buffer from the wrapping Buffer. This will copy the data. Finally get a reference to the new Buffer's ArrayBuffer.
This is the most straighforward way I can find. The most efficient? Perhaps.
const wrappingBuffer = Buffer.from(arrayBuffer)
const copiedBuffer = Buffer.from(wrappingBuffer)
const copiedArrayBuffer = copiedBuffer.buffer
In some cases (like webaudio Audiobuffers) you only have a reference to the 2 arrays.
So if you have array1 as a float32Array and array2 as a float32Array,
you must do an element by element copy.
To do so you can use different methods.
var ib=z.inputBuffer.getChannelData(0);
var ob=z.outputBuffer.getChannelData(0);
this
ib.forEach((chd,i)=>ob[i]=chd);
or this nicer and probably faster
ob.set(ib);
That's because Array.set populates an existing array with multiple data (even from another array)
Some of the operations above only do "shallow" copies. When working with workers and transferable arrays, you need to do a deep copy.
function copyTypedArray(original, deep){
var copy;
var kon = original.constructor;
if(deep){
var len = original.length;
copy = new kon(len);
for (var k=len; --k;) {
copy[k] = original[k];
}
} else {
var sBuf = original.buffer;
copy = new kon(sBuf);
copy.set(original);
}
return copy;
}
HINT (for the confused): Typed Arrays contain an ArrayBuffer, which can be obtained via the "buffer" property.
var arr = new Float32Array(8);
arr.buffer <-- this is an ArrayBuffer
If you're in the browser you can do:
const copy = structuredClone(buffer);
new Uint16Array(ArrayBuffer, byteOffset, length);
For Uint16 (word) byteOffset can only be 0, 2, 4, 6 etc. How access to 2nd, 4th byte? (byteOffset = 1, 3 etc)
DataView is solution for Chrome but not for FireFox (dont know about opera at all).
You can convert the buffer into a Uint8Array (for separate bytes), and then read from that array:
var a = new Uint16Array(10);
// fill `a` with data
for(var i = 0; i < 10; i++) a[i] = i * 10;
var b = new Uint8Array(a.buffer); // `b` contains bytes of `a`, e.g. use `b[1]`
var orig = new Uint8Array(buffer);
var sub = new Uint16Array(orig.subarray(1, 2)); // from index 1 to 2, so second byte
var word = sub[0]; // 2nd byte as Uint16
About your posted case:
var buf = new ArrayBuffer(65535);
var u8s = new Uint8Array(buf);
u8s[0] = 0x01;
u8s[1] = 0x02;
u8s[2] = 0x03;
var x = new Uint8Array(u8s.subarray(1, 3)).buffer; // buffer from subarray
// 1 to 3 because 1 word is 2 bytes
sub = new Uint16Array(x); // create Uint16Array from it
sub[0]; // 770, which is: (3 << 8) | 2 (it is big-endian)