We Keep Coding

how to convert javascript BigInt to Uint8Array with negative values considered

I have a java backend that's reading/writing arbitrary precision numbers with javascript front-end using bytes.
On Java side I just do:
new BigInteger("123").toByteArray()
The doc says:
Returns a byte array containing the two's-complement
representation of this BigInteger.
On javascript side I need to read the bytes into javascript's BigInt, and vise versa.
I have the javascript reading the bytes into BigInt as bellow:
const hex = Buffer.from(bytes).toString('hex')
let big = BigInt('0x' + hex)
if (a[0] & 0x80) {
const negative = BigInt('0x1' + '0'.repeat(hex.length))
big -= negative
}
But converting BigInt to bytes seems to be very tricky. I looked around and found that other solutions all only cover positive numbers but not negative numbers

After digging around and understanding two's-complement representation, I'm answering my own question:
const big0 = BigInt(0)
const big1 = BigInt(1)
const big8 = BigInt(8)
function bigToUint8Array(big: bigint) {
if (big < big0) {
// work out how long is the big int in bits and add 1
const bits: bigint = (BigInt(big.toString(2).length) / big8 + big1) * big8
// create a BigInt that's 100000... of length of big + 1
const prefix1: bigint = big1 << bits
big += prefix1
}
let hex = big.toString(16)
if (hex.length % 2) {
hex = '0' + hex
}
const len = hex.length / 2
const u8 = new Uint8Array(len)
var i = 0
var j = 0
while (i < len) {
u8[i] = parseInt(hex.slice(j, j + 2), 16)
i += 1
j += 2
}
return u8
}

Converting Double to Bytes and Long to Bytes gives different byte arrays [duplicate]

Is there any way I can read bytes of a float value in JS? What I need is to write a raw FLOAT or DOUBLE value into some binary format I need to make, so is there any way to get a byte-by-byte IEEE 754 representation? And same question for writing of course.

You can do it with typed arrays:
var buffer = new ArrayBuffer(4);
var intView = new Int32Array(buffer);
var floatView = new Float32Array(buffer);
floatView[0] = Math.PI
console.log(intView[0].toString(2)); //bits of the 32 bit float
Or another way:
var view = new DataView(new ArrayBuffer(4));
view.setFloat32(0, Math.PI);
console.log(view.getInt32(0).toString(2)); //bits of the 32 bit float
Not sure what browser support is like though

I've created an expansion of Miloš's solution that should be a bit faster, assuming TypedArrays are not an option of course (in my case I'm working with an environment where they're not available):
function Bytes2Float32(bytes) {
var sign = (bytes & 0x80000000) ? -1 : 1;
var exponent = ((bytes >> 23) & 0xFF) - 127;
var significand = (bytes & ~(-1 << 23));
if (exponent == 128)
return sign * ((significand) ? Number.NaN : Number.POSITIVE_INFINITY);
if (exponent == -127) {
if (significand == 0) return sign * 0.0;
exponent = -126;
significand /= (1 << 22);
} else significand = (significand | (1 << 23)) / (1 << 23);
return sign * significand * Math.pow(2, exponent);
}
Given an integer containing 4 bytes holding an IEEE-754 32-bit single precision float, this will produce the (roughly) correct JavaScript number value without using any loops.

Koolinc's snippet is good if you need a solution that powerful, but if you need it for limited use you are better off writing your own code. I wrote the following function for converting a string hex representation of bytes to a float:
function decodeFloat(data) {
var binary = parseInt(data, 16).toString(2);
if (binary.length < 32)
binary = ('00000000000000000000000000000000'+binary).substr(binary.length);
var sign = (binary.charAt(0) == '1')?-1:1;
var exponent = parseInt(binary.substr(1, 8), 2) - 127;
var significandBase = binary.substr(9);
var significandBin = '1'+significandBase;
var i = 0;
var val = 1;
var significand = 0;
if (exponent == -127) {
if (significandBase.indexOf('1') == -1)
return 0;
else {
exponent = -126;
significandBin = '0'+significandBase;
}
}
while (i < significandBin.length) {
significand += val * parseInt(significandBin.charAt(i));
val = val / 2;
i++;
}
return sign * significand * Math.pow(2, exponent);
}
There are detailed explanations of algorithms used to convert in both directions for all formats of floating points on wikipedia, and it is easy to use those to write your own code. Converting from a number to bytes should be more difficult because you need to normalize the number first.

I had a similar problem, I wanted to convert any javascript number to a Buffer and then parse it back without stringifying it.
function numberToBuffer(num) {
const buf = new Buffer(8)
buf.writeDoubleLE(num, 0)
return buf
}
Use example:
// convert a number to buffer
const buf = numberToBuffer(3.14)
// and then from a Buffer
buf.readDoubleLE(0) === 3.14
This works on current Node LTS (4.3.1) and up. didn't test in lower versions.

Would this snippet help?
var parser = new BinaryParser
,forty = parser.encodeFloat(40.0,2,8)
,twenty = parser.encodeFloat(20.0,2,8);
console.log(parser.decodeFloat(forty,2,8).toFixed(1)); //=> 40.0
console.log(parser.decodeFloat(twenty,2,8).toFixed(1)); //=> 20.0

I expect you could figure it out (blech), but I assume you're asking if there's something built-in. Not as far as I've ever heard; see sections 8.5 and 15.7 of the spec.

64-bit IEEE 754 float to its binary representation and back:
// float64ToOctets(123.456) -> [64, 94, 221, 47, 26, 159, 190, 119]
function float64ToOctets(number) {
const buffer = new ArrayBuffer(8);
new DataView(buffer).setFloat64(0, number, false);
return [].slice.call(new Uint8Array(buffer));
}
// octetsToFloat64([64, 94, 221, 47, 26, 159, 190, 119]) -> 123.456
function octetsToFloat64(octets) {
const buffer = new ArrayBuffer(8);
new Uint8Array(buffer).set(octets);
return new DataView(buffer).getFloat64(0, false);
}
// intToBinaryString(8) -> "00001000"
function intToBinaryString(i, length) {
return i.toString(2).padStart(8, "0");
}
// binaryStringToInt("00001000") -> 8
function binaryStringToInt(b) {
return parseInt(b, 2);
}
function octetsToBinaryString(octets) {
return octets.map((i) => intToBinaryString(i)).join("");
}
function float64ToBinaryString(number) {
return octetsToBinaryString(float64ToOctets(number));
}
function binaryStringToFloat64(string) {
return octetsToFloat64(string.match(/.{8}/g).map(binaryStringToInt));
}
console.log(float64ToBinaryString(123.123))
console.log(binaryStringToFloat64(float64ToBinaryString(123.123)))
console.log(binaryStringToFloat64(float64ToBinaryString(123.123)) === 123.123)
This is a slightly modified version this MIT-licensed code: https://github.com/bartaz/ieee754-visualization/blob/master/src/ieee754.js

JavaScript set four bits

I got following bit pattern:
1000 0001 (129)
I now want to set the last four bits after my favor (1 - 10, 0x1 - 0xA):
1000 0010
or
1000 1000
I have actually no idea how I can accomplish this. I could read out the first four bits:
var buff = new Buffer(1);
buff[0] = 129;
var isFirstBitSet = (buff[0] & 128) == 128;
var isSecondBitSet = (buff[0] & 64) == 40;
var isThirdBitSet = (buff[0] & 32) === 32;
var isFourthBitSet = (buff[0] & 16) === 16;
var buff[0] = 0xA;
if (isFirstBitSet) {
buff[0] = buff[0] & 128;
}
and map then on a new one but I think it is self explained that this is crap.

You can set the low four bits of an integer by first ANDing the integer with 0xfffffff0 and then ORing it with your four-bit value.
function setLowFour(n, lowFour) {
return (n & 0xfffffff0) | (lowFour & 0xf);
}
Note that JavaScript doesn't really have an integer type. The bitwise operations force the values to be integers, but they're really still stored as floating point numbers.
edit — I think it actually works too :-) setLowFour(1025, 12) returns 1036. How's that for unit testing?

Javascript: Parsing Double [duplicate]

I would like to convert a number in base 10 with fraction to a number in base 16.
var myno = 28.5;
var convno = myno.toString(16);
alert(convno);
All is well there. Now I want to convert it back to decimal.
But now I cannot write:
var orgno = parseInt(convno, 16);
alert(orgno);
As it doesn't return the decimal part.
And I cannot use parseFloat, since per MDC, the syntax of parseFloat is
parseFloat(str);
It wouldn't have been a problem if I had to convert back to int, since parseInt's syntax is
parseInt(str [, radix]);
So what is an alternative for this?
Disclaimer: I thought it was a trivial question, but googling didn't give me any answers.
This question made me ask the above question.

Another possibility is to parse the digits separately, splitting the string up in two and treating both parts as ints during the conversion and then add them back together.
function parseFloat(str, radix)
{
var parts = str.split(".");
if ( parts.length > 1 )
{
return parseInt(parts[0], radix) + parseInt(parts[1], radix) / Math.pow(radix, parts[1].length);
}
return parseInt(parts[0], radix);
}
var myno = 28.4382;
var convno = myno.toString(16);
var f = parseFloat(convno, 16);
console.log(myno + " -> " + convno + " -> " + f);

Try this.
The string may be raw data (simple text) with four characters (0 - 255) or
a hex string "0xFFFFFFFF" four bytes in length.
jsfiddle.net
var str = '0x3F160008';
function parseFloat(str) {
var float = 0, sign, order, mantissa, exp,
int = 0, multi = 1;
if (/^0x/.exec(str)) {
int = parseInt(str, 16);
}
else {
for (var i = str.length -1; i >=0; i -= 1) {
if (str.charCodeAt(i) > 255) {
console.log('Wrong string parameter');
return false;
}
int += str.charCodeAt(i) * multi;
multi *= 256;
}
}
sign = (int >>> 31) ? -1 : 1;
exp = (int >>> 23 & 0xff) - 127;
mantissa = ((int & 0x7fffff) + 0x800000).toString(2);
for (i=0; i<mantissa.length; i+=1) {
float += parseInt(mantissa[i]) ? Math.pow(2, exp) : 0;
exp--;
}
return float*sign;
}

Please try this:
function hex2dec(hex) {
hex = hex.split(/\./);
var len = hex[1].length;
hex[1] = parseInt(hex[1], 16);
hex[1] *= Math.pow(16, -len);
return parseInt(hex[0], 16) + hex[1];
}
function hex2dec(hex) {
hex = hex.split(/\./);
var len = hex[1].length;
hex[1] = parseInt(hex[1], 16);
hex[1] *= Math.pow(16, -len);
return parseInt(hex[0], 16) + hex[1];
}
// ----------
// TEST
// ----------
function calc(hex) {
let dec = hex2dec(hex);
msg.innerHTML = `dec: <b>${dec}</b><br>hex test: <b>${dec.toString(16)}</b>`
}
let init="bad.a55";
inp.value=init;
calc(init);
<input oninput="calc(this.value)" id="inp" /><div id="msg"></div>

I combined Mark's and Kent's answers to make an overloaded parseFloat function that takes an argument for the radix (much simpler and more versatile):
function parseFloat(string, radix)
{
// Split the string at the decimal point
string = string.split(/\./);
// If there is nothing before the decimal point, make it 0
if (string[0] == '') {
string[0] = "0";
}
// If there was a decimal point & something after it
if (string.length > 1 && string[1] != '') {
var fractionLength = string[1].length;
string[1] = parseInt(string[1], radix);
string[1] *= Math.pow(radix, -fractionLength);
return parseInt(string[0], radix) + string[1];
}
// If there wasn't a decimal point or there was but nothing was after it
return parseInt(string[0], radix);
}

Try this:
Decide how many digits of precision you need after the decimal point.
Multiply your original number by that power of 16 (e.g. 256 if you want two digits).
Convert it as an integer.
Put the decimal point in manually according to what you decided in step 1.
Reverse the steps to convert back.
Take out the decimal point, remembering where it was.
Convert the hex to decimal in integer form.
Divide the result by the the appropriate power of 16 (16^n, where n is the number of digits after the decimal point you took out in step 1).
A simple example:
Convert decimal 23.5 into hex, and want one digit after the decimal point after conversion.
23.5 x 16 = 376.
Converted to hex = 0x178.
Answer in base 16: 17.8
Now convert back to decimal:
Take out the decimal point: 0x178
Convert to decimal: 376
Divide by 16: 23.5

I'm not sure what hexadecimal format you wanted to parse there. Was this something like: "a1.2c"?
Floats are commonly stored in hexadecimal format using the IEEE 754 standard. That standard doesn't use any dots (which don't exist in pure hexadecimal alphabet). Instead of that there are three groups of bits of predefined length (1 + 8 + 23 = 32 bits in total ─ double uses 64 bits).
I've written the following function for parsing such a numbers into float:
function hex2float(num) {
var sign = (num & 0x80000000) ? -1 : 1;
var exponent = ((num >> 23) & 0xff) - 127;
var mantissa = 1 + ((num & 0x7fffff) / 0x7fffff);
return sign * mantissa * Math.pow(2, exponent);
}

Here is a size-improvement of Mark Eirich's answer:
function hex2dec(hex) {
let h = hex.split(/\./);
return ('0x'+h[1])*(16**-h[1].length)+ +('0x'+h[0]);
}
function hex2dec(hex) {
let h = hex.split(/\./);
return ('0x'+h[1])*(16**-h[1].length)+ +('0x'+h[0]);
}
function calc(hex) {
let dec = hex2dec(hex);
msg.innerHTML = `dec: <b>${dec}</b><br>hex test: <b>${dec.toString(16)}</b>`
}
let init = "bad.a55";
inp.value = init;
calc(init);
<input oninput="calc(this.value)" id="inp" /><div id="msg"></div>

private hexStringToFloat(hexString: string): number {
return Buffer.from(hexString, 'hex').readFloatBE(0);
}

Someone might find this useful.
bytes to Float32
function Int2Float32(bytes) {
var sign = (bytes & 0x80000000) ? -1 : 1;
var exponent = ((bytes >> 23) & 0xFF) - 127;
var significand = (bytes & ~(-1 << 23));
if (exponent == 128)
return sign * ((significand) ? Number.NaN : Number.POSITIVE_INFINITY);
if (exponent == -127) {
if (significand === 0) return sign * 0.0;
exponent = -126;
significand /= (1 << 22);
} else significand = (significand | (1 << 23)) / (1 << 23);
return sign * significand * Math.pow(2, exponent);
}

How to do bitwise AND in javascript on variables that are longer than 32 bit?

I have 2 numbers in javascript that I want to bit and. They both are 33bit long
in C#:
((4294967296 & 4294967296 )==0) is false
but in javascript:
((4294967296 & 4294967296 )==0) is true
4294967296 is ((long)1) << 32
As I understand it, it is due to the fact that javascript converts values to int32 when performing bit wise operations.
How do I work around this?
Any suggestions on how to replace bit and with a set of other math operations so that bits are not lost?

Here's a fun function for arbitrarily large integers:
function BitwiseAndLarge(val1, val2) {
var shift = 0, result = 0;
var mask = ~((~0) << 30); // Gives us a bit mask like 01111..1 (30 ones)
var divisor = 1 << 30; // To work with the bit mask, we need to clear bits at a time
while( (val1 != 0) && (val2 != 0) ) {
var rs = (mask & val1) & (mask & val2);
val1 = Math.floor(val1 / divisor); // val1 >>> 30
val2 = Math.floor(val2 / divisor); // val2 >>> 30
for(var i = shift++; i--;) {
rs *= divisor; // rs << 30
}
result += rs;
}
return result;
}
Assuming that the system handles at least 30-bit bitwise operations properly.

You could split each of the vars into 2 32-bit values (like a high word and low word), then do a bitwise operation on both pairs.
The script below runs as a Windows .js script. You can replace WScript.Echo() with alert() for Web.
var a = 4294967296;
var b = 4294967296;
var w = 4294967296; // 2^32
var aHI = a / w;
var aLO = a % w;
var bHI = b / w;
var bLO = b % w;
WScript.Echo((aHI & bHI) * w + (aLO & bLO));

There are several BigInteger librairy in Javascript, but none of them offer bitwise operation you need at the moment. If you are motivated and really need that functionality you can modify one of those librairy and add a method to do so. They already offer a good code base to work with huge number.
You can find a list of the BigInteger librairy in Javascript in this question :
Huge Integer JavaScript Library

The simplest bit-wise AND, that works up to JavaScript's maximum number
JavaScript's max integer value is 2^53 for internal reasons (it's a double float). If you need more there are good libraries for that big integer handling.
2^53 is 9,007,199,254,740,992, or about 9,000 trillion (~9 quadrillion).
// Works with values up to 2^53
function bitwiseAnd_53bit(value1, value2) {
const maxInt32Bits = 4294967296; // 2^32
const value1_highBits = value1 / maxInt32Bits;
const value1_lowBits = value1 % maxInt32Bits;
const value2_highBits = value2 / maxInt32Bits;
const value2_lowBits = value2 % maxInt32Bits;
return (value1_highBits & value2_highBits) * maxInt32Bits + (value1_lowBits & value2_lowBits)
}

Ran into this problem today and this is what I came up with:
function bitwiseAnd(firstNumber, secondNumber) {
let // convert the numbers to binary strings
firstBitArray = (firstNumber).toString(2),
secondBitArray = (secondNumber).toString(2),
resultedBitArray = [],
// get the length of the largest number
maxLength = Math.max(firstBitArray.length, secondBitArray.length);
//add zero fill ahead in case the binary strings have different lengths
//so we can have strings equal in length and compare bit by bit
firstBitArray = firstBitArray.padStart(maxLength,'0');
secondBitArray = secondBitArray.padStart(maxLength,'0');
// bit by bit comparison
for(let i = 0; i < maxLength; i++) {
resultedBitArray.push(parseInt(firstBitArray[i]) && secondBitArray[i]);
}
//concat the result array back to a string and parse the binary string back to an integer
return parseInt(resultedBitArray.join(''),2);
}
Hope this helps anyone else who runs into this problem.