What would be the best way to read an Int64BE from a node.js buffer into a Number primitive, like readInt32BE reads an Int32?
I know that I'll lose precision with numbers +/- 9'007'199'254'740'992, but i won't get such high numbers in the protocol I want to implement.
Javascript uses only 64 bit double precision floats. To read a long number you have to read two 32 bit integers and shift the high 32 bits to the left. Also note that there possibly is an information loss for long values not in the range of 9007199254740992 <= x <= -9007199254740992 since the internal representation uses 1 bit for the sign and 11 bits for the exponent.
Since the low part can be negative but must be treated as unsigned, a correction is added.
function readInt64BEasFloat(buffer, offset) {
var low = readInt32BE(buffer, offset + 4);
var n = readInt32BE(buffer, offset) * 4294967296.0 + low;
if (low < 0) n += 4294967296;
return n;
}
Don't try and code the conversion yourself, use a tested version like node-int64.
var Int64 = require('node-int64');
function readInt64BEasFloat(buffer, offset) {
var int64 = new Int64(buffer, offset);
return int64.toNumber(true);
}
In the latest Node.js (12.0.0), you can use buf.readBigInt64BE :))
Related
I've been trying to convert a hash to a float number, so far I have only achieved to generate a number from it.
How would I convert a SHA256 Hash to a float number ranging from 0.0 to 1.0?
How I got a number,
var hash = 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855';
var number = hash.substr(0, 8);
number = parseInt(number, 16);
number = Math.abs(number) % 11;
console.log('Number (0 - 10): ' + number);
Something like this:
var hash = 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855';
var number = 0.0;
for(let i=0;i<hash.length;i++) {
number += parseInt(hash.charAt(i), 16) / 16;
}
number /= hash.length;
console.log('number = ' + number);
In this example, I give the equal weight to each of the 32 numbers (which seems reasonable since a hash is really not a number). But, you can give different weights to different bytes. For instance, you can give a weight of 2^31 to the first byte, and 2^30 for the second byte, etc. and normalize the final number accordingly.
Edit: You can also do the comparison for multiple bytes at a time. E.g., 16 numbers with each coming from 2 bytes, or 8 numbers with each coming from 4 bytes, etc. As you increase the "unit" size, the resulting final number will have a "higher fidelity" to the original hash value.
The comment by Mörre contains the best answer. Simply take the first 52-bits of the hash (or any convenient 52 bits of the hash) and assign it to a javascript number. A 52-bit integer can be stored exactly as an IEEE 754 double. Then simply divide the number by 252, which should only effect the exponent bits of the number. The result is a floating point number x, 0 <= x < 1, with as much entropy as is possible given the constraints.
I'm currently generating UUIDs in Javascript with this function (Create GUID / UUID in JavaScript?):
lucid.uuid = function() {
return 'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx'.replace(/[xy]/g, function(c) {
var r = Math.random()*16|0, v = c == 'x' ? r : (r&0x3|0x8);
return v.toString(16);
});
}
I understand that all the randomness is only coming from Javascript's Math.random() function, and I don't care if it meets an RFC for a UUID. What I want is to pack as much randomness into as few bytes as possible in a Javascript string. The above function gives about 128 bits of randomness. How small of a string (as measured in UTF8 bytes sent over the wire in an HTTP POST) could I fit 128 bits into in Javascript? And how would I generate such a string?
Edit: This string will be part of a JSON object when sent to the server, so characters that need to be escaped in a string are not very helpful.
Here is one potential function I came up with. The seed string is the set of unreserved URL characters (66 of them). I prefix the randomness with about a year's worth of 1-second-resolution timestamp data, which is helpful since the collision space for my particular application is only filled up reasonably slowly over time (only at MOST a few hundred of these generated per second in an extreme case).
uuidDense = function() {
var seed = 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789-_.~';
//Start the UUID with 4 digits of seed from the current date/time in seconds
//(which is almost a year worth of second data).
var seconds = Math.floor((new Date().getTime())/1000);
var ret = seed[seconds % seed.length];
ret += seed[Math.floor(seconds/=seed.length) % seed.length];
ret += seed[Math.floor(seconds/=seed.length) % seed.length];
ret += seed[Math.floor(seconds/=seed.length) % seed.length];
for(var i = 0; i < 8; i++)
ret += seed[Math.random()*seed.length|0];
return ret;
}
Thoughts?
128 bits = 16 bytes -> base64 -> 16*3/2 = will give you string of 24 characters (versus 36 chars that you have)
You also can use base85 for better density but that will require URL encode so you may get even worse results than you have.
Your question is somewhat contradictory. Javascript strings use UCS-2 (fixed 16-bit characters) for their internal representation. However UTF-8 is variable width, but for encoding purposes I believe the most compact form would be to use 1-byte UTF8 characters, which only require the most significant bit be zero. I.e. you could pack 128 bits into 128 * 8/7 = 147 bits.
Converting to bytes and rounding up, you could do this in 19 characters.
I need to do the following in JavaScript and so far been unable to find solutions to do it seamlessly:
Grab two integers in a specific order and pack them like Python's struct module.
This packed value, (bonus for supporting different endianness than host) will be turned into a 64 bit float (double). They must be arbitrary thus I might get an exponent representation of the integer (say, they could be 0xdeadbeef and 500):
In exp form:
1.0883076389305e-311
1.0883076389305000 * 10 ^ - 311
I need to convert it to the arbitrary precision, non-exponent form, so:
0.0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000108830763893050000000000000000000000000000000000000000000000000000000000000000000000000000
That number converted to a string :)
I haven't found a way to do this in Javascript and I have to output some numbers like that which must support arbitrary precision, or at least, of a scale up to the 1024 exponent (or, say 400) of doubles.
Thanks!!
Note: I do need the "packing/unpacking' to be a faithful representation of those two numbers converted to a double/64bit float. But I don't care about, say, exporting to a string or raw buffer. As long as I get an arbitrary precision double representation for the double it's all fine.
1: Khronos has a specification in progress for a DataView interface as part of the WebGL TypedArray requirements, which combined with Int32Array and Float64Array would let you write your two ints into a buffer, and read them back out as a double.
Unfortunately browser support for this isn't common yet - to test your browser visit http://html5test.com/ and look at the section entitled "Native binary data".
Without the TypedArray support above I don't think there's any way to do this using bit-twiddling since Javascript's bit operators treat numbers as 32-bit unsigned values, so you'd have no access to the higher-order bits.
2: double variables don't have any specific form, IEE754 is just an internal representation.
3: that's the point at which you can attempt to show the actual precision. Unfortunately the built-in method, e.g. Number.toFixed(), doesn't support showinng more than 20 decimal places. You will need to parse the exponential form and manually construct a string with the appropriate number of leading zeros.
NB - the exponent range of a double is 2^1024, not 10^1024, hence the real limit is actually ~1.0E±308 - your example figure is smaller than that range.
EDIT actually, there might be a way, but I can't guarantee the precision of this:
take your two integers, call them hi and lo.
extract the exponent - exp = (hi >> 20) & 0x7ff
extract the sign - sign = (hi >> 31)
extract the mantissa - ((hi & 0xfffff) * Math.pow(2, 32) + lo) / Math.pow(2, 52)
result = (1 + m) * (Math.pow(2.0, exp - 1023))
if (sign) result *= -1
EDIT 2 - it works! See http://jsfiddle.net/alnitak/assXS/
var hex2double = function(input) {
var hi = parseInt(input.substring(0, 8), 16);
var lo = parseInt(input.substring(8 ), 16);
var p32 = 0x100000000;
var p52 = 0x10000000000000;
var exp = (hi >> 20) & 0x7ff;
var sign = (hi >> 31);
var m = 1 + ((hi & 0xfffff) * p32 + lo) / p52;
m = exp ? (m + 1) : (m * 2.0);
return (sign ? -1 : 1) * m * Math.pow(2, exp - 1023);
};
Enter a floating point number at http://babbage.cs.qc.edu/IEEE-754/Decimal.html, take the resulting hex string from the bottom row of output, and pass it to the function above. You should see an alert containing the original value.
EDIT 3 code fixed to account for the special case when the exponent bits are all zero.
I think you need a big number library for JavaScript such as http://jsfromhell.com/classes/bignumber.
I'm writing a function to extend a number with sign to a wider bit length. This is a very frequently used action in the PowerPC instruction set. This is what I have so far:
function exts(value, from, to) {
return (value | something_goes_here);
}
value is the integer input, from is the number of bits that the value is using, and to is the target bit length.
What is the most efficient way to create a number that has to - from bits set to 1, followed by from bits set to 0?
Ignoring the fact that JavaScript has no 0b number syntax, for example, if I called
exts(0b1010101010, 10, 14)
I would want the function to OR the value with 0b11110000000000, returning a sign-extended result of 0b11111010101010.
A number containing p one bits followed by q zero bits can be generated via
((1<<p)-1)<<q
thus in your case
((1<<(to-from))-1)<<from
or much shorter
(1<<to)-(1<<from)
if you have the number 2^q (= 1 shifted left by q) represented as an integer of width p + q bits, it has the representation:
0...010...0
p-1 q
then 2^q - 1 has the representation
0...01...1
p q
which is exactly the opposite of you want. So just flip the bits
hence what you want is NOT((1 LEFT SHIFT by q) - 1)
= ~((1 << q) - 1) in c notation
I am not overly familiar with binary mathematics in JavaScript... But if you need to OR a number with 0b11110000000000, then I assume you would just convert that to decimal (which would get you 15360), and do value | 15360.
Relevant info that you may find useful: parseInt("11110000000000", 2) converts a binary number (specified as a string) to a decimal number, and (15360).toString(2) converts a decimal number (15360 in this case) to a binary number (the result is a string).
Revised solution
There's probably a more elegant and mathematical method, but here's a quick-and-dirty solution:
var S = "";
for(var i=0;i<p;i++)
S += "1";
for(i=0;i<q;i++)
S += "0";
S = parseInt(S, 2); // convert to decimal
I am trying to perform something that is brain-dead simple in any other language but not javascript: get the bits out of float (and the other way around).
In C/C++ it would be something like
float a = 3.1415;
int b = *((int*)&a);
and vise-versa
int a = 1000;
float b = *((float*)&a);
In C# you can use the BitConverter
...floatBits or something alike in Java... Even in VB6 for Christ's sake you can memcpy a float32 into an int32. How on earth can I translate between and int and a float in javascript?
function DoubleToIEEE(f)
{
var buf = new ArrayBuffer(8);
(new Float64Array(buf))[0] = f;
return [ (new Uint32Array(buf))[0] ,(new Uint32Array(buf))[1] ];
}
You certainly don't get anything low-level like that in JavaScript. It would be extremely dangerous to allow recasting and pointer-frobbing in a language that has to be safe for untrusted potential-attacker web sites to use.
If you want to get a 32-bit IEEE754 representation of a single-precision value in a Number (which remember is not an int either; the only number type you get in JavaScript is double), you will have to make it yourself by fiddling the sign, exponent and mantissa bits together. There's example code here.
function FloatToIEEE(f)
{
var buf = new ArrayBuffer(4);
(new Float32Array(buf))[0] = f;
return (new Uint32Array(buf))[0];
}
Unfortunately, this doesn't work with doubles and in old browsers.
JavaScript uses double (IEEE 754) to represent all numbers
double consists of [sign, exponent(11bit), mantissa(52bit)] fields.
Value of number is computed using formula (-1)^sign * (1.mantissa) * 2^(exponent - 1023). (1.mantissa - means that we take bits of mantissa add 1 at the beginning and tread that value as number, e.g. if mantissa = 101 we get number 1.101 (bin) = 1 + 1/2 + 1/8 (dec) = 1.625 (dec).
We can get value of sign bit testing if number is greater than zero. There is a small issue with 0 here because double have +0 and -0 values, but we can distinguish these two by computing 1/value and checking if value is +Inf or -Inf.
Since 1 <= 1.mantissa < 2 we can get value of exponent using Math.log2 e.g. Math.floor(Math.log2(666.0)) = 9 so exponent is exponent - 1023 = 9 and exponent = 1032, which in binary is (1032).toString(2) = "10000001000"
After we get exponent we can scale number to zero exponent without changing mantissa, value = value / Math.pow(2, Math.floor(Math.log2(666.0))), now value represents number (-1)^sign * (1.mantissa). If we ignore sign and multiply that by 2^52 we get integer value that have same bits as 1.mantissa: ((666 / Math.pow(2, Math.floor(Math.log2(666)))) * Math.pow(2, 52)).toString(2) = "10100110100000000000000000000000000000000000000000000" (we must ignore leading 1).
After some string concat's you will get what you want
This is only proof of concept, we didn't discuss denormalized numbers or special values such as NaN - but I think it can be expanded to account for these cases too.
#bensiu answers is fine, but if find yourself using some old JS interpreter you can use this approach.
Like the other posters have said, JavaScript is loose typed, so there is no differentiation in data types from float to int or vice versa.
However, what you're looking for is
float to int:
Math.floor( 3.9 ); // result: 3 (truncate everything past .) or
Math.round( 3.9 ); // result: 4 (round to nearest whole number)
Depending on which you'd like. In C/C++ it would essentially be using Math.floor to convert to integer from float.
int to float:
var a = 10;
a.toFixed( 3 ); // result: 10.000