I'm having trouble understanding how shifting works. I would expect that a and b would be the same but that's not the case:
a = 0xff000000;
console.log(a.toString(16));
b = 0xff << 24;
console.log(b.toString(16));
resulting in:
ff000000
-1000000
I came to this code while trying to create a 32bit number from 4 bytes.
Bitwise operators convert their operands to signed 32 bit numbers. That means the most significant bit is the sign bit, which gives you only 31 bits for the number value.
0xff000000 by itself is interpreted as 64bit floating point value. But truncating this to a 32bit signed integer produces a negative value since the most significant bit is 1:
0xff000000.toString(2);
> "11111111000000000000000000000000"
(0xff000000 | 0).toString(16)
> -1000000
According to Bitwise operations on 32-bit unsigned ints? you can use >>> 0 to convert the value back to an unsigned value:
0xff << 24 >>> 0
> 4278190080
From the spec:
The result is an unsigned 32-bit integer.
So it turns out this is as per the spec. Bit shift operators return signed, 32-bit integer results.
The result is a signed 32-bit integer.
From the latest ECMAScript spec.
Because your number is already 8 bits long, shifting it left by 24 bits and then interpreting that as a signed integer means that the leading 1 bit is seen as making it a negative number.
Related
I have a u64 (unsigned integer) stored in 8 bytes of memory. Clearly the range is 0-2^64 integers.
I am converting it to a javascript number by turning each byte into hex and making a hex string:
let s = '0x'
s += buffer.slice(0,1).toString("hex")
s += buffer.slice(1,2).toString("hex")
...
n = parseInt(s)
Works great for everything I have done so far.
But when I look at how javascript stores numbers, I become unsure. Javascript uses 8 bytes for numbers, but treats all numbers the same. This internal javascript "number" representation can also hold floating point numbers.
Can a javascript number store all integers from 0 to 2^64? seems not.
At what point do I get into trouble?
What do people do to get round this?
An unsigned 64 bit integer has the range of a 0 to 18.446.744.073.709.551.615.
You could use the Number wrapper object with the .MAX_VALUE property, it represents the maximum numeric value representable in JavaScript.
The JavaScript Number type is a double-precision 64-bit binary format IEEE 754 value, like double in Java or C#.
General Info:
Integers in JS:
JavaScript has only floating-point numbers. Integers appear internally in two ways. First, most JavaScript engines store a small enough number without a decimal fraction as an integer (with, for example, 31 bits) and maintain that representation as long as possible. They have to switch back to a floating point representation if a number’s magnitude grows too large or if a decimal fraction appears.
Second, the ECMAScript specification has integer operators: namely, all of the bitwise operators. Those operators convert their operands to 32-bit integers and return 32-bit integers. For the specification, integer only means that the numbers don’t have a decimal fraction, and 32-bit means that they are within a certain range. For engines, 32-bit integer means that an actual integer (non-floating-point) representation can usually be introduced or maintained.
Ranges of integers
Internally, the following ranges of integers are important in JavaScript:
Safe integers [1], the largest practically usable range of integers that JavaScript supports:
53 bits plus a sign, range (−2^53, 2^53) which relates to (+/-) 9.007.199.254.740.992
Array indices [2]:
32 bits, unsigned
Maximum length: 2^32−1
Range of indices: [0, 2^32−1) (excluding the maximum length!)
Bitwise operands [3]:
Unsigned right shift operator (>>>): 32 bits, unsigned, range [0, 2^32)
All other bitwise operators: 32 bits, including a sign, range [−2^31, 2^31)
“Char codes”, UTF-16 code units as numbers:
Accepted by String.fromCharCode()
Returned by String.prototype.charCodeAt()
16 bit, unsigned
References:
[1] Safe integers in JavaScript
[2] Arrays in JavaScript
[3] Label bitwise_ops
Source: https://2ality.com/2014/02/javascript-integers.html
First, (-1 >>> 0) === (2**32 - 1) which I expect is due to adding a new zero to the left, thus converting the number into 33-bit number?
But, Why is (-1 >>> 32) === (2**32 - 1) as well, while I expect it (after shifting the 32-bit number 32 times and replacing the Most Significant Bits with zeros) to be 0.
Shouldn't it be equal ((-1 >>> 31) >>> 1) === 0? or Am I missing something?
When you execute (-1 >>> 0) you are executing an unsigned right shift. The unsigned here is key. Per the spec, the result of >>> is always unsigned. -1 is represented as the two's compliment of 1. This in binary is all 1s (In an 8 bit system it'd be 11111111).
So now you are making it unsigned by executing >>> 0. You are saying, "shift the binary representation of -1, which is all 1s, by zero bits (make no changes), but make it return an unsigned number.” So, you get the value of all 1s. Go to any javascript console in a browser and type:
console.log(2**32 - 1) //4294967295
// 0b means binary representation, and it can have a negative sign
console.log(0b11111111111111111111111111111111) //4294967295
console.log(-0b1 >>> 0) //4294967295
Remember 2 ** any number minus 1 is always all ones in binary. It's the same number of ones as the power you raised two to. So 2**32 - 1 is 32 1s. For example, two to the 3rd power (eight) minus one (seven) is 111 in binary.
So for the next one (-1 >>> 32) === (2**32 - 1).... let's look at a few things. We know the binary representation of -1 is all 1s. Then shift it right one digit and you get the same value as having all 1s but precede it with a zero (and return an unsigned number).
console.log(-1 >>> 1) //2147483647
console.log(0b01111111111111111111111111111111) //2147483647
And keep shifting until you have 31 zeros and a single 1 at the end.
console.log(-1 >>> 31) //1
This makes sense to me, we have 31 0s and a single 1 now for our 32 bits.
So then you hit the weird case, shifting one more time should make zero right?
Per the spec:
6.1.6.1.11 Number::unsignedRightShift ( x, y )
Let lnum be ! ToInt32(x).
Let rnum be ! ToUint32(y).
Let shiftCount be the result of masking out all but the least significant 5 bits of rnum, that is, compute rnum & 0x1F.
Return the result of performing a zero-filling right shift of lnum by shiftCount bits. Vacated bits are filled with zero. The result is an unsigned 32-bit integer.
So we know we already have -1, which is all 1s in twos compliment. And we are going to shift it per the last step of the docs by shiftCount bits (which we think is 32). And shiftCount is:
Let shiftCount be the result of masking out all but the least significant 5 bits of rnum, that is, compute rnum & 0x1F.
So what is rnum & 0x1F? Well & means a bitwise AND operation. lnum is the number left of the >>> and rnum is the number right of it. So we are saying 32 AND 0x1F. Remember 32 is 100000. 0x is hexadecimal where each character can be represented by 4 bits. 1 is 0001 and F is 1111. So 0x1F is 00011111 or 11111 (31 in base 10, 2**5 - 1 also).
console.log(0x1F) //31 (which is 11111)
32: 100000 &
0x1F: 011111
---------
000000
The number of bits to shift if zero. This is because the leading 1 in 32 is not part of the 5 most significant bits! 32 is six bits. So we take 32 1s and shift it zero bits! That's why. The answer is still 32 1s.
On the example -1 >>> 31 this made sense because 31 is <= 5 bits. So we did
31: 11111 &
0x1F: 11111
-------
11111
And shifted it 31 bits.... as expected.
Let's test this further.... let's do
console.log(-1 >>> 33) //2147483647
console.log(-1 >>> 1) //2147483647
That makes sense, just shift it one bit.
33: 100001 &
0x1F: 011111
---------
00001
So, go over 5 bits with a bitwise operator and get confused. Want to play stump the dummy with a person who hasn't researched the ECMAScript to answer a stackoverflow post? Just ask why are these the same.
console.log(-1 >>> 24033) //2147483647
console.log(-1 >>> 1) //2147483647
Well of course it's because
console.log(0b101110111100001) // 24033
console.log(0b000000000000001) // 1
// ^^^^^ I only care about these bits!!!
When you do (-1 >>> 0), you are turning the sign bit into zero while keeping the rest of the number the same, therefore ending up as 2**32 - 1.
The next behaviour is documented in the ECMAScript specification. The actual number of shifts is going to be "the result of masking out all but the least significant 5 bits of rnum, that is, compute rnum & 0x1F".
Since 32 & 0x1F === 0, both of your results will be identical.
Javascript
1<<31
-2147483648
1<<32
1
Python
1<<31
2147483648
1<<32
4294967296
Is this related to max int?
But 4294967296 not Bigger then the max int in js.
An integer in JavaScript is actually an IEEE 754 64bit float number. But an integer in Python may be a simple integer or a bignum.
All bit operations in JavaScript was defined on 32bit signed / unsigned integers. When you do these operations, the two operand was first converted to 32 bit integers, and the result will always be a 32 bit integer.
If you want multiple a number with 232, you should do 1 * 2 ** 32 (or 1 * Math.pow(2, 32) in ES5) instead of this one.
Python has builtin bignum support, which support all bit operations such as shift left. As a result, you may shift a number with any (reasonable) bits and it may be greater than 232.
The case is the following:
/// returns 406913024, but should 417018740736
alert(6363201 << 16);
What is wrong? I tried the same in ruby and it returns the correct value (http://www.miniwebtool.com/bitwise-calculator/bit-shift/?data_type=10&number=6363201&place=16&operator=Shift+Left)
Quoting from MDN Left Shift Operator
This operator shifts the first operand the specified number of bits to
the left. Excess bits shifted off to the left are discarded. Zero bits
are shifted in from the right.
Quoting Bitwise Shift Operators
Shift operators convert their operands to 32-bit integers in
big-endian order and return a result of the same type as the left
operand. The right operand should be less than 32, but if not only the
low five bits will be used.
Binary of 6363201 is 11000010001100001000001.
When you left shift 6363201 << 16, it becomes 417018740736 which in binary is 110000100011000010000010000000000000000
Now, 32 bits from the least significant bits are retained, the actual bits retained are 00011000010000010000000000000000 which corresponds to 406913024
In JavaScript, you are dealing with 32-bit numbers.
6363201 << 16 results in 110000100011000010000010000000000000000, which are 39 bits. Shedding off the first 7 bits (since you are shifting from right to left, you end up with 00011000010000010000000000000000, which a (binary) parseInt of it will show you that is 406913024, not 417018740736.
Anyone knows why javascript Number.toString function does not represents negative numbers correctly?
//If you try
(-3).toString(2); //shows "-11"
// but if you fake a bit shift operation it works as expected
(-3 >>> 0).toString(2); // print "11111111111111111111111111111101"
I am really curious why it doesn't work properly or what is the reason it works this way?
I've searched it but didn't find anything that helps.
Short answer:
The toString() function takes the decimal, converts it
to binary and adds a "-" sign.
A zero fill right shift converts it's operands to signed 32-bit
integers in two complements format.
A more detailed answer:
Question 1:
//If you try
(-3).toString(2); //show "-11"
It's in the function .toString(). When you output a number via .toString():
Syntax
numObj.toString([radix])
If the numObj is negative, the sign is preserved. This is the case
even if the radix is 2; the string returned is the positive binary
representation of the numObj preceded by a - sign, not the two's
complement of the numObj.
It takes the decimal, converts it to binary and adds a "-" sign.
Base 10 "3" converted to base 2 is "11"
Add a sign gives us "-11"
Question 2:
// but if you fake a bit shift operation it works as expected
(-3 >>> 0).toString(2); // print "11111111111111111111111111111101"
A zero fill right shift converts it's operands to signed 32-bit integers. The result of that operation is always an unsigned 32-bit integer.
The operands of all bitwise operators are converted to signed 32-bit
integers in two's complement format.
-3 >>> 0 (right logical shift) coerces its arguments to unsigned integers, which is why you get the 32-bit two's complement representation of -3.
http://en.wikipedia.org/wiki/Two%27s_complement
http://en.wikipedia.org/wiki/Logical_shift
var binary = (-3 >>> 0).toString(2); // coerced to uint32
console.log(binary);
console.log(parseInt(binary, 2) >> 0); // to int32
on jsfiddle
output is
11111111111111111111111111111101
-3
.toString() is designed to return the sign of the number in the string representation. See EcmaScript 2015, section 7.1.12.1:
If m is less than zero, return the String concatenation of the String "-" and ToString(−m).
This rule is no different for when a radix is passed as argument, as can be concluded from section 20.1.3.6:
Return the String representation of this Number value using the radix specified by radixNumber. [...] the algorithm should be a generalization of that specified in 7.1.12.1.
Once that is understood, the surprising thing is more as to why it does not do the same with -3 >>> 0.
But that behaviour has actually nothing to do with .toString(2), as the value is already different before calling it:
console.log (-3 >>> 0); // 4294967293
It is the consequence of how the >>> operator behaves.
It does not help either that (at the time of writing) the information on mdn is not entirely correct. It says:
The operands of all bitwise operators are converted to signed 32-bit integers in two's complement format.
But this is not true for all bitwise operators. The >>> operator is an exception to the rule. This is clear from the evaluation process specified in EcmaScript 2015, section 12.5.8.1:
Let lnum be ToUint32(lval).
The ToUint32 operation has a step where the operand is mapped into the unsigned 32 bit range:
Let int32bit be int modulo 232.
When you apply the above mentioned modulo operation (not to be confused with JavaScript's % operator) to the example value of -3, you get indeed 4294967293.
As -3 and 4294967293 are evidently not the same number, it is no surprise that (-3).toString(2) is not the same as (4294967293).toString(2).
Just to summarize a few points here, if the other answers are a little confusing:
what we want to obtain is the string representation of a negative number in binary representation; this means the string should show a signed binary number (using 2's complement)
the expression (-3 >>> 0).toString(2), let's call it A, does the job; but we want to know why and how it works
had we used var num = -3; num.toString(-3) we would have gotten -11, which is simply the unsigned binary representation of the number 3 with a negative sign in front, which is not what we want
expression A works like this:
1) (-3 >>> 0)
The >>> operation takes the left operand (-3), which is a signed integer, and simply shifts the bits 0 positions to the left (so the bits are unchanged), and the unsigned number corresponding to these unchanged bits.
The bit sequence of the signed number -3 is the same bit sequence as the unsigned number 4294967293, which is what node gives us if we simply type -3 >>> 0 into the REPL.
2) (-3 >>> 0).toString
Now, if we call toString on this unsigned number, we will just get the string representation of the bits of the number, which is the same sequence of bits as -3.
What we effectively did was say "hey toString, you have normal behavior when I tell you to print out the bits of an unsigned integer, so since I want to print out a signed integer, I'll just convert it to an unsigned integer, and you print the bits out for me."
Daan's answer explains it well.
toString(2) does not really convert the number to two's complement, instead it just do simple translation of the number to its positive binary form, while preserve the sign of it.
Example
Assume the given input is -15,
1. negative sign will be preserved
2. `15` in binary is 1111, therefore (-15).toString(2) gives output
-1111 (this is not in 2's complement!)
We know that in 2's complement of -15 in 32 bits is
11111111 11111111 11111111 11110001
Therefore in order to get the binary form of (-15), we can actually convert it to unsigned 32 bits integer using the unsigned right shift >>>, before passing it to toString(2) to print out the binary form. This is the reason we do (-15 >>> 0).toString(2) which will give us 11111111111111111111111111110001, the correct binary representation of -15 in 2's complement.