Related
I'm using this BigInteger.js for some calculations:
let myBigInt = bigInt(20).pow(200) // gets 160693804425899027554196209234116260252220299378279283530137600000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
I'd like to apply the logarithm to the big integer but in the docs I could not find any matching function. How can I implement a log(baseN, valueX) function for the BigInteger.js library?
Note: let myLogarithm = myBigInt.log(baseN) is not a valid implementation.
Note: After a lot of try&error I did found a working solution my own and I will post it here because I'm pretty sure there are a few more people then me that also gots faced with the same issue right there. So I hope, I could help :)
Have a look at wikipedia, as I did because theres a very nice article about baseConversion.
Below you can find a function for Math.log(base, value) that is able to calculate the log(base) from a value.
Math.log = (function() {
var log = Math.log;
return function(base, n) {
return log(n)/(base ? log(base) : 1);
};
})();
To calculate the logarithmToBaseN for bigInt-values just use this line of code:
let logarithmToBaseN = (myBigInt.toString().length * Math.log(baseN, 10) + Math.log(baseN, parseFloat("0." + myBigInt))) - 1);
Edit: This soltuion is a tiny workaround bacause parseFloat("0." + myBigInt) converts a big value like 100000 to a really small one like 0.100000,... what causes that it will be in integer precision.
According to #Jonas W's comment: The solution is very accurate for lower bases like (5, 10, ...) combined with low values like 10, 1000, 100000 - but for really big values like bigInt(20).pow(200) is it not.
Note: Using parseFloat (IEEE 754 double precision floating-point) means, you have a maximum of 52 bits of precision, which is a bit more than 15 decimal places. After that - the accuracy will be killed.
Note: For really big values bigInt(20).pow(200) combined with really big Bases like 100*(and more) it seems to be pretty accurate again.
Greetings, jonas.
Let me be brief. I'm trying to calculate
alert((Math.pow(7,35))%71)
but it gives me 61, when the result must be 70. What's wrong?
As others mentioned before with regards to using Math.pow(7,35), this resulting number is way too big for Javascript to handle.
To resolve your problem you need to use an external javascript library.
(Or write your own ;) )
Here are some examples of Javascript libraries that handle big numbers.
BigNum
Bignumber
I hope it helps.
The number you're using is too big for javascript. The max size of an int is 2^53 -- which is less than 7^35.
The only value which requires more precision is an intermediate result. So the problem can also be avoided without the need for higher precision variables when you have an algorithm that doesn't need the problematic intermediate result.
The following formula can be useful for that:
(a.b) % c = (a % c)(b % c) % c
This means Math.pow(7,35)%71 = ((Math.pow(7,17)%71) * (Math.pow(7,18)%71)) % 71.
Now the intermediate results are smaller, but might still be too big. So we need to split up further and to apply the modula operator on smaller intermediate results.
So you can do something like this:
Math.pow((Math.pow(7,7)%71),5)%71
But you probably need to do this for integer numbers wich are variable (otherwise, you could have avoided the problem by hardcoding the result).
So, you need to have an idea about the range of values you can expect and to define an algoritm that splits up the power calculation into pieces that will always have results that are small enough when the input is within the expected range.
And whatever you choose for a calculation like this, calculation with higher precision variables (using a specialized library) or a specilized algoritm, you should always assert the input to make sure your calculation returns a value only when you are sure it can deliver a correct value. Otherwise your implementation becomes unreliable for anyone who isn't aware of the range in which it is valid. Otherwise return an exception with a message that tells clearly which input or combination of inputs is the problem and why.
I am looking for a portable algorithm for creating a hashCode for binary data. None of the binary data is very long -- I am Avro-encoding keys for use in kafka.KeyedMessages -- we're probably talking anywhere from 2 to 100 bytes in length, but most of the keys are in the 4 to 8 byte range.
So far, my best solution is to convert the data to a hex string, and then do a hashCode of that. I'm able to make that work in both Scala and JavaScript. Assuming I have defined b: Array[Byte], the Scala looks like this:
b.map("%02X" format _).mkString.hashCode
It's a little more elaborate in JavaScript -- luckily someone already ported the basic hashCode algorithm to JavaScript -- but the point is being able to create a Hex string to represent the binary data, I can ensure the hashing algorithm works off the same inputs.
On the other hand, I have to create an object twice the size of the original just to create the hashCode. Luckily most of my data is tiny, but still -- there has to be a better way to do this.
Instead of padding the data as its hex value, I presume you could just coerce the binary data into a String so the String has the same number of bytes as the binary data. It would be all garbled, more control characters than printable characters, but it would be a string nonetheless. Do you run into portability issues though? Endian-ness, Unicode, etc.
Incidentally, if you got this far reading and don't already know this -- you can't just do:
val b: Array[Byte] = ...
b.hashCode
Luckily I already knew that before I started, because I ran into that one early on.
Update
Based on the first answer given, it appears at first blush that java.util.Arrays.hashCode(Array[Byte]) would do the trick. However, if you follow the javadoc trail, you'll see that this is the algorithm behind it, which is as based on the algorithm for List and the algorithm for byte combined.
int hashCode = 1;
for (byte e : list) hashCode = 31*hashCode + (e==null ? 0 : e.intValue());
As you can see, all it's doing is creating a Long representing the value. At a certain point, the number gets too big and it wraps around. This is not very portable. I can get it to work for JavaScript, but you have to import the npm module long. If you do, it looks like this:
function bufferHashCode(buffer) {
const Long = require('long');
var hashCode = new Long(1);
for (var value of buff.values()) { hashCode = hashCode.multiply(31).add(value) }
return hashCode
}
bufferHashCode(new Buffer([1,2,3]));
// hashCode = Long { low: 30817, high: 0, unsigned: false }
And you do get the same results when the data wraps around, sort of, though I'm not sure why. In Scala:
java.util.Arrays.hashCode(Array[Byte](1,2,3,4,5,6,7,8,9,10))
// res30: Int = -975991962
Note that the result is an Int. In JavaScript:
bufferHashCode(new Buffer([1,2,3,4,5,6,7,8,9,10]);
// hashCode = Long { low: -975991962, high: 197407, unsigned: false }
So I have to take the low bytes and ignore the high, but otherwise I get the same results.
This functionality is already available in Java standard library, look at the Arrays.hashCode() method.
Because your binary data are Array[Byte], here is how you can verify it works:
println(java.util.Arrays.hashCode(Array[Byte](1,2,3))) // prints 30817
println(java.util.Arrays.hashCode(Array[Byte](1,2,3))) // prints 30817
println(java.util.Arrays.hashCode(Array[Byte](2,2,3))) // prints 31778
Update: It is not true that the Java implementation boxes the bytes. Of course, there is conversion to int, but there's no way around that. This is the Java implementation:
public static int hashCode(byte a[]) {
if (a == null) return 0;
int result = 1;
for (byte element : a) result = 31 * result + element;
return result;
}
Update 2
If what you need is a JavaScript implementation that gives the same results as a Scala/Java implementation, than you can extend the algorithm by, e.g., taking only the rightmost 31 bits:
def hashCode(a: Array[Byte]): Int = {
if (a == null) {
0
} else {
var hash = 1
var i: Int = 0
while (i < a.length) {
hash = 31 * hash + a(i)
hash = hash & Int.MaxValue // taking only the rightmost 31 bits
i += 1
}
hash
}
}
and JavaScript:
var hashCode = function(arr) {
if (arr == null) return 0;
var hash = 1;
for (var i = 0; i < arr.length; i++) {
hash = hash * 31 + arr[i]
hash = hash % 0x80000000 // taking only the rightmost 31 bits in integer representation
}
return hash;
}
Why do the two implementations produce the same results? In Java, integer overflow behaves as if the addition was performed without loss of precision and then bits higher than 32 got thrown away and & Int.MaxValue throws away the 32nd bit. In JavaScript, there is no loss of precision for integers up to 253 which is a limit the expression 31 * hash + a(i) never exceeds. % 0x80000000 then behaves as taking the rightmost 31 bits. The case without overflows is obvious.
This is the meat of algorithm used in the Java library:
int result 1;
for (byte element : a) result = 31 * result + element;
You comment:
this algorithm isn't very portable
Incorrect. If we are talking about Java, then provided that we all agree on the type of the result, then the algorithm is 100% portable.
Yes the computation overflows, but it overflows exactly the same way on all valid implementations of the Java language. A Java int is specified to be 32 bits signed two's complement, and the behavior of the operators when overflow occurs is well-defined ... and the same for all implementations. (The same goes for long ... though the size is different, obviously.)
I'm not an expert, but my understanding is that Scala's numeric types have the same properties as Java. Javascript is different, being based on IEE 754 double precision floating point. However, with case you should be able to code the Java algorithm portably in Javascript. (I think #Mifeet's version is wrong ...)
How do I generate cryptographically secure random numbers in javascript?
There's been discussion at WHATWG on adding this to the window.crypto object. You can read the discussion and check out the proposed API and webkit bug (22049).
Just tested the following code in Chrome to get a random byte:
(function(){
var buf = new Uint8Array(1);
window.crypto.getRandomValues(buf);
alert(buf[0]);
})();
In order, I think your best bets are:
window.crypto.getRandomValues or window.msCrypto.getRandomValues
The sjcl library's randomWords function (http://crypto.stanford.edu/sjcl/)
The isaac library's random number generator (which is seeded by Math.random, so not really cryptographically secure) (https://github.com/rubycon/isaac.js)
window.crypto.getRandomValues has been implemented in Chrome for a while now, and relatively recently in Firefox as well. Unfortunately, Internet Explorer 10 and before do not implement the function. IE 11 has window.msCrypto, which accomplishes the same thing. sjcl has a great random number generator seeded from mouse movements, but there's always a chance that either the mouse won't have moved sufficiently to seed the generator, or that the user is on a mobile device where there is no mouse movement whatsoever. Thus, I recommend having a fallback case where you can still get a non-secure random number if there is no choice. Here's how I've handled this:
function GetRandomWords (wordCount) {
var randomWords;
// First we're going to try to use a built-in CSPRNG
if (window.crypto && window.crypto.getRandomValues) {
randomWords = new Int32Array(wordCount);
window.crypto.getRandomValues(randomWords);
}
// Because of course IE calls it msCrypto instead of being standard
else if (window.msCrypto && window.msCrypto.getRandomValues) {
randomWords = new Int32Array(wordCount);
window.msCrypto.getRandomValues(randomWords);
}
// So, no built-in functionality - bummer. If the user has wiggled the mouse enough,
// sjcl might help us out here
else if (sjcl.random.isReady()) {
randomWords = sjcl.random.randomWords(wordCount);
}
// Last resort - we'll use isaac.js to get a random number. It's seeded from Math.random(),
// so this isn't ideal, but it'll still greatly increase the space of guesses a hacker would
// have to make to crack the password.
else {
randomWords = [];
for (var i = 0; i < wordCount; i++) {
randomWords.push(isaac.rand());
}
}
return randomWords;
};
You'll need to include sjcl.js and isaac.js for that implementation, and be sure to start the sjcl entropy collector as soon as your page is loaded:
sjcl.random.startCollectors();
sjcl is dual-licensed BSD and GPL, while isaac.js is MIT, so it's perfectly safe to use either of those in any project. As mentioned in another answer, clipperz is another option, however for whatever bizarre reason, it is licensed under the AGPL. I have yet to see anyone who seems to understand what implications that has for a JavaScript library, but I'd universally avoid it.
One way to improve the code I've posted might be to store the state of the isaac random number generator in localStorage, so it isn't reseeded every time the page is loaded. Isaac will generate a random sequence, but for cryptography purposes, the seed is all-important. Seeding with Math.random is bad, but at least a little less bad if it isn't necessarily on every page load.
You can for instance use mouse movement as seed for random numbers, read out time and mouse position whenever the onmousemove event happens, feed that data to a whitening function and you will have some first class random at hand. Though do make sure that user has moved the mouse sufficiently before you use the data.
Edit: I have myself played a bit with the concept by making a password generator, I wouldn't guarantee that my whitening function is flawless, but being constantly reseeded I'm pretty sure that it's plenty for the job: ebusiness.hopto.org/generator.htm
Edit2: It now sort of works with smartphones, but only by disabling touch functionality while the entropy is gathered. Android won't work properly any other way.
Use window.crypto.getRandomValues, like this:
var random_num = new Uint8Array(2048 / 8); // 2048 = number length in bits
window.crypto.getRandomValues(random_num);
This is supported in all modern browsers and uses the operating system's random generator (e.g. /dev/urandom). If you need IE11 compatibility, you have to use their prefixed implementation viavar crypto = window.crypto || window.msCrypto; crypto.getRandomValues(..) though.
Note that the window.crypto API can also generate keys outright, which may be the better option.
Crypto-strong
to get cryptographic strong number from range [0, 1) (similar to Math.random()) use crypto:
let random = ()=> crypto.getRandomValues(new Uint32Array(1))[0]/2**32;
console.log( random() );
You might want to try
http://sourceforge.net/projects/clipperzlib/
It has an implementation of Fortuna which is a cryptographically secure random number generator. (Take a look at src/js/Clipperz/Crypto/PRNG.js). It appears to use the mouse as a source of randomness as well.
First of all, you need a source of entropy. For example, movement of the mouse, password, or any other. But all of these sources are very far from random, and guarantee you 20 bits of entropy, rarely more. The next step that you need to take is to use the mechanism like "Password-Based KDF" it will make computationally difficult to distinguish data from random.
Many years ago, you had to implement your own random number generator and seed it with entropy collected by mouse movement and timing information. This was the Phlogiston Era of JavaScript cryptography. These days we have window.crypto to work with.
If you need a random integer, random-number-csprng is a great choice. It securely generates a series of random bytes and then converts it into an unbiased random integer.
const randomInt = require("random-number-csprng");
(async function() {
let random = randomInt(10, 30);
console.log(`Your random number: ${random}`);
})();
If you need a random floating point number, you'll need to do a little more work. Generally, though, secure randomness is an integer problem, not a floating point problem.
I know i'm late to the party, but if you don't want to deal with the math of getting a cryptographically secure random value, i recommend using rando.js. it's a super small 2kb library that'll give you a decimal, pick something from an array, or whatever else you want- all cryptographically secure.
It's on npm too.
Here's a sample I copied from the GitHub, but it does more than this if you want to go there and read about it more.
console.log(rando()); //a floating-point number between 0 and 1 (could be exactly 0, but never exactly 1)
console.log(rando(5)); //an integer between 0 and 5 (could be 0 or 5)
console.log(rando(5, 10)); //a random integer between 5 and 10 (could be 5 or 10)
console.log(rando(5, "float")); //a floating-point number between 0 and 5 (could be exactly 0, but never exactly 5)
console.log(rando(5, 10, "float")); //a floating-point number between 5 and 10 (could be exactly 5, but never exactly 10)
console.log(rando(true, false)); //either true or false
console.log(rando(["a", "b"])); //{index:..., value:...} object representing a value of the provided array OR false if array is empty
console.log(rando({a: 1, b: 2})); //{key:..., value:...} object representing a property of the provided object OR false if object has no properties
console.log(rando("Gee willikers!")); //a character from the provided string OR false if the string is empty. Reoccurring characters will naturally form a more likely return value
console.log(rando(null)); //ANY invalid arguments return false
<script src="https://randojs.com/2.0.0.js"></script>
If you need large amounts, here's what I would do:
// Max value of random number length
const randLen = 16384
var randomId = randLen
var randomArray = new Uint32Array(randLen)
function random32() {
if (randomId === randLen) {
randomId = 0
return crypto.getRandomValues(randomArray)[randomId++] * 2.3283064365386963e-10
}
return randomArray[randomId++] * 2.3283064365386963e-10
}
function random64() {
if (randomId === randLen || randomId === randLen - 1) {
randomId = 0
crypto.getRandomValues(randomArray)
}
return randomArray[randomId++] * 2.3283064365386963e-10 + randomArray[randomId++] * 5.421010862427522e-20
}
console.log(random32())
console.log(random64())
How do I generate cryptographically secure random numbers in javascript?
There's been discussion at WHATWG on adding this to the window.crypto object. You can read the discussion and check out the proposed API and webkit bug (22049).
Just tested the following code in Chrome to get a random byte:
(function(){
var buf = new Uint8Array(1);
window.crypto.getRandomValues(buf);
alert(buf[0]);
})();
In order, I think your best bets are:
window.crypto.getRandomValues or window.msCrypto.getRandomValues
The sjcl library's randomWords function (http://crypto.stanford.edu/sjcl/)
The isaac library's random number generator (which is seeded by Math.random, so not really cryptographically secure) (https://github.com/rubycon/isaac.js)
window.crypto.getRandomValues has been implemented in Chrome for a while now, and relatively recently in Firefox as well. Unfortunately, Internet Explorer 10 and before do not implement the function. IE 11 has window.msCrypto, which accomplishes the same thing. sjcl has a great random number generator seeded from mouse movements, but there's always a chance that either the mouse won't have moved sufficiently to seed the generator, or that the user is on a mobile device where there is no mouse movement whatsoever. Thus, I recommend having a fallback case where you can still get a non-secure random number if there is no choice. Here's how I've handled this:
function GetRandomWords (wordCount) {
var randomWords;
// First we're going to try to use a built-in CSPRNG
if (window.crypto && window.crypto.getRandomValues) {
randomWords = new Int32Array(wordCount);
window.crypto.getRandomValues(randomWords);
}
// Because of course IE calls it msCrypto instead of being standard
else if (window.msCrypto && window.msCrypto.getRandomValues) {
randomWords = new Int32Array(wordCount);
window.msCrypto.getRandomValues(randomWords);
}
// So, no built-in functionality - bummer. If the user has wiggled the mouse enough,
// sjcl might help us out here
else if (sjcl.random.isReady()) {
randomWords = sjcl.random.randomWords(wordCount);
}
// Last resort - we'll use isaac.js to get a random number. It's seeded from Math.random(),
// so this isn't ideal, but it'll still greatly increase the space of guesses a hacker would
// have to make to crack the password.
else {
randomWords = [];
for (var i = 0; i < wordCount; i++) {
randomWords.push(isaac.rand());
}
}
return randomWords;
};
You'll need to include sjcl.js and isaac.js for that implementation, and be sure to start the sjcl entropy collector as soon as your page is loaded:
sjcl.random.startCollectors();
sjcl is dual-licensed BSD and GPL, while isaac.js is MIT, so it's perfectly safe to use either of those in any project. As mentioned in another answer, clipperz is another option, however for whatever bizarre reason, it is licensed under the AGPL. I have yet to see anyone who seems to understand what implications that has for a JavaScript library, but I'd universally avoid it.
One way to improve the code I've posted might be to store the state of the isaac random number generator in localStorage, so it isn't reseeded every time the page is loaded. Isaac will generate a random sequence, but for cryptography purposes, the seed is all-important. Seeding with Math.random is bad, but at least a little less bad if it isn't necessarily on every page load.
You can for instance use mouse movement as seed for random numbers, read out time and mouse position whenever the onmousemove event happens, feed that data to a whitening function and you will have some first class random at hand. Though do make sure that user has moved the mouse sufficiently before you use the data.
Edit: I have myself played a bit with the concept by making a password generator, I wouldn't guarantee that my whitening function is flawless, but being constantly reseeded I'm pretty sure that it's plenty for the job: ebusiness.hopto.org/generator.htm
Edit2: It now sort of works with smartphones, but only by disabling touch functionality while the entropy is gathered. Android won't work properly any other way.
Use window.crypto.getRandomValues, like this:
var random_num = new Uint8Array(2048 / 8); // 2048 = number length in bits
window.crypto.getRandomValues(random_num);
This is supported in all modern browsers and uses the operating system's random generator (e.g. /dev/urandom). If you need IE11 compatibility, you have to use their prefixed implementation viavar crypto = window.crypto || window.msCrypto; crypto.getRandomValues(..) though.
Note that the window.crypto API can also generate keys outright, which may be the better option.
Crypto-strong
to get cryptographic strong number from range [0, 1) (similar to Math.random()) use crypto:
let random = ()=> crypto.getRandomValues(new Uint32Array(1))[0]/2**32;
console.log( random() );
You might want to try
http://sourceforge.net/projects/clipperzlib/
It has an implementation of Fortuna which is a cryptographically secure random number generator. (Take a look at src/js/Clipperz/Crypto/PRNG.js). It appears to use the mouse as a source of randomness as well.
First of all, you need a source of entropy. For example, movement of the mouse, password, or any other. But all of these sources are very far from random, and guarantee you 20 bits of entropy, rarely more. The next step that you need to take is to use the mechanism like "Password-Based KDF" it will make computationally difficult to distinguish data from random.
Many years ago, you had to implement your own random number generator and seed it with entropy collected by mouse movement and timing information. This was the Phlogiston Era of JavaScript cryptography. These days we have window.crypto to work with.
If you need a random integer, random-number-csprng is a great choice. It securely generates a series of random bytes and then converts it into an unbiased random integer.
const randomInt = require("random-number-csprng");
(async function() {
let random = randomInt(10, 30);
console.log(`Your random number: ${random}`);
})();
If you need a random floating point number, you'll need to do a little more work. Generally, though, secure randomness is an integer problem, not a floating point problem.
I know i'm late to the party, but if you don't want to deal with the math of getting a cryptographically secure random value, i recommend using rando.js. it's a super small 2kb library that'll give you a decimal, pick something from an array, or whatever else you want- all cryptographically secure.
It's on npm too.
Here's a sample I copied from the GitHub, but it does more than this if you want to go there and read about it more.
console.log(rando()); //a floating-point number between 0 and 1 (could be exactly 0, but never exactly 1)
console.log(rando(5)); //an integer between 0 and 5 (could be 0 or 5)
console.log(rando(5, 10)); //a random integer between 5 and 10 (could be 5 or 10)
console.log(rando(5, "float")); //a floating-point number between 0 and 5 (could be exactly 0, but never exactly 5)
console.log(rando(5, 10, "float")); //a floating-point number between 5 and 10 (could be exactly 5, but never exactly 10)
console.log(rando(true, false)); //either true or false
console.log(rando(["a", "b"])); //{index:..., value:...} object representing a value of the provided array OR false if array is empty
console.log(rando({a: 1, b: 2})); //{key:..., value:...} object representing a property of the provided object OR false if object has no properties
console.log(rando("Gee willikers!")); //a character from the provided string OR false if the string is empty. Reoccurring characters will naturally form a more likely return value
console.log(rando(null)); //ANY invalid arguments return false
<script src="https://randojs.com/2.0.0.js"></script>
If you need large amounts, here's what I would do:
// Max value of random number length
const randLen = 16384
var randomId = randLen
var randomArray = new Uint32Array(randLen)
function random32() {
if (randomId === randLen) {
randomId = 0
return crypto.getRandomValues(randomArray)[randomId++] * 2.3283064365386963e-10
}
return randomArray[randomId++] * 2.3283064365386963e-10
}
function random64() {
if (randomId === randLen || randomId === randLen - 1) {
randomId = 0
crypto.getRandomValues(randomArray)
}
return randomArray[randomId++] * 2.3283064365386963e-10 + randomArray[randomId++] * 5.421010862427522e-20
}
console.log(random32())
console.log(random64())