We Keep Coding

How to deal with big numeric values from response in javascript?

I have to read a JSON response which contains the value which is greater than MAX_SAFE_INTEGER without loss of precision. Like I had values
value1 = 232333433534534634534411
And I have to process that value and convert to
value2 = +232333433534534634534411.0000
without any loss of precision?

This can't be done with standard JSON.parse:
JSON.parse('{"prop": 232333433534534634534411}')
// {prop: 2.3233343353453462e+23}
If you have control over the API producing the JSON
You can use a string instead, then create a BigInt out of it (assuming legacy browser support isn't required).
const result = JSON.parse('{"prop": "232333433534534634534411"}')
const int = BigInt(result.prop)
// 232333433534534634534411n
If you need to perform decimal arithmetic on this result with a precision of 4 decimal places, you can multiply it by 10000n, for example:
const tenThousandths = int * 10000n // 2323334335345346345344110000n
const sum = tenThousandths + 55000n + 21n * 2n // 2323334335345346345344165042n
const fractionalPart = sum % 10000n // 5042n
const wholePart = sum / 10000n // 232333433534534634534416n (floor division)
const reStringified = `${wholePart}.${fractionalPart}` // "232333433534534634534416.5042"
const newJson = JSON.stringify({prop: reStringified})
// '{"prop":"232333433534534634534416.5042"}'
For legacy browser support, you could do similar with a library such as BigInteger.js.
If you don't control the API
In this case, you'll need a custom JSON parsing library, such as lossless JSON.

You could split by dot and treat the digits after the dot.
const convert = string => {
const values = string.split('.');
values[1] = (values[1] || '').padEnd(4, 0).slice(0, 4);
return values.join('.');
};
console.log(convert('232333433534534634534411'));
console.log(convert('232333433534534634534411.12345'));

JavaScript: reading 3 bytes Buffer as an integer

Let's say I have a hex data stream, which I want to divide into 3-bytes blocks which I need to read as an integer.
For example: given a hex string 01be638119704d4b9a I need to read the first three bytes 01be63 and read it as integer 114275. This is what I got:
var sample = '01be638119704d4b9a';
var buffer = new Buffer(sample, 'hex');
var bufferChunk = buffer.slice(0, 3);
var decimal = bufferChunk.readUInt32BE(0);
The readUInt32BE works perfectly for 4-bytes data, but here I obviously get:
RangeError: index out of range
at checkOffset (buffer.js:494:11)
at Buffer.readUInt32BE (buffer.js:568:5)
How do I read 3-bytes as integer correctly?

If you are using node.js v0.12+ or io.js, there is buffer.readUIntBE() which allows a variable number of bytes:
var decimal = buffer.readUIntBE(0, 3);
(Note that it's readUIntBE for Big Endian and readUIntLE for Little Endian).
Otherwise if you're on an older version of node, you will have to do it manually (check bounds first of course):
var decimal = (buffer[0] << 16) + (buffer[1] << 8) + buffer[2];

I'm using this, if someone knows something wrong with it, please advise;
const integer = parseInt(buffer.toString("hex"), 16)

you should convert three byte to four byte.
function three(var sample){
var buffer = new Buffer(sample, 'hex');
var buf = new Buffer(1);
buf[0] = 0x0;
return Buffer.concat([buf, buffer.slice(0, 3)]).readUInt32BE();
}
You can try this function.

Convert 64 bit Steam ID to 32 bit account ID

How do I convert a 64 bit steam ID to a 32 bit account ID? Steam says to take the first 32 bits of the number, but how do you do this in Node?
Do I need to use BigNumber to store the 64 bit int?

To convert a 64 bit Steam ID to a 32 bit Account ID, you can just subtract 76561197960265728 from the 64 bit id.
This requires bigNumber in node:
bignumber = require("bignumber.js");
console.log(bignumber('76561197991791363').minus('76561197960265728'))

I had the same issue but didn't want to use any library like bignumber.js as my project was quite small and will be used in a web browser. In the end I came up with this elegant solution:
function steamID64toSteamID32 (steamID64) {
return Number(steamID64.substr(-16,16)) - 6561197960265728
}
How it works:
To get the lower 32 bits we need to convert the SteamID64 string to a number, but because JavaScript has a precision limit of 57 bits the SteamID64 will be erroneously rounded. The workaround is to truncate the leftmost digits to get a 16 digit number, which uses at most 54 bits and will therefore retain its precision in Javascript. This is acceptable because the leftmost digits comes from the higher 32 bits which will be zeroed anyway, so nothing of value will be lost.
To zero the remaining higher bits we subtract the decimal number they're representing. If we assume that every SteamID64 we're converting to be in the public universe this decimal number will be constant and can be computed like this:
1. 0b00000001000100000000000000000001 0b00000000000000000000000000000000 = 76561197960265728
2. Number('76561197960265728'.substr(-16,16)) = 6561197960265728

Here's what I came up with. I started learning JavaScript yesterday (coming from a C++ background, not very accustomed to working without types), so correct me if I did something derpy with the language. I tested it with my own steam ID and it seems to work.
// NOTE: Functions can take in a steamID in its packed 64-bit form
// (community ID starting with 765), its modern form with or without
// either or both brackets, and its legacy form. SteamID's that
// contain letters (e.g. STEAM_0... or [U:1...) are not case-sensitive.
// Dependencies: BigInteger library, available from http://silentmatt.com/biginteger/
// Global variable used by all conversion functions
var STEAM_BASELINE = '76561197960265728';
// IN: String containing a steamID in any of the 3 formats
// OUT: String containing the steamID as a community ID (64-bit packed ID)
function ConvertToPacked(inputID)
{
var output = "unknown";
// From packed
if(inputID.match(/^765/) && inputID.length > 15)
{
output = inputID;
}
// From modern
else if(inputID.match(/^\[U:1:/i) || inputID.match(/^U:1:/i))
{
var numericPortion = inputID.replace(/^\[U:1:|^U:1:/i,'').replace(/\]/,'');
output = BigInteger.add(numericPortion, STEAM_BASELINE).toString();
}
// From legacy
else if(inputID.match(/^STEAM_0:[0-1]:/i))
{
var splitID = inputID.split(":");
var product = BigInteger.multiply(splitID[2],2);
var sum = BigInteger.add(product, STEAM_BASELINE);
output = BigInteger.add(sum, splitID[1]).toString();
}
return output;
}
// IN: String containing a steamID in any of the 3 formats
// OUT: String containing the steamID in the modern format (e.g. [U:1:123456])
function ConvertToModern(inputID)
{
var output = "unknown";
// From packed
if(inputID.match(/^765/) && inputID.length > 15)
{
output = "[U:1:" + BigInteger.subtract(inputID, STEAM_BASELINE).toString() + "]";
}
// From modern
else if(inputID.match(/^\[U:1:/i) || inputID.match(/^U:1:/i))
{
var numericPortion = inputID.replace(/^\[U:1:|^U:1:/i,'').replace(/\]/,'');
output = "[U:1:" + numericPortion + "]";
}
// From legacy
else if(inputID.match(/^STEAM_0:[0-1]:/i))
{
var splitID = inputID.split(":");
var numeric = BigInteger.add(BigInteger.multiply(splitID[2],2), splitID[1]);
output = "[U:1:" + numeric.toString() + "]";
}
return output;
}
// IN: String containing a steamID in any of the 3 formats
// OUT: String containing the steamID in the legacy format (e.g. STEAM_0:0:123456)
function ConvertToLegacy(inputID)
{
var output = "unknown"
// From packed
if(inputID.match(/^765/) && inputID.length > 15)
{
var z = BigInteger.divide(BigInteger.subtract(inputID, STEAM_BASELINE), 2);
var y = BigInteger.remainder(inputID, 2);
output = 'STEAM_0:' + y.toString() + ':' + z.toString();
}
// From modern
else if(inputID.match(/^\[U:1:/i) || inputID.match(/^U:1:/i))
{
var numericPortion = inputID.replace(/^\[U:1:|^U:1:/i,'').replace(/\]/,'');
var z = BigInteger.divide(numericPortion, 2);
var y = BigInteger.remainder(numericPortion, 2);
output = 'STEAM_0:' + y.toString() + ':' + z.toString();
}
// From legacy
else if(inputID.match(/^STEAM_0:[0-1]:/i))
{
output = inputID.toUpperCase();
}
return output;
}

Generating a unique ID in Javascript

I want to implement a saving system similar to Imgur where if a user presses a button a unique 5 character value is returned. Here is what I have so far:
The database backend uses auto-incrementing ID's starting at 5308416. I use a modified Radix function (see below) to convert these numerical ID's into characters. I use a reverse function to lookup character ID's back to numerical database ID's.
function genID (value)
{
var alphabet = "23456789BCDFGHJKLMNPRSTVWXYZbcdfghjkmnpqrstvwxyz";
var result = "";
var length = alphabet.length;
while (value > 0)
{
result = alphabet[value % length] + result;
value = Math.floor (value / length);
}
return result;
}
The problem is that these generated ID's are very much predictable. My question is, how can I make the generated ID's seem random but still unique (so I can look them up in the database as numbers). I was thinking of using some encryption algorithm but not sure where to start. Any help or suggestions would be appreciated (maybe there is a better way of doing this also).

Do you have to be able to go both ways (i.e. convert an integer to it's hash and back again)? If you can store the hash and lookup the content that way, then it's relatively easy to create a function that produces a hard-to-guess, but complete hash space. You use primes to generate a sequence that only repeats once all possible permutations are exhausted.
The following PHP example is from my own code, adapted from this site:
function hash($len = 6) {
$base = 36;
$gp = array(1,23,809,28837,1038073,37370257 /*,1345328833*/);
$maxlen = count($gp);
$len = $len > ($maxlen-1) ? ($maxlen-1) : $len;
while($len < $maxlen && pow($base,$len) < $this->ID) $len++;
if($len >= $maxlen) throw new Exception($this->ID." out of range (max ".pow($base,$maxlen-1).")");
$ceil = pow($base,$len);
$prime = $gp[$len];
$dechash = ($this->ID * $prime) % $ceil;
$hash = base_convert($dechash, 10, $base);
return str_pad($hash, $len, "0", STR_PAD_LEFT);
}
It would be easy enough to implement that in JavaScript, but ideally you wouldn't need too - you'd have an insert trigger on your table that populated a hash field with the result of that algorithm (adapted for SQL, of course).

A non-predictable, but unique ID can be made by combining your server-side auto-incrementing number with either a current date/time nugget or with a random number. The server-side auto-incrementing number guarantees uniqueness and the date/time nugget or random number removes the predictability.
For a unique ID in string form that takes the server-side unique number as input and where you add the date/time nugget on the client you can do this:
function genID(serverNum) {
return(serverNum + "" + (new Date).getTime());
}
Or using a random number:
function genID(serverNum) {
return(serverNum + "" + Math.floor(Math.random() * 100000));
}
But, it might be best to add the date/time element on the server and just store that whole unique ID in the database there.

Best way to generate unique ids client-side (with Javascript)

I need to generate unique ids in the browser. Currently, I'm using this:
Math.floor(Math.random() * 10000000000000001)
I'd like to use the current UNIX time ((new Date).getTime()), but I'm worried that if two clients generate ids at the exact same time, they wouldn't be unique.
Can I use the current UNIX time (I'd like to because that way ids would store more information)? If not, what's the best way to do this (maybe UNIX time + 2 random digits?)

you can create a GUID using the following links:
http://softwareas.com/guid0-a-javascript-guid-generator
Create GUID / UUID in JavaScript?
This will maximise your chance of "uniqueness."
Alternatively, if it is a secure page, you can concatenate the date/time with the username to prevent multiple simultaneous generated values.

https://github.com/uuidjs/uuid provides RFC compliant UUIDs based on either timestamp or random #'s. Single-file with no dependencies, supports timestamp or random #-based UUIDs, uses native APIs for crypto-quality random numbers if available, plus other goodies.

In modern browser you can use crypto:
var array = new Uint32Array(1);
window.crypto.getRandomValues(array);
console.log(array);

var c = 1;
function cuniq() {
var d = new Date(),
m = d.getMilliseconds() + "",
u = ++d + m + (++c === 10000 ? (c = 1) : c);
return u;
}

Here is my javascript code to generate guid. It does quick hex mapping and very efficient:
AuthenticationContext.prototype._guid = function () {
// RFC4122: The version 4 UUID is meant for generating UUIDs from truly-random or
// pseudo-random numbers.
// The algorithm is as follows:
// Set the two most significant bits (bits 6 and 7) of the
// clock_seq_hi_and_reserved to zero and one, respectively.
// Set the four most significant bits (bits 12 through 15) of the
// time_hi_and_version field to the 4-bit version number from
// Section 4.1.3. Version4
// Set all the other bits to randomly (or pseudo-randomly) chosen
// values.
// UUID = time-low "-" time-mid "-"time-high-and-version "-"clock-seq-reserved and low(2hexOctet)"-" node
// time-low = 4hexOctet
// time-mid = 2hexOctet
// time-high-and-version = 2hexOctet
// clock-seq-and-reserved = hexOctet:
// clock-seq-low = hexOctet
// node = 6hexOctet
// Format: xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx
// y could be 1000, 1001, 1010, 1011 since most significant two bits needs to be 10
// y values are 8, 9, A, B
var guidHolder = 'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx';
var hex = '0123456789abcdef';
var r = 0;
var guidResponse = "";
for (var i = 0; i < 36; i++) {
if (guidHolder[i] !== '-' && guidHolder[i] !== '4') {
// each x and y needs to be random
r = Math.random() * 16 | 0;
}
if (guidHolder[i] === 'x') {
guidResponse += hex[r];
} else if (guidHolder[i] === 'y') {
// clock-seq-and-reserved first hex is filtered and remaining hex values are random
r &= 0x3; // bit and with 0011 to set pos 2 to zero ?0??
r |= 0x8; // set pos 3 to 1 as 1???
guidResponse += hex[r];
} else {
guidResponse += guidHolder[i];
}
}
return guidResponse;
};

You can always run a test against existing IDs in the set to accept or reject the generated random number recursively.
for example:
const randomID = function(){
let id = Math.floor(Math.random() * 10000000000000001) + new Date();
if (idObjectArray.contains(id)) {
randomID;
} else {
idObjectArray.push(id);
}
};
This example assumes you would just be pushing the id into a 1D array, but you get the idea. There shouldn't be many collisions given the uniqueness of the random number with the date, so it should be efficient.

There are two ways to achieve this
js const id = Date.now().toString()
While this does not guarantee uniqueness (When you are creating multiple objects within 1ms), this will work on a practical level, since it is usually not long before the objects on the client are sent to a real server.
If you wanted to create multiple records withing 1ms, I suggest using the code below
const { randomBytes } = require("crypto");
// 32 Characters
const id = randomBytes(16).toString("hex");
It works similar to a uuid4 without needing to add an external library (Assuming you have access to NodeJs at some point)