I want to write out to, and read in ieee format float/double values from a binary byte stream in a javascript module. Anyone know of how one would go about doing this? an equivalent to the doubleToRawLongBits and longBitsToDouble functionality in java.
You'll want to use a typed array for such things. Store your doubles into a Float64Array, then access the raw bytes from the underlying ArrayBuffer (and use bit shifts on those if you need single bits). If you're doing something more complicated, want to mix multiple types, or even control the endianness, overlay it with a DataView.
Related
I'm pushing the limits of HTML5 here.
I have this problem: I have an javascript array of a billion doubles (or ints), anyways A LOT of numbers. I want to store this in the HTML5 localStorage.
You may say, hey, just use JSON.Stringify, BUT, JSON.Stringify produces a huge 200MB string. Because, a number (0.03910319 for example), is stored as a string (so each number is taking up some bytes instead of just a few bytes for the whole number).
I was thinking about base64 encoding the numbers in the array first, and then applying JSON.stringify?
Or is it for example better to JSON.Stringify and then GZip or use some compression function?
Come up with your creative ideas to encode/decode an javascript array of A BILLION ints/doubles in an efficient matter to a localStorage variable.
TensorFlowJS
I looked at TensorflowJS, my array is basically a 1-D Tensor. Tensorflow has some storage capabilities for models... Maybe that is a feasible solution.
For anyone who is also dealing with this problem:
I used a Float32Array (javascript typed array) for my data.
A Float32Array is easily stored in IndexedDB using https://github.com/localForage/localForage
I have numbers generated from Javascript code and I want to store them in PostgreSQL table. I have legacy table where the whole JSON object is stored as JSONB type and in the new table I'd like to flatten the JSON to separate columns.
Ideally I want to avoid loss of precision as much as possible. Especially I'd like to avoid turning JS integer numbers into float numbers and vice versa. In other words inserting integer and getting back float is something I'd like to mitigate (if possible).
So far I've experimented with DOUBLE PRECISION and NUMERIC types. I think NUMERIC is better fit because documentation states that within the implementation limits there is no loss in precision. On the other hand DOUBLE PRECISION will be probably faster for numeric operations. I plan to do a lot of statistical operations.
I am not sure which one to choose. What is the optimal or recommended PostgreSQL data type with regards to maximum compatibility JavaScript Number type?
I am not JavaScript expert, but what I found on net, then JavaScript uses 64bit floats. It is same like DOUBLE PRECISION type - 8bytes like 8bytes.
The number is larger than 9223372036854775807 - too big for NumberLong, which is mongo's native 64-bit long type. What's the best way to do this/the best field type?
Is it possible to preserve any of the querying functionality of a smaller integer (such as {$lt})?
The big numbers are being generated by bignumber.js, and I'm using mongoose to interact with mongoDb.
I'm afraid that the only viable/safe option would be to store such big numbers as a string and serialize it back and forth between the application and MongoDB when read/written. However, you will loose the ability to use MongoDB built-in functions that work with numbers (you can still cast the values to numbers, but it won't be safe anymore).
I'm getting PCM audio samples in Float32Arrays. But my application can only support data with a 16 bit length so I need to convert.
I'm not sure how to do this since there is no typed array for a Float16Array and I've never worked with data on the bit level. Could someone explain or show how this could be done?
Firstoff, JavaScript is dynamically typed and has no concept of type length / size.
It is a matter of the runtime what type sizes may be used internally on the native platform, but that does not necessarily limit the possible values i.e the emulated type length if you will, inside the JavaScript runtime environment.
Also, probably you need 16 bit int to pass it to an audio device. Would make more sense.
So, I would narrow down your question to: how to convert float to int in JavaScript and the answer is here: How do I convert a float number to a whole number in JavaScript?
BUT if you really want to convert the precision of your values: What you could do is multiply by X, and round that ("convert to int", see link above), and then divide again by X. However this is not like converting a float of one size to another size because you dont take into account the mantisse/exponent representation of a float inside the actual memory. My solution only chops off some precision (quantization if you will).
Anyway, JavaScript is not a platform where you handle memory representation of types; it is already messy enough to handle the difference between float and int ;)
Looking for a library to work with large numbers on javascript (bigger than 2^53) I checked a couple of questions (JavaScript large number library? and Is there a bignum library for JavaScript?) and then tinkered a little bit with javascript-bignum.js and big.js, but the thing is that with I am unable to represent odd numbers, since both
Big(9007199254740995);
and
SchemeNumber.fn["string->number"](9007199254740995);
return
9007199254740996
rather than
9007199254740995
as I would expect.
So, is it that I am doing something wrong? Or there's no way to represent large odd numbers?
When you say this
Big(9007199254740995)
you are not giving the bignum library a chance! Your numeric literal is first parsed by pure JS, in which that number isn't exactly representable. You can see this simply with
window.alert(9007199254740995);
which alerts 9007199254740996.
In order to let your chosen bignum library successfully represent this number, you will need to pass it as a string, for example:
Big('9007199254740995')
should get you this exact number, as a bignum.