We Keep Coding

Convert Base64 String to Javascript Uint16Array?

I'm working in an Javascript application that receives a base64 array. This array encodes a 16 bits per pixel raw image.
Thus, I would like to do some calculations in it. For this, I need to unpack this Base64 string in a Uint16Array, so I can iterate over pixels and perform the required calculations.
What are my options for doing that?

After some hours looking for a solution, I found a way for doing this:
function getData()
{
fetch("test_data/img_base64.txt")
.then(res => res.text())
.then((out) => {
rawString = window.atob(out);
uint8Array = new Uint8Array(rawString.length);
for(var i = 0; i < rawString.length; i++)
{
uint8Array[i] = rawString.charCodeAt(i);
}
uint16Array = new Uint16Array(uint8Array.buffer);
console.log(uint16Array);
})
.catch(err => { throw err });
}
First I fetch my base64 string from a file. Then using window.atob it is converted to a JavaScript string. After this, I need to fill a Uint8Array with every byte loaded from the string. Finally, I had to convert this Uint8Array, into the final Uint16Array.
That was tough to achieve exactly what I was looking. But I have found it.

You can use this function that converts a base64 string into a binary Uint16 Array
var BASE64_MARKER = ';base64,';
function convertDataURIToBinary(dataURI) {
var base64Index = dataURI.indexOf(BASE64_MARKER) + BASE64_MARKER.length;
var base64 = dataURI.substring(base64Index);
var raw = window.atob(base64);
var rawLength = raw.length;
var array = new Uint16Array(new ArrayBuffer(rawLength));
for(i = 0; i < rawLength; i++) {
array[i] = raw.charCodeAt(i);
}
return array;
}
If you're targeting Firefox and feeling adventurous you can shorten the function down to this:
var BASE64_MARKER = ';base64,';
function convertDataURIToBinaryFF(dataURI) {
var base64Index = dataURI.indexOf(BASE64_MARKER) + BASE64_MARKER.length;
var raw = window.atob(dataURI.substring(base64Index));
return Uint8Array.from(Array.prototype.map.call(raw, function(x) {
return x.charCodeAt(0);
}));
};

Looping through an array that has binary numbers and writing true if the number is 1 or false if the number is 0

I am trying to make a webpage that makes an encrypted letter by first parsing a single character in ascii then parsing the ascii into binary then putting the binary into an array. After putting it into an array I have to loop through the array and write true for "1" or false for "0". Then I have to output to the page. an example of what the output would look like if you put in the letter "a" would be "false,true,true,false,false,false,false,true"
Update: I have added the "loop" in order to make sense of my problem
$(document).ready(function()
{
var output = document.getElementById("output");
var strQuestion = "Enter ONE character, matey!";
var strStandard = "J";
var chrCharacter = "";
var chrLength = 0;
var array = [];
var arrayLength = 0;
while (chrLength != 1)
{
chrCharacter = prompt(strQuestion, strStandard);
chrLength = chrCharacter.length;
}
intAscii = parseAscii(chrCharacter);
strBin = parseBin(intAscii);
array = strBin.split("");
for (i = 0; i < arrayLength; i++ )
{
if (array[i] = 0)
{
array[i] = false;
}
else if (array[i] = 1)
{
array[i] = true;
}
}
output.innerHTML = array;
}); //end document.ready
/*****
Purpose: Converts a character into ascii
Parameters: single character / letter
Return: integer representing an ascii value
*****/
function parseAscii(chrCharacter)
{
intAscii = chrCharacter.charCodeAt(0);
return intAscii;
}
/*****
Purpose: Takes the ascii code and turns it into binary
Parameters: single integer representing an ascii value
Return: binary, base 2 representation of the number passed to this function
*****/
function parseBin(intAscii)
{
strBin = parseInt(intAscii, 10).toString(2);
if(strBin.length < 8)
{
var intPlaceHolders = 8 - strBin.length;
for(var i = 0; i < intPlaceHolders; i++)
{
strBin = "0" + strBin;
}
}
return strBin;
}

I would convert the array with binaries to an array with boolean values wich you can joint together to a string that can be shown on the webpage.
array = [1,1,0,0,1]
// This will map over the items and perform an type conversion
var booleanArray = array.map(Boolean)
// Join all the items together as a string
Var booleanString = booleanArray.join(", ")
output.innerHTML = booleanString
`
I didn't test it, but it should work if I didn't make any typo's.
Btw, I dont think that this is what they ment with looping. But it's definitely a way to get the job done.

If I understand your question correctly, you can convert your array of ones and zeros (binary) to values of ture and false using the map function and using innerHTML to add the output to the DOM:
See example below:
// Populate myBinaryArray using your ascii method to get the follow:
let myBinaryArray = [1, 0, 0, 1, 1, 0, 1];
document.body.innerHTML += myBinaryArray.map(bit => !(!bit));

Node Red javascript Convert a HEX string to a ACSII string [duplicate]

How to convert from Hex string to ASCII string in JavaScript?
Ex:
32343630 it will be 2460

function hex2a(hexx) {
var hex = hexx.toString();//force conversion
var str = '';
for (var i = 0; i < hex.length; i += 2)
str += String.fromCharCode(parseInt(hex.substr(i, 2), 16));
return str;
}
hex2a('32343630'); // returns '2460'

Another way to do it (if you use Node.js):
var input = '32343630';
const output = Buffer.from(input, 'hex');
log(input + " -> " + output); // Result: 32343630 -> 2460

For completeness sake the reverse function:
function a2hex(str) {
var arr = [];
for (var i = 0, l = str.length; i < l; i ++) {
var hex = Number(str.charCodeAt(i)).toString(16);
arr.push(hex);
}
return arr.join('');
}
a2hex('2460'); //returns 32343630

You can use this..
var asciiVal = "32343630".match(/.{1,2}/g).map(function(v){
return String.fromCharCode(parseInt(v, 16));
}).join('');
document.write(asciiVal);

** for Hexa to String**
let input = '32343630';
Note : let output = new Buffer(input, 'hex'); // this is deprecated
let buf = Buffer.from(input, "hex");
let data = buf.toString("utf8");

I found a useful function present in web3 library.
var hexString = "0x1231ac"
string strValue = web3.toAscii(hexString)
Update: Newer version of web3 has this function in utils
The functions now resides in utils:
var hexString = "0x1231ac"
string strValue = web3.utils.hexToAscii(hexString)

I've found that the above solution will not work if you have to deal with control characters like 02 (STX) or 03 (ETX), anything under 10 will be read as a single digit and throw off everything after. I ran into this problem trying to parse through serial communications. So, I first took the hex string received and put it in a buffer object then converted the hex string into an array of the strings like so:
buf = Buffer.from(data, 'hex');
l = Buffer.byteLength(buf,'hex');
for (i=0; i<l; i++){
char = buf.toString('hex', i, i+1);
msgArray.push(char);
}
Then .join it
message = msgArray.join('');
then I created a hexToAscii function just like in #Delan Azabani's answer above...
function hexToAscii(str){
hexString = str;
strOut = '';
for (x = 0; x < hexString.length; x += 2) {
strOut += String.fromCharCode(parseInt(hexString.substr(x, 2), 16));
}
return strOut;
}
then called the hexToAscii function on 'message'
message = hexToAscii(message);
This approach also allowed me to iterate through the array and slice into the different parts of the transmission using the control characters so I could then deal with only the part of the data I wanted.
Hope this helps someone else!

console.log(
"68656c6c6f20776f726c6421".match(/.{1,2}/g).reduce((acc,char)=>acc+String.fromCharCode(parseInt(char, 16)),"")
)

An optimized version of the implementation of the reverse function proposed by #michieljoris (according to the comments of #Beterraba and #Mala):
function a2hex(str) {
var hex = '';
for (var i = 0, l = str.length; i < l; i++) {
var hexx = Number(str.charCodeAt(i)).toString(16);
hex += (hexx.length > 1 && hexx || '0' + hexx);
}
return hex;
}
alert(a2hex('2460')); // display 32343630

I use this one, it seems more clear to me as I also receive data with spaces like '30 31 38 30 38 30' and the output is 018080
hexToString(hex: string): string {
return hex.split(' ').map(s => string.fromCharCode(parseInt(s,16))).join('');
}

Convert large array of integers to unicode string and then back to array of integers in node.js

I have some data which is represented as an array of integers and can be up to 200 000 elements. The integer value can vary from 0 to 200 000.
To emulate this data (for debugging purposes) I can do the following:
let data = [];
let len = 200000
for (let i = 0; i < len; i++) {
data[i] = i;
}
To convert this array of integers as an unicode string I perform this:
let dataAsText = data.map((e) => {
return String.fromCodePoint(e);
}).join('');
When I want to convert back to an array of integers the array appears to be longer:
let dataBack = dataAsText.split('').map((e) => {
return e.codePointAt(e);
});
console.log(dataBack.length);
How does it come ? What is wrong ?
Extra information:
I use codePointAt/fromCodePoint because it can deal with all unicode values (up to 21 bits) while charCodeAt/fromCharCode fails.
Using, for example, .join('123') and .split('123') will make that the variable dataBack is the same length as data. But this isn't an elegant solution because the size of the string dataAsText will unnecessarily be very large.
If let len is equal or less to 65536 (which is 2^16 or 16 bits max value) then everything works fine. Which is strange ?
EDIT:
I use codePoint because I need to convert the data as unicode text so that the data is short.
More about codePoint vs charCode with an example:
If we convert 150000 to a character then back to an integer with codePoint:
console.log(String.fromCodePoint("150000").codePointAt(0));
this gives us 150000 which is correct. Doing the same with charCode fails and prints 18928 (and not 150000):
console.log(String.fromCharCode("150000").charCodeAt(0));

That's because higher code point values will yield 2 words, as can be seen in this snippet:
var s = String.fromCodePoint(0x2F804)
console.log(s); // Shows one character
console.log('length = ', s.length); // 2, because encoding is \uD87E\uDC04
var i = s.codePointAt(0);
console.log('CodePoint value at 0: ', i); // correct
var i = s.codePointAt(1); // Should not do this, it starts in the middle of a sequence!
console.log('CodePoint value at 1: ', i); // misleading
In your code things go wrong when you do split, as there the words making up the string are all split, discarding the fact that some pairs are intended to combine into a single character.
You can use the ES6 solution to this, where the spread syntax takes this into account:
let dataBack = [...dataAsText].map((e, i) => {
// etc.
Now your counts will be the same.
Example:
// (Only 20 instead of 200000)
let data = [];
for (let i = 199980; i < 200000; i++) {
data.push(i);
}
let dataAsText = data.map(e => String.fromCodePoint(e)).join("");
console.log("String length: " + dataAsText.length);
let dataBack = [...dataAsText].map(e => e.codePointAt(0));
console.log(dataBack);
Surrogates
Be aware that in the range 0 ... 65535 there are ranges reserved for so-called surrogates, which only represent a character when combined with another value. You should not iterate over those expecting that these values represent a character on their own. So in your original code, this will be another source for error.
To fix this, you should really skip over those values:
for (let i = 0; i < len; i++) {
if (i < 0xd800 || i > 0xdfff) data.push(i);
}
In fact, there are many other code points that do not represent a character.

I have a feeling split doesn't work with unicode values, a quick test above 65536 shows that they become double the length after splitting
Perhaps look at this post and answers, as they ask a similar question

I don't think you want charPointAt (or charCodeAt) at all. To convert a number to a string, just use String; to have a single delimited string with all the values, use a delimiter (like ,); to convert it back to a number, use the appropriate one of Number, the unary +, parseInt, or parseFloat (in your case, Number or + probably):
// Only 20 instead of 200000
let data = [];
for (let i = 199980; i < 200000; i++) {
data.push(i);
}
let dataAsText = data.join(",");
console.log(dataAsText);
let dataBack = dataAsText.split(",").map(Number);
console.log(dataBack);
If your goal with codePointAt is to keep the dataAsText string short, then you can do that, but you can't use split to recreate the array because JavaScript strings are UTF-16 (effectively) and split("") will split at each 16-bit code unit rather than keeping code points together.
A delimiter would help there too:
// Again, only 20 instead of 200000
let data = [];
for (let i = 199980; i < 200000; i++) {
data.push(i);
}
let dataAsText = data.map(e => String.fromCodePoint(e)).join(",");
console.log("String length: " + dataAsText.length);
let dataBack = dataAsText.split(",").map(e => e.codePointAt(0));
console.log(dataBack);

If you're looking for a way to encode a list of integers so that you can safely transmit it over a network, node Buffers with base64 encoding might be a better option:
let data = [];
for (let i = 0; i < 200000; i++) {
data.push(i);
}
// encoding
var ta = new Int32Array(data);
var buf = Buffer.from(ta.buffer);
var encoded = buf.toString('base64');
// decoding
var buf = Buffer.from(encoded, 'base64');
var ta = new Uint32Array(buf.buffer, buf.byteOffset, buf.byteLength >> 2);
var decoded = Array.from(ta);
// same?
console.log(decoded.join() == data.join())
Your original approach won't work because not every integer has a corresponding code point in unicode.
UPD: if you don't need the data to be binary-safe, no need for base64, just store the buffer as is:
// saving
var ta = new Int32Array(data);
fs.writeFileSync('whatever', Buffer.from(ta.buffer));
// loading
var buf = fs.readFileSync('whatever');
var loadedData = Array.from(new Uint32Array(buf.buffer, buf.byteOffset, buf.byteLength >> 2));
// same?
console.log(loadedData.join() == data.join())

How to convert from Hex to ASCII in JavaScript?

How to convert from Hex string to ASCII string in JavaScript?
Ex:
32343630 it will be 2460

function hex2a(hexx) {
var hex = hexx.toString();//force conversion
var str = '';
for (var i = 0; i < hex.length; i += 2)
str += String.fromCharCode(parseInt(hex.substr(i, 2), 16));
return str;
}
hex2a('32343630'); // returns '2460'

Another way to do it (if you use Node.js):
var input = '32343630';
const output = Buffer.from(input, 'hex');
log(input + " -> " + output); // Result: 32343630 -> 2460

For completeness sake the reverse function:
function a2hex(str) {
var arr = [];
for (var i = 0, l = str.length; i < l; i ++) {
var hex = Number(str.charCodeAt(i)).toString(16);
arr.push(hex);
}
return arr.join('');
}
a2hex('2460'); //returns 32343630

You can use this..
var asciiVal = "32343630".match(/.{1,2}/g).map(function(v){
return String.fromCharCode(parseInt(v, 16));
}).join('');
document.write(asciiVal);

** for Hexa to String**
let input = '32343630';
Note : let output = new Buffer(input, 'hex'); // this is deprecated
let buf = Buffer.from(input, "hex");
let data = buf.toString("utf8");

I found a useful function present in web3 library.
var hexString = "0x1231ac"
string strValue = web3.toAscii(hexString)
Update: Newer version of web3 has this function in utils
The functions now resides in utils:
var hexString = "0x1231ac"
string strValue = web3.utils.hexToAscii(hexString)

I've found that the above solution will not work if you have to deal with control characters like 02 (STX) or 03 (ETX), anything under 10 will be read as a single digit and throw off everything after. I ran into this problem trying to parse through serial communications. So, I first took the hex string received and put it in a buffer object then converted the hex string into an array of the strings like so:
buf = Buffer.from(data, 'hex');
l = Buffer.byteLength(buf,'hex');
for (i=0; i<l; i++){
char = buf.toString('hex', i, i+1);
msgArray.push(char);
}
Then .join it
message = msgArray.join('');
then I created a hexToAscii function just like in #Delan Azabani's answer above...
function hexToAscii(str){
hexString = str;
strOut = '';
for (x = 0; x < hexString.length; x += 2) {
strOut += String.fromCharCode(parseInt(hexString.substr(x, 2), 16));
}
return strOut;
}
then called the hexToAscii function on 'message'
message = hexToAscii(message);
This approach also allowed me to iterate through the array and slice into the different parts of the transmission using the control characters so I could then deal with only the part of the data I wanted.
Hope this helps someone else!

console.log(
"68656c6c6f20776f726c6421".match(/.{1,2}/g).reduce((acc,char)=>acc+String.fromCharCode(parseInt(char, 16)),"")
)

An optimized version of the implementation of the reverse function proposed by #michieljoris (according to the comments of #Beterraba and #Mala):
function a2hex(str) {
var hex = '';
for (var i = 0, l = str.length; i < l; i++) {
var hexx = Number(str.charCodeAt(i)).toString(16);
hex += (hexx.length > 1 && hexx || '0' + hexx);
}
return hex;
}
alert(a2hex('2460')); // display 32343630

I use this one, it seems more clear to me as I also receive data with spaces like '30 31 38 30 38 30' and the output is 018080
hexToString(hex: string): string {
return hex.split(' ').map(s => string.fromCharCode(parseInt(s,16))).join('');
}