We Keep Coding

Proper hexadecimal conversion to text/ascii

Trying to figure out how to get my qr scanners to output text to be used in my website.
My qr reader is outputting text that needs to be rendered with event.target.value.getuintsomeintegerhere. I cannot get this to convert to text unless I plop it into some hex decoder online.
I do notice the comment about using TextDecoder to process the data, but it still sort of takes me back to the same place off not really understanding what's going on with this sort of thing...
For example
0x6d 0x6f 0x6f 0x0d should read 'moo'
I understand that the encoding is "utf-8" but not what that means or how to modify the code so that it outputs things correctly...
Here's a chunk of the code that seems to be where i need the most help:
function handleNotifications(event) {
let value = event.target.value;
let a = [];
// Convert raw data bytes to hex values just for the sake of showing something.
// In the "real" world, you'd use data.getUint8, data.getUint16 or even
// TextDecoder to process raw data bytes.
for (let i = 0; i < value.byteLength; i++) {
a.push('0x' + ('00' + value.getUint8(i).toString(16)).slice(-2));
}
log(hex2ascii('> ' + a.join(' ')));
}

It can all be vastly simplified
const h2a = str => str && str.replace(/ ?0x/g,"") // get rid of the space and 0x
.match(/[a-zA-Z0-9]{1,2}/g) // get the 00-FF parts
.map(v => String.fromCharCode(parseInt(v, 16))) // convert each to in using hex radix and get the ascii from the int
.join(''); // string them together
document.getElementById("hex")
.addEventListener("input", e => document.getElementById("ascii").value = h2a(e.target.value))
<input type="text" value="" id="hex" /> Hex (paste or scan - for example cut and paste 0x6d 0x6f 0x6f 0x0d)
<hr/>
<input type="text" value="" id="ascii" /> Ascii

function handleNotifications(event) {
let value = event.target.value;
let a = [];
// Convert raw data bytes to hex values just for the sake of showing something.
// In the "real" world, you'd use data.getUint8, data.getUint16 or even
// TextDecoder to process raw data bytes.
for (let i = 0; i < value.byteLength; i++) {
a.push('0x' + ('00' + value.getUint8(i).toString(16)).slice(-2));
}
log(hex2ascii('> ' + a.join(' ')));
}
function hex2ascii(hexString) {
const numberArray = hexString.slice(2) // Get the string without the first two characters (removes '> ')
.split(' ') // Split the string on spaces.
.map(hex => Number(hex)); // Create a new array where each element of the hex array is converted to a JavaScript Number
// Now we have an array of Numbers.
const byteArray = new Uint8Array(numberArray); // Make an array of bytes from the number array This is necessary to use the TextDecoder.
const string = (new TextDecoder()).decode(byteArray); // Create a TextDecoder, and decode the byte array to get a string.
return string;
}
console.log(hex2ascii("> 0x6d 0x6f 0x6f 0x0d"));
This snippet only logs the received data every 2 messages, concatenating the data received between them. For example, if the first message said moo, and the second message said I'm a cow!, the text moo I'm a cow! would be logged after receiving the second message.
let messageCount = 0;
let currentMessage = "";
function handleNotifications(event) {
let value = event.target.value;
let a = [];
// Convert raw data bytes to hex values just for the sake of showing something.
// In the "real" world, you'd use data.getUint8, data.getUint16 or even
// TextDecoder to process raw data bytes.
for (let i = 0; i < value.byteLength; i++) {
a.push('0x' + ('00' + value.getUint8(i).toString(16)).slice(-2));
}
currentMessage += hex2ascii('> ' + a.join(' '));
if (messageCount < 1) messageCount++;
else {
messageCount = 0;
log(currentMessage);
currentMessage = "";
}
}
function hex2ascii(hexString) {
const numberArray = hexString.slice(2) // Get the string without the first two characters (removes '> ')
.split(' ') // Split the string on spaces.
.map(hex => Number(hex)); // Create a new array where each element of the hex array is converted to a JavaScript Number
// Now we have an array of Numbers.
const byteArray = new Uint8Array(numberArray); // Make an array of bytes from the number array This is necessary to use the TextDecoder.
const string = (new TextDecoder()).decode(byteArray); // Create a TextDecoder, and decode the byte array to get a string.
return string;
}
console.log(hex2ascii("> 0x6d 0x6f 0x6f 0x0d"));

Add = or == to a base64 if necessary

I am using the Ionic Choose plugin to take a file in an app, this one in the resolve returns me a base64 of the file I chose, the problem is that this base64 never ends with = or == when necessary in some cases.
When trying to send this base64 to the server it returns an error because the = or == is missing at the end of the string.
So I get the base64
this.chooser.getFile()
.then((file) => {
let base64 = file.dataURI;
})
.catch((err) => {
console.log(err);
});
Can I somehow detect if base64 needs either = or == to complete the string?

Base64 'blocks' consists of 4 characters, so if the final block contains less than 4 characters it gets padded with "=" sign, so it can have anywhere from 0 to 2 "=" signs in them, that's why the length of the dataURI needs to be a multiple of 4, You can do something like this:
let base64 = file.dataURI;
const len = base64.length;
const modifiedBase64 = base64 + "=".repeat(len%4);

getting 4 byte from a file and convert it to timestamp

I am facing issue in getting data from a file as a buffer and then converting it into a long sample code is
var fs = require('fs');
fs.open('24.mkt', 'r', function(status, fd) {
if (status) {
console.log(status.message);
return;
}
var buffer = new Buffer(4);
fs.read(fd, buffer, 0, 4, 0, function(err, num) {
console.log(buffer.values());
});
});
file link - > https://archive.org/download/kamo_24/24.mkt
first 4 bytes contains Timestamp in Long in 4 Bytes

You can use node.js's Buffer.readInt32BE function. It reads 4 bytes in the given order (Big, or Little endian) to a variable, starting at the offset parameter:
// Unix timestamp now: 1492079016
var buffer = Buffer.from([0x58, 0xEF, 0x51, 0xA8]);
var timestamp = buffer.readInt32BE(0);
process.stdout.write(timestamp.toString());

You probably want to use readUInt32BE and/or readUInt32LE (of buffer) to convert buffer values to number.
You may also try to converting a values in a buffer to numeric value using node-bigint or node-bignum (it's probably overkill for a 4 bytes case but if you need to deal with a bigger numbers it may suit the need), both allow creation from buffer in a similar form (just be aware of options differences):
bignum.fromBuffer(buf, opts)
// or
bigint.fromBuffer(buf, opts)

javascript: smallest JSON.stringify for Float32Array?

FireFox 46.0.1: I am using 3rd-party (easyrtc) software to send 15KB chunks of Float32Arrays between peers. Easyrtc insists that the data be JSON-able. Unfortunately, JSON.stringify yields a string more than twice as long as the original data: 16384 bytes of data becomes a string of length 35755. Below is my test code followed by the console output. What if anything can I do to reduce the stringify'd size? Is there a way to send the values only (no keys)? Can I use the 'replacer' argument to send only the values, and if so, don't I need to use a replacer on the corresponding JSON.parse on the receiving end?
var g_testBufferNBytes = 4096 * 4;
var g_testBuffer = new ArrayBuffer(g_testBufferNBytes);
var g_testBufferView = new Float32Array(g_testBuffer);
console.log("array byte length " + g_testBuffer.byteLength);
console.log("view byte length " + g_testBufferView.byteLength);
var j = JSON.stringify(g_testBufferView);
console.log("j length " + j.length);
var newBuf = JSON.parse(j);
console.log("newBuf length " + Object.keys(newBuf).length);
CONSOLE:
array byte length 16384
view byte length 16384
j length 35755
newBuf length 4096

Yes
ES6: Assume that your data are in let f32 = g_testBufferView (array Float32Array) ) - whe can save it as JSON array in at leas 4 ways:
// code
let f32json = JSON.stringify(f32);
let f32jsonArr = JSON.stringify(Array.from(f32));
let f32base64 = btoa(String.fromCharCode(...(new Uint8Array(f32.buffer))));
let f32base128 = ... // not trivial, look below
// decode
let df32json = new Float32Array(Object.values(JSON.parse(f32json)));
let df32jsonArr = new Float32Array(JSON.parse(f32jsonArr));
let df32base64 = new Float32Array(new Uint8Array([...atob(f32base64)].map(c => c.charCodeAt(0))).buffer);
let df32base128 = ... // not trivial, look below
Note that Object.values return values sorted by numeric keys (look here).
Here is working example. You can also use base128 do decode but I not use in this example (to not complicate it) - more details here.
If your Float32Array- f32 has 4096 elements equals to 0.3 then:
f32 has 16384 bytes,
f32json (j from your question) has 109483 bytes (which is >6x bigger than f32)
f32jsonArr has 81921 bytes (which is >5x bigger than f32)
f32base64 has 21848 bytes(which is ~1.3x bigger than f32)
f32base128 has 18725 bytes (whis is <1.15x bigger than f32) but chrome will send ~2x bigger request (depends on input data)
If your Float32Array- f32 has 4096 elements equals integer from 1 to 9 then:
f32 has 16384 bytes - CONST,
f32json (j from your question) has 35755 bytes (which is >2x bigger than f32)
f32jsonArr has 8193 bytes (which is 2x SMALLER (sic!) than f32)
f32base64 has 21848 bytes - CONST (which is ~1.3x bigger than f32)
f32base128 has 18725 bytes - CONST (whis is <1.15x bigger than f32) but chrome will send ~2x bigger request (depends on input data)
Conclusion
The smallest result which not depends of array values (result size is constant) we get for f32base64 ~33% bigger than input array size. For f32base128 - it contains valid JSON (string) which is something about <15% bigger than input, but chrome during sending increase this size (look here - on 'update' section). So use f32base64 - this is probably the smallest JSON that you can get without more sophisticated methods.

Support [ and ] characters in PDU mode

I am writing application in nodejs for sending and recieving sms in PDU mode.I use wavecom GSM modem(7-bit encoding) to send sms. It also support 8 bit(AT+CSMP=1,167,0,8) encoding scheme.
I can send alpha numeric character properly.But I can't send some character like ([,],| etc).
Here string :
AT+CMGS=14
0001030C911989890878800004015B
Text String : [
But I recieve some junk character. Any Idea?
And how to send multipart sms. I have refer this, and this but I does not get desired output. can anyone suggest 8-bit(7-bit encoding scheme) text encoding scheme?
Please Help me...

According to this page (see section Sending an Unicode SMS message), the 8bit encoding is in fact UCS-2.
I don't know enough about nodejs to give you the full implementation, but here is a .NET sample:
string EncodeSmsText(string text)
{
// Convert input string to a sequence of bytes in BigEndian UCS-2 encoding
// 'Hi' -> [0, 72, 0, 105]
var bytes = Encoding.BigEndianUnicode.GetBytes(text);
// Encode bytes to hex representation
// [0, 72, 0, 105] -> '00480069'
return BitConverter.ToString(bytes).Replace("-", "");
}
Please note that according to this post my code will not work for characters encoded as surrogate pairs, because Encoding.BigEndianEncoding is UTF-16 (not UCS-2).
Edit
Here is NodeJS version that uses the built-in UCS2 converter in Buffer class:
function swapBytes(buffer) {
var l = buffer.length;
if (l & 0x01) {
throw new Error('Buffer length must be even');
}
for (var i = 0; i < l; i += 2) {
var a = buffer[i];
buffer[i] = buffer[i+1];
buffer[i+1] = a;
}
return buffer;
}
function encodeSmsText(input) {
var ucs2le = new Buffer(input, 'ucs2');
var ucs2be = swapBytes(ucs2le);
return ucs2be.toString('hex');
}
console.log(encodeSmsText('Hi'));
Inspired by these SO answers:
Node.JS Big-Endian UCS-2
How to do Base64 encoding in node.js?

Thanks,
Finally I got the answer :)
This characters([,],|) are encode as two characters like
[ is encode as 1B1E (Combination of escape character and < sign)
] is encode as 1B20 (Combination of escap character and > sign)
So whenever I fond such characters I replaced it with corresponding value then I use 7 bit encoding. It's Work good...
So my encoding string for [ is
> AT+CMGS=15
> 0001000C911989890878800000021B1E
And for "[hello]"
> AT+CMGS=21
> 0001000C911989890878800000091B1EBACC66BF373E
Thanks once again..