TextEncoder and TextDecoder not perfect inverses of each other - javascript

I received this answer to my previous question about encoding strings. My hope in asking that question was to get some reversible way of shifting between a string and its representation as an array of bytes like in Python 3.
I ran into a problem with one particular Uint8Array though:
var encoder = new TextEncoder();
var decoder = new TextDecoder(encoder.encoding);
var s = [248, 35, 45, 41, 178, 175, 190, 62, 134, 39];
var t = Array.from(decoder.decode(encoder.encode(Uint8Array(s)));
I expected the value of t to be [248, 35, 45, 41, 178, 175, 190, 62, 134, 39]. Instead, it is [239, 191, 189, 35, 45, 41, 239, 191, 189, 239, 191, 189, 239, 191, 189, 62, 239, 191, 189, 39]. The person who posted the answer was temporarily suspended from the site, so I cannot resolve this through commenting on his answer.

change var t = Array.from(decoder.decode(encoder.encode(Uint8Array(s))); to
var t = JSON.parse('['+decoder.decode(encoder.encode(new Uint8Array(s)))+']');.
var encoder = new TextEncoder();
var decoder = new TextDecoder(encoder.encoding);
var s = [248, 35, 45, 41, 178, 175, 190, 62, 134, 39];
var t = JSON.parse('['+decoder.decode(encoder.encode(new Uint8Array(s)))+']');
console.log(t);

Related

Unable to convert bytes Uint8Array to string in javascript

I am unable to decrypt this Uint8Array
Uint8Array(32) [
174, 157, 255, 238, 54, 143, 97, 132,
70, 243, 7, 249, 98, 188, 68, 170,
53, 82, 78, 9, 96, 226, 182, 160,
131, 79, 3, 147, 153, 34, 205, 162
]
using this code
const string = new TextDecoder().decode(hash);
console.log(string);
I get this: ����Kc2Jk�&an↕�9���&��h-Lﺠ�Ԋ
I've also tried with many online converters but it says there's some problem with the byte array. This byte array is a response from an API.
Where Am I wrong? How can I convert it properly?

How to convert the object of Uint8Array to string?

I have same problem as this question. I see the answer but I couldn't understand how I can do it. Is there any other suggestions?
My code is:
var eccrypto = require("eccrypto");
var w,actual;
var publicKey = Buffer.from([4, 86, 82, 58, 244, 11, 140, 41, 132, 245, 184, 162, 163, 98, 49, 119, 168, 235, 252, 50, 6, 91, 147, 191, 190, 61, 65, 63, 101, 164, 132, 213, 188, 106, 26, 203, 171, 215, 240, 151, 7, 193, 10, 151, 103, 107, 1, 135, 117, 225, 5, 41, 55, 57, 18, 205, 98, 178, 82, 135, 170, 111, 188, 98, 57],'hex');
var privateKey= Buffer.from([238, 239, 199, 101, 188, 134, 13, 13, 195, 172, 125, 168, 225, 189, 72, 148, 225, 200, 127, 218, 204, 11, 150, 146, 180, 243, 195, 109, 200, 119, 50, 20],'hex');
eccrypto.encrypt(publicKey, Buffer.from("message")).then(function(encrypted) {
console.log(encrypted)
let encoded =JSON.stringify(encrypted)
w=encoded;
console.log(encoded)
actual = JSON.parse((encoded))
console.log(actual)
});
eccrypto.decrypt(privateKey,actual).then(function(plaintext) {
console.log("Message to part B:", plaintext.toString());
});
When I use actual variable I have this error:
Uncaught (in promise) Error: Bad public key
this the output of encrypted :
this the output of encoded :
and this the output of actual "there is some things changes i think, is not it ?":
thank you advance.
While parsing, you need to send a reviver function, which conditionally parses the object. If you see how encoded looks, you will understand why the reviver function needs to be written this way.
I think you need to remove the "hex" encoding parameter from your Buffer.from.

having issues in running sample code for public-key cryptography in JS

newbie to JS and trying to reproduce the result of this code, [1] : bob encrypt the message for alice. In [2]: Alice decrypt the message. The private/public key is generated in [1] here
const bob = nacl.box.keyPair()
const alice = nacl.box.keyPair()
, how do I pass these keys in [2], since console.log does not print these keys?
// reading Alice key pair from secret key
const alice = nacl.box.keyPair.fromSecretKey(/* Uint8Array with 32-byte secret key */)
// reading Bob public key
const bob = {publicKey: /* Uint8Array with 32-byte secret key */}`
Many thanks for your input.
[1]
const nacl = require('tweetnacl')
nacl.util = require('tweetnacl-util')
// generating key pairs
const bob = nacl.box.keyPair()
const alice = nacl.box.keyPair()
// generating one time nonce for encryption
const nonce = nacl.randomBytes(24)
// message for Alice
const utf8 = 'Hello Alice'
// Bob encrypts message for Alice
const box = nacl.box(
nacl.util.decodeUTF8(utf8),
nonce,
alice.publicKey,
bob.secretKey
)
// somehow send this message to Alice
const message = {box, nonce}
[2]
const nacl = require('tweetnacl')
nacl.util = require('tweetnacl-util')
// reading Alice key pair from secret key
const alice = nacl.box.keyPair.fromSecretKey(/* Uint8Array with 32-byte secret key */)
// reading Bob public key
const bob = {publicKey: /* Uint8Array with 32-byte secret key */}
// const message = ... the message object from Bob
// Alice decrypts message from Bob
const payload = nacl.box.open(message.box, message.nonce, bob.publicKey, alice.secretKey)
const utf8 = nacl.util.encodeUTF8(payload) // <-- 'Hello Alice'
So,
I don't know why your code isn't working but it's probably wrong somewhere.
I work in applied crypto, and I wrote up the following example to try and help you.
tweetnacl is fine and all, but in production systems you should use something like libsodium.
This code I wrote up shows how to do elliptic curve Diffie-Hellman and EdDSA signature in a JS binding for libsodium. Also it will print bobs pub and private key.
To get this lib, you need to do:
npm install sodium-native first...
function toHexString(byteArray) {
return Array.from(byteArray, function(byte) {
return ('0' + (byte & 0xFF).toString(16)).slice(-2);
}).join('')
}
//ECDH using curve25519
//totally fresh empty arrays
var alicePub = new Uint8Array(32);
var alicePriv = new Uint8Array(32);
var bobPub = new Uint8Array(32);
var bobPriv = new Uint8Array(32);
//fresh array to hold generated keys
var rxKey = new Uint8Array(sodium.crypto_kx_SESSIONKEYBYTES);
var txKey = new Uint8Array(sodium.crypto_kx_SESSIONKEYBYTES);
var otherRxKey = new Uint8Array(sodium.crypto_kx_SESSIONKEYBYTES);
var otherTxKey = new Uint8Array(sodium.crypto_kx_SESSIONKEYBYTES);
//generate key pairs for Alice and bob
sodium.crypto_kx_keypair(alicePub, alicePriv);
sodium.crypto_kx_keypair(bobPub, bobPriv);
//generate shared keys using ECDH
sodium.crypto_kx_client_session_keys(rxKey, txKey, alicePub, alicePriv, bobPub)
sodium.crypto_kx_server_session_keys(otherRxKey,otherTxKey, bobPub, bobPriv, alicePub)
console.log("ECDH Key Exchange, rx, tx")
console.log(rxKey, txKey)
console.log(otherRxKey, otherTxKey)
console.log(bobPub, bobPriv)
//signature using Ed25519
var publicKey = new Uint8Array(sodium.crypto_sign_PUBLICKEYBYTES);
var privateKey = new Uint8Array(sodium.crypto_sign_SECRETKEYBYTES);
sodium.crypto_sign_keypair(publicKey, privateKey);
var message = new Uint8Array([0,0,1,2,128]);
var similarMessage = new Uint8Array([0,1,1,2,128]);
//sign and message
var signedMessage = new Uint8Array(sodium.crypto_sign_BYTES + message.length)
sodium.crypto_sign(signedMessage, message, privateKey);
console.log("signedMessage: %s", signedMessage);
//verify it works
var bool = sodium.crypto_sign_verify_detached(signedMessage, message, publicKey)
var bool2 = sodium.crypto_sign_verify_detached(signedMessage, similarMessage, publicKey)
console.log("Real message:%s",message, bool);
console.log("Fake message:%s",similarMessage, bool2);
Output:
ECDH Key Exchange, rx, tx
Uint8Array(32) [
67, 47, 133, 36, 228, 190, 228, 192,
193, 16, 142, 142, 25, 97, 124, 2,
4, 210, 194, 51, 29, 25, 148, 70,
50, 30, 215, 107, 154, 52, 19, 180
] Uint8Array(32) [
219, 28, 88, 238, 33, 174, 171, 81,
173, 197, 38, 38, 254, 160, 142, 138,
66, 216, 5, 65, 238, 94, 62, 183,
89, 61, 151, 137, 220, 108, 4, 161
]
Uint8Array(32) [
219, 28, 88, 238, 33, 174, 171, 81,
173, 197, 38, 38, 254, 160, 142, 138,
66, 216, 5, 65, 238, 94, 62, 183,
89, 61, 151, 137, 220, 108, 4, 161
] Uint8Array(32) [
67, 47, 133, 36, 228, 190, 228, 192,
193, 16, 142, 142, 25, 97, 124, 2,
4, 210, 194, 51, 29, 25, 148, 70,
50, 30, 215, 107, 154, 52, 19, 180
]
bobs keys
Uint8Array(32) [
235, 23, 39, 184, 8, 74, 147, 55,
88, 230, 185, 51, 137, 47, 12, 246,
247, 235, 204, 57, 99, 78, 143, 131,
161, 24, 71, 185, 171, 23, 30, 215
] Uint8Array(32) [
79, 224, 52, 64, 235, 231, 82, 236,
10, 169, 210, 156, 249, 182, 60, 27,
173, 160, 150, 31, 8, 20, 217, 215,
58, 172, 185, 230, 185, 185, 67, 0
]
signedMessage: 134,97,230,121,91,96,87,17,1,55,95,20,176,7,203,124,116,236,31,140,173,32,129,206,82,174,214,188,177,45,30,80,230,6,203,204,140,216,228,69,106,134,196,175,95,139,140,116,239,27,235,49,245,24,130,129,154,175,126,127,206,163,206,11,0,0,1,2,128
Real message:0,0,1,2,128 true
Fake message:0,1,1,2,128 false
So as you can see I get out bobs key pair, but really, elliptic curves are used for two main things:
1) to digitally sign things
2) to agree a key for symmetric (AES/ChaCha/Salsa) encryption
Remember, the public key in ECC is just a point on the curve.
The private key is just a large number which you scalar multiple by a special point on the curve called the generator to get the public key.
This code represents the type of approach we see in production systems, it's side channel resistant and uses grown up libs.

Browser buffer to string conversion is not same in browser and nodejs

I have encountered an interesting issue.
I'm using node v8.1.4
I have the following buffer.
[ 191, 164, 235, 131, 30, 28, 164, 179, 101, 138, 94, 36, 115, 176, 83, 193, 9, 177, 85, 228, 189, 193, 127, 71, 165, 16, 211, 132, 228, 241, 57, 207, 254, 152, 122, 98, 100, 71, 67, 100, 29, 218, 165, 101, 25, 17, 177, 173, 92, 173, 162, 186, 198, 1, 80, 94, 228, 165, 124, 171, 78, 49, 145, 158 ]
When i try to convert it to utf8 using nodejs and using browser i get different results. even length of string is not the same.
Is there a way to convert string to utf8 in browser same way as node js do?
It seems that some characters that some sequence which nodejs replace to U+FFFD are more lengthy than the replaced sequence in browser. so output utf8 string is different
Code i use in browser and in nodejs is same
i have buffer object tmpString
tmpString.toString('utf-8')
tmpString.toString('utf-8').length differs in browser and nodejs for the same source bytes.
In nodejs i use native buffer implementation, for browser webpack loads polyfill (feross/buffer i think)
i think more accurately would say that i try to interpret buffer bytes as UTF8 string.
Have you tried the TextEncoder/TextDecoder APIs? I've used them for converting strings in both nodejs and the browser and haven't seen any differences.
E.g.:
const encoder = new TextEncoder('utf-8');
const decoder = new TextDecoder('utf-8');
const foo = 'Hello world!';
const encoded = encoder.encode(foo);
console.log(encoded);
const decoded = decoder.decode(encoded);
console.log(decoded);

Crypto.js decrypt with key and iv (vector) in byte arrays

I have to decrypt some strings which are AES encrypted.
Example encrypted string: 129212143036071008133136215105140171136216244116
I have a key, and a vector (iv) supplied to me in a byte-array format:
Key: [ 123, 217, 20, 11, 24, 26, 85, 45, 114, 184, 27, 162, 37, 115, 222, 209, 241, 24, 175, 144, 175, 53, 196, 29, 24, 23, 17, 218, 131, 226, 53, 209 ]
Vector (iv): [ 146, 66, 191, 151, 23, 3, 113, 119, 231, 131, 133, 112, 79, 32, 114, 136 ]
I should be able to decrypt the string and get:
Correct output: testtest
I'm trying to use Crypto.js but I can't find a way to use the supplied key and vector. I can't find a way to convert the byte-arrays to hex.
var encrypted = '129212143036071008133136215105140171136216244116';
var key = CryptoJS.enc.Hex.parse([ 123, 217, 20, 11, 24, 26, 85, 45, 114, 184, 27, 162, 37, 115, 222, 209, 241, 24, 175, 144, 175, 53, 196, 29, 24, 23, 17, 218, 131, 226, 53, 209 ]);
var iv = CryptoJS.enc.Hex.parse([ 146, 66, 191, 151, 23, 3, 113, 119, 231, 131, 133, 112, 79, 32, 114, 136 ]);
var decrypted = CryptoJS.AES.decrypt(encrypted, key, { iv: iv });
console.log('Output: '+decrypted.toString(CryptoJS.enc.Utf8)); //Should be "testtest"
I would be so grateful if anyone could show me how to decrypt the example string using the key and vector with Crypto.js OR any other js method.
Thanks so much for any help,
Kind regards
I can't manage to decrypt your original string, but I can successful use it to encrypt and decrypt a new string. In your initial implementation, an error occurs in aes.js because it expects key and iv to be strings rather than arrays. I have corrected this code example below:
//var encrypted = '129212143036071008133136215105140171136216244116';
var key = CryptoJS.enc.Hex.parse(CryptoJS.lib.ByteArray([123, 217, 20, 11, 24, 26, 85, 45, 114, 184, 27, 162, 37, 115, 222, 209, 241, 24, 175, 144, 175, 53, 196, 29, 24, 23, 17, 218, 131, 226, 53, 209]));
var iv = CryptoJS.enc.Hex.parse(CryptoJS.lib.ByteArray([146, 66, 191, 151, 23, 3, 113, 119, 231, 131, 133, 112, 79, 32, 114, 136]));
var message = 'testest'
var encrypted = CryptoJS.AES.encrypt(message, key, {
'iv': iv
});
var decrypted = CryptoJS.AES.decrypt(encrypted, key, {
'iv': iv
});
document.write('Output: ' + decrypted.toString(CryptoJS.enc.Utf8)); //Should be "testtest"
<script src="http://crypto-js.googlecode.com/svn/tags/3.1.2/build/rollups/aes.js"></script>
<script src="http://crypto-js.googlecode.com/svn/tags/3.1.2/build/components/pad-nopadding-min.js"></script>
<script src="https://greasyfork.org/scripts/6696-cryptojs-lib-bytearray/code/CryptoJSlibByteArray.js"></script>
Bit of an old post, and I think the API has changed somewhat since then as it now uses WordArray rather than ByteArray.
But I was running into the same issue. Turns out, you can simply provide a base64-encoded string into decrypt(). So all you need to do is to create a base64-encoded string from your original message. I'm not sure what format your message is in now, but I'm going to assume it's a hex string.
var origCipher = "129212143036071008133136215105140171136216244116"; // is this supposed to be a hex string?
var origKey = [ 123, 217, 20, 11, 24, 26, 85, 45, 114, 184, 27, 162, 37, 115, 222, 209, 241, 24, 175, 144, 175, 53, 196, 29, 24, 23, 17, 218, 131, 226, 53, 209 ];
var origIV = [ 146, 66, 191, 151, 23, 3, 113, 119, 231, 131, 133, 112, 79, 32, 114, 136 ];
var key = CryptoJS.lib.WordArray.create(new UInt8Array(origKey));
var iv = CryptoJS.lib.WordArray.create(new UInt8Array(origIV));
var cipher = CryptoJS.enc.Base64.stringify(CryptoJS.enc.Hex.parse(origCipher));
var plain = CryptoJS.AES.decrypt(cipher, key, { iv: iv });
var plaintext = CryptoJS.enc.UTF8.stringify(plain);
Above code doesn't result in a correct result, though, so your input ciphertext is probably not a hex string.
I ended up using a .net ashx generic handler, to which i post the data over ssl, and the server-side decryption takes place using the simpleAES class.
This class uses block-size as a parameter which seems to make the difference and using this approach I was able to decrypt all the strings i needed to.
More info about simpleAES here: https://github.com/huanlin/YetAnotherLibrary/blob/master/Source/Yalib/Cryptography/SimpleAes.cs
Thanks again for the help!

Categories