I'm creating a Buffer from a long string of hex but getting some size mismatching (when I try to decrypt it). Sure enough when I run the following code:
var hexData = fs.readFileSync(fileName).toString().trim();
var bufferData = new Buffer(hexData, 'hex');
console.log(Buffer.byteLength(hexData, 'hex'));
console.log(bufferData.length);
my output is:
232548
30
Why is the whole string not being loaded into the Buffer?
EDIT: I noticed I was being silly and should be doing
var bufferData = fs.readFileSync(fileName, 'hex');
But the length of that is 930194!
Assuming you meant hexData and not data when you build bufferData, Buffer.byteLength seems to happily accept malformed hex, whereas the Buffer constructor will strip it out of the buffer. Consider:
> Buffer.byteLength('ff00junk', 'hex')
4
> var b = new Buffer('ff00junk', 'hex')
> b.length
2
> b
<Buffer ff 00>
Perhaps your file contains invalid hex?
Related
I'm getting a binary file from my computer in javascript. And this binary file contains some long numbers in the following:
20
56599565 6559565
65656589 6595956
26989565 5656595
...
My question is that how can I get these numbers?
I have a binary file that contains numbers, its created with serialization on c++. And the first 40 bytes of this binary file is empty. After these 40 bytes have passed, my numbers are starting to come.
My solution:
After reading the file successfully I have an ArrayBuffer right? And I know that my numbers start from the 40. bytes. So I sliced it.
// rawData ArrayBuffer(197283) {}
var frameCount = rawData.slice(40,48);
var frameCountView = new Float64Array(frameCount);
And then what should I do?
I think we can get the Int8Array from Arraybuffer;
// rawData ArrayBuffer(197283) {}
var fileContent = rawData.slice(40);
var frameCountView = new Int64Array(fileContent);
var byteArray = new BigUint64Array(fileContent);
Consider the code:
const CryptoJS = require("crypto-js");
var key = CryptoJS.enc.Hex.parse("000102030405060708090a0b0c0d0e0f");
var iv = CryptoJS.enc.Hex.parse("101112131415161718191a1b1c1d1e1f");
// encrypt
var aesEncryptor = CryptoJS.algo.AES.createEncryptor(key, { iv: iv });
var ciphertextPart1 = aesEncryptor.process("Message Part 1");
var ciphertextPart2 = aesEncryptor.process("Message Part 2");
var ciphertextPart3 = aesEncryptor.process("Message Part 3");
var ciphertextPart4 = aesEncryptor.finalize();
// decrypt
var aesDecryptor = CryptoJS.algo.AES.createDecryptor(key, { iv: iv });
var plaintextPart1 = aesDecryptor.process(ciphertextPart1);
var plaintextPart2 = aesDecryptor.process(ciphertextPart2);
var plaintextPart3 = aesDecryptor.process(ciphertextPart3);
var plaintextPart4 = aesDecryptor.process(ciphertextPart4);
var plaintextPart5 = aesDecryptor.finalize();
console.log(plaintextPart5.toString());
Source: https://cryptojs.gitbook.io/docs/#ciphers
Maybe I am wrong, but I expected the messages to be decrypted.
The output is actually:
61676520506172742033
I have no idea what that output means and where that comes from.
If I print out another part, the same issue:
console.log(plaintextPart4.toString());
Output:
7373616765205061727420324d657373
Discussion
One comment says that I am wrong on my assumption regarding how the method works, a nice answer could nicely correct me as well! I know the difference, from my studies, between one-way encryption and by parts: it makes no sense to me the way it is actually working, it seems a bug. My ideia is that you could keep adding message to be encrypted, as we can do with hashing, in the case of hashing (from the same abovementioned source), it seems to work, why not for encryption as well?
I honestly expected this example to work since it is from the official documentation, and they do not mention anything.
Comprehensive documentation is a rare beast.
it makes no sense to me the way it is actually working, it seems a bug
It works like this:
const CryptoJS = require("crypto-js");
const original_message = " ...0, ...1, ...2, ...3, ...4, ...5, ...6, ...7, ...8, ...9, ...The End!"
var key = CryptoJS.enc.Hex.parse("000102030405060708090a0b0c0d0e0f");
var iv = CryptoJS.enc.Hex.parse("101112131415161718191a1b1c1d1e1f");
const aesEncryptor = CryptoJS.algo.AES.createEncryptor(key, { iv: iv });
const aesDecryptor = CryptoJS.algo.AES.createDecryptor(key, { iv: iv });
const fragment_size = 25
const e_acc = []
for(let i = 0 ; i*fragment_size < original_message.length ; ++i ) {
const slice = original_message.slice(i*fragment_size, i*fragment_size + fragment_size)
console.log("slice to encrypt", slice)
e_acc.push(aesEncryptor.process(slice));
}
e_acc.push(aesEncryptor.finalize());
let message = ""
for(let i = 0 ; i !== e_acc.length ; ++i ) {
message += aesDecryptor.process(e_acc[i]).toString(CryptoJS.enc.Utf8)
console.log("intermidiate message", message)
}
message += aesDecryptor.finalize().toString(CryptoJS.enc.Utf8)
console.log("full message", message)
[...] it supposes to be a trick that you can add information by parts
[...]
My idea is that you could keep adding message to be encrypted
And you can, the only thing is - you can't decipher parts of messages that do not fill a block. You can only get as much as fits into a chain of complete blocks.
Consider the code
const sum1 = solver.process("1+22+3")
const sum2 = solver.process("03")
const sum3 = solver.finalize()
Here you can get the intermediate sum1 - sum of 1 and 22. But you can't add 3 right away, because, as you can see later on, it is not actually 3 but 303. Same thing with sum2 - solver do not know yet if the 303 should be added or maybe it would become 3031234.... So the only thing the solver can do is to remember the 3 and then the 303, but not use it. Until you call finalize.
aesEncryptor is like the solver but much more complicated.
Let's get back to the aesDecryptor and why all of plaintextPart* contain only the parts of the message... Why do you need "Progressive Ciphering" in the first place? Probably because you are sending fragments over the net? The key word is fragments - they purposely do not contain all the previous parts of the message as they could weight gigabytes!
in the case of hashing [...], it seems to work, why not for encryption as well?
Hashing is a quite different thing. You can not expect that AES has all the same properties as hashing, as you can not expect that hashing has all the same properties as, for example, summation.
The .process method returns any new chunks of ciphertext that were generated by processing the plaintext given as the argument to .process along with any plaintext that was left over from earlier encryptor activity.
In this case the cipher algorithm is AES. Specifically it's AES-128, because the key used to create the encryptor is 128 bits long. AES-128 consumes plaintext in 128-bit (16-byte) blocks and emits 128 bits (16 bytes) of ciphertext for each plaintext block. This block-based processing is what produces the results that you didn't understand.
What happens with your program is:
var ciphertextPart1 = aesEncryptor.process("Message Part 1");
The encryptor is given 14 bytes of input. That's not enough to allow it to generate a ciphertext block, so .process returns an empty ciphertext result which you store in ciphertextPart1. The encryptor stores the unprocessed 14 bytes internally.
var ciphertextPart2 = aesEncryptor.process("Message Part 2");
This gives the encryptor a further 14 bytes. It appends those to the 14 bytes left over from the previous call, so it now has a total of 28 unprocessed bytes. It processes as many of those bytes as it can. That is, it processes the first 16 of those bytes ("Message Part1Me") and returns the ciphertext block for those 16 bytes, which you store in cipherText2. The encryptor now contains 12 unprocessed bytes.
var ciphertextPart3 = aesEncryptor.process("Message Part 3");
This gives the encryptor another 14 bytes. It now has 26 unprocessed bytes. It processes the first 16 of those bytes ("ssage Part2Mess") and returns the ciphertext block for those 16 bytes, which you store in cipherText3. The encryptor now contains 10 unprocessed bytes.
var ciphertextPart4 = aesEncryptor.finalize();
This forces the encryptor to process any unprocessed bytes. It can only work on 16-byte blocks, so it adds 6 bytes of padding to the remaining 10 unprocessed plaintext bytes ("age Part 3"), encrypts that block and returns the ciphertext for that block. You store that ciphertext block as ciphertextPart4.
And now you decrypt the ciphertext blocks.
var plaintextPart1 = aesDecryptor.process(ciphertextPart1);
cipherTextPart1 was an empty block, so plaintextPart1 will be empty and obviously the decryptor will retain no unprocessed ciphertext.
var plaintextPart2 = aesDecryptor.process(ciphertextPart2);
cipherTextPart2 contained the encrypted version of the first 16 bytes of the plaintext, so plaintextPart2 will contain "Message Part 1Me". The ciphertext input was exactly 16 bytes long, therefore the decryptor contains no unprocessed ciphertext.
var plaintextPart3 = aesDecryptor.process(ciphertextPart3);
cipherTextPart3 contained the encrypted version of the next 16 bytes of the plaintext, so plaintextPart3 will contain "ssage Part 2Mess". Again the decryptor is holding no unprocessed ciphertext.
var plaintextPart4 = aesDecryptor.process(ciphertextPart4);
cipherTextPart4 contained the encrypted version of the final 10 bytes of the plaintext, so plaintextPart3 will contain "age Part 3". No unprocessed ciphertext remains in the decryptor.
var plaintextPart5 = aesDecryptor.finalize();
The is no unprocessed ciphertext remaining in the decryptor, so finalize has no work to do and plaintextPart5 will be empty.
I am using aes-js library to test AES encryption . When I try to decrypt a ciphertext, I am getting the following error
Error: unsupported array-like object
at coerceArray (/data/data/com.suspendresume/files/nodejs-project/node_modules/aes-js/index.js:51:15)
at new ModeOfOperationCBC (/data/data/com.suspendresume/files/nodejs-project/node_modules/aes-js/index.js:442:33)
at MyEmitter.rn_bridge.channel.on (/data/data/com.suspendresume/files/nodejs-project/main.js:76:15)
at emitOne (events.js:115:13)
at MyEmitter.emit (events.js:210:7)
at Immediate.setImmediate [as _onImmediate] (/data/data/com.suspendresume/files/nodejs-builtin_modules/rn-bridge/index.js:14:13)
at runCallback (timers.js:781:20)
at tryOnImmediate (timers.js:743:5)
at processImmediate [as _immediateCallback] (timers.js:714:5)
The following is my code
encryptedHex ='yRe2x6Gf2uVzfesp1I7ISkkAjTo2xoH2SPSqXzdWKHg+HhosYblfTFUJVoPVgpyf'
iv= 'ec8902010adc3d63';
key='aa54c24fae5e52a5861c80f466a90922'
key= aesjs.utils.hex.toBytes(key)
// When ready to decrypt the hex string, convert it back to bytes
var encryptedBytes = aesjs.utils.hex.toBytes(encryptedHex);
// The cipher-block chaining mode of operation maintains internal
// state, so to decrypt a new instance must be instantiated.
var aesCbc = new aesjs.ModeOfOperation.cbc(key, iv);
var decryptedBytes = aesCbc.decrypt(encryptedBytes);
// Convert our bytes back into text
var decryptedText = aesjs.utils.utf8.fromBytes(decryptedBytes);
console.log(decryptedText);
Please help me to solve this problem
There are a few problems with your code:
The Initialization Vector must also be passed as an array.
The Initialization Vector must be of length 16 bytes (that is 32 in hex). Yours is only 8 bytes.
The variable called encryptedHex doesn't contain hex but it contains base64. Hex only allows numbers and the letters A-F, but the string contains other letters and even +.
I have solved these issues like this:
Convert it to bytes using aesjs.utils.hex.toBytes
Since I don't have the original 16 bytes IV, I can't decrypt your text. Instead, I have created a new encrypted text, IV and Key to demonstrate that the code works.
I convert the base64 string into an Uint8Array using this snippet.
Here's the result. It should print Hello in the console:
encryptedB64 = 'kcGz8P/m0lRRRxcT3tJiSw==';
iv = '6162636465666768696a6b6c6d6e6f70';
iv = aesjs.utils.hex.toBytes(iv);
key = '31323334353637383930313233343536';
key = aesjs.utils.hex.toBytes(key);
// When ready to decrypt the base64 string, convert it back to bytes
var encryptedBytes = Uint8Array.from(atob(encryptedB64), c => c.charCodeAt(0));
// The cipher-block chaining mode of operation maintains internal
// state, so to decrypt a new instance must be instantiated.
var aesCbc = new aesjs.ModeOfOperation.cbc(key, iv);
var decryptedBytes = aesCbc.decrypt(encryptedBytes);
// Convert our bytes back into text
var decryptedText = aesjs.utils.utf8.fromBytes(decryptedBytes);
console.log(decryptedText.trimEnd('\0'));
<script src="https://cdnjs.cloudflare.com/ajax/libs/aes-js/3.1.2/index.min.js"></script>
I have a BMP image and I need to send it to a device via TCP/IP. We already have the C library in our company, which can handle this, but I need to do it in JavaScript. Unfortunately, I don't have access to the source code of the library, nor to the device system.
This is a Base64 string of a sample image (a black&white checkmark):
Qk2+AAAAAAAAAD4AAAAoAAAAIAAAACAAAAABAAEAAAAAAIAAAADEDgAAxA4AAAAAAAAAAAAAAAAAAP///wD//////+H////A////gH///wB///4AP//8AD//+AAf//AAD//gAA//yAAH/5wAB/+8EAP/fjAB/394Af+++AD/vPwAf8P+AH///gA///9AH///v5///7/P///fx///7+P//+/z///38f//+/n///38///+/f///v3///97////hw==
I use window.atob and I am encoding it into HEX. I use this JS function:
function toHex(str) {
var result = '';
for (var i = 0; i < str.length; i++) {
result += str.charCodeAt(i).toString(16);
}
return result;
}
var str = window.atob(base64img);
var result = toHex(str);
It gives me this result, which is almost expected result:
424dbe00000003e0002800020000200001010000080000c4e00c4e00000000000000ffffff0ffffffffffe1ffffffc0ffffff807fffff07ffffe03ffffc03ffff801ffff00fffe00fffc807ff9c07ffbc103ff7e301ff7f781ffbef80ffbcfc07fc3fe07ffffe03fffff401fffffbf9fffffbfcfffffdfc7ffffefe3ffffeff3fffff7f1fffffbf9fffffdfcfffffefdfffffefdffffff7bffffff87
The library sends the exact same image correctly (the device accepts the message). This is how it looks like (copied from log):
be00424dbe000000000000003e000000280000002000000020000000010001000000000080000000c40e0000c40e0000000000000000000000000000ffffff00ffffffffffe1ffffffc0ffffff807fffff007ffffe003ffffc003ffff8001ffff0000fffe0000fffc80007ff9c0007ffbc10101003ff7e3001ff7f7801ffbef800ffbcfc007fc3fe007ffffe003fffff401fffffbf9fffffbfcfffffdfc7ffffefe3ffffeff3fffff7f1fffffbf9fffffdfcfffffefdfffffefdffffff7bffffff87
So this is what I need to get from Base64 in my JavaScript. Is it even possible? Or am I missing something?
Documentation for the library says the image has to be 2B binary data (Little Endian). I don't understand it. Should I encode the image any other way?
One option would be encode each byte separately to ensure correct endianness.
img = "Qk2+AAAAAAAAAD4AAAAoAAAAIAAAACAAAAABAAEAAAAAAIAAAADEDgAAxA4AAAAAAAAAAAAAAAAAAP///wD//////+H////A////gH///wB///4AP//8AD//+AAf//AAD//gAA//yAAH/5wAB/+8EAP/fjAB/394Af+++AD/vPwAf8P+AH///gA///9AH///v5///7/P///fx///7+P//+/z///38f//+/n///38///+/f///v3///97////hw=="
str = atob(img)
buf = []
function hex(str, pos) {
return ('000' + (str.charCodeAt(pos) || 0).toString(16)).substr(-2);
}
for (var i = 0; i < str.length; i+= 4) {
buf.push(hex(str, i+2));
buf.push(hex(str, i+3));
buf.push(hex(str, i+0));
buf.push(hex(str, i+1));
}
console.log(buf.join(''))
This doesn't match your desired output exactly, are you sure it corresponds to the given base64 string?
On the other side, your initial output looks better, 424d are correct bytes to start a BMP file (BM signature), while be00 are not.
This is a Base64 string of a sample image (a black&white checkmark)...
It gives me this result, which is almost expected result:
424dbe00000003e0002800020000200001010000080000c4e00c4e00000000000000ffffff0ffffffffffe1ffffffc0ffffff807fffff07ffffe03ffffc03ffff801ffff00fffe00fffc807ff9c07ffbc103ff7e301ff7f781ffbef80ffbcfc07fc3fe07ffffe03fffff401fffffbf9fffffbfcfffffdfc7ffffefe3ffffeff3fffff7f1fffffbf9fffffdfcfffffefdfffffefdffffff7bffffff87
What do you mean by "almost expected result"?, those bytes make the image below (save as .bmp).
Your expected "a black&white checkmark" image :
PS:
The correct .bmp file has 190 bytes. Your library version (copied from logs) gives result of 194 bytes.
The library's version of bytes has a beginning 2-byte short for file size. That is BE 00 (= 190 when read as endian of 00 BE) followed by rest of Bitmap file bytes (190 total bytes). This makes total of 190 bytes + the 2 bytes of short. Then it adds these two mysterious 10 and 10 bytes starting at position 114. Grand total of 194 bytes.
To me... the library is corrupting the already fine image bytes, but you say the device accepts it?
Does it accept the "almost expected result" hex bytes too?
I'm using the following to convert an image to a base64 encoded string. On the client website in javascript:
var reader = new FileReader();
reader.onloadend = function () {
data64 = reader.result;
};
reader.readAsDataURL(myFile);
Now I pass this data to the server, which does the following:
var data = Convert.FromBase64String(data64);
However this results in a format exception:
The format of s is invalid. s contains a non-base-64 character, more than two padding characters, or a non-white space-character among the padding characters.
My input file is one of the sample images found on Windows 7 -> My Pictures/Sample Pictures/Tulips.jpeg
How can I attempt to debug what is causing the problem for such a large result output?
Okay, I have worked around this by using reader.readAsBinaryString instead and then converting this using btoa.
This seems to be accepted fine in Convert.FromBase64String
I experienced the same issue and founds out that my complete dataurl contained not only padding characters at the end, but also padding characters in the middle of the dataurl.
I used the following code to fix the base64string (but it still has a bug):
private static string getCleanedBase64String(string base64String)
{
string tempString = base64String.Replace("_", "/").Replace("-", "+").Replace("=", "");
return tempString + new string('=', (4 - tempString.Length % 4) % 4);
}