Related
I have a simple program which is decompressing a byte array in C#.
There is the code of "Decompress" func:
private byte[] Decompress(byte[] compressed)
{
using var from = new MemoryStream(compressed);
using var to = new MemoryStream();
using var gZipStream = new GZipStream(from, CompressionMode.Decompress);
gZipStream.CopyTo(to);
return to.ToArray();
}
and I have two byte arrays
var first = new byte[] { 31, 139, 8, 0, 0, 0, 0, 0, 0, 10, 45, 202, 65, 10, 128, 48, 12, 5, 209, 171, 132, 28, 195, 219, 68, 251, 91, 11, 38, 41, 105, 92, 137, 119, 23, 197, 221, 240, 152, 139, 39, 80, 120, 185, 88, 13, 234, 214, 55, 94, 88, 197, 154, 83, 115, 73, 154, 103, 84, 217, 64, 56, 48, 118, 177, 164, 130, 57, 164, 7, 5, 20, 186, 226, 141, 22, 72, 90, 97, 168, 61, 41, 251, 167, 7, 124, 72, 20, 250, 167, 233, 145, 124, 223, 15, 199, 8, 21, 161, 110, 0, 0, 0 };
var second = new byte[] { 31, 139, 8, 0, 0, 0, 0, 0, 0, 10, 1, 80, 0, 175, 255, 34, 127, 33, 66, 194, 252, 100, 6, 77, 155, 56, 146, 208, 225, 255, 219, 203, 33, 122, 153, 14, 246, 130, 28, 163, 183, 127, 145, 96, 71, 208, 206, 123, 9, 122, 133, 204, 255, 86, 191, 73, 197, 91, 247, 121, 156, 180, 101, 176, 125, 158, 123, 114, 192, 252, 70, 193, 148, 180, 183, 200, 146, 79, 121, 135, 152, 90, 247, 107, 168, 157, 128, 84, 121, 4, 122, 92, 145, 45, 223, 177, 179, 189, 149, 80, 0, 0, 0 }
my next step is:
var decompressed = Decompress(first);
var result = Encoding.UTF8.GetString(decompressed);
Output: {"seed":{"mnemonic":"mango goat surface elephant despair remember regret benefit timber leopard member sort"}}
var decompressed = Decompress(second);
var result = Encoding.UTF8.GetString(decompressed);
Output: "!B??d♠M?8??????!z???∟???`G??{ z???V?I?[?y??e?}?{r??F?????Oy??Z?k???Ty♦z\?-?
The structure of these arrays is the same... length (103), first 10 elements and last 3 elements. But first array decompresses with func Decompress fine and the second one is very bad. I don't know why it's happening. Someone explain where is my mistake.
I tried to decompress with python using gzip but I have the same result. There is the code
import gzip
first = [31, 139, 8, 0, 0, 0, 0, 0, 0, 10, 45, 202, 65, 10, 128, 48, 12, 5, 209, 171, 132, 28, 195, 219, 68, 251, 91, 11, 38, 41, 105, 92, 137, 119, 23, 197, 221, 240, 152, 139, 39, 80, 120, 185, 88, 13, 234, 214, 55, 94, 88, 197, 154, 83, 115, 73, 154, 103, 84, 217, 64, 56, 48, 118, 177, 164, 130, 57, 164, 7, 5, 20, 186, 226, 141, 22, 72, 90, 97, 168, 61, 41, 251, 167, 7, 124, 72, 20, 250, 167, 233, 145, 124, 223, 15, 199, 8, 21, 161, 110, 0, 0, 0 ];
second = [31, 139, 8, 0, 0, 0, 0, 0, 0, 10, 1, 80, 0, 175, 255, 34, 127, 33, 66, 194, 252, 100, 6, 77, 155, 56, 146, 208, 225, 255, 219, 203, 33, 122, 153, 14, 246, 130, 28, 163, 183, 127, 145, 96, 71, 208, 206, 123, 9, 122, 133, 204, 255, 86, 191, 73, 197, 91, 247, 121, 156, 180, 101, 176, 125, 158, 123, 114, 192, 252, 70, 193, 148, 180, 183, 200, 146, 79, 121, 135, 152, 90, 247, 107, 168, 157, 128, 84, 121, 4, 122, 92, 145, 45, 223, 177, 179, 189, 149, 80, 0, 0, 0 ];
bytes_of_values = bytes(first)
decompressed_block = gzip.decompress(bytes_of_values)
print(decompressed_block.decode('utf8', 'ignore'))
#Output: {"seed":{"mnemonic":"mango goat surface elephant despair remember regret benefit timber leopard member sort"}}
bytes_of_values = bytes(second)
decompressed_block = gzip.decompress(bytes_of_values)
print(decompressed_block.decode('utf8', 'ignore'))
#Output: "!Bd♠M8!z∟`G{ zVI[ye}{rFȒOyZkTy♦z\-
UPDATE #1
FIRST ARRAY was expanded with this JS function.
const crypto = require('crypto')
module.exports = function (size ) {
return {
expand (data ) {
const buffer = crypto.randomBytes(data.length < size - 4 ? size : data.length + 4)
buffer.writeUInt32BE(data.length, 0)
data.copy(buffer, 4, 0)
return buffer
},
shrink (buffer ) {
const dataLen = buffer.readUInt32BE(0)
return buffer.slice(4, dataLen + 4)
}
}
}
Where size = 2 ** 15 (2^15 = 32768); data = {"seed":{"mnemonic":"mango goat surface elephant despair remember regret benefit timber leopard member sort"}}. In output we have a Buffer with size 32768. After that I take this Buffer and convert to string like "[1, 2, 3 ,4 ,5, etc.]" (remind that size is 32768) and pass this to C#. In C# I use this code:
var shrink = BinaryPrimitives.ReadUInt32BigEndian(data); //data == array with size 32768 (string from JS)
//shrink == 103 (integer)
var segment = new ArraySegment<byte>(data, 4, (int)shrink).ToArray();
// segment == FIRST ARRAY
after following these steps I get the FIRST ARRAY and it decompresses fine. The structure of the first array are identical to SECOND ARRAY. And I understood that SECOND ARRAY uses the same algorithm. But it doesn't unpack.
I want to create with transforming old array.
arr1 = [ 4, 26, 40, 53, 58, 68, 73, 86, 91, 114, 119, 132, 137, 147, 152, 165, 170, 183, 188, 201, 206, 208, 215, 220, 233, 238, 250, 255, 267, 272, 284, 289, 301, 306, 318, 323 ]
I want to create like
newArr = [[4,5,6,7,8,9,10 ..., 24,25], [26,27,28 ..., 39], [40,...,52], ...]
As you see in arr1, the number is the beginning, and the next item - 1 is the last number. How I can do like this. The last number is 325 in arr1
I tried chunk_array, but it does not work.
You could use map and Array.from and then get all except the last one with slice(0, -1).
const arr = [4, 26, 40, 53, 58, 68, 73, 86, 91, 114, 119, 132, 137, 147, 152, 165, 170, 183, 188, 201, 206, 208, 215, 220, 233, 238, 250, 255, 267, 272, 284, 289, 301, 306, 318, 323]
const result = arr.map((e, i) => Array.from(Array((arr[i + 1] || e) - e), (_, j) => e + j)).slice(0, -1)
console.log(result)
1) You can use map and Array.from to achieve the desired result
const arr1 = [
4, 26, 40, 53, 58, 68, 73, 86, 91, 114, 119, 132, 137, 147, 152, 165, 170,
183, 188, 201, 206, 208, 215, 220, 233, 238, 250, 255, 267, 272, 284, 289,
301, 306, 318, 323,
];
const result = arr1.map((el, i, src) => {
const start = el, end = src[i + 1];
return Array.from({ length: end - start }, (_, i) => i + start);
});
result.pop();
console.log(result);
2) You can also do as
const arr1 = [
4, 26, 40, 53, 58, 68, 73, 86, 91, 114, 119, 132, 137, 147, 152, 165, 170,
183, 188, 201, 206, 208, 215, 220, 233, 238, 250, 255, 267, 272, 284, 289,
301, 306, 318, 323,
];
const result = arr1.map((el, i, src) => {
const start = el, end = src[i + 1] ?? start;
return Array(end - start)
.fill(0)
.map((_, i) => i + start);
});
result.pop();
console.log(result);
3) You can also use reduce here:
const arr1 = [
4, 26, 40, 53, 58, 68, 73, 86, 91, 114, 119, 132, 137, 147, 152, 165, 170,
183, 188, 201, 206, 208, 215, 220, 233, 238, 250, 255, 267, 272, 284, 289,
301, 306, 318, 323,
];
const result = arr1.reduce((acc, start, i, src) => {
const temp = Array((src[i + 1] ?? start) - start)
.fill(0)
.map((_, i) => i + start);
if (temp.length) acc.push(temp);
return acc;
}, []);
console.log(result);
const data = [ 4, 26, 40, 53, 58, 68, 73, 86, 91, 114, 119, 132, 137, 147, 152, 165, 170, 183, 188, 201, 206, 208, 215, 220, 233, 238, 250, 255, 267, 272, 284, 289, 301, 306, 318, 323 ]
const result = data.map((n, index, array) => Array.from({ length: ( array[index+1] || n+1 ) - n }, (_, i) => i+n));
console.log(result);
.as-console-wrapper { max-height: 100% !important; top: 0 }
i am trying to inflate a buffer with the zlib module in NodeJs. I had the problem that i always got the "unexpected end of file" error so i used my original data and let nodejs deflate it to compare the byte values.
(1) So this is the value that i have:
120, 156, 106, 46, 97, 96, 96, 96, 73, 97, 96, 76, 1, 49, 152, 82, 24, 216, 226, 75, 138, 18, 147, 83, 115, 128, 60, 198, 92, 32, 161, 17, 173, 148, 158, 88, 146, 90, 158, 88, 169, 91, 80, 148, 162, 155, 155, 152, 153, 167, 91, 104, 96, 94, 172, 164, 83, 173, 148, 155, 153, 92, 148, 95, 172, 100, 101, 160, 103, 80, 27, 155, 149, 194, 32, 148, 145, 154, 88, 84, 146, 148, 154, 88, 18, 159, 153, 87, 146, 90, 84, 150, 152, 147, 196, 192, 176, 80, 41, 133, 129, 41, 191, 32, 145, 11, 104, 75, 113, 10, 3, 115, 94, 102, 14, 144, 85, 2, 97, 1, 0, 0, 0, 255, 255
(2) And this is what zlib.deflateSync gave me:
120, 156, 1, 125, 0, 130, 255, 120, 156, 106, 46, 97, 96, 96, 96, 73, 97, 96, 76, 1, 49, 152, 82, 24, 216, 226, 75, 138, 18, 147, 83, 115, 128, 60, 198, 92, 32, 161, 17, 173, 148, 158, 88, 146, 90, 158, 88, 169, 91, 80, 148, 162, 155, 155, 152, 153, 167, 91, 104, 96, 94, 172, 164, 83, 173, 148, 155, 153, 92, 148, 95, 172, 100, 101, 160, 103, 80, 27, 155, 149, 194, 32, 148, 145, 154, 88, 84, 146, 148, 154, 88, 18, 159, 153, 87, 146, 90, 84, 150, 152, 147, 196, 192, 176, 80, 41, 133, 129, 41, 191, 32, 145, 11, 104, 75, 113, 10, 3, 115, 94, 102, 14, 144, 85, 2, 97, 1, 0, 0, 0, 255, 255, 164, 112, 54, 45, (bold = same)
So as far as i can tell from the RFC the first two bytes are the compression method and flags and therefor both have it. The other bold part is the compressed data. So far so good.
The first part that doesn't match is the DICTID part. But according to the RFC it is only four bytes long so what are the other three bytes (255, 120, 156)? The last part that is not bold is the ADLER32 checksum of the compressed data.
Question: Is zlib (nodejs) somehow able to inflate data without the DICTID and ADLER32 checksum?
Current Code Snippet:
const data = /* Array from (1) */;
const buffer = Buffer.from(data);
const out = zlib.inflateSync(buffer);
Output: Error: unexpected end of file
Background: What i am ultimately trying to do is the following but in javascript instead of python.
Note: The command line program zlib-flate is somehow able to inflate the bytes from (1) so i am assuming i do something wrong.
After hours of trail and error i finally figured it out what the problem was.
To be able to inflate the bytes from (1) you have to set the finishFlush option:
const result = zlib.inflateSync(buffer, {finishFlush: zlib.constants.Z_SYNC_FLUSH});
I am using eccrypto.js library for public key encryption.
Here I want to encrypt the message of A using public key of B and then let B to decrypt using private key of B.
In the library this is done by:
// Encrypting the message for B.
eccrypto.encrypt(publicKeyB, Buffer.from("msg to b")).then(function(encrypted) {
// B decrypting the message.
eccrypto.decrypt(privateKeyB, encrypted).then(function(plaintext) {
console.log("Message to part B:", plaintext.toString());
});
});
if I console log the encrypted value I got the object as:
{iv: Uint8Array(16), ephemPublicKey: Uint8Array(65), ciphertext: Uint8Array(48), mac: Uint8Array(32)}
ciphertext: Uint8Array(48) [13, 240, 10, 109, 88, 109, 108, 153, 213, 115, 40, 237, 66, 232, 251, 120, 27, 67, 119, 231, 17, 143, 78, 69, 43, 76, 214, 74, 132, 127, 220, 131, 44, 144, 221, 133, 48, 124, 239, 158, 226, 22, 119, 200, 170, 101, 241, 82]
ephemPublicKey: Uint8Array(65) [4, 84, 253, 207, 251, 2, 157, 203, 14, 233, 166, 216, 107, 1, 23, 90, 229, 209, 150, 58, 95, 253, 214, 183, 148, 167, 224, 15, 224, 244, 176, 165, 84, 121, 70, 4, 175, 186, 189, 104, 211, 207, 255, 195, 20, 128, 200, 237, 7, 9, 173, 234, 14, 208, 208, 68, 46, 76, 38, 26, 107, 41, 10, 188, 108]
iv: Uint8Array(16) [230, 246, 79, 17, 203, 191, 117, 7, 57, 149, 198, 68, 193, 220, 159, 56]
mac: Uint8Array(32) [202, 77, 212, 211, 27, 186, 174, 106, 211, 145, 100, 81, 100, 68, 61, 172, 175, 188, 213, 49, 63, 92, 172, 83, 30, 22, 47, 93, 60, 215, 33, 116]
Note: I have to store this in json format so i have converted it into json data.
Now if I convert this object to json and recover the json data the recovered object is not same.
Json object:
{
"iv":{"type":"Buffer","data":[226,253,245,0,227,222,47,37,65,177,171,68,201,142,242,35]},
"ephemPublicKey":{"type":"Buffer","data":[4,9,137,99,138,202,169,89,90,209,92,130,156,105,170,132,192,250,88,232,15,250,33,107,38,13,129,178,21,237,77,136,215,39,215,123,140,226,102,98,39,110,192,209,79,214,138,83,174,192,100,183,157,44,56,128,38,52,170,244,42,213,199,57,232]},
"ciphertext":{"type":"Buffer","data":[135,147,187,164,109,39,204,244,195,161,65,24,178,160,132,146,200,35,113,120,164,140,20,223,225,104,23,111,13,155,193,26,35,73,236,77,209,246,85,16,77,30,250,122,206,242,111,63]},
"mac":{"type":"Buffer","data":[79,195,220,150,230,150,13,187,9,131,12,81,151,107,29,216,138,143,85,52,153,71,179,167,243,141,107,88,97,206,110,107]}}
The recovered object (JSON.parse(encrypted)) is:
{iv: {…}, ephemPublicKey: {…}, ciphertext: {…}, mac: {…}}
ciphertext:
data: (48) [135, 147, 187, 164, 109, 39, 204, 244, 195, 161, 65, 24, 178, 160, 132, 146, 200, 35, 113, 120, 164, 140, 20, 223, 225, 104, 23, 111, 13, 155, 193, 26, 35, 73, 236, 77, 209, 246, 85, 16, 77, 30, 250, 122, 206, 242, 111, 63]
type: "Buffer"
__proto__: Object
ephemPublicKey:
data: (65) [4, 9, 137, 99, 138, 202, 169, 89, 90, 209, 92, 130, 156, 105, 170, 132, 192, 250, 88, 232, 15, 250, 33, 107, 38, 13, 129, 178, 21, 237, 77, 136, 215, 39, 215, 123, 140, 226, 102, 98, 39, 110, 192, 209, 79, 214, 138, 83, 174, 192, 100, 183, 157, 44, 56, 128, 38, 52, 170, 244, 42, 213, 199, 57, 232]
type: "Buffer"
__proto__: Object
iv:
data: (16) [226, 253, 245, 0, 227, 222, 47, 37, 65, 177, 171, 68, 201, 142, 242, 35]
type: "Buffer"
__proto__: Object
mac:
data: (32) [79, 195, 220, 150, 230, 150, 13, 187, 9, 131, 12, 81, 151, 107, 29, 216, 138, 143, 85, 52, 153, 71, 179, 167, 243, 141, 107, 88, 97, 206, 110, 107]
type: "Buffer"
__proto__: Object
__proto__: Object
if i used the recovered object to decrypt I am getting the error as bad public key.
My code is:
let PublicKey = Buffer.from("0418c7ced07c0c17f42b132747c70fddb6b31ea0ad349c2e9f800f48f0a73c2ea028d41b239077a48136ce546f9d2811bf1ec311c56e6a41f33906a1fc2472e451", 'hex')
eccrypto.encrypt(PublicKey, Buffer.from('Message to encrypt').then(function (encrypted) {
console.log(encrypted)
let encoded = JSON.stringify(encrypted)
console.log(encoded)
var actual = JSON.parse((encoded))
console.log(actual)
let pk = Buffer.from("9a2d66404b69023c2c45da81ca4b696a8234b7ae53ea6b7ffc0d6bdd0e0e3279", 'hex')
eccrypto.decrypt(pk, actual).then(function (plaintext) {
console.log("Message to part B:", plaintext.toString());
});
});
I am getting error as bad public key.
If i used the encrypted variable instead of acutal in the eccrypto.decrypt(pk,encrypted)...
then i get the decrypted value.
You can add a second argument to JSON.parse to allow you to inject code to help interpret the JSON as it's deserialized. It's called a reviver in the documentation.
You will get each key, value pair in the object.
Here is some sample code:
JSON.parse('{"p": 5}', (key, value) =>
typeof value === 'number'
? value * 2 // return value * 2 for numbers
: value // return everything else unchanged
);
Here is mode info on MDN:
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/parse
In your example, I think you want to identify the values as type buffer and then replace them with UInt8Array
or create a new object like this:
const newWorkingObject = {
ciphertext: Buffer.from([...recoveredObject.ciphertext.data]),
ephemPublicKey: Buffer.from([...recoveredObject.ephemPublicKey.data]),
iv: Buffer.from([...recoveredObject.iv.data]),
mac: Buffer.from([...recoveredObject.mac.data]),
}
I can convert Unit8Array to hex by using below code
var bkh = {
publicKey: new Uint8Array([91, 221, 234, 40, 144, 246, 91, 187, 154, 76,
60, 178, 204, 81, 35, 195, 254, 114, 246, 88, 90, 170, 68, 97, 199,
170, 72, 36, 107, 66, 206, 9]
),
secretKey: new Uint8Array([64, 68, 196, 103, 210, 179, 166, 40, 187,
150, 167, 233, 144, 206, 64, 26, 77, 133, 70, 238, 232, 227, 133,
83, 149, 202, 213, 41, 152, 243, 237, 41]
)
}
let hex = Buffer.from(bkh.publicKey).toString('hex');
console.log('master key',hex)
How can I convert this hex value back to Unit8Array in node
var hex = '5bddea2890f65bbb9a4c3cb2cc5123c3fe72f6585aaa4461c7aa48246b42ce09'
new Uint8Array(Buffer.from(hex, 'hex'))
// or
Uint8Array.from(Buffer.from(hex, 'hex'))