We Keep Coding

Crypto-js encryption and Python decryption using HKDF key

Based on the example provided here on how to establish a shared secret and derived key between JS (Crypto-JS) and Python, I can end up with the same shared secret and derived key on both ends.
However, when I try to encrypt as below, I cannot find a way to properly decrypt from Python. My understanding is that probably I am messing with the padding or salts and hashes.
const payload = "hello"
var iv = CryptoJS.enc.Utf8.parse("1020304050607080");
var test = CryptoJS.AES.encrypt(
payload,
derived_key,
{iv: iv, mode: CryptoJS.mode.CBC}
).toString();
console.log(test)
Output "y+In4kriw0qy4lji6/x14g=="
Python (one of the attempts):
from Crypto.Cipher import AES
from Crypto.Util.Padding import pad,unpad
iv = "1020304050607080"
test_enc = "y+In4kriw0qy4lji6/x14g=="
enc = base64.b64decode(test_enc)
cipher = AES.new(derived_key, AES.MODE_CBC, iv.encode('utf-8'))
print(base64.b64decode(cipher.decrypt(enc)))
print(unpad(cipher.decrypt(enc),16))
Any guidance here would be greatly appreciated as I am stuck for quite some time.
(I have encryption working using a password, but struggling with HKDF).
EDIT:
Here is the full Python code:
from cryptography.hazmat.primitives.asymmetric import ec
from cryptography.hazmat.primitives.kdf.hkdf import HKDF
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives import serialization
import base64
from Crypto.Cipher import AES
from Crypto.Util.Padding import pad,unpad
def deriveKey():
server_pkcs8 = b'''-----BEGIN PRIVATE KEY-----
MIG2AgEAMBAGByqGSM49AgEGBSuBBAAiBIGeMIGbAgEBBDBReGpDVmoVTzxNbJx6
aL4L9z1EdB91eonAmAw7mKDocLfCJITXZPUAmM46c6AipTmhZANiAAR3t96P0ZhU
jtW3rHkHpeGu4e+YT+ufMiMeanE/w8p+d9aCslvIbZyBBzeZ/266yqTUUoiYDzqv
Hb5q8rz7vEgr3DG4XfHYpCqfE2nttQGK3emHKGnvY239AteZkdwMpcs=
-----END PRIVATE KEY-----'''
client_x509 = b'''-----BEGIN PUBLIC KEY-----
MHYwEAYHKoZIzj0CAQYFK4EEACIDYgAEm0xeyy3nVnYpOpx/CV/FnlNEdWUZaqtB
AGf7flKxXEjmlSUjseYzCd566sLpNg56Gw6hcFx+rWTLGR4eDRWfmwlXhyUasuEg
mb0BQf8XJLBdvadb9eFx2CP1yjBsiy8e
-----END PUBLIC KEY-----'''
client_public_key = serialization.load_pem_public_key(client_x509)
server_private_key = serialization.load_pem_private_key(server_pkcs8, password=None)
shared_secret = server_private_key.exchange(ec.ECDH(), client_public_key)
print('Shared secret: ' + base64.b64encode(shared_secret).decode('utf8')) # Shared secret: xbU6oDHMTYj3O71liM5KEJof3/0P4HlHJ28k7qtdqU/36llCizIlOWXtj8v+IngF
salt_bytes = "12345678".encode('utf-8')
info_bytes = "abc".encode('utf-8')
derived_key = HKDF(
algorithm=hashes.SHA256(),
length=32,
salt=salt_bytes,
info=info_bytes,
).derive(shared_secret)
print('Derived key: ' + base64.b64encode(derived_key).decode('utf8'))
return derived_key
derived_key = deriveKey()
iv = "1020304050607080"
test_enc = "y+In4kriw0qy4lji6/x14g=="
enc = base64.b64decode(test_enc)
cipher = AES.new(derived_key, AES.MODE_CBC, iv.encode('utf-8'))
print(base64.b64decode(cipher.decrypt(enc)))
print(unpad(cipher.decrypt(enc),16))

The issue is that the key is not passed correctly in the CryptoJS code.
The posted Python code generates LefjQ2pEXmiy/nNZvEJ43i8hJuaAnzbA1Cbn1hOuAgA= as Base64-encoded key. This must be imported in the CryptoJS code using the Base64 encoder:
const payload = "hello"
var derived_key = CryptoJS.enc.Base64.parse("LefjQ2pEXmiy/nNZvEJ43i8hJuaAnzbA1Cbn1hOuAgA=")
var iv = CryptoJS.enc.Utf8.parse("1020304050607080");
var test = CryptoJS.AES.encrypt(payload, derived_key, {iv: iv, mode: CryptoJS.mode.CBC}).toString();
document.getElementById("ct").innerHTML = test; // bLdmGA+HLLyFEVtBEuCzVg==
<script src="https://cdnjs.cloudflare.com/ajax/libs/crypto-js/4.1.1/crypto-js.min.js"></script>
<p style="font-family:'Courier New', monospace;" id="ct"></p>
The hereby generated ciphertext bLdmGA+HLLyFEVtBEuCzVg== can be decrypted with the Python code:
from Crypto.Cipher import AES
from Crypto.Util.Padding import unpad
import base64
test_enc = "bLdmGA+HLLyFEVtBEuCzVg=="
enc = base64.b64decode(test_enc)
derived_key = base64.b64decode("LefjQ2pEXmiy/nNZvEJ43i8hJuaAnzbA1Cbn1hOuAgA=")
iv = "1020304050607080"
cipher = AES.new(derived_key, AES.MODE_CBC, iv.encode('utf-8'))
print(unpad(cipher.decrypt(enc),16)) # b'hello'
Note that for security reasons, a static IV should not be used so that key/IV pairs are not repeated.

Ruby and other languages(Python, JavaScript) different hmac sha256 result

I have Ruby 2.5.3 code that create hmac with sha256
require 'openssl'
key = '4629de5def93d6a2abea6afa9bd5476d9c6cbc04223f9a2f7e517b535dde3e25'
message = 'lucas'
hash = OpenSSL::HMAC.hexdigest('sha256', key, message)
hash => ba2e2505c6f302fb3c40bea4491d95bacd96c3d12e8fbe50197ca431165fcee2
But the result is different from Python & JavaScript code. What should I do in Ruby code to have the same result as from the others?
Python
import hmac
import hashlib
import binascii
message = "lucas"
key = "4629de5def93d6a2abea6afa9bd5476d9c6cbc04223f9a2f7e517b535dde3e25"
hash = hmac.new(
binascii.unhexlify(bytearray(key, "utf-8")),
msg=message.encode('utf-8'),
digestmod=hashlib.sha256
).hexdigest()
hash => 99427c7bba36a6902c5fd6383f2fb0214d19b81023296b4bd6b9e024836afea2
JavaScript
const crypto = require('crypto');
const message = 'lucas';
const key = '4629de5def93d6a2abea6afa9bd5476d9c6cbc04223f9a2f7e517b535dde3e25';
const hash = crypto.createHmac('sha256', Buffer.from(key, 'hex'))
.update(message)
.digest('hex');
hash => 99427c7bba36a6902c5fd6383f2fb0214d19b81023296b4bd6b9e024836afea2

In Python and JS you are using the "key" as hexstring, means that the hexstring is converted to a binary format. In Ruby the key is used without conversion. – Michael Fehr
This gave me the solution.
key = '4629de5def93d6a2abea6afa9bd5476d9c6cbc04223f9a2f7e517b535dde3e25'
message = 'lucas'
puts OpenSSL::HMAC.hexdigest('sha256', [key].pack('H*') , message)
hash => 99427c7bba36a6902c5fd6383f2fb0214d19b81023296b4bd6b9e024836afea2
thank you!

Different encryption values from NodeJs Crypto and CryptoJS library

I am doing Encryption and Decryption using both NodeJS Crypto and CryptoJS library in both front-end and backend. I am able to do the encryption using NodeJS Crypto which works perfectly fine and the encrypted string is also consistent with the Java encryption. But when I use, CryptoJS library the encryption string does not as expected. I provide below the standalone Typescript code.
Please help me. I am using aes-256-cfb8 Algorithm. Is it because of this Algorithm which may not be supported by CryptoJS library? Please give me some direction, I am struck now.
import * as crypto from "crypto";
import CryptoJS, { mode } from "crypto-js";
export class Test3 {
private static readonly CRYPTO_ALGORITHM = 'aes-256-cfb8'
private static readonly DEFAULT_IV: string = "0123456789123456";
// NodeJS Crypto - Works fine
public encryptValue(valueToEncrypt: string, secretKey: string): string {
let encryptedValue: string = '';
const md5 = crypto.createHash('md5').update(secretKey).digest('hex');
const key: string = md5;
console.log("MD5 Value : ", md5);
const iv = Buffer.from(Test3.DEFAULT_IV)
console.log("iv: ", iv)
const cipher = crypto.createCipheriv(Test3.CRYPTO_ALGORITHM, key, iv);
let encrypted = cipher.update(valueToEncrypt, 'utf8', 'base64');
console.log("encrypted ::: ", encrypted);
encrypted += cipher.final('base64');
encryptedValue = encrypted;
return encryptedValue;
}
// CryptoJS library - does not work
public check3(valueToEncrypt: string, secretKey: string): void {
const hash = CryptoJS.MD5(secretKey);
const md5Key: string = hash.toString(CryptoJS.enc.Hex)
console.log("MD5 Value: ", md5Key); // Upto correct
const IV11 = CryptoJS.enc.Utf8.parse(Test3.DEFAULT_IV);
const key11 = CryptoJS.enc.Utf8.parse(md5Key);
const data = CryptoJS.enc.Utf8.parse(valueToEncrypt);
const enc1 = CryptoJS.AES.encrypt(data, key11, {
format: CryptoJS.format.OpenSSL,
iv: IV11,
mode: CryptoJS.mode.CFB,
padding: CryptoJS.pad.NoPadding
});
console.log("Cipher text11: ", enc1.toString());
}
}
const text2Encrypt = "A brown lazy dog";
const someSecretKey: string = "projectId" + "~" + "India";
const test = new Test3();
// Using NodeJS Crypto
const enc = test.encryptValue(text2Encrypt, someSecretKey);
console.log("Encrypted Value Using NodeJS Crypto: ", enc); // ===> nJG4Fk27JZ7E5klLqFAt
// Using CryptoJS
console.log("****************** Using CryptoJS ******************")
test.check3(text2Encrypt, someSecretKey);
Please help me for the method check3() of the above class. Also I have checked many links in SO.

You suspect right. Both codes use different variants of the CFB mode. For CFB a segment size must be defined. The segment size of the CFB mode corresponds to the bits encrypted per encryption step, see CFB.
The crypto module of NodeJS supports several segment sizes, e.g. 8 bits (aes-256-cfb8) or 128 bits (aes-256-cfb), while CryptoJS only supports a segment size of 128 bits (CryptoJS.mode.CFB).
If the segment size is changed to 128 bits (aes-256-cfb) in the crypto module, both ciphertexts match.
Edit:
In this post you will find an extension of CryptoJS for the CFB mode, so that also a variable segment size, e.g. 8 bits, is supported.

CryptoJS to Pycrypto AES [duplicate]

I have a password which is encrypt from JavaScript via
var password = 'sample'
var passphrase ='sample_passphrase'
CryptoJS.AES.encrypt(password, passphrase)
Then I tried to decrypt the password comes from JavaScript in Python:
from Crypto.Cipher import AES
import base64
PADDING = '\0'
pad_it = lambda s: s+(16 - len(s)%16)*PADDING
key = 'sample_passphrase'
iv='11.0.0.101' #------> here is my question, how can I get this iv to restore password, what should I put here?
key=pad_it(key) #------> should I add padding to keys and iv?
iv=pad_it(iv) ##
source = 'sample'
generator = AES.new(key, AES.MODE_CFB,iv)
crypt = generator.encrypt(pad_it(source))
cryptedStr = base64.b64encode(crypt)
print cryptedStr
generator = AES.new(key, AES.MODE_CBC,iv)
recovery = generator.decrypt(crypt)
print recovery.rstrip(PADDING)
I checked JS from browser console, it shows IV in CryptoJS.AES.encrypt(password, passphrase) is a object with some attributes( like sigBytes:16, words: [-44073646, -1300128421, 1939444916, 881316061]). It seems generated randomly.
From one web page, it tells me that JS has two way to encrypt password
(reference link ):
a. crypto.createCipher(algorithm, password)
b. crypto.createCipheriv(algorithm, key, iv)
What I saw in JavaScript should be option a. However, only option b is equivalent to AES.new() in python.
The questions are:
How can I restore this password in Python without changing JavaScript code?
If I need IV in Python, how can I get it from the password that is used in JavaScript?

You will have to implement OpenSSL's EVP_BytesToKey, because that is what CryptoJS uses to derive the key and IV from the provided password, but pyCrypto only supports the key+IV type encryption. CryptoJS also generates a random salt which also must be send to the server. If the ciphertext object is converted to a string, then it uses automatically an OpenSSL-compatible format which includes the random salt.
var data = "Some semi-long text for testing";
var password = "some password";
var ctObj = CryptoJS.AES.encrypt(data, password);
var ctStr = ctObj.toString();
out.innerHTML = ctStr;
<script src="https://cdn.rawgit.com/CryptoStore/crypto-js/3.1.2/build/rollups/aes.js"></script>
<div id="out"></div>
Possible output:
U2FsdGVkX1+ATH716DgsfPGjzmvhr+7+pzYfUzR+25u0D7Z5Lw04IJ+LmvPXJMpz
CryptoJS defaults to 256 bit key size for AES, PKCS#7 padding and CBC mode. AES has a 128 bit block size which is also the IV size. This means that we have to request 32+16 = 48 byte from EVP_BytesToKey. I've found a semi-functional implementation here and extended it further.
Here is the full Python (tested with 2.7 and 3.4) code, which is compatible with CryptoJS:
from Cryptodome import Random
from Cryptodome.Cipher import AES
import base64
from hashlib import md5
BLOCK_SIZE = 16
def pad(data):
length = BLOCK_SIZE - (len(data) % BLOCK_SIZE)
return data + (chr(length)*length).encode()
def unpad(data):
return data[:-(data[-1] if type(data[-1]) == int else ord(data[-1]))]
def bytes_to_key(data, salt, output=48):
# extended from https://gist.github.com/gsakkis/4546068
assert len(salt) == 8, len(salt)
data += salt
key = md5(data).digest()
final_key = key
while len(final_key) < output:
key = md5(key + data).digest()
final_key += key
return final_key[:output]
def encrypt(message, passphrase):
salt = Random.new().read(8)
key_iv = bytes_to_key(passphrase, salt, 32+16)
key = key_iv[:32]
iv = key_iv[32:]
aes = AES.new(key, AES.MODE_CBC, iv)
return base64.b64encode(b"Salted__" + salt + aes.encrypt(pad(message)))
def decrypt(encrypted, passphrase):
encrypted = base64.b64decode(encrypted)
assert encrypted[0:8] == b"Salted__"
salt = encrypted[8:16]
key_iv = bytes_to_key(passphrase, salt, 32+16)
key = key_iv[:32]
iv = key_iv[32:]
aes = AES.new(key, AES.MODE_CBC, iv)
return unpad(aes.decrypt(encrypted[16:]))
password = "some password".encode()
ct_b64 = "U2FsdGVkX1+ATH716DgsfPGjzmvhr+7+pzYfUzR+25u0D7Z5Lw04IJ+LmvPXJMpz"
pt = decrypt(ct_b64, password)
print("pt", pt)
print("pt", decrypt(encrypt(pt, password), password))
Similar code can be found in my answers for Java and PHP.
JavaScript AES encryption in the browser without HTTPS is simple obfuscation and does not provide any real security, because the key must be transmitted alongside the ciphertext.
[UPDATE]:
You should use pycryptodome instead of pycrypto because pycrypto(latest pypi version is 2.6.1) no longer maintained and it has vulnerabilities CVE-2013-7459 and CVE-2018-6594 (CVE warning reported by github). I choose pycryptodomex package here(Cryptodome replace Crypto in code) instead of pycryptodome package to avoid conflict name with Crypto from pycrypto package.

Encrypt in python - decrypt in Javascript

I have need to simply encrypt some text in python and being able to decrypt in JavaScrypt.
So far I have in python:
from Crypto import Random
from Crypto.Cipher import AES
import base64
BLOCK_SIZE = 16
key = "1234567890123456" # want to be 16 chars
textToEncrypt = "This is text to encrypt"
def encrypt(message, passphrase):
# passphrase MUST be 16, 24 or 32 bytes long, how can I do that ?
IV = Random.new().read(BLOCK_SIZE)
aes = AES.new(passphrase, AES.MODE_CFB, IV)
return base64.b64encode(aes.encrypt(message))
def decrypt(encrypted, passphrase):
IV = Random.new().read(BLOCK_SIZE)
aes = AES.new(passphrase, AES.MODE_CFB, IV)
return aes.decrypt(base64.b64decode(encrypted))
print encrypt( textToEncrypt, key )
this is producing text: ZF9as5JII5TlqcB5tAd4sxPuBXd5TrgE
in JavaScript:
<script src="http://crypto-js.googlecode.com/svn/tags/3.1.2/build/rollups/aes.js"></script>
<script>
var decrypted = CryptoJS.AES.decrypt( "ZF9as5JII5TlqcB5tAd4sxPuBXd5TrgE", "1234567890123456");
console.log ( decrypted.toString( CryptoJS.enc.Utf8 ) );
</script>
however it does not produce original string (empty string instead).
What I am doing wrong ?
Is it focusing on AES is a best idea - I will be happy if I have some kind of encryption that will blur data.

There are many problems with your Python code and CryptoJS code:
You use a random IV to encrypt some plaintext in Python. If you want to retrieve that plaintext, you need to use the same IV during decryption. The plaintext cannot be recovered without the IV. Usually the IV is simply prepended to the ciphertext, because it doesn't have to be secret. So you need to read the IV during decryption and not generate a new one.
You use CBC mode in CryptoJS (default) instead of CFB mode. The mode has to be the same. The other tricky part is that CFB mode is parametrized with a segment size. PyCrypto uses by default 8-bit segments (CFB8), but CryptoJS is only implemented for fixed segments of 128-bit (CFB128). Since the PyCrypto version is variable, you need to change that.
The CryptoJS decrypt() function expects as ciphertext either an OpenSSL formatted string or a CipherParams object. Since you don't have an OpenSSL formatted string, you have to convert the ciphertext into an object.
The key for CryptoJS is expected to be a WordArray and not a string.
Use the same padding. PyCrypto doesn't pad the plaintext if CFB8 is used, but padding is needed when CFB128 is used. CryptoJS uses PKCS#7 padding by default, so you only need to implement that padding in python.
Python code (for version 2):
def pad(data):
length = 16 - (len(data) % 16)
return data + chr(length)*length
def unpad(data):
return data[:-ord(data[-1])]
def encrypt(message, passphrase):
IV = Random.new().read(BLOCK_SIZE)
aes = AES.new(passphrase, AES.MODE_CFB, IV, segment_size=128)
return base64.b64encode(IV + aes.encrypt(pad(message)))
def decrypt(encrypted, passphrase):
encrypted = base64.b64decode(encrypted)
IV = encrypted[:BLOCK_SIZE]
aes = AES.new(passphrase, AES.MODE_CFB, IV, segment_size=128)
return unpad(aes.decrypt(encrypted[BLOCK_SIZE:]))
JavaScript code:
<script src="https://cdn.rawgit.com/CryptoStore/crypto-js/3.1.2/build/rollups/aes.js"></script>
<script src="https://cdn.rawgit.com/CryptoStore/crypto-js/3.1.2/build/components/mode-cfb-min.js"></script>
<script>
var base64ciphertextFromPython = "...";
var ciphertext = CryptoJS.enc.Base64.parse(base64ciphertextFromPython);
// split iv and ciphertext
var iv = ciphertext.clone();
iv.sigBytes = 16;
iv.clamp();
ciphertext.words.splice(0, 4); // delete 4 words = 16 bytes
ciphertext.sigBytes -= 16;
var key = CryptoJS.enc.Utf8.parse("1234567890123456");
// decryption
var decrypted = CryptoJS.AES.decrypt({ciphertext: ciphertext}, key, {
iv: iv,
mode: CryptoJS.mode.CFB
});
console.log ( decrypted.toString(CryptoJS.enc.Utf8));
</script>
Other considerations:
It seems that you want to use a passphrase as a key. Passphrases are usually human readable, but keys are not. You can derive a key from a passphrase with functions such as PBKDF2, bcrypt or scrypt.
The code above is not fully secure, because it lacks authentication. Unauthenticated ciphertexts may lead to viable attacks and unnoticed data manipulation. Usually the an encrypt-then-MAC scheme is employed with a good MAC function such as HMAC-SHA256.

(1 Year later but I hope this works for someone)
First of all, thanks Artjom B. your post helps me a lot. And Like OP, I have the same same problem Python server endonding and Javascript client decoding. This was my solution:
Python 3.x (Server)
I used an excplicit PKCS7 encode for padding, why? because I want to be sure Im using the same padding enconding and decoding, this is the link where I found it http://programmerin.blogspot.com.co/2011/08/python-padding-with-pkcs7.html .
Then, like Artjom B. said, be sure about your segment size, IV size and AES mode (CBC for me),
This is the code:
def encrypt_val(clear_text):
master_key = '1234567890123456'
encoder = PKCS7Encoder()
raw = encoder.encode(clear_text)
iv = Random.new().read( 16 )
cipher = AES.new( master_key, AES.MODE_CBC, iv, segment_size=128 )
return base64.b64encode( iv + cipher.encrypt( raw ) )
Note than your are enconding on base64 the concatenation of IV and encryption data.
Javascript (client)
function decryptMsg (data) {
master_key = '1234567890123456';
// Decode the base64 data so we can separate iv and crypt text.
var rawData = atob(data);
// Split by 16 because my IV size
var iv = rawData.substring(0, 16);
var crypttext = rawData.substring(16);
//Parsers
crypttext = CryptoJS.enc.Latin1.parse(crypttext);
iv = CryptoJS.enc.Latin1.parse(iv);
key = CryptoJS.enc.Utf8.parse(master_key);
// Decrypt
var plaintextArray = CryptoJS.AES.decrypt(
{ ciphertext: crypttext},
key,
{iv: iv, mode: CryptoJS.mode.CBC, padding: CryptoJS.pad.Pkcs7}
);
// Can be Utf8 too
output_plaintext = CryptoJS.enc.Latin1.stringify(plaintextArray);
console.log("plain text : " + output_plaintext);
}
One of my main problem was keep in mind all kind of encoding and decoding data, for example, I didn't know that the master_key on client side was to be parse with Utf8.

//First pip install pycryptodome -- (pycrypto is obsolete and gives issues)
// pip install pkcs7
from Crypto import Random
from Crypto.Cipher import AES
import base64
from pkcs7 import PKCS7Encoder
from app_settings.views import retrieve_settings # my custom settings
app_secrets = retrieve_settings(file_name='secrets');
def encrypt_data(text_data):
#limit to 16 bytes because my encryption key was too long
#yours could just be 'abcdefghwhatever'
encryption_key = app_secrets['ENCRYPTION_KEY'][:16];
#convert to bytes. same as bytes(encryption_key, 'utf-8')
encryption_key = str.encode(encryption_key);
#pad
encoder = PKCS7Encoder();
raw = encoder.encode(text_data) # Padding
iv = Random.new().read(AES.block_size ) #AES.block_size defaults to 16
# no need to set segment_size=BLAH
cipher = AES.new( encryption_key, AES.MODE_CBC, iv )
encrypted_text = base64.b64encode( iv + cipher.encrypt( str.encode(raw) ) )
return encrypted_text;