Python Code:
signature = hmac.new(bytearray.fromhex(key), data.encode('utf-8'), hashlib.sha256).hexdigest()
Solutions That I have tried
var compute_hmac = crypto.createHmac('sha256', key).update(data).digest('hex');
var compute_hmac = crypto.createHmac('sha256', Buffer.from(key, 'hex').toString()).update(data).digest('hex');
const hmac = crypto.createHmac('sha256', Buffer.from(key, 'hex'))
Trying to validate webhook signatures of the following API
https://developer.close.com/topics/webhooks/
data is the payload received, the same thing is passed to python and JS code. But somehow, hex digest of python code is validated and hex code of JS code is entirely different.
Please refer to API link mentioned above (webhook signatures) to understand what I'm trying to achieve
Pass directly the keybuffer instead of adding .toString() to it
var compute_hmac = crypto.createHmac('sha256', Buffer.from(key, 'hex')).update(data).digest('hex');
py code
import hashlib
import hmac
key ="A1FF92";
data = "hello"
signature = hmac.new(bytearray.fromhex(key), data.encode('utf-8'), hashlib.sha256).hexdigest()
//78a1151ddd4f298a134e4625362af2ab8ef4bd49719e17053ec1eadd4cbf1bab
node code
var crypto = require("crypto")
var key = "A1FF92"
var data="hello";
var compute_hmac = crypto.createHmac('sha256', Buffer.from(key, 'hex')).update(data).digest('hex');
// 78a1151ddd4f298a134e4625362af2ab8ef4bd49719e17053ec1eadd4cbf1bab
The scenario is that I have a JS script that creates a HMAC for a user provided input and I want to compute the same HMAC for the same input using python. To make things clearer, consider the following JS and Python code snippets.
Javascript
<script src="https://cdnjs.cloudflare.com/ajax/libs/crypto-js/3.1.9-1/crypto-js.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/crypto-js/3.1.9-1/hmac-sha256.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/crypto-js/3.1.9-1/enc-base64.min.js"></script>
<script>
var secretAccessKey = "bAvW5O18eSrxke4I7eFcrnrDJkN+wKQmx9aSHuMZQ0w=";
var stringtoSign = "Test";
// Generate HMAC SHA256 signature
var secretAccessKeyBase64 = CryptoJS.enc.Base64.parse(secretAccessKey);
var hash = CryptoJS.HmacSHA256(stringtoSign, secretAccessKeyBase64);
var signature = CryptoJS.enc.Base64.stringify(hash);
</script>
Python
stringToSign = "Test"
secretAccessKey = "bAvW5O18eSrxke4I7eFcrnrDJkN+wKQmx9aSHuMZQ0w="
secretAccessKeyBase64 = base64.b64decode(secretAccessKey).hex()
keyBytes = bytes(secretAccessKeyBase64, 'utf-8')
stringToSignBytes = bytes(stringToSign, 'utf-8')
signatureHash = hmac.new(keyBytes, stringToSignBytes, digestmod=hashlib.sha256).digest()
signature = base64.b64encode(signatureHash)
print(signature)
The Javascript code gives me b+1wRzDODA85vyDZkXByPIKO5qmnjCRNF5gZFi33/Ic=, while python gives me the value b'SsZ4bcYe3op1nGU6bySzlSc9kgg9Kgp37qzF15s2zNc='
Why is my python code generating a different HMAC for (seemingly) identical inputs that was provided to the JS script? Is there anyway to obtain the HMAC value outputted by the JS code using python?
You are using a Base64 encoded value as secret in Javascript, whereas in Python you use the plain text secret.
<script>
var secretAccessKeyBase64 = "secret";
var hash = CryptoJS.HmacSHA256("Message", secretAccessKeyBase64);
var hashInBase64 = CryptoJS.enc.Base64.stringify(hash);
document.write(hashInBase64);
</script>
This prints out the same value as the Python code:
qnR8UCqJggD55PohusaBNviGoOJ67HC6Btry4qXLVZc=
Edit:
base64 returns a byte-object there is no need to convert it to hex():
stringToSign = "Test"
secretAccessKey = "bAvW5O18eSrxke4I7eFcrnrDJkN+wKQmx9aSHuMZQ0w="
secretAccessKeyBase64 = base64.b64decode(secretAccessKey)
keyBytes = secretAccessKeyBase64
stringToSignBytes = bytes(stringToSign, 'utf-8')
signatureHash = hmac.new(keyBytes, stringToSignBytes, digestmod=hashlib.sha256).digest()
signature = base64.b64encode(signatureHash)
print(signature)
Prints correctly:
b'b+1wRzDODA85vyDZkXByPIKO5qmnjCRNF5gZFi33/Ic='
Code snippets:
php:
$shaVal = '59bc125840733ea828f42e276661b01e177f1414';
$enc = base64_encode(pack('H*', $shaVal));
echo $enc;
//prints => WbwSWEBzPqgo9C4nZmGwHhd/FBQ=
and in Javascipt I used buffer npm module
let Buffer = require('buffer').Buffer;
let shaVal = '59bc125840733ea828f42e276661b01e177f1414';
//function similar to php's pack() and returns binary data
let bData = Buffer.from(shaVal, 'hex').toString();
console.log('bData ', bData)
//encode with base64
let val64 = Buffer.from(bData, 'binary').toString('base64');
console.log('base 64 encode ', val64)
//prints => Wf0SWEBzPv0o/S4nZmH9Hhd/FBQ=
How can I get the exact same output printed by php?
Note: Both options showing binary data as Y�X#s>�(�.'fa�
It's because PHP pack returns string, where as javascript buffer returns Array.
This answer might help : https://stackoverflow.com/a/41962257/3086531
I have code that is working in my PHP app. In the PHP I sign the url with the following code:
private static function __getHash($string)
{
return hash_hmac('sha1', $string, self::$__secretKey, true);
}
I am attempting to sign the URL in the same way in a Node.js application. This is what I'm trying:
S3.prototype.getHash = function(string){
var key = this.secret_key;
var hmac = crypto.createHash('sha1', key);
hmac.update(string);
return hmac.digest('binary');
};
However, I am getting the following error:
The request signature we calculated does not match the signature you provided. Check your key and signing method.
Do these pieces of code do the same thing? Am I missing something?
This answer from Chris is good if you are porting hash_hmac with the last parameter being true. In this case, binary is produced, as is the case with Chris's javascript.
To add to that, this example:
$sign = hash_hmac('sha512', $post_data, $secret);
Would be ported with a function like so in nodejs:
const crypto = require("crypto");
function signHmacSha512(key, str) {
let hmac = crypto.createHmac("sha512", key);
let signed = hmac.update(Buffer.from(str, 'utf-8')).digest("hex");
return signed
}
The difference here being that when you leave off the last argument to hash_hmac (or set it to something not true), it behaves as defined in the PHP docs:
When set to TRUE, outputs raw binary data. FALSE outputs lowercase hexits.
In order to do this with node.js we use digest('hex') as you can see in the snippet.
The primary problem here is that you are using createHash which creates a hash, rather than createHmac which creates an HMAC.
Change createHash to createHmac and you should find it produces the same result.
This is the output you should expect:
chris /tmp/hmac $ cat node.js
var crypto = require('crypto');
var key = 'abcd';
var data = 'wxyz';
function getHash(string){
var hmac = crypto.createHmac('sha1', key);
hmac.update(string);
return hmac.digest('binary');
};
process.stdout.write(getHash(data));
chris /tmp/hmac $ cat php.php
<?php
$key = "abcd";
$data = "wxyz";
function __getHash($string)
{
global $key;
return hash_hmac('sha1', $string, $key, true);
}
echo utf8_encode(__getHash($data));
chris /tmp/hmac $ node node.js | base64
WsOKw4xgw4jDlFHDl3jDuEPDuCfCmsOFwoDCrsK/w6ka
chris /tmp/hmac $ php php.php | base64
WsOKw4xgw4jDlFHDl3jDuEPDuCfCmsOFwoDCrsK/w6ka
I want to get compress layer data from tmx file . Who knows libraries for decompress gzip and zlib string in javascript ? I try zlib but it doesn't work for me . Ex , layer data in tmx file is :
<data encoding="base64" compression="zlib">
eJztwTEBAAAAwqD1T20JT6AAAHgaCWAAAQ==
</data>
My javascript code is
var base64Data = "eJztwTEBAAAAwqD1T20JT6AAAHgaCWAAAQ==";
var compressData = atob(base64Data);
var inflate = new Zlib.Inflate(compressData);
var output = inflate.decompress();
It runs with displays message error "unsupported compression method" . But I try decompress with online tool as http://i-tools.org/gzip , it returns correct string.
Pako is a full and modern Zlib port.
Here is a very simple example and you can work from there.
Get pako.js and you can decompress byteArray like so:
<html>
<head>
<title>Gunzipping binary gzipped string</title>
<script type="text/javascript" src="pako.js"></script>
<script type="text/javascript">
// Get datastream as Array, for example:
var charData = [31,139,8,0,0,0,0,0,0,3,5,193,219,13,0,16,16,4,192,86,214,151,102,52,33,110,35,66,108,226,60,218,55,147,164,238,24,173,19,143,241,18,85,27,58,203,57,46,29,25,198,34,163,193,247,106,179,134,15,50,167,173,148,48,0,0,0];
// Turn number array into byte-array
var binData = new Uint8Array(charData);
// Pako magic
var data = pako.inflate(binData);
// Convert gunzipped byteArray back to ascii string:
var strData = String.fromCharCode.apply(null, new Uint16Array(data));
// Output to console
console.log(strData);
</script>
</head>
<body>
Open up the developer console.
</body>
</html>
Running example: http://jsfiddle.net/9yH7M/
Alternatively you can base64 encode the array before you send it over as the Array takes up a lot of overhead when sending as JSON or XML. Decode likewise:
// Get some base64 encoded binary data from the server. Imagine we got this:
var b64Data = 'H4sIAAAAAAAAAwXB2w0AEBAEwFbWl2Y0IW4jQmziPNo3k6TuGK0Tj/ESVRs6yzkuHRnGIqPB92qzhg8yp62UMAAAAA==';
// Decode base64 (convert ascii to binary)
var strData = atob(b64Data);
// Convert binary string to character-number array
var charData = strData.split('').map(function(x){return x.charCodeAt(0);});
// Turn number array into byte-array
var binData = new Uint8Array(charData);
// Pako magic
var data = pako.inflate(binData);
// Convert gunzipped byteArray back to ascii string:
var strData = String.fromCharCode.apply(null, new Uint16Array(data));
// Output to console
console.log(strData);
Running example: http://jsfiddle.net/9yH7M/1/
To go more advanced, here is the pako API documentation.
I can solve my problem by zlib . I fix my code as below
var base64Data = "eJztwTEBAAAAwqD1T20JT6AAAHgaCWAAAQ==";
var compressData = atob(base64Data);
var compressData = compressData.split('').map(function(e) {
return e.charCodeAt(0);
});
var inflate = new Zlib.Inflate(compressData);
var output = inflate.decompress();
For anyone using Ruby on Rails, who wants to send compressed encoded data to the browser, then uncompress it via Javascript on the browser, I've combined both excellent answers above into the following solution. Here's the Rails server code in my application controller which compresses and encodes a string before sending it the browser via a #variable to a .html.erb file:
require 'zlib'
require 'base64'
def compressor (some_string)
Base64.encode64(Zlib::Deflate.deflate(some_string))
end
Here's the Javascript function, which uses pako.min.js:
function uncompress(input_field){
base64data = document.getElementById(input_field).innerText;
compressData = atob(base64data);
compressData = compressData.split('').map(function(e) {
return e.charCodeAt(0);
});
binData = new Uint8Array(compressData);
data = pako.inflate(binData);
return String.fromCharCode.apply(null, new Uint16Array(data));
}
Here's a javascript call to that uncompress function, which wants to unencode and uncompress data stored inside a hidden HTML field:
my_answer = uncompress('my_hidden_field');
Here's the entry in the Rails application.js file to call pako.min.js, which is in the /vendor/assets/javascripts directory:
//= require pako.min
And I got the pako.min.js file from here:
https://github.com/nodeca/pako/tree/master/dist
All works at my end, anyway! :-)
I was sending data from a Python script and trying to decode it in JS. Here's what I had to do:
Python
import base64
import json
import urllib.parse
import zlib
...
data_object = {
'_id': '_id',
...
}
compressed_details = base64.b64encode(zlib.compress(bytes(json.dumps(data_object), 'utf-8'))).decode("ascii")
urlsafe_object = urllib.parse.quote(str(compressed_details))#.replace('%', '\%') # you likely don't need this last part
final_URL = f'https://my.domain.com?data_object={urlsafe_object}'
...
JS
// npm install this
import pako from 'pako';
...
const urlParams = new URLSearchParams(window.location.search);
const data_object = urlParams.get('data_object');
if (data_object) {
const compressedData = Uint8Array.from(window.atob(data_object), (c) => c.charCodeAt(0));
originalObject = JSON.parse(pako.inflate(compressedData, { to: 'string' }));
};
...