We Keep Coding

AWS Signature V2: generate a signature for a Query request in Javascript

i am trying to re-create AWS signature version 2 authentication on javascript, what i have right now is
String.prototype.getBytes = () => {
return this.toString()
.split('')
.map((i) => i.charCodeAt(0));
};
let key = 'redacted_access_key_id';
const bytes = key.getBytes();
let signingKey = crypto.HmacSHA256(bytes, key);
let data = JSON.stringify({ lang: 'en', pageNumber: 0, pageSize: 20 });
const contentMd5 = crypto.MD5(data).toString();
data = data.getBytes();
signingKey = crypto.HmacSHA256(data, key);
const result = Buffer.from(signingKey.toString()).toString('base64');
Which outputs something like
ZGY0MmI3MDVjNmJlNzY5ZWYwZjU1ZTc5MDhhOGNkYzI3ZWVjYzQ5ODBmY2M1NGI5NTc2MmVmNTY1NzEwNjhhMA== 
which is incorrect, because the hash should be exactly 28 characters in length. Now the AWS signature version 2 auth docs show how it is being made, but only in java
import java.security.SignatureException;
import javax.crypto.Mac;
import javax.crypto.spec.SecretKeySpec;
import com.amazonaws.util.*;
/**
* This class defines common routines for generating
* authentication signatures for AWS Platform requests.
*/
public class Signature {
private static final String HMAC_SHA256_ALGORITHM = "HmacSHA256";
public static String calculateRFC2104HMAC(String data, String key)
throws java.security.SignatureException
{
String result;
try {
// Get an hmac_sha256 key from the raw key bytes.
SecretKeySpec signingKey = new SecretKeySpec(key.getBytes("UTF-8"), HMAC_SHA256_ALGORITHM);
// Get an hmac_sha256 Mac instance and initialize with the signing key.
Mac mac = Mac.getInstance(HMAC_SHA256_ALGORITHM);
mac.init(signingKey);
// Compute the hmac on input data bytes.
byte[] rawHmac = mac.doFinal(data.getBytes("UTF-8"));
// Base64-encode the hmac by using the utility in the SDK
result = BinaryUtils.toBase64(rawHmac);
} catch (Exception e) {
throw new SignatureException("Failed to generate HMAC : " + e.getMessage());
}
return result;
}
}
I am trying to recreate this exact same code in javascript but something is wrong. Can someone please help me with this, i cant find any examples in javascript.
Thank you.

The following code is the equivalent of the Java version of calculateRFC2104HMAC in JS.
const CryptoJS = require('crypto-js');
const calculateRFC2104HMAC = (data, key) => {
const rawHmac = CryptoJS.HmacSHA256(CryptoJS.enc.Utf8.parse(data), CryptoJS.enc.Utf8.parse(key));
return CryptoJS.enc.Base64.stringify(rawHmac);
}
Sample usage based on the example on AWS Signature V2 page
const urlSafeSignature = (data, key) => encodeURIComponent(calculateRFC2104HMAC(data, key));
const data =
`GET
elasticmapreduce.amazonaws.com
/
AWSAccessKeyId=AKIAIOSFODNN7EXAMPLE&Action=DescribeJobFlows&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2011-10-03T15%3A19%3A30&Version=2009-03-31`
const key = `wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY`
console.log(urlSafeSignature(data, key));
The documentation advises to use AWS Signature V4 which has a AWS published library on NPM here. The AWS signed requests are for AWS Services and the signature in the request helps validating the request, prevents replay attacks. I'm not sure what you are trying to send in the following code and for which AWS service.
let data = JSON.stringify({ lang: 'en', pageNumber: 0, pageSize: 20 });
You must provide all details required to sign a request as per the AWS documentation.

No Response After Running Python Script Using Python-Shell

I have a nodeJS application and I need to run a python script in order to get a certain response. I am using python-shell in order to do that, but I am getting no response.
I have tried also using a child-process, same response.
Here I call the python script:
var ps = require('python-shell');
ps.PythonShell.run('./face_detect.py', array1, function (err, data) {
if (err) req.send(err);
req.send(data.toString())
});
This is a snippet of my python script:
import cv2
import sys
import os
import numpy as np
students = sys.argv[1]
# get the names and put them in an array ---> subjects
imagePath = "class/welcome.jpg"
cascPath = "haarcascade_frontalface_alt.xml"
faceCascade = cv2.CascadeClassifier(cascPath)
.....
for (x, y, w, h) in faces:
num = 0
crop_img = cv2.UMat(image[y-40:y+h+100,x-40:x+h+40])
cv2.imwrite("face" + str(num) + ".jpg", crop_img)
test_img = cv2.imread("face" + str(num) + ".jpg")
num = num + 1
predicted_img1 = predict(test_img)
absences["A"] = 1
for name, a in absences.items():
if a == 0:
noshow.append(name)
print(noshow)
cv2.waitKey(0)
I expect it to return an array.
Can anyone help me with this?

The correct syntax for passing argument from Nodejs python-shell to Python script is:
ps.PythonShell.run('./face_detect.py', { args: array1 }, function (err, data) { ... })
Here the value of sys.argv[1] in your Python script will not contain the Nodejs array1 value because you don't set the args property in your PythonShell options.
Note also this should probably be res.send instead of req.send, depending on your program, and I advise you to return if there is an error to prevent "headers already sent" exception.

InvalidKey at Kraken API with JS, not with Python

My code to get my Balance from the Kraken API does work in Python (based on the krakenex library), but not in the JS version (based loosely off the kraken-api library, but with the crypto library substituted for crypto-js). The error is always: Invalid Key.
Even when I copy the headers and the nonce sent by the Python client into Postman, I get Invalid Key.
I believe the signature and nonce to be valid, because when they are not, Kraken retorts that either the signature or nonce are invalid.
Is there anything else that Javascript's fetch does differently than Python3 requests? Because the body and headers are otherwise identical.
JS code that generates auth data:
const getMessageSignature = (path, request, secret, nonce) => {
// API-Sign = Message signature using HMAC-SHA512 of (URI path + SHA256(nonce + POST data)) and base64 decoded secret API key
const message = qs.stringify(request);
console.log(message);
const secret_buffer = btoa(secret);
const hash = CryptoJS.algo.SHA256.create();
const hmac = CryptoJS.algo.HMAC.create(CryptoJS.algo.SHA512, secret_buffer);
const hash_digest = hash.update(nonce + message).finalize().toString(CryptoJS.enc.Hex);
const hmac_digest = hmac.update(path + hash_digest).finalize().toString(CryptoJS.enc.Base64);
// CANNOT USE ORIGINAL LIB CODE (Buffer, got and crypto not supported)
// const secret_buffer = new Buffer(secret, 'base64');
// const hash = new crypto.createHash('sha256');
// const hmac = new crypto.createHmac('sha512', secret_buffer);
// const hash_digest = hash.update(nonce + message).digest('binary');
// const hmac_digest = hmac.update(path + hash_digest, 'binary').digest('base64');
return hmac_digest;
};
Update:
In fact, the following observations are weird:
correct key + correct signature = "incorrect key"
incorrect key + correct signature = "incorrect key"
incorrect key + incorrect signature = "incorrect key"
correct key + incorrect signature = "invalid signature"
what gives?
Update2
Seems the requests are identical (other than the signature and nonce of course, which will and should change with every request).

Turns out it was the signature after all and Kraken simply doesn't give very accurate responses (which makes some sense, but is a pain if you're trying to figure something out). Finally I was able to rewrite the code using CryptoJS only:
const getMessageSignature = (path, request, secret, nonce) => {
// API-Sign = Message signature using HMAC-SHA512 of (URI path + SHA256(nonce + POST data)) and base64 decoded secret API key
const message = JSON.stringify(request);
const hash = CryptoJS.SHA256(nonce + message);
const secret_buffer = CryptoJS.enc.Base64.parse(secret);
const hmac = CryptoJS.algo.HMAC.create(CryptoJS.algo.SHA512, secret_buffer);
hmac.update(path, secret_buffer);
hmac.update(hash, secret_buffer);
return hmac.finalize().toString(CryptoJS.enc.Base64);
};
This yields a correct signature and Kraken no longer complains. Zzah.

Validate Facebook signed_request signature in Javascript

I'm building a Facebook Page app in Classic ASP. I've been unable to match the signature that Facebook passes into the app as the first part of the POSTed signed_request.
Because there are few libraries for cryptography in VBScript, I'm using server side Javascript and the crypto-js library from https://code.google.com/archive/p/crypto-js/
I've tried to translate the PHP code example from Facebook's docs at https://developers.facebook.com/docs/games/gamesonfacebook/login#parsingsr into Javascript. I can generate an HMAC SHA256 hash of the signed_request payload but that doesn't match the signed_request signature.
I think the problem is that Facebook's signature is in a different format. It looks to be binary (~1抚Ö.....) while the HMAC SHA256 hash I'm generating is a hexadecimal string (7f7e8f5f.....). In Facebook's PHP example the hash_hmac function uses the raw binary parameter. So I think I need to either convert Facebook's signature to hexadecimal or my signature to binary in order to do an "apples-to-apples" comparison and get a match.
Here's my code:
/* Use the libraries from https://code.google.com/archive/p/crypto-js/
crypto-js/crypto-js.min.js
crypto-js/hmac-sha256.min.js
crypto-js/enc-base64.min.js
*/
var signedRequest = Request.queryString("signed_request")
var FB_APP_SECRET = "459f038.....";
var arSR = signedRequest.split(".");
var encodedSig = arSR[0];
var encodedPayload = arSR[1];
var payload = base64UrlDecode(encodedPayload);
var sig = base64UrlDecode(encodedSig);
var expectedSig;
expectedSig = CryptoJS.HmacSHA256(encodedPayload, FB_APP_SECRET); // Unaltered payload string; no match
expectedSig = CryptoJS.HmacSHA256(payload, FB_APP_SECRET); // base64-decoded payload string; no match
if (sig == expectedSig) {
Response.write(payload);
} else {
Response.write("Bad signature");
}
function base64UrlDecode(input) {
// Replace characters and convert from base64.
return Base64.decode(input.replace("-", "+").replace("_", "/"));
}

After looking into the crypto-js documentation about encoding I found the solution. The de-/encoding methods provided by crypto-js are listed under 'Encoders' at the bottom of https://code.google.com/archive/p/crypto-js/ (Thanks for the nudge, CBroe.)
The solution was to use .toString() on the signatures. It seems like crypto-js uses a word format that was preventing a comparison match. I did also switch to using the base64 decoding provided by crypto-js in order to stick with one library.
Here's my updated code:
/* Use the libraries from https://code.google.com/archive/p/crypto-js/
crypto-js/crypto-js.min.js
crypto-js/hmac-sha256.min.js
crypto-js/enc-base64.min.js
*/
var signedRequest = Request.queryString("signed_request")
var FB_APP_SECRET = "459f038.....";
var arSR = signedRequest.split(".");
var encodedSig = arSR[0];
var encodedPayload = arSR[1];
var payload = base64UrlDecode(encodedPayload);
var sig = base64UrlDecode(encodedSig);
var expectedSig = CryptoJS.HmacSHA256(encodedPayload, FB_APP_SECRET); /******** Correct payload */
if (sig.toString() != expectedSig.toString()) { /******* Use .toString() to convert to normal strings */
Response.write(payload);
} else {
Response.write("Bad signature");
}
function base64UrlDecode(input) {
return CryptoJS.enc.Base64.parse( /******** Decode */
input.replace("-", "+").replace("_", "/") // Replace characters
);
}

I recently implemented this for their required user data deletion webhook. No external dependencies needed anymore:
const crypto = require('crypto');
function parseSignedRequest(signedRequest, secret) {
const [signatureReceived, encodedPayload] = signedRequest.split('.', 2);
const payload = b64decode(encodedPayload)
const data = JSON.parse(payload);
const hmac = crypto.createHmac('sha256', secret).update(payload);
const expectedSignature = hmac.digest('base64');
if (signatureReceived === expectedSignature) {
return data;
} else {
throw new Error("Signature mismatch");
}
}
function b64decode(data) {
const buff = Buffer.from(data, 'base64');
return buff.toString('ascii');
}
It's a translation of their example PHP code. I also have a repo setup with tests.

I found this worked for me.
const crypto = require('crypto')
const _atob = (str) => Buffer.from(str, 'base64').toString('binary')
const parseSignedRequest = (signed_request, app_secret) => {
const [encoded_sig, payload] = signed_request.split('.')
const json = _atob(payload)
const data = JSON.parse(json)
if (!data.algorithm || data.algorithm.toUpperCase() !== 'HMAC-SHA256') {
return {error: true, type: 'Unknown algorithm. Expected HMAC-SHA256'}
}
// check sig
const expected_sig = crypto.createHmac('sha256', config.facebook.app_secret)
.update(payload).digest('base64')
.replace(/\+/g, '-').replace(/\//g, '_')
.replace(/=/g, '')
if (encoded_sig !== expected_sig) {
return ({error: true, type: 'invalid_signature'})
}
return {error: false, parsedRequest: data}
}
const {error, type, parsedRequest} = parseSignedRequest(signed_request)

Verify PKCS#7 (PEM) signature / unpack data in node.js

I get a PKCS#7 crypto package from a 3rd party system.
The package is not compressed and not encrypted, PEM-encoded, signed with X.509 certificate.
I also have a PEM cert file from the provider.
The data inside is XML
I need to do the following in Node.JS:
extract the data
verify the signature
A sample package (no sensitive info, data refers to our qa system) http://pastebin.com/7ay7F99e

OK, finally got it.
First of all, PKCS messages are complex structures binary-encoded using ASN1.
Second, they can be serialized to binary files (DER encoding) or text PEM files using Base64 encoding.
Third, PKCS#7 format specifies several package types from which my is called Signed Data. These formats are distinguished by OBJECT IDENTIFIER value in the beginning of the ASN1 object (1st element of the wrapper sequence) — you can go to http://lapo.it/asn1js/ and paste the package text for the fully parsed structure.
Next, we need to parse the package (Base64 -> ASN1 -> some object representation). Unfortunately, there's no npm package for that. I found quite a good project forge that is not published to npm registry (though npm-compatible). It parsed PEM format but the resulting tree is quite an unpleasant thing to traverse. Based on their Encrypted Data and Enveloped Data implementations I created partial implementation of Signed Data in my own fork. UPD: my pull request was later merged to the forge project.
Now finally we have the whole thing parsed.
At that point I found a great (and probably the only on the whole web) explanative article on signed PKCS#7 verification: http://qistoph.blogspot.com/2012/01/manual-verify-pkcs7-signed-data-with.html
I was able to extract and successfully decode the signature from the file, but the hash inside was different from the data's hash. God bless Chris who explained what actually happens.
The data signing process is 2-step:
original content's hash is calculated
a set of "Authorized Attributes" is constructed including: type of the data singed, signing time and data hash
Then the set from step 2 is signed using the signer's private key.
Due to PKCS#7 specifics this set of attributes is stored inside of the context-specific constructed type (class=0x80, type=0) but should be signed and validated as normal SET (class=0, type=17).
As Chris mentions (https://stackoverflow.com/a/16154756/108533) this only verifies that the attributes in the package are valid. We should also validate the actual data hash against the digest attribute.
So finally here's a code doing validation (cert.pem is a certificate file that the provider sent me, package is a PEM-encoded message I got from them over HTTP POST):
var fs = require('fs');
var crypto = require('crypto');
var forge = require('forge');
var pkcs7 = forge.pkcs7;
var asn1 = forge.asn1;
var oids = forge.pki.oids;
var folder = '/a/path/to/files/';
var pkg = fs.readFileSync(folder + 'package').toString();
var cert = fs.readFileSync(folder + 'cert.pem').toString();
var res = true;
try {
var msg = pkcs7.messageFromPem(pkg);
var attrs = msg.rawCapture.authenticatedAttributes;
var set = asn1.create(asn1.Class.UNIVERSAL, asn1.Type.SET, true, attrs);
var buf = Buffer.from(asn1.toDer(set).data, 'binary');
var sig = msg.rawCapture.signature;
var v = crypto.createVerify('RSA-SHA1');
v.update(buf);
if (!v.verify(cert, sig)) {
console.log('Wrong authorized attributes!');
res = false;
}
var h = crypto.createHash('SHA1');
var data = msg.rawCapture.content.value[0].value[0].value;
h.update(data);
var attrDigest = null;
for (var i = 0, l = attrs.length; i < l; ++i) {
if (asn1.derToOid(attrs[i].value[0].value) === oids.messageDigest) {
attrDigest = attrs[i].value[1].value[0].value;
}
}
var dataDigest = h.digest();
if (dataDigest !== attrDigest) {
console.log('Wrong content digest');
res = false;
}
}
catch (_e) {
console.dir(_e);
res = false;
}
if (res) {
console.log("It's OK");
}

Your answer is a big step in the right direction. You are however missing out an essential part of the validation!
You should verify the hash of the data against the digest contained in the signed attributes. Otherwise it would be possible for someone to replace the content with malicious data. Try for example validating the following 'package' with your code (and have a look at the content): http://pastebin.com/kaZ2XQQc
I'm not much of a NodeJS developer (this is actually my first try :p), but here's a suggestion to help you get started.
var fs = require('fs');
var crypto = require('crypto');
var pkcs7 = require('./js/pkcs7'); // forge from my own fork
var asn1 = require('./js/asn1');
var folder = '';
var pkg = fs.readFileSync(folder + 'package').toString();
var cert = fs.readFileSync(folder + 'cert.pem').toString();
try {
var msg = pkcs7.messageFromPem(pkg);
var attrs = msg.rawCapture.authenticatedAttributes; // got the list of auth attrs
var set = asn1.create(asn1.Class.UNIVERSAL, asn1.Type.SET, true, attrs); // packed them inside of the SET object
var buf = new Buffer(asn1.toDer(set).data, 'binary'); // DO NOT forget 'binary', otherwise it tries to interpret bytes as UTF-8 chars
var sig = msg.rawCapture.signature;
var shasum = crypto.createHash('sha1'); // better be based on msg.rawCapture.digestAlgorithms
shasum.update(msg.rawCapture.content.value[0].value[0].value);
for(var n in attrs) {
var attrib = attrs[n].value;
var attrib_type = attrib[0].value;
var attrib_value = attrib[1].value[0].value;
if(attrib_type == "\x2a\x86\x48\x86\xf7\x0d\x01\x09\x04") { // better would be to use the OID (1.2.840.113549.1.9.4)
if(shasum.digest('binary') == attrib_value) {
console.log('hash matches');
var v = crypto.createVerify('RSA-SHA1');
v.update(buf);
console.log(v.verify(cert, sig)); // -> should type true
} else {
console.log('hash mismatch');
}
}
}
}
catch (_e) {
console.dir(_e);
}

based on inspiration form this answer, I've implemented a sample for signing and verifying pdf files using node-signpdf and node-forge.