We Keep Coding

Validate Google Extended Access Article signature (Python)

Google Extended Access
Google Extended Access provides access to news article beyond the Publisher paywall if the user signs in via Google OAuth. The number of access is based on metering. More info.
The news article is redirected to the publisher where it's determined if it's a valid visit from Google Showcase panel. One of the steps of validation is the verification of the signature sent along in the URL parameters (gaa_sig).
The signature is generated from the base URL and the other three GAA parameters, which you can use to verify that this is a valid visit from a Showcase panel.
Verification Steps
Transform the gaa_sig value from its "web safe" format by replacing all '' characters with '+', and the '' characters with '/'.
Base64 decode the transformed value from step 1.
Remove the gaa_sig URL parameter from the URL to form the data to verify.
Loop through the latest JSON web keys (JWKs):
For development: https://play.google.com/newsstand/api/v3/articleaccess/publicsigningkey/dev
Verify the signature value from step2 with the data from step 3 and the keys from step 4.
Working Code in Node
const { subtle } = require('crypto').webcrypto;
const fetch = (...args) => import('node-fetch').then(({default: fetch}) => fetch(...args));
GOOGLE_JSON_WEB_KEYS = "https://play.google.com/newsstand/api/v3/articleaccess/publicsigningkey/dev"
async function getPlaySigningKeys() {
let response = await fetch(GOOGLE_JSON_WEB_KEYS);
let data = await response.json();
return data;
}
async function verifyGaaUrlSignature(url, keys){
let urlObj = new URL(url);
let params = new URLSearchParams(urlObj.search);
let sigB64Str = params.get('gaa_sig');
if (!sigB64Str) {
return false;
}
let sigBuffer = Buffer.from(sigB64Str.replace(/-/g,'+').replace(/_/g,'/'), 'base64');
params.delete('gaa_sig');
let data = new TextEncoder().encode(urlObj.origin + urlObj.pathname + '?' + params);
async function verifySig(key) {
let cryptoKey = await subtle.importKey(
'jwk', key, {name: 'ECDSA', namedCurve: key.crv}, true, key.key_ops);
return subtle.verify(
{name: 'ECDSA', hash: 'SHA-256'}, cryptoKey, sigBuffer, data);
}
const results = await Promise.all(keys.map(verifySig));
return results.includes(true);
}
async function run() {
const playSigningKeys = await getPlaySigningKeys();
url = "https://www.bbc.com/?gaa_at=la&gaa_n=ATKjfPG-7F6PGpCXtPZFAfqigovblSKOl3G6jduKn8zWcjHMSu-a3wQ1ub-mKBl47rjP&gaa_ts=630be8f6&gaa_sig=ZGvbOCFg5J_zGAtd6R39YbEEYjcoarQ7AaAjQPsAae5jikZTjX57_Ja3vVyp8bUIcUbftI5dQdTP7gtwtIC3eQ%3D%3D"
const isValidSig = await verifyGaaUrlSignature(url, playSigningKeys.keys);
console.log('Valid Signature: ' + isValidSig);
}
if (require.main === module) {
run();
}
Steps to produce the parameters and signature
Generate the GAA parameters for the required url by appending it to the below url. The updated url will be redirected to the with the GAA parameters.
https://play.google.com/newsstand/api/v3/articleaccess?testurl=
Example:
https://play.google.com/newsstand/api/v3/articleaccess?testurl=https://bbc.com
Redirected to
https://www.bbc.com/?gaa_at=la&gaa_n=ATKjfPG-7F6PGpCXtPZFAfqigovblSKOl3G6jduKn8zWcjHMSu-a3wQ1ub-mKBl47rjP&gaa_ts=630be8f6&gaa_sig=ZGvbOCFg5J_zGAtd6R39YbEEYjcoarQ7AaAjQPsAae5jikZTjX57_Ja3vVyp8bUIcUbftI5dQdTP7gtwtIC3eQ%3D%3D
What would be the working code in Python?
I tried looking into python-jose. I couldn't figure out the equivalent. The Web API for SubtleCrypto.verify().

Sign a message with EdDSA algorithm in Javascript to get JWT

I need to get JWT with EdDSA algorithm to be able to use an API. I have the private key to sign the message and I could do that with PHP with the next library: https://github.com/firebase/php-jwt (you can see the example with EdDSA at README). Now I need to do the same in JS but I didn't find the way to get JWT with a given secret key (encoded base 64) like that (only an example is not the real secretKey):
const secretKey = Dm2xriMD6riJagld4WCA6zWqtuWh40UzT/ZKO0pZgtHATOt0pGw90jG8BQHCE3EOjiCkFR2/gaW6JWi+3nZp8A==
I tried a lot of libraries like jose, js-nacl, crypto, libsodium, etc. And I am really close to get the JWT with libsodium library, now I attach the code:
const base64url = require("base64url");
const _sodium = require("libsodium-wrappers");
const moment = require("moment");
const getJWT = async () => {
await _sodium.ready;
const sodium = _sodium;
const privateKey =
"Dm2xriMD6riJagld4WCA6zWqtuWh40UzT/ZKO0pZgtHATOt0pGw90jG8BQHCE3EOjiCkFR2/gaW6JWi+3nZp8A==";
const payload = {
iss: "test",
aud: "test.com",
iat: 1650101178,
exp: 1650101278,
sub: "12345678-1234-1234-1234-123456789123"
};
const { msg, keyAscii} = encode(payload, privateKey, "EdDSA");
const signature = sodium.crypto_sign_detached(msg, keyDecoded); //returns Uint8Array(64)
//Here is the problem.
};
const encode = (payload, key, alg) => {
const header = {
typ: "JWT",
alg //'EdDSA'
};
const headerBase64URL = base64url(JSON.stringify(header));
const payloadBase64URL = base64url(JSON.stringify(payload));
const headerAndPayloadBase64URL = `${headerBase64URL}.${payloadBase64URL}`;
const keyAscii= Buffer.from(key, "base64").toString("ascii");
return {headerAndPayloadBase64URL , keyAscii}
};
The problem is in the sodium.crypto_sign_detached function because it returns an Uint8Array(64) signature and and I need the JWT like that:
eyJ0eXAiOiJKV1QiLCJhbGciOiJFZERTQSJ9.eyJpc3MiOiJ0ZXN0IiwiYXVkIjoidGVzdC5jb20iLCJpYXQiOjE2NTAxMDExNzgsImV4cCI6MTY1MDEwMTI3OCwic3ViIjoiMTIzNDU2NzgtMTIzNC0xMjM0LTEyMzQtMTIzNDU2Nzg5MTIzIn0.f7WG_02UKljrMeVVOTNNBAGxtLXJUT_8QAnujNhomV18Pn5cU-0lHRgVlmRttOlqI7Iol_fHut3C4AOXxDGnAQ
How can I change the Uint8Array(64) to get the signature in a right format to get the JWT? I tried with base64, base64url, hex, text, ascii, etc and the final JWT is not valid (because the signature is wrong).
If you compare my code with the code that I mentioned with PHP is very similar but the function sodium.crypto_sign_detached returns Uint8Array(64) at JS library and the same function in PHP returns an string and I can get the token.
Or maybe there a way to adapt my given private key for use in other library (like crypto or jose where I received an error for the private key format)
Thank you!

In the posted NodeJS code there are the following issues:
crypto_sign_detached() returns the signature as a Uint8Array, which can be imported with Buffer.from() and converted to a Base64 string with base64url().
Concatenating headerAndPayloadBase64URL and the Base64url encoded signature with a . as separator gives the JWT you are looking for.
The raw private key must not be decoded with 'ascii', as this generally corrupts the data. Instead, it should simply be handled as buffer. Note: If for some reason a conversion to a string is required, use 'binary' as encoding, which produces a byte string (however, this is not an option with crypto_sign_detached() as this function expects a buffer).
With these changes, the following NodeJS code results:
const _sodium = require('libsodium-wrappers');
const base64url = require("base64url");
const getJWT = async () => {
await _sodium.ready;
const sodium = _sodium;
const privateKey = "Dm2xriMD6riJagld4WCA6zWqtuWh40UzT/ZKO0pZgtHATOt0pGw90jG8BQHCE3EOjiCkFR2/gaW6JWi+3nZp8A==";
const payload = {
iss: "test",
aud: "test.com",
iat: 1650101178,
exp: 1650101278,
sub: "12345678-1234-1234-1234-123456789123"
};
const {headerAndPayloadBase64URL, keyBuf} = encode(payload, privateKey, "EdDSA");
const signature = sodium.crypto_sign_detached(headerAndPayloadBase64URL, keyBuf);
const signatureBase64url = base64url(Buffer.from(signature));
console.log(`${headerAndPayloadBase64URL}.${signatureBase64url}`) // eyJ0eXAiOiJKV1QiLCJhbGciOiJFZERTQSJ9.eyJpc3MiOiJ0ZXN0IiwiYXVkIjoidGVzdC5jb20iLCJpYXQiOjE2NTAxMDExNzgsImV4cCI6MTY1MDEwMTI3OCwic3ViIjoiMTIzNDU2NzgtMTIzNC0xMjM0LTEyMzQtMTIzNDU2Nzg5MTIzIn0.f7WG_02UKljrMeVVOTNNBAGxtLXJUT_8QAnujNhomV18Pn5cU-0lHRgVlmRttOlqI7Iol_fHut3C4AOXxDGnAQ
};
const encode = (payload, key, alg) => {
const header = {
typ: "JWT",
alg //'EdDSA'
};
const headerBase64URL = base64url(JSON.stringify(header));
const payloadBase64URL = base64url(JSON.stringify(payload));
const headerAndPayloadBase64URL = `${headerBase64URL}.${payloadBase64URL}`;
const keyBuf = Buffer.from(key, "base64");
return {headerAndPayloadBase64URL, keyBuf};
};
getJWT();
Test:
Since Ed25519 is deterministic, the NodeJS code can be checked by comparing both JWTs: If, as in the above NodeJS code, the same header and payload are used as in the PHP code, the same signature and thus the same JWT is generated as by the PHP code, namely:
eyJ0eXAiOiJKV1QiLCJhbGciOiJFZERTQSJ9.eyJpc3MiOiJ0ZXN0IiwiYXVkIjoidGVzdC5jb20iLCJpYXQiOjE2NTAxMDExNzgsImV4cCI6MTY1MDEwMTI3OCwic3ViIjoiMTIzNDU2NzgtMTIzNC0xMjM0LTEyMzQtMTIzNDU2Nzg5MTIzIn0.f7WG_02UKljrMeVVOTNNBAGxtLXJUT_8QAnujNhomV18Pn5cU-0lHRgVlmRttOlqI7Iol_fHut3C4AOXxDGnAQ
which shows that the NodeJS code works.
Note that instead of the moment package, Date.now() could be used. This will return the time in milliseconds, so the value has to be divided by 1000, e.g. Math.round(Date.now()/1000), but saves a dependency.

AWS Signature V2: generate a signature for a Query request in Javascript

i am trying to re-create AWS signature version 2 authentication on javascript, what i have right now is
String.prototype.getBytes = () => {
return this.toString()
.split('')
.map((i) => i.charCodeAt(0));
};
let key = 'redacted_access_key_id';
const bytes = key.getBytes();
let signingKey = crypto.HmacSHA256(bytes, key);
let data = JSON.stringify({ lang: 'en', pageNumber: 0, pageSize: 20 });
const contentMd5 = crypto.MD5(data).toString();
data = data.getBytes();
signingKey = crypto.HmacSHA256(data, key);
const result = Buffer.from(signingKey.toString()).toString('base64');
Which outputs something like
ZGY0MmI3MDVjNmJlNzY5ZWYwZjU1ZTc5MDhhOGNkYzI3ZWVjYzQ5ODBmY2M1NGI5NTc2MmVmNTY1NzEwNjhhMA== 
which is incorrect, because the hash should be exactly 28 characters in length. Now the AWS signature version 2 auth docs show how it is being made, but only in java
import java.security.SignatureException;
import javax.crypto.Mac;
import javax.crypto.spec.SecretKeySpec;
import com.amazonaws.util.*;
/**
* This class defines common routines for generating
* authentication signatures for AWS Platform requests.
*/
public class Signature {
private static final String HMAC_SHA256_ALGORITHM = "HmacSHA256";
public static String calculateRFC2104HMAC(String data, String key)
throws java.security.SignatureException
{
String result;
try {
// Get an hmac_sha256 key from the raw key bytes.
SecretKeySpec signingKey = new SecretKeySpec(key.getBytes("UTF-8"), HMAC_SHA256_ALGORITHM);
// Get an hmac_sha256 Mac instance and initialize with the signing key.
Mac mac = Mac.getInstance(HMAC_SHA256_ALGORITHM);
mac.init(signingKey);
// Compute the hmac on input data bytes.
byte[] rawHmac = mac.doFinal(data.getBytes("UTF-8"));
// Base64-encode the hmac by using the utility in the SDK
result = BinaryUtils.toBase64(rawHmac);
} catch (Exception e) {
throw new SignatureException("Failed to generate HMAC : " + e.getMessage());
}
return result;
}
}
I am trying to recreate this exact same code in javascript but something is wrong. Can someone please help me with this, i cant find any examples in javascript.
Thank you.

The following code is the equivalent of the Java version of calculateRFC2104HMAC in JS.
const CryptoJS = require('crypto-js');
const calculateRFC2104HMAC = (data, key) => {
const rawHmac = CryptoJS.HmacSHA256(CryptoJS.enc.Utf8.parse(data), CryptoJS.enc.Utf8.parse(key));
return CryptoJS.enc.Base64.stringify(rawHmac);
}
Sample usage based on the example on AWS Signature V2 page
const urlSafeSignature = (data, key) => encodeURIComponent(calculateRFC2104HMAC(data, key));
const data =
`GET
elasticmapreduce.amazonaws.com
/
AWSAccessKeyId=AKIAIOSFODNN7EXAMPLE&Action=DescribeJobFlows&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2011-10-03T15%3A19%3A30&Version=2009-03-31`
const key = `wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY`
console.log(urlSafeSignature(data, key));
The documentation advises to use AWS Signature V4 which has a AWS published library on NPM here. The AWS signed requests are for AWS Services and the signature in the request helps validating the request, prevents replay attacks. I'm not sure what you are trying to send in the following code and for which AWS service.
let data = JSON.stringify({ lang: 'en', pageNumber: 0, pageSize: 20 });
You must provide all details required to sign a request as per the AWS documentation.

Electron - write file before open save dialog

I'm using electron to develop an app. after some encryption operations are done, I need to show a dialog to the user to save the file. The filename I want to give to the file is a random hash but I have no success also with this. I'm trying with this code but the file will not be saved. How I can fix this?
const downloadPath = app.getPath('downloads')
ipcMain.on('encryptFiles', (event, data) => {
let output = [];
const password = data.password;
data.files.forEach( (file) => {
const buffer = fs.readFileSync(file.path);
const dataURI = dauria.getBase64DataURI(buffer, file.type);
const encrypted = CryptoJS.AES.encrypt(dataURI, password).toString();
output.push(encrypted);
})
const filename = hash.createHash('md5').toString('hex');
console.log(filename)
const response = output.join(' :: ');
dialog.showSaveDialog({title: 'Save encrypted file', defaultPath: downloadPath }, () => {
fs.writeFile(`${filename}.mfs`, response, (err) => console.log(err) )
})
})

The problem you're experiencing is resulting from the asynchronous nature of Electron's UI functions: They do not take callback functions, but return promises instead. Thus, you do not have to pass in a callback function, but rather handle the promise's resolution. Note that this only applies to Electron >= version 6. If you however run an older version of Electron, your code would be correct -- but then you should really update to a newer version (Electron v6 was released well over a year ago).
Adapting your code like below can be a starting point to solve your problem. However, since you do not state how you generate the hash (where does hash.createHash come from?; did you forget to declare/import hash?; did you forget to pass any message string?; are you using hash as an alias for NodeJS' crypto module?), it is (at this time) impossible to debug why you do not get any output from console.log (filename) (I assume you mean this by "in the code, the random filename will not be created"). Once you provide more details on this problem, I'd be happy to update this answer accordingly.
As for the default filename: As per the Electron documentation, you can pass a file path into dialog.showSaveDialog () to provide the user with a default filename.
The file type extension you're using should also actually be passed with the file extension into the save dialog. Also passing this file extension as a filter into the dialog will prevent users from selecting any other file type, which is ultimately what you're also currently doing by appending it to the filename.
Also, you could utilise CryptoJS for the filename generation: Given some arbitrary string, which could really be random bytes, you could do: filename = CryptoJS.MD5 ('some text here') + '.mfs'; However, remember to choose the input string wisely. MD5 has been broken and should thus no longer be used to store secrets -- using any known information which is crucial for the encryption of the files you're storing (such as data.password) is inherently insecure. There are some good examples on how to create random strings in JavaScript around the internet, along with this answer here on SO.
Taking all these issues into account, one might end up with the following code:
const downloadPath = app.getPath('downloads'),
path = require('path');
ipcMain.on('encryptFiles', (event, data) => {
let output = [];
const password = data.password;
data.files.forEach((file) => {
const buffer = fs.readFileSync(file.path);
const dataURI = dauria.getBase64DataURI(buffer, file.type);
const encrypted = CryptoJS.AES.encrypt(dataURI, password).toString();
output.push(encrypted);
})
// not working:
// const filename = hash.createHash('md5').toString('hex') + '.mfs';
// alternative requiring more research on your end
const filename = CryptoJS.MD5('replace me with some random bytes') + '.mfs';
console.log(filename);
const response = output.join(' :: ');
dialog.showSaveDialog(
{
title: 'Save encrypted file',
defaultPath: path.format ({ dir: downloadPath, base: filename }), // construct a proper path
filters: [{ name: 'Encrypted File (*.mfs)', extensions: ['mfs'] }] // filter the possible files
}
).then ((result) => {
if (result.canceled) return; // discard the result altogether; user has clicked "cancel"
else {
var filePath = result.filePath;
if (!filePath.endsWith('.mfs')) {
// This is an additional safety check which should not actually trigger.
// However, generally appending a file extension to a filename is not a
// good idea, as they would be (possibly) doubled without this check.
filePath += '.mfs';
}
fs.writeFile(filePath, response, (err) => console.log(err) )
}
}).catch ((err) => {
console.log (err);
});
})

Error Uploading to AWS S3 via POST in browser?

Based On The AWS in Browser-Based Uploads Using POST Docs, I'm attempting to upload audio files to my bucket without having to pull in an entire SDK. I'm using Vue.js. When I make a request, here is the error I get back:
<Error>
<Code>InvalidRequest</Code>
<Message>The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.</Message>
<RequestId>7FE397A138CF89</RequestId>
<HostId>rEx4jk6vj363wlVGrGqutyDkMNeUhi6DizAXhAiIWrIpG8Rups1rLFGO4Dge5loeNj</HostId>
</Error>
CODE TO CREATE A POLICY, SIGNING KEY & SIGNATURE
Policy Func - creates JSON obj of conditions & returns Utf-8 & Base64 encoded version (as per AWS Doc's)
getPolicy (date) {
let moment = this.$moment(date) // using moment.js
let formattedDate = moment.format('YYYYMMDD')
let obj = {
"expiration": date,
"conditions": [
{"bucket": "test-bucket"},
{"acl": "public-read"},
{"key": "test.mp3"},
["starts-with", "$Content-Type", "audio/"],
["content-length-range", 1048579, 3000000000],
{"x-amz-server-side-encryption": "AES256"},
{"x-amz-credential": `KJIAI4OQHKZGIBSQY5TQ/${formattedDate}/us-east-2/s3/aws4_request`},
{"x-amz-algorithm": "AWS4-HMAC-SHA256"},
{"x-amz-date": formatedDate}
]
}
let string = JSON.stringify(obj)
let utf8 = encodeURI(string)
let base64 = btoa(utf8)
return base64
}
Signing Key Func - creates an HmacSHA256 Signing Key (as per AWS Doc's)
getSigningKey (date) {
// AWSSecretAccessKeyId (obviously this is a dummy)
let key = '+eo98jdkXTjOYO2weY84m2vzCV63vMI6yGvC097R'
let dateKey = crypto.HmacSHA256(date, `AWS4${key}`)
let dateRegionKey = crypto.HmacSHA256('us-east-2', dateKey)
let dateRegionServiceKey = crypto.HmacSHA256('s3', dateRegionKey)
let signingKey = crypto.HmacSHA256('aws4_request', dateRegionServiceKey)
return signingKey.toString()
}
Signature Func - creates a hex-encoded, HmacSHA256 signature (as per AWS Doc's)
getSignature (date) {
let policy = this.getPolicy(date)
let signingKey = this.getSigningKey(date)
let signature = crypto.HmacSHA256(policy, signingKey)
let hexEncodedSignature = signature.toString(hex)
return hexEncodedSignature
}
AJAX REQUEST MADE USING AXIOS
uploadFile (file) {
const date = new Date().toISOString()
let moment = this.$moment(date)
let formattedDate = b.format('YYYYMMDD')
const policy = this.getPolicy(date)
const signature = this.getSignature(date)
const form = new FormData()
form.append('key', 'test.mp3')
form.append('acl', 'public-read')
form.append('Content-Type', 'audio/*')
form.append('x-amz-server-side-encryption', 'AES256')
form.append('X-Amz-Credential', `KJIAI4OQHKZGIBSQY5TQ/${formattedDate}/us-east-2/s3/aws4_request`)
form.append('X-Amz-Algorithm', 'AWS4-HMAC-SHA256')
form.append('X-Amz-Date', formattedDate)
form.append('AWSAccessKeyId', 'KJIAI4OQHKZGIBSQY5TQ')
form.append('Policy', policy)
form.append('Signature', signature)
form.append('file', file)
return axios.post('https://test-bucket.s3.us-east-2.amazonaws.com/', form).then((response) => {
// do something
}
WHAT AM I DOING WRONG TO RECEIVE THE RESPONSE ERROR?