I'm trying to use Postman to send a GET http request which contains a parameter which is dynamically generated by taking the full request query string (everything to the right of the question mark in the URL, after URL encoding), concatenating a previously assigned shared secret key, and then performing a SHA-1 hash of the resulting string.
I would use a Pre-request Script to achieve this.
Thank you.
I actually found a solution and would like to share it.
var params = [
["client_id", "222"]
,["account_id", ""]
];
// Build the request body string from the Postman request.data object
var requestBody = "";
var firstpass = true;
for(var i=0;i < params.length; i++) {
if(!firstpass){
requestBody += "&";
}
requestBody += params[i][0] + "=" + params[i][1];
firstpass = false;
postman.setGlobalVariable(params[i][0], params[i][1]);
}
requestBody += postman.getEnvironmentVariable("sharedSecretKey");
postman.setGlobalVariable("requestBody", requestBody);
var mac = "";
if(requestBody){
// SHA1 hash
mac = CryptoJS.SHA1(requestBody);
}
postman.setGlobalVariable("mac", mac);
Then I just need to set the parameters in the URL :
{{baseUrl}}/get?client_id={{client_id}}&account_id={{account_id}}&mac={{mac}}
where {{baseUrl}} is an environment variable
and {{client_id}}, {{account_id}} are global variables
Hope it can be helpful to someone.
Thank you.
Inspired by this answer, I used the following Postman pre-request script to create a SHA1 hash of a request.
Note that request.data is an implied variable and the CryptoJS library are provided by the Postman Sandbox for pre-request scripts.
const hash = CryptoJS.HmacSHA1(request.data, 'yourSecret').toString();
pm.globals.set('hash', hash);
You can now reference the hash value as a postman global variable using {{hash}} syntax.
Creating X-Hub-Signature Header like GitHub API Webhook Requests
My purpose in all this was to simulate the X-Hub-Signature header provided by the GitHub Webhooks API because my web service validates all webhook payloads to match the signature. So for me to test my web service, I also needed postman to generate a valid signature header.
Here's an adaptation of the above code snippet for generating the X-Hub-Signature request header value.
In GitHub, I set a webhook secret for my GitHub App.
In Postman, I created an environment and added the key=value pair GITHUB_WEBHOOK_SECRET with the value I specified when I created my GitHub App.
In Postman, I used the following pre-request script. It set the computed hash as a global variable.
const hash = CryptoJS.HmacSHA1(
request.data,
pm.environment.get('GITHUB_WEBHOOK_SECRET')
).toString();
pm.globals.set('X-HUB-SIGNATURE', 'sha1=' + hash);
In Postman, I reference the global hash variable as a header in my requests, just like the GitHub Webhooks API will.
Related
Problem
I have uploaded a set of images (blobs) to a private Azure Blob Storage account, but when I try to access them, I am faced with the following error.
GET https://<account-name>.blob.core.windows.net/<container-name>/<blob-name> 403 (Server failed
to authenticate the request. Make sure the value of Authorization header is formed correctly
including the signature.)
I don't have any problems uploading this data as this is done through the server-side using a Django app. I wish to be able to successfully retrieve this uploaded blob data using client-side JavaScript.
Background
I have thoroughly read through and implemented the steps from the Microsoft Azure documentation for authorizing access to my private account via the use of Shared Keys. This includes everything from constructing my signature string to hashing this data using the HMAC SHA-256 algorithm, as detailed in the link above.
I am running everything on Docker containers except for the client-side Vue-based interface which is attempting to invoke the Get Blob API endpoint, as you will see below.
Minimum Reproducible Example
The code that raises this error is as follows:
// Add imports
const crypto = require('crypto');
const axios = require('axios');
// Set Azure blob storage data
const account = "<azure-blob-storage-private-account-name>"
const version = "2020-04-08"
const blob = "<blob-name>"
const container = "<container-name>"
const blob_uri = `https://${account}.blob.core.windows.net/${container}/${blob}`;
const today = new Date().toGMTString();
// Construct signature string
const CanonicalisedHeaders = `x-ms-date:${today}\nx-ms-version:${version}\n`;
const CanonicalisedResource = `/${account}/${container}/${blob}`;
const StringToSign = `GET\n\n\n\n\n\n\n\n\n\n\n\n` + CanonicalisedHeaders + CanonicalisedResource;
// Hash string using HMAC Sha-256 and encode to base64
const key = "<shared-access-key-in-base64>";
const utf8encoded = Buffer.from(key, 'base64').toString('utf8');
const signature = crypto.createHmac('sha256', utf8encoded).update(StringToSign).digest("base64");
// Construct the headers and invoke the API call
const blob_config = {
headers: {
"Authorization": `SharedKey ${account}:${signature}`,
"x-ms-date": today,
"x-ms-version": version
}
}
await axios.get(blob_uri, blob_config)
.then((data) => console.log(data))
.catch((error) => console.log(error.message));
What I have tried
I have tried the following, but none of them have helped me resolve the issue at hand.
Updated CORS settings to avoid CORS-related 403 Forbidden Access issues.
Regenerated my key and connection strings.
Checked the DateTime settings on my local machine and on my Docker containers to ensure they are on the correct GMT time.
Checked that my signature string's components (canonicalized headers, resources, etc.) are constructed according to the rules defined here.
Read through similar StackOverflow and Azure forum posts in search of a solution.
Please try by changing the following lines of code:
const utf8encoded = Buffer.from(key, 'base64').toString('utf8');
const signature = crypto.createHmac('sha256', utf8encoded).update(StringToSign).digest("base64");
to
const keyBuffer = Buffer.from(key, 'base64');
const signature = crypto.createHmac('sha256', keyBuffer).update(StringToSign).digest("base64");
I don't think you need to convert the key buffer to a UTF8 encoded string.
Few other things:
Considering you're using it in the browser, there's a massive security risk as you're exposing your storage keys to your users.
Is there a reason you're using REST API directly instead of using Azure Storage Blob SDK?
In browser-based environments, you should be using Shared Access Signature based authorization instead of Shared Access Key based authorization.
I am trying to manually verify web-hook:
const stripeSecret = createHmac('sha256', STRIPE_SIGNING_SECRET)
.update(event.body)
.digest('hex');
if(stripeSecret !== keyFromHeader) {
throw err();
}
But it is not matched with Stripe secret key which is received in header.
Here is event data which I am also trying to use in Stripe API (it also fails):
it's not event.body you should hash According to the documentation (https://stripe.com/docs/webhooks/signatures#verify-manually
) its a concatenation of :
The timestamp (as a string)
The character .
The actual JSON payload (i.e., the request body) => JSON.stringify(req.body)
you will need to parse this to get the timestamp (the xxxxx in the "t=xxxxx" part)
const sig = request.headers['stripe-signature'];
if you give me a sample stripe signature header i can try a code sample.
event.body might not be enough — it's very common in Node server environments for that to be a "parsed" version of the incoming request body, and instead you need to make sure to access the actual raw JSON string in the body — that's what Stripe's signature is computed against. It can be a little tricky!
https://github.com/stripe/stripe-node/tree/master/examples/webhook-signing
(and many examples contributed for various frameworks at https://github.com/stripe/stripe-node/issues/341 )
Also, is there a specific reason to do this manually and not just use Stripe's Node library? :)
I have the following code in Google Apps Script which retrieves CSV data from a webpage via HTTP using basic authentication and places it into a spreadsheet:
CSVImport.gs
function parseCSVtoSheet(sheetName, url)
{
// Credentials
var username = "myusername";
var password = "mypassword";
var header = "Basic " + Utilities.base64Encode(username + ":" + password);
// Setting the authorization header for basic HTTP authentication
var options = {
"headers": {
"Authorization": header
}
};
// Getting the ID of the sheet with the name passed as parameter
var spreadsheet = SpreadsheetApp.getActive();
var sheet = spreadsheet.getSheetByName(sheetName);
var sheetId = sheet.getSheetId();
// Getting the CSV data and placing it into the spreadsheet
var csvContent = UrlFetchApp.fetch(url, options).getContentText();
var resource = {requests: [{pasteData: {data: csvContent, coordinate: {sheetId: sheetId}, delimiter: ","}}]};
Sheets.Spreadsheets.batchUpdate(resource, spreadsheet.getId());
}
This has been working up until recently where randomly I get the following error on the UrlFetchApp.fetch line:
Exception: Unexpected error: http://www.myurl.com/data/myfile.csv (line 21, file "CSVImport")
I have tried:
Putting the credentials directly in the URL instead of in an Authorization header (I received a different error saying "Login information disallowed").
Encoding the credentials to base64 right when I pass it into the headers object (didn't work, same error).
Removing authentication altogether (predictably I received a 401 response from the HTTP page).
I'm not sure what else to try and why this randomly broke down all of a sudden. Any advice?
This is related to a new bug, see here
Many users are affected, I recommend you to "star" the issue to increase visibility and hopefully accelerate the process.
I had the same situation. At that time, I could noticed that when the built-in function of Google Spreadsheet is used for the URL, the values can be retrieved. In that case, as the current workaround, I used the following flow.
Put a formula of =IMPORTDATA(URL).
Retrieve the values from the sheet.
When above flow is reflected to your URL of http://www.myurl.com/data/myfile.csv, it becomes as follows.
About basic authorization for URL:
When I saw your script, I confirmed that you are using the basic authorization. In this case, the user name and password can be used for the URL like http://username:password#www.myurl.com/data/myfile.csv.
From your script, when the values of username and password are myusername and mypassword, respectively, you can use the URL as http://myusername:mypassword#www.myurl.com/data/myfile.csv.
Here, there is an important point. If the specific characters are included in username and password, please do the url encode for them.
Sample script:
function myFunction() {
const url = "http://myusername:mypassword#www.myurl.com/data/myfile.csv"; // This is your URL.
// Retrieve the values from URL.
var spreadsheet = SpreadsheetApp.getActive();
var sheet = spreadsheet.getSheetByName(sheetName);
sheet.clear();
var range = sheet.getRange("A1");
range.setFormula(`=IMPORTDATA("${url}")`);
// Retrieve the values from sheet to an array.
SpreadsheetApp.flush();
var values = sheet.getDataRange().getValues();
range.clear();
console.log(values)
}
When above script is run, the values from the URL are put to the sheet, and the values are retrieved as 2 dimensional array for values. If you want to leave only values without the formula, I think that you can copy and paste the values.
In this answer, I used IMPORTDATA. But for each situation, other functions might be suitable. In that case, please check them.
Note:
This is the current workaround. So when this issue was removed, I think that you can use your original script.
References:
IMPORTDATA
setFormula()
Disable Chrome V8 Runtime Engine until Google fix this.
To disable:
From Menu click on Run > Disable new Apps Script runtime powered by Chrome V8
As per #346 from the official issue tracker https://issuetracker.google.com/issues/175141974
hey guys I have found a possible solution that is working for me try
inputting empty {} like this
UrlFetchApp.fetch(apiLink, {});
it worked for me
I tried this, and it too is working for me. Works even when using the "new Apps Script runtime powered by Chrome V8".
Using the SAP Cloud SDK JavaScript (#sap-cloud-sdk/core version 1.26.1, #sap/cloud-sdk-vdm-product-service version 1.19.0) ODATA filters are not percent encoded in the URL query part.
Example: (Assuming a product with description "ä_description" exists)
The following example does not retrieve this product description:
const term = 'ä_description';
const destination = getDestinationInformation(); //get destination information from somewhere
const results = await ProductDescription.requestBuilder()
.getAll()
.filter(ProductDescription.PRODUCT_DESCRIPTION.equals(term))
.execute(destination);
This snippet produces the following request URL:
"https://<host>/sap/opu/odata/sap/API_PRODUCT_SRV/A_ProductDescription?$format=json&$filter=(ProductDescription eq 'ä_description')"
When performing percent-encoding on the search term (const term = encodeURIComponent('ä_description');) the following request URL is generated:
"https://<host>/sap/opu/odata/sap/API_PRODUCT_SRV/A_ProductDescription?$format=json&$filter=(ProductDescription eq '%C3%A4_description')"
This returns the expected result.
I haven't checked but this may affect other VDM packages as well.
Should the SDK itself or the SDK user perform URL encoding? I want to avoid double encoding.
Thanks in advance,
ujj
The version 1.27.0 has been released last week. The SDK handles the url encoding from this version. Please try it. See release notes here.
I need to find how many parameters sent in an URL.
How can I determine the number of parameters sent via URL in nodeJS?
use req.query
if Your URL is like localhost:3000?param1=¶m2=
var params = req.query;
van length = Object.keys(params).length;
the length is 2.
If using express framework
req.params // can be used
See the documentation here