I have a private Google Spreadsheet and I’m trying to access it programmatically using Google Visualization/Google Charts. I've created the service account, and I have tried using the google-auth-library and googleapis npm packages to create an access token that I can then use to access the spreadsheet. But, when I try to use that access token to read from the spreadsheet, the request fails with HTTP code 401 (Unauthorized). What do I need to do in order to make this work?
This is my code:
const { auth } = require('google-auth-library');
const keys = require('./jwt.keys.json');
const id = '{Spreadsheet ID}';
const sheetId = '{Sheet ID}';
const query = "{Query}";
async function getClient(){
const client = auth.fromJSON(keys);
client.scopes = ['https://www.googleapis.com/auth/spreadsheets.readonly'];
console.log(client);
return client;
}
async function main(){
const client = await getClient();
const token = await client.getAccessToken();
console.log(token);
console.log(token.token);
const url = `https://docs.google.com/spreadsheets/d/${id}/gviz/tq?tqx=out:csv&tq=${encodeURIComponent(query)}&access_token=${token.token}#gid=${sheetId}`;
const res = await client.request({url});
console.log(res);
}
main().catch(console.error);
When I saw your script, I thought that it is required modifying the scope. I had got the same situation as you (the status code 401 using your endpoint). Unfortunately, it seems that your endpoint cannot be used using the scope of https://www.googleapis.com/auth/spreadsheets.readonly. So, for example, how about changing it as follows, and testing it again?
From:
https://www.googleapis.com/auth/spreadsheets.readonly
To:
https://www.googleapis.com/auth/spreadsheets
or
https://www.googleapis.com/auth/drive.readonly
Note:
When I tested your endpoint using the modified scope, I confirmed that no error occurred. But if you tested it and when an error occurs, please check other part, again.
Related
Problem
I have uploaded a set of images (blobs) to a private Azure Blob Storage account, but when I try to access them, I am faced with the following error.
GET https://<account-name>.blob.core.windows.net/<container-name>/<blob-name> 403 (Server failed
to authenticate the request. Make sure the value of Authorization header is formed correctly
including the signature.)
I don't have any problems uploading this data as this is done through the server-side using a Django app. I wish to be able to successfully retrieve this uploaded blob data using client-side JavaScript.
Background
I have thoroughly read through and implemented the steps from the Microsoft Azure documentation for authorizing access to my private account via the use of Shared Keys. This includes everything from constructing my signature string to hashing this data using the HMAC SHA-256 algorithm, as detailed in the link above.
I am running everything on Docker containers except for the client-side Vue-based interface which is attempting to invoke the Get Blob API endpoint, as you will see below.
Minimum Reproducible Example
The code that raises this error is as follows:
// Add imports
const crypto = require('crypto');
const axios = require('axios');
// Set Azure blob storage data
const account = "<azure-blob-storage-private-account-name>"
const version = "2020-04-08"
const blob = "<blob-name>"
const container = "<container-name>"
const blob_uri = `https://${account}.blob.core.windows.net/${container}/${blob}`;
const today = new Date().toGMTString();
// Construct signature string
const CanonicalisedHeaders = `x-ms-date:${today}\nx-ms-version:${version}\n`;
const CanonicalisedResource = `/${account}/${container}/${blob}`;
const StringToSign = `GET\n\n\n\n\n\n\n\n\n\n\n\n` + CanonicalisedHeaders + CanonicalisedResource;
// Hash string using HMAC Sha-256 and encode to base64
const key = "<shared-access-key-in-base64>";
const utf8encoded = Buffer.from(key, 'base64').toString('utf8');
const signature = crypto.createHmac('sha256', utf8encoded).update(StringToSign).digest("base64");
// Construct the headers and invoke the API call
const blob_config = {
headers: {
"Authorization": `SharedKey ${account}:${signature}`,
"x-ms-date": today,
"x-ms-version": version
}
}
await axios.get(blob_uri, blob_config)
.then((data) => console.log(data))
.catch((error) => console.log(error.message));
What I have tried
I have tried the following, but none of them have helped me resolve the issue at hand.
Updated CORS settings to avoid CORS-related 403 Forbidden Access issues.
Regenerated my key and connection strings.
Checked the DateTime settings on my local machine and on my Docker containers to ensure they are on the correct GMT time.
Checked that my signature string's components (canonicalized headers, resources, etc.) are constructed according to the rules defined here.
Read through similar StackOverflow and Azure forum posts in search of a solution.
Please try by changing the following lines of code:
const utf8encoded = Buffer.from(key, 'base64').toString('utf8');
const signature = crypto.createHmac('sha256', utf8encoded).update(StringToSign).digest("base64");
to
const keyBuffer = Buffer.from(key, 'base64');
const signature = crypto.createHmac('sha256', keyBuffer).update(StringToSign).digest("base64");
I don't think you need to convert the key buffer to a UTF8 encoded string.
Few other things:
Considering you're using it in the browser, there's a massive security risk as you're exposing your storage keys to your users.
Is there a reason you're using REST API directly instead of using Azure Storage Blob SDK?
In browser-based environments, you should be using Shared Access Signature based authorization instead of Shared Access Key based authorization.
looking for some AWS Javascript SDK help.
I am in the situation that my User Pool is defined in a separate account to my Lambda which needs to send a DescribeUserPoolClient command.
Code snippet below:
import {
CognitoIdentityProviderClient,
DescribeUserPoolClientCommand,
} from '#aws-sdk/client-cognito-identity-provider';
export async function describeUserPoolClient(userPoolClientId: string, userPoolId: string) {
const cognitoClient = new CognitoIdentityProviderClient({});
const describeUserPoolClientCommand = new DescribeUserPoolClientCommand({
ClientId: userPoolClientId,
UserPoolId: userPoolId,
});
const userPoolClient = await cognitoClient.send(describeUserPoolClientCommand);
return userPoolClient;
}
Since I can only provide a userPoolId and not a full ARN, I can't see a way to send this request cross-account without assuming a role in the other account which makes my local testing a nightmare with getting roles and policies set up.
Can anyone see another way of doing this? Thanks for your help.
I've asked this other question here that leads me to believe, by default, the JavaScript AWS SDK looks for credentials in a number of places in your environment without you having to do anything. The order of places it checks is listed here: https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/setting-credentials-node.html
I've got some working code that connects to AWS Athena. I can only get it to work if I hardcode the credentials manually, which seems to contradict the documentation above. Here is my code:
export const getAthena = (): AWS.Athena => {
if (process.env["LOCAL_MODE"] === "true") {
const awsCredentials = {
region: "us-east-1",
accessKeyId: awsCredentialsParser("aws_access_key_id"),
secretAccessKey: awsCredentialsParser("aws_secret_access_key"),
sessionKey: awsCredentialsParser("aws_session_token")
};
AWS.config.update(awsCredentials);
let credential = new AWS.Credentials({
accessKeyId: awsCredentials.accessKeyId,
secretAccessKey: awsCredentials.secretAccessKey,
sessionToken: awsCredentials.sessionKey
});
return new AWS.Athena({credentials: credential, signatureCache: false});
} else {
const awsCredentials1 = {
region: "us-east-1",
accessKeyId: undefined,
secretAccessKey: undefined,
sessionKey: undefined
};
AWS.config.update(awsCredentials1);
return new AWS.Athena({credentials: undefined, signatureCache: false});
}
};
export const awsCredentialsParser = (key: string): string => {
const homeDirectory = os.homedir();
const awsCredentials = fs.readFileSync(homeDirectory + "/.aws/credentials", {encoding: "UTF8"});
const awsCredentialLines = awsCredentials.split("\n");
const lineThatStartsWithKey = awsCredentialLines.filter((line) => line.startsWith(key))[0];
return lineThatStartsWithKey.split(" = ")[1];
};
As you can see, I am using an environment variable called "LOCAL_MODE". If this is set to true, it grabs credentials from my shared credential file. Whereas, if you're not in local mode, it sets all credentials to undefined and relies on the IAM role instead. Isn't the documentation saying I don't have to do this?
But, if I change my code to this, any call to athena hangs until it times out:
export const getAthena = (): AWS.Athena => {
return new AWS.Athena();
};
If I set the timeout to a really large number, it eventually will let me know that I have invalid credentials.
According to the documentation, shouldn't the 2nd example be find credentials the same way the 1st does? Why does the 2nd example hang? I don't want to have to write the code above. How do I get my code to work like the examples?
Am I somehow creating AWS.Athena() the wrong way in the 2nd example?
How do I troubleshoot this to figure out why it's hanging?
According to the documentation, shouldn't the bottom example be doing the same thing as the top?
So after an investigation it seems that this (i.e. the failure on your second snippet) is because you don't have the [default] profile in your .aws/credentials file. Which is a special profile. I assume that the client uses empty strings (or nulls or something) when he can't find it. Which I find amusing to be honest (should throw an exception).
Anyway, to fix that either rename the profile you have to [default] or setup a different profile in your code. Here's the relevant docs:
https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/loading-node-credentials-shared.html
I recommend using the AWS_PROFILE environment variable. Will make your code more portable.
If you're using an IAM role, you don't need to explicitly supply any credentials, null or otherwise:
const AWS = require('aws-sdk');
const athena = new AWS.Athena();
const params = { ... };
const rc = await athena.startQueryExecution(params).promise();
In fact, this works with credentials supplied via local environment variables or in a credentials/config file too. There's a chain of credentials providers that the SDK will try, one by one.
const functions = require('firebase-functions');
import iapReceiptValidator from 'iap-receipt-validator';
const password = 'abcdefghijkac64beb900889e1'; // Shared Secret from iTunes connect
const production = false; // use sandbox or production url for validation
async validate(receiptData) {
try {
const validationData = await validateReceipt(receiptData);
// check if Auto-Renewable Subscription is still valid
validationData['latest_receipt_info'][0].expires_date > today
} catch(err) {
console.log(err.valid, err.error, err.message)
}
}
exports.validateReceipt = functions.https.onRequest((req, res) => {
const validateReceipt = iapReceiptValidator(password, production);
}
This is what I have so far. I am using this external library. I am using firebase cloud functions and I am on flame plan. I want to make POST request to send receipt JSON object to app store server for auto-renewable purchase verification. I am not sure how to go forward with this. Can i please get some help in regards to structuring my cloud function. The firebase documentation isn't very helpful.
Appreciate any help!
I am making a server side Facebook Graph API call to the all_mutual_friends edge: https://developers.facebook.com/docs/graph-api/reference/user-context/all_mutual_friends/
The call works when the two users are friends, but returns no useful data when they users aren't friends. According to the docs, this is because I must sign the call with the appsecret_proof parameter. No matter what I try, I am not able to successfully pass this parameter. I am using jsrsasign running on Parse. I have tried every configuration of using the access token as the message and my appSecret as the key, and vice versa. I have also tried multiple combinations of utf8 and hex. Every time I receive the error: invalid appsecret_proof provided in the API argument
Code:
var Signer = require("cloud/vendor/jsrsasign/lib/jsrsasign.js");
var userId = request.params.userId;
var accessToken = request.params.accessToken;
var appSecret = "redactedStringPastedFromFacebook";
var signer = new Signer.Mac({alg: "hmacsha256", pass: appSecret});
var appSecretString = signer.doFinalString(accessToken);
var appSecretHex = signer.doFinalHex(accessToken);
var graphRequestURL = "https://graph.facebook.com/v2.5/" + userId;
var fields = "?fields=context.fields(all_mutual_friends.fields(name,picture.width(200).height(200)))";
//var authorization = "&access_token=" + accessToken; //this works, but only for existing friends
var authorization = "&access_token=" + accessToken + "&appsecret_proof=" + appSecretHex;
return Parse.Cloud.httpRequest({
url: graphRequestURL + fields + authorization,
method: "GET",
})
Most examples I have seen are in PHP or Python and the crypto routines are a bit more clear. This works in that both appSecretString and appSecretHex don't throw errors and look reasonable, however the values are always rejected by Facebook.
Notes:
I have triple checked the App Secret value provided by Facebook
I have been approved by Facebook to use the all_mutual_friends feature, which is a requirement for this particular call
I am using Parse, which isn't Node, and can't use NPM modules that have external dependencies, which is why I am using jsrsasign. I also tried using CryptoJS directly, but it is no longer maintained and doesn't have proper module support and jsrsasign seems to wrap it anyway.
Here it is:
import CryptoJS from 'crypto-js';
const accessToken = <your_page_access_token>;
const clientSecret = <your_app_client_secret>;
const appsecretProof = CryptoJS.HmacSHA256(accessToken, clientSecret).toString(CryptoJS.enc.Hex);