Problem
I have uploaded a set of images (blobs) to a private Azure Blob Storage account, but when I try to access them, I am faced with the following error.
GET https://<account-name>.blob.core.windows.net/<container-name>/<blob-name> 403 (Server failed
to authenticate the request. Make sure the value of Authorization header is formed correctly
including the signature.)
I don't have any problems uploading this data as this is done through the server-side using a Django app. I wish to be able to successfully retrieve this uploaded blob data using client-side JavaScript.
Background
I have thoroughly read through and implemented the steps from the Microsoft Azure documentation for authorizing access to my private account via the use of Shared Keys. This includes everything from constructing my signature string to hashing this data using the HMAC SHA-256 algorithm, as detailed in the link above.
I am running everything on Docker containers except for the client-side Vue-based interface which is attempting to invoke the Get Blob API endpoint, as you will see below.
Minimum Reproducible Example
The code that raises this error is as follows:
// Add imports
const crypto = require('crypto');
const axios = require('axios');
// Set Azure blob storage data
const account = "<azure-blob-storage-private-account-name>"
const version = "2020-04-08"
const blob = "<blob-name>"
const container = "<container-name>"
const blob_uri = `https://${account}.blob.core.windows.net/${container}/${blob}`;
const today = new Date().toGMTString();
// Construct signature string
const CanonicalisedHeaders = `x-ms-date:${today}\nx-ms-version:${version}\n`;
const CanonicalisedResource = `/${account}/${container}/${blob}`;
const StringToSign = `GET\n\n\n\n\n\n\n\n\n\n\n\n` + CanonicalisedHeaders + CanonicalisedResource;
// Hash string using HMAC Sha-256 and encode to base64
const key = "<shared-access-key-in-base64>";
const utf8encoded = Buffer.from(key, 'base64').toString('utf8');
const signature = crypto.createHmac('sha256', utf8encoded).update(StringToSign).digest("base64");
// Construct the headers and invoke the API call
const blob_config = {
headers: {
"Authorization": `SharedKey ${account}:${signature}`,
"x-ms-date": today,
"x-ms-version": version
}
}
await axios.get(blob_uri, blob_config)
.then((data) => console.log(data))
.catch((error) => console.log(error.message));
What I have tried
I have tried the following, but none of them have helped me resolve the issue at hand.
Updated CORS settings to avoid CORS-related 403 Forbidden Access issues.
Regenerated my key and connection strings.
Checked the DateTime settings on my local machine and on my Docker containers to ensure they are on the correct GMT time.
Checked that my signature string's components (canonicalized headers, resources, etc.) are constructed according to the rules defined here.
Read through similar StackOverflow and Azure forum posts in search of a solution.
Please try by changing the following lines of code:
const utf8encoded = Buffer.from(key, 'base64').toString('utf8');
const signature = crypto.createHmac('sha256', utf8encoded).update(StringToSign).digest("base64");
to
const keyBuffer = Buffer.from(key, 'base64');
const signature = crypto.createHmac('sha256', keyBuffer).update(StringToSign).digest("base64");
I don't think you need to convert the key buffer to a UTF8 encoded string.
Few other things:
Considering you're using it in the browser, there's a massive security risk as you're exposing your storage keys to your users.
Is there a reason you're using REST API directly instead of using Azure Storage Blob SDK?
In browser-based environments, you should be using Shared Access Signature based authorization instead of Shared Access Key based authorization.
Related
I have a private Google Spreadsheet and I’m trying to access it programmatically using Google Visualization/Google Charts. I've created the service account, and I have tried using the google-auth-library and googleapis npm packages to create an access token that I can then use to access the spreadsheet. But, when I try to use that access token to read from the spreadsheet, the request fails with HTTP code 401 (Unauthorized). What do I need to do in order to make this work?
This is my code:
const { auth } = require('google-auth-library');
const keys = require('./jwt.keys.json');
const id = '{Spreadsheet ID}';
const sheetId = '{Sheet ID}';
const query = "{Query}";
async function getClient(){
const client = auth.fromJSON(keys);
client.scopes = ['https://www.googleapis.com/auth/spreadsheets.readonly'];
console.log(client);
return client;
}
async function main(){
const client = await getClient();
const token = await client.getAccessToken();
console.log(token);
console.log(token.token);
const url = `https://docs.google.com/spreadsheets/d/${id}/gviz/tq?tqx=out:csv&tq=${encodeURIComponent(query)}&access_token=${token.token}#gid=${sheetId}`;
const res = await client.request({url});
console.log(res);
}
main().catch(console.error);
When I saw your script, I thought that it is required modifying the scope. I had got the same situation as you (the status code 401 using your endpoint). Unfortunately, it seems that your endpoint cannot be used using the scope of https://www.googleapis.com/auth/spreadsheets.readonly. So, for example, how about changing it as follows, and testing it again?
From:
https://www.googleapis.com/auth/spreadsheets.readonly
To:
https://www.googleapis.com/auth/spreadsheets
or
https://www.googleapis.com/auth/drive.readonly
Note:
When I tested your endpoint using the modified scope, I confirmed that no error occurred. But if you tested it and when an error occurs, please check other part, again.
Got stuck while trying to download an attachment of a work item (Azure DevOps).
I'm Using node.js 'azure-devops-node-api' client (https://www.npmjs.com/package/azure-devops-node-api) to interact with ADO API. I get a certain workItem using WorkItemTracking client (wit):
let workItem = await witApi.getWorkItem(1234, undefined, undefined, WorkItemExpand.All);
let attachment = await witApi.getAttachmentContent(attachmentId, fileName, projectId, true);
Documentation states that getAttachmentContent method downloads an attachment (https://github.com/microsoft/azure-devops-node-api/blob/ff820b2dd0c9a09cf09e64e94d3f95818a77249d/api/WorkItemTrackingApi.ts#L392), as a return value I'm getting a ReadableStream which i tried to write to a file using standard fs module:
fs.writeFile('WordDoc.docx', attachment, function (err) {if (err) return console.log(err);});
File is created but is empty. While debugging i also see that attachment variable has type ReadableStream but inside has lot's of properties and values, among them there is a buffer which i actually would like to extract and pass to fs.writeFile but can't reach them
What am i doing wrong ?
I believe you should write using WritableStream. As the getAttachmentContent is returning a ReadableStream. Below is the pseudo code. It might work
let readableStream = await witApi.getAttachmentContent(attachmentId, fileName, projectId, true);
let writableStream = fs.createWriteStream('./WordDoc.docx');
readableStream.pipe(writableStream);
I flawlessly use azure APIs in my Note.js project.
Login:
const MsRest = require('ms-rest-azure');
MsRest.loginWithServicePrincipalSecret(keys.appId, keys.pass, keys.tenantId);
Create resource group:
const { ResourceManagementClient } = require('azure-arm-resource');
const resourceClient = new ResourceManagementClient(credentials, subscriptionId);
resourceClient.resourceGroups.createOrUpdate(groupName, groupParameters);
It works flawlessly, and so do azure-arm-authorization, azure-arm-compute and azure-arm-network modules.
However, I do not manage to use azure-graph API:
const GraphkManagementClient = require('azure-graph');
const client = new GraphkManagementClient(credentials, subscriptionId);
return client.users.get(principalID);
The last line throws an error:
Access Token missing or malformed
Active Directory Graph service is tenant based and has a different token audience compared to other Azure services like resource manger, compute etc. where all these has subscription based token audience. so, the token that is acquired will work for other Azure Services but not for Graph and thus you received the respective token error. Please refer to https://github.com/Azure/azure-sdk-for-node/tree/master/lib/services/graphManagement#how-to-use to understand the implementation of Graph with node.js
I have to upload an image to the firebase storage. I'm not using the web version of storage (I shouldn't use it). I am using the firebase admin.
No problem, I upload the file without difficulty and I get the result in the variable "file".
and if I access the firebase storage console, the image is there. all right.
return admin.storage().bucket().upload(filePath, {destination: 'demo/images/restaurantCover.jpg',
metadata:{contentType: 'image/jpeg'}
public: true
}).then(file =>{
console.log(`file --> ${JSON.stringify(file, null, 2)}`);
let url = file["0"].metadata.mediaLink; // image url
return resolve(res.status(200).send({data:file})); // huge data
}) ;
Now, I have some questions.
Why so much information and so many objects as a response to the upload () method? Reviewing the immense object, I found a property called mediaLink inside metadata and it is the download url of the image. but...
Why is the url different from the one shown by firebase? Why can not I find the downloadURL property?
How can get the url of firebase?
firebase: https://firebasestorage.googleapis.com/v0/b/myfirebaseapp.appspot.com/o/demo%2Fimages%2Fthumb_restaurant.jpg?alt=media&token=bee96b71-2094-4492-96aa-87469363dd2e
mediaLink: https://www.googleapis.com/download/storage/v1/b/myfirebaseapp.appspot.com/o/demo%2Fimages%2Frestaurant.jpg?generation=1530193601730593&alt=media
If I use the mediaLink url is there any problem with different urls? (read, update from ios and Web Client)
Looking at Google Cloud Storage: Node.js Client documentation, they have a link to sample code which shows exactly how to do this. Also, see the File class documentation example (below)
// Imports the Google Cloud client library
const Storage = require('#google-cloud/storage');
// Creates a client
const storage = new Storage();
/**
* TODO(developer): Uncomment the following lines before running the sample.
*/
// const bucketName = 'Name of a bucket, e.g. my-bucket';
// const filename = 'File to access, e.g. file.txt';
// Gets the metadata for the file
storage
.bucket(bucketName)
.file(filename)
.getMetadata()
.then(results => {
const metadata = results[0];
console.log(`File: ${metadata.name}`);
console.log(`Bucket: ${metadata.bucket}`);
console.log(`Storage class: ${metadata.storageClass}`);
console.log(`Self link: ${metadata.selfLink}`);
console.log(`ID: ${metadata.id}`);
console.log(`Size: ${metadata.size}`);
console.log(`Updated: ${metadata.updated}`);
console.log(`Generation: ${metadata.generation}`);
console.log(`Metageneration: ${metadata.metageneration}`);
console.log(`Etag: ${metadata.etag}`);
console.log(`Owner: ${metadata.owner}`);
console.log(`Component count: ${metadata.component_count}`);
console.log(`Crc32c: ${metadata.crc32c}`);
console.log(`md5Hash: ${metadata.md5Hash}`);
console.log(`Cache-control: ${metadata.cacheControl}`);
console.log(`Content-type: ${metadata.contentType}`);
console.log(`Content-disposition: ${metadata.contentDisposition}`);
console.log(`Content-encoding: ${metadata.contentEncoding}`);
console.log(`Content-language: ${metadata.contentLanguage}`);
console.log(`Metadata: ${metadata.metadata}`);
console.log(`Media link: ${metadata.mediaLink}`);
})
.catch(err => {
console.error('ERROR:', err);
});
I am making a server side Facebook Graph API call to the all_mutual_friends edge: https://developers.facebook.com/docs/graph-api/reference/user-context/all_mutual_friends/
The call works when the two users are friends, but returns no useful data when they users aren't friends. According to the docs, this is because I must sign the call with the appsecret_proof parameter. No matter what I try, I am not able to successfully pass this parameter. I am using jsrsasign running on Parse. I have tried every configuration of using the access token as the message and my appSecret as the key, and vice versa. I have also tried multiple combinations of utf8 and hex. Every time I receive the error: invalid appsecret_proof provided in the API argument
Code:
var Signer = require("cloud/vendor/jsrsasign/lib/jsrsasign.js");
var userId = request.params.userId;
var accessToken = request.params.accessToken;
var appSecret = "redactedStringPastedFromFacebook";
var signer = new Signer.Mac({alg: "hmacsha256", pass: appSecret});
var appSecretString = signer.doFinalString(accessToken);
var appSecretHex = signer.doFinalHex(accessToken);
var graphRequestURL = "https://graph.facebook.com/v2.5/" + userId;
var fields = "?fields=context.fields(all_mutual_friends.fields(name,picture.width(200).height(200)))";
//var authorization = "&access_token=" + accessToken; //this works, but only for existing friends
var authorization = "&access_token=" + accessToken + "&appsecret_proof=" + appSecretHex;
return Parse.Cloud.httpRequest({
url: graphRequestURL + fields + authorization,
method: "GET",
})
Most examples I have seen are in PHP or Python and the crypto routines are a bit more clear. This works in that both appSecretString and appSecretHex don't throw errors and look reasonable, however the values are always rejected by Facebook.
Notes:
I have triple checked the App Secret value provided by Facebook
I have been approved by Facebook to use the all_mutual_friends feature, which is a requirement for this particular call
I am using Parse, which isn't Node, and can't use NPM modules that have external dependencies, which is why I am using jsrsasign. I also tried using CryptoJS directly, but it is no longer maintained and doesn't have proper module support and jsrsasign seems to wrap it anyway.
Here it is:
import CryptoJS from 'crypto-js';
const accessToken = <your_page_access_token>;
const clientSecret = <your_app_client_secret>;
const appsecretProof = CryptoJS.HmacSHA256(accessToken, clientSecret).toString(CryptoJS.enc.Hex);