I want to upload a file to S3, preferably not through a backend server, but only through browser.
In AWS example, to create an S3 client, I need to provide secret key and secret id:
const S3 = new AWS.S3({
accessKeyId: <ACCESS_KEY_ID>,
secretAccessKey: <ACCESS_KEY_SECRET>,
region: <AWS_REGION>
});
But I don't want to expose my access keys. The website is hosted in CloudFront, is it possible to setup permission between the CloudFront and S3 bucket so that I don't need to provide credentials in JavaScript?
For security reasons you should generally have some way to audit the source of an object upload to S3 (it could afterall be anything if you're allowing public access).
Rather than allowing completely public access it might be better to setup AWS Cognito and use it on your frontend.
You'd have the choice for one of the below scenarios:
User signs in via a cognito user
The anonymous user is used.
Both of these options will generate temporary credentials that can be used by your frontend. This will prevent a user making a note of them and keeping them forever, ensure you lockdown the permissions to be a putObject only for that specific S3 bucket.
An alternative approach is to use API Gateway and Lambda to generate a presigned URL that can support uploads. For more information on this approach take a look at this article.
To enhance your security it is probably worth adding an S3 event to trigger a Lambda that will validate the object after it has been uploaded.
You need some kind of backend in order to hide your credentials effectively. Since you're already in the AWS universe, you could use an AWS Lambda function that returns a so-called pre-signed upload url to your frontend which you can use to upload your binaries.
More info:
AWS S3 Presigned URLs
You need to use S3 presigned URLs. Your backend generates a special URL that contains a signature and send it to the browser. Then the browser can use that URL to send a POST request (or a PUT request, but that is a signed PUT URL) to upload the file directly to the S3 bucket. No access keys are exposed to the client.
You still need to add access keys to your backend, which can be through roles if it is inside AWS. But these keys don't reach the browser, only the signature.
I've written about this topic extensively and also made code examples you can check.
Related
I'm new to web development and server requests, I have a url to an amazon s3 bucket and I have all the required values, such as key, policy, security token, signature, and the file I want to upload etc. I was wondering how I would actually upload the file to the s3 bucket in javascript. Would I use:
$.post, or something else and what would the parameters be for the request and the format. Currently I keep getting error 412 or 403.
Thanks
I would suggest using the AWS Javascript SDK which is a library of functions for interacting with AWS service endpoints. If you attempt to post directly to a service endpoint or s3 URL then you will be expected to have signed the call using the standard AWS Signature v4 which can be hard to implement and is done for you in the SDK libraries.
The SDK will also correctly integrate with IAM permissions which you typically need to have correctly setup to write to a bucket.
A full example is provided here
I have a static website hosted on an S3 bucket. The website takes a file as an input and uploads it to another S3 bucket. I don't want to keep the bucket as public. How can I upload a file to a non-public bucket? Currently, I'm using my access key to upload a file, but I don't want to expose the access key in my Javascript code for authentication. My objective is to upload a file to a non-public bucket without exposing my access credentials.
Edit1:
I used the amazon API gateway with Lambda to generate pre-signed S3 URL. This way my credentials don't get exposed.
You can use environment variables to hide API Keys for Javascript. React and Node examples
You could also use AWS Parameter Store to store the API Keys. There are examples of how to integrate with node.js and other frameworks.
I am trying to get hold of my image objects from S3, from my JavaScript frontend application.
Acording to the documentation, these are the steps required:
import * as AWS from "aws-sdk";
AWS.config.update({accesKeyId, secretAccesKey, region});
let s3 = new AWS.S3();
And then, you can get the objects like so:
function listObjects(bucketName, folderName) {
return new Promise((resolve) => {
s3.listObjects({Bucket: bucketName, Prefix: folderName}).promise()
.then((data) => {
resolve(data.Contents);
})
});
}
All seems to work correctly, but what worries me is that I also need to keep the accessKeyId and the secretAccessKey in my frontend application, in order to access the bucket.
How does one secure the bucket, or access the objects without providing these confidential data?
You're right to worry. Any one will be able to take the credentials out of your app. There's a few approaches to this:
if the objects aren't actually sensitive, then there's nothing lost if the credential can only take the actions you wish to allow everyone. For that matter you should be able to get rid of the need for credentials all together if you set the permissions on your bucket properly .. I think that includes list permissions if necessary.
if the objects are sensitive, then you already have some sort of authentication system for your users. IF you're using Oauth accounts to auth ( google, amazon, facebook ,etc ) then you can use AWS Cognito to generate short lived AWS credentials that are associated to that user, which would allow you to differentiate permissions between users ... it's pretty slick and a great fit if already using oauth. IF you're not using oauth, consider whether you should be. It's far more secure than having to hande your own auth creds layer for your users. https://aws.amazon.com/cognito/
if you don't want to or can't user cognito, you can still assume an AWS role from the backend and generate temporary credentials that automatically expire in anywhere from 15 minutes to 1 hour or more and then pass those credentials to the front end. I'd call it "poor man's cognito" but I think it's probably actually more expensive to run the infra to provide the service than cognito costs.
Or, as #Tomasz Swinder suggests, you can simply proxy the requests through your application, resolving the asset the user requests to an s3 resource and pulling it in your backend and then serving to your user. This is an inferior solution in most cases because your servers are farther away from the end user than s3's endpoints are likely to be. And, you have to run infrastructure to proxy. But, that having been said, it has it's place.
Finally, pre-signed s3 urls may be a good fit for your application. Typically a backend would sign the s3 urls directly before providing them to the user. The signature is enough to authorize the operation ( which can be PUT or GET) but doesn't itself contain the private key used to sign - in other words, presigned urls provide an authorized URL but not the credentials used to authorize them, so they're a great way to provide ad hoc authorization to s3.
Overall it's really awesome to have a backend-free application, and for that you're going to need a 3rd party auth and something like cognito. but once you start using it, you can then use all sorts of aws services to provide what would otherwise be done by a backend. Just be careful with permissions because aws is all pay as you go and there's usually no capability to limit the calls to a service to make sure a cruel internet user strives to drive up your AWS bill by making tons of calls with the temporary creds you've provided them. One notable exception to that is API Gateway, which does allow per user rate limits and therefore is a great fit for a cognito-authorized serverless backend.
Also bear in mind that LISTing s3 objects is both much slower, and much more expensive ( still cheap per op, but 10x ) than GETing s3 objects, so it's usually best to avoid calling lIST whenever possible. I'm just throwing that out there, I suspect you're just doing that to test out the s3 connection.
Could you request that through your server? or is it a static site?
If this is a static site then, You can create IAM User for s3 that can only read the content that you are going to use and show in the fronted anyway.
One method we use is to store the credentials in a .env file and use dotenv (https://github.com/motdotla/dotenv) to read in the variables. These can then be accessed through process.env. For example, the .env file would contain:
AWSKEY=1234567abcdefg
AWSSECRET=hijklmn7654321
REGION=eu-west
Then in your code you would then call require('dotenv').load() to read the environment variables. You then access them as:
AWS.config.update({process.env.AWSKEY, process.env.AWSSECRET, process.env.REGION});
Make sure that the .env file is not committed into your repo. If you want you could have a env.example and instructions on how to create a .env when creating either a dev or production install.
As for securing the bucket, you can do so by restricting the read/write access to an IAM user that owns the AWS key/secret pair.
I'm completely new to AWS and have some security issue.
I want to allow my user to upload a profile picture and I want to save it in S3.
My code looks like this:
import AWS_S3 from 'aws-sdk/clients/s3';
import config from '../../config';
const myS3Credentials = {
accessKeyId: config('AWSS3AccessKeyID'),
secretAcccessKey: config('AWSS3SecretAccessKey'),
};
console.log('myS3Credentials:', myS3Credentials);
const S3 = new AWS_S3({
credentials: myS3Credentials,
region: config('AWSS3Region'),
});
All of the variables (like AWSS3Region, and my credentials) are set up in the .env file. But here, I'm exposing them in the code. How to avoid that? Or should I set up some bucket permissions?
You can use the aws sdk getSignedUrl method which is exposed on the S3 object. This would allow you to upload to your bucket directly from the client without exposing your access tokens. You can keep your access tokens safe by leaving them in the .env file and keeping that file out of your repo.
Creating signed urls will require creating an endpoint on your server that would return the signed URL. From there you would perform a put request containing the image. I have created a gist with an example. In the gist there is client and server code. https://gist.github.com/pizza-r0b/35be6dd3e992ef1ebb2159772cb768c0
You should never, ever send/expose AWS access tokens directly in your client code.
Put that code on a server and make calls to your server code, which in turn makes calls to AWS.
On your server, you should never use hardcoded access keys either. Use environmental variables to get the access tokens, as Alejandro stated in his answer below.
Its strongly discouraged to store permanent credentials in your client code to upload files to S3. There are several approaches to handle this securely.
Get temporary access credentials from your backend using AWS STS SDK or using a service like AWS Cognito.
Use AWS CloudFront Signed URLs or Signed Cookies to send back to the client from Server so that, using them you can upload files to S3.
Few references are listed below to get you started with Signed Urls.
Uploading Objects Using Pre-Signed URLs
Node.js module to create and sign URLs to access private resources on Amazon S3
Just call on your code process.env.<YOUR_KEY>.
I allow users to upload to s3 directly without the need to go through the server ; Everything works perfectly; my only worry is the security.
In my javascript code I check for file extensions . However I know in javascript code Users can manipulate script- in my case to allow upload of xml files - (since client side upload) and thus would'nt they be able to replace the crossdomain.xml in my bucket and accordingly be able to control my bucket?
Note: I am using the bucket owner access key and secret key.
Update:
Any possible approaches to overcome this issue...?
If you are not adverse to running additional resources, you can accomplish this by running a Token Vending Machine.
Here's the gist:
Your token vending machine (TVM) runs as a reduced privileged user.
Your client code still uploads directly to S3, but it needs to contact your TVM to get temporary, user-specific token to access your bucket when the user logs in
The TVM calls the Amazon Security Token Service to create temporary credentials for your user to access S3
The S3 API uses the temporary credentials when it makes requests to upload/download
You can define polices on your buckets to limit what areas of the bucket each user can access
A simple example of creating a "dropbox"-like service using per-user access is detailed here.