I allow users to upload to s3 directly without the need to go through the server ; Everything works perfectly; my only worry is the security.
In my javascript code I check for file extensions . However I know in javascript code Users can manipulate script- in my case to allow upload of xml files - (since client side upload) and thus would'nt they be able to replace the crossdomain.xml in my bucket and accordingly be able to control my bucket?
Note: I am using the bucket owner access key and secret key.
Update:
Any possible approaches to overcome this issue...?
If you are not adverse to running additional resources, you can accomplish this by running a Token Vending Machine.
Here's the gist:
Your token vending machine (TVM) runs as a reduced privileged user.
Your client code still uploads directly to S3, but it needs to contact your TVM to get temporary, user-specific token to access your bucket when the user logs in
The TVM calls the Amazon Security Token Service to create temporary credentials for your user to access S3
The S3 API uses the temporary credentials when it makes requests to upload/download
You can define polices on your buckets to limit what areas of the bucket each user can access
A simple example of creating a "dropbox"-like service using per-user access is detailed here.
Related
I want to upload a file to S3, preferably not through a backend server, but only through browser.
In AWS example, to create an S3 client, I need to provide secret key and secret id:
const S3 = new AWS.S3({
accessKeyId: <ACCESS_KEY_ID>,
secretAccessKey: <ACCESS_KEY_SECRET>,
region: <AWS_REGION>
});
But I don't want to expose my access keys. The website is hosted in CloudFront, is it possible to setup permission between the CloudFront and S3 bucket so that I don't need to provide credentials in JavaScript?
For security reasons you should generally have some way to audit the source of an object upload to S3 (it could afterall be anything if you're allowing public access).
Rather than allowing completely public access it might be better to setup AWS Cognito and use it on your frontend.
You'd have the choice for one of the below scenarios:
User signs in via a cognito user
The anonymous user is used.
Both of these options will generate temporary credentials that can be used by your frontend. This will prevent a user making a note of them and keeping them forever, ensure you lockdown the permissions to be a putObject only for that specific S3 bucket.
An alternative approach is to use API Gateway and Lambda to generate a presigned URL that can support uploads. For more information on this approach take a look at this article.
To enhance your security it is probably worth adding an S3 event to trigger a Lambda that will validate the object after it has been uploaded.
You need some kind of backend in order to hide your credentials effectively. Since you're already in the AWS universe, you could use an AWS Lambda function that returns a so-called pre-signed upload url to your frontend which you can use to upload your binaries.
More info:
AWS S3 Presigned URLs
You need to use S3 presigned URLs. Your backend generates a special URL that contains a signature and send it to the browser. Then the browser can use that URL to send a POST request (or a PUT request, but that is a signed PUT URL) to upload the file directly to the S3 bucket. No access keys are exposed to the client.
You still need to add access keys to your backend, which can be through roles if it is inside AWS. But these keys don't reach the browser, only the signature.
I've written about this topic extensively and also made code examples you can check.
I am trying to get hold of my image objects from S3, from my JavaScript frontend application.
Acording to the documentation, these are the steps required:
import * as AWS from "aws-sdk";
AWS.config.update({accesKeyId, secretAccesKey, region});
let s3 = new AWS.S3();
And then, you can get the objects like so:
function listObjects(bucketName, folderName) {
return new Promise((resolve) => {
s3.listObjects({Bucket: bucketName, Prefix: folderName}).promise()
.then((data) => {
resolve(data.Contents);
})
});
}
All seems to work correctly, but what worries me is that I also need to keep the accessKeyId and the secretAccessKey in my frontend application, in order to access the bucket.
How does one secure the bucket, or access the objects without providing these confidential data?
You're right to worry. Any one will be able to take the credentials out of your app. There's a few approaches to this:
if the objects aren't actually sensitive, then there's nothing lost if the credential can only take the actions you wish to allow everyone. For that matter you should be able to get rid of the need for credentials all together if you set the permissions on your bucket properly .. I think that includes list permissions if necessary.
if the objects are sensitive, then you already have some sort of authentication system for your users. IF you're using Oauth accounts to auth ( google, amazon, facebook ,etc ) then you can use AWS Cognito to generate short lived AWS credentials that are associated to that user, which would allow you to differentiate permissions between users ... it's pretty slick and a great fit if already using oauth. IF you're not using oauth, consider whether you should be. It's far more secure than having to hande your own auth creds layer for your users. https://aws.amazon.com/cognito/
if you don't want to or can't user cognito, you can still assume an AWS role from the backend and generate temporary credentials that automatically expire in anywhere from 15 minutes to 1 hour or more and then pass those credentials to the front end. I'd call it "poor man's cognito" but I think it's probably actually more expensive to run the infra to provide the service than cognito costs.
Or, as #Tomasz Swinder suggests, you can simply proxy the requests through your application, resolving the asset the user requests to an s3 resource and pulling it in your backend and then serving to your user. This is an inferior solution in most cases because your servers are farther away from the end user than s3's endpoints are likely to be. And, you have to run infrastructure to proxy. But, that having been said, it has it's place.
Finally, pre-signed s3 urls may be a good fit for your application. Typically a backend would sign the s3 urls directly before providing them to the user. The signature is enough to authorize the operation ( which can be PUT or GET) but doesn't itself contain the private key used to sign - in other words, presigned urls provide an authorized URL but not the credentials used to authorize them, so they're a great way to provide ad hoc authorization to s3.
Overall it's really awesome to have a backend-free application, and for that you're going to need a 3rd party auth and something like cognito. but once you start using it, you can then use all sorts of aws services to provide what would otherwise be done by a backend. Just be careful with permissions because aws is all pay as you go and there's usually no capability to limit the calls to a service to make sure a cruel internet user strives to drive up your AWS bill by making tons of calls with the temporary creds you've provided them. One notable exception to that is API Gateway, which does allow per user rate limits and therefore is a great fit for a cognito-authorized serverless backend.
Also bear in mind that LISTing s3 objects is both much slower, and much more expensive ( still cheap per op, but 10x ) than GETing s3 objects, so it's usually best to avoid calling lIST whenever possible. I'm just throwing that out there, I suspect you're just doing that to test out the s3 connection.
Could you request that through your server? or is it a static site?
If this is a static site then, You can create IAM User for s3 that can only read the content that you are going to use and show in the fronted anyway.
One method we use is to store the credentials in a .env file and use dotenv (https://github.com/motdotla/dotenv) to read in the variables. These can then be accessed through process.env. For example, the .env file would contain:
AWSKEY=1234567abcdefg
AWSSECRET=hijklmn7654321
REGION=eu-west
Then in your code you would then call require('dotenv').load() to read the environment variables. You then access them as:
AWS.config.update({process.env.AWSKEY, process.env.AWSSECRET, process.env.REGION});
Make sure that the .env file is not committed into your repo. If you want you could have a env.example and instructions on how to create a .env when creating either a dev or production install.
As for securing the bucket, you can do so by restricting the read/write access to an IAM user that owns the AWS key/secret pair.
I am trying to show objects from an S3 bucket that is not public. In order to do this I would have to provide the access and secret keys to AWS.
I have this fiddle (without the keys) but it does not work when I enter the correct keys: http://jsfiddle.net/jsp3wzbu/
<section ng-app data-ng-controller="myCtrl">
<img ng-src="{{s3url}}" id="myimg">
</section>
also, how is security handled? I would not want to store the access/secret keys in my client code because users will see it. My server code keeps these keys in environment variables and I fear that if I share them with my client side JS code, then they will be exposed. Is there any other way for me to show the S3 object on the browser? ....Can the server provide the images as base64 json and the client side code renders it?
There are multiple approaches you can follow to achieve this.
Use S3 Signed Urls.
Use AWS STS to generate temporary access credentials from Backend.
Use AWS Cognito Federated Identities to Generate temporary access credentials.
Use CloudFront Signed Urls or Cookies.
Note: Storing or sending permanent IAM credentials to the client side is not recommended.
Here is how I handle providing access to the contents of a private S3 bucket.
I use IAM roles for my EC2 instances. I do not store AWS credentials on the EC2 instance.
I require the user to login. You can use a home brew login setup (database), Cognito or another IDP such as Google or Facebook.
From my back-end code I generate presigned URLs that expire in 15 minutes. If the URLs are for large files, I adjust the timeout to be longer based upon the size of the file assuming a slow Internet connection.
In the JavaScript for my HTML pages, I refresh the URLs before the 15 minutes expires (usually every 5 minutes via AJAX). This can be done via a simple refresh page or (better) by using AJAX to just refresh the URLs. This handles users that leave a page open for a long period of time.
I'm completely new to AWS and have some security issue.
I want to allow my user to upload a profile picture and I want to save it in S3.
My code looks like this:
import AWS_S3 from 'aws-sdk/clients/s3';
import config from '../../config';
const myS3Credentials = {
accessKeyId: config('AWSS3AccessKeyID'),
secretAcccessKey: config('AWSS3SecretAccessKey'),
};
console.log('myS3Credentials:', myS3Credentials);
const S3 = new AWS_S3({
credentials: myS3Credentials,
region: config('AWSS3Region'),
});
All of the variables (like AWSS3Region, and my credentials) are set up in the .env file. But here, I'm exposing them in the code. How to avoid that? Or should I set up some bucket permissions?
You can use the aws sdk getSignedUrl method which is exposed on the S3 object. This would allow you to upload to your bucket directly from the client without exposing your access tokens. You can keep your access tokens safe by leaving them in the .env file and keeping that file out of your repo.
Creating signed urls will require creating an endpoint on your server that would return the signed URL. From there you would perform a put request containing the image. I have created a gist with an example. In the gist there is client and server code. https://gist.github.com/pizza-r0b/35be6dd3e992ef1ebb2159772cb768c0
You should never, ever send/expose AWS access tokens directly in your client code.
Put that code on a server and make calls to your server code, which in turn makes calls to AWS.
On your server, you should never use hardcoded access keys either. Use environmental variables to get the access tokens, as Alejandro stated in his answer below.
Its strongly discouraged to store permanent credentials in your client code to upload files to S3. There are several approaches to handle this securely.
Get temporary access credentials from your backend using AWS STS SDK or using a service like AWS Cognito.
Use AWS CloudFront Signed URLs or Signed Cookies to send back to the client from Server so that, using them you can upload files to S3.
Few references are listed below to get you started with Signed Urls.
Uploading Objects Using Pre-Signed URLs
Node.js module to create and sign URLs to access private resources on Amazon S3
Just call on your code process.env.<YOUR_KEY>.
I receive images for processing on my site and wanna omit uploading to local server and further adding to Dropbox folder from it by implementing direct upload to Dropbox from web browser.
Dropbox API is pretty simple, but has one problem - I need to expose API key to end user, and that key allows to download all pics from my account.
Then if I use Dropbox API - one user can download images uploaded by others, and it's not acceptable scenario.
Is there any way to bypass limitation with exposing full access API key to website end users?
Unfortunately, no, in order to upload directly to Dropbox via the Dropbox API, the client needs the access token. Further, the Dropbox API doesn't offer an upload-only permission.
Fundamentally, clients can't keep secrets, so any access token that exists client-side (in this case, in the browser) can be extracted and abused.
Edit:
The Dropbox API now offers some new pieces of functionality that may be useful here:
A) The Dropbox API now offers "scopes", which you can use to configure an app or access token to only a limited set of functionality, such as the ability to write but not read files.
You can find more information about the release in our blog post here:
https://dropbox.tech/developers/now-available--scoped-apps-and-enhanced-permissions
B) The Dropbox API now offers the /2/files/get_temporary_upload_link endpoint, which can be used to get a URL that a client can POST to in order to upload to a Dropbox account without an access token:
https://www.dropbox.com/developers/documentation/http/documentation#files-get_temporary_upload_link