We Keep Coding

How to use an S3 pre-signed POST url?

Actually, this is the first time that I'm using s3 for uploading files. I have heard about pre-signed urls But apparently, I can't set a limitation for file size so I found "pre-signed post urls" but it's a little bit wierd!! Surprisingly I didn't find any example. maybe it's not what I want.
I'm getting pre-signed post url from the server:
const { S3 } = require("aws-sdk");
const s3 = new S3({
accessKeyId: accessKey,
secretAccessKey: secretKey,
endpoint: api,
s3ForcePathStyle: true,
signatureVersion: "v4",
});
app.post("/get-url", (req, res) => {
const key = `user/${uuri.v4()}.png`;
const params = {
Bucket: "bucketName",
Fields: {
Key: key,
ContentType: "image/png",
},
};
s3.createPresignedPost(params, function (err, data) {
if (err) {
console.error("Presigning post data encountered an error", err);
} else {
res.json({ url: data.url });
}
});
});
The weird thing is that the url that I get is not like a pre-signed url. it's just the endpoint followed by the bucket name. no query parameter. no option.
As you might guess, i can't use this url:
await axios.put(url, file, {
headers: {
"Content-Type": "image/png",
},
});
I do not even know if I should use post or two requests.
I tried both, Nothing happens. Maybe the pre-signed post url is not like pre-signed url!
At least show me an example! I can't found any.

You are on the right track, but you need to change the method you are invoking. The AWS S3 API docs of the createPresignedPost() that you are currently using states that:
Get a pre-signed POST policy to support uploading to S3 directly from an HTML form.
Try change this method to either getSignedUrl():
Get a pre-signed URL for a given operation name.
const params = { Bucket: 'bucket', Key: 'key' };
s3.getSignedUrl('putObject', params, function (err, url) {
if (err) {
console.error("Presigning post data encountered an error", err);
} else {
res.json({ url });
}
});
or synchronously:
const params = { Bucket: 'bucket', Key: 'key' };
const url = s3.getSignedUrl('putObject', params)
res.json({ url });
Alternatively, use a promise by executing getSignedUrlPromise():
Returns a 'thenable' promise that will be resolved with a pre-signed URL for a given operation name.
const params = { Bucket: 'bucket', Key: 'key' };
s3.getSignedUrlPromise('putObject', params)
.then(url => {
res.json({ url });
}, err => {
console.error("Presigning post data encountered an error", err);
});
Please also read the notes parts of the API documentation to make sure that you understand the limitations of each method.

AWS S3 bucket returns access denied on ec2 instance usage, even with attached policies and permissions

I have hosted a loopback api on ec-2 server which will download the contents of an s3 bucket folder. When the bucket policy has
"Principal": {
"AWS": "*"
},
The download will work this way,
but on changing the principle to
"Principal": {
"AWS": "arn:aws:iam::<account-number>:role/<ec2-role>"
},
the server throws, access denied error with code 403.
My bucket policy is as:
{
"Version": "2012-10-17",
"Id": "Policy1566920715903",
"Statement": [
{
"Sid": "Stmt1566920713528",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::<account-number>:role/<ec2-role>"
},
"Action": "s3:*",
"Resource": "arn:aws:s3:::<bucket-name>/*"
}
]
}
The "block all public access" on my bucket is turned off.
The ec2 role that I have attached to my instance has full s3 access.
The uploader of the bucket object I am trying to access is my account itself.
The ACL for bucket owner has all the permissions turned on.
The IAM user running the ec2 instance also has s3 full access.
I have read various questions here on stack overflow including :
AWS bucket policy- permission denied
AWS S3 Bucket Access from EC2
Grant EC2 instance access to S3 Bucket
also,
https://aws.amazon.com/premiumsupport/knowledge-center/s3-troubleshoot-403/
https://aws.amazon.com/blogs/security/iam-policies-and-bucket-policies-and-acls-oh-my-controlling-access-to-s3-resources/
and checked on countless AWS documentation, but nothing seems to work.
Error on the server:
message: 'Access Denied',
code: 'AccessDenied',
region: null,
time: 2019-08-29T11:14:30.083Z,
requestId: '98B0580B0D2A00F3',
extendedRequestId:'s5K0fPsew96Mf8c2d3R8xj0M85ICY/gNL5wu0ZthpTwO1jgLAccfVee/J7QXZDSXXLmXioVNQwE=',
cfId: undefined,
statusCode: 403,
retryable: false,
retryDelay: 89.70255005732741 }
The code to download from s3 bucket:
module.exports = function(app) {
var myParser = require('body-parser');
app.use(myParser.urlencoded({extended: true}));
app.post('/downloadZip', function(request, response) {
const s3Zip = require('s3-zip');
const AWS = require('aws-sdk');
const region = 'us-east-1';
const bucket = '<bucket-name';
const filename = new Date().toISOString().replace(/:/g,'-').replace(/T|Z/g,'_') + '_recordings_dump.zip';
const folder =request.body.userId + '/';
const fileArray = JSON.parse(request.body.files);
console.log('Request body=', request.body);
response.set('content-type', 'application/zip');
response.set('Content-Disposition', 'attachment; filename=' + filename);
s3Zip.archive({region: region, bucket: bucket, preserveFolderStructure: true, debug: true },folder, fileArray).pipe(response);
});
};
I dont want to keep the bucket items have public access.

AWS: Access Denied when trying to load presigned url (direct fileupload browser)

I'm attempting to use a presigned URL but I keep getting a 403 Forbidden Access Denied despite setting up everything as I believe that I'm supposed to. I want to upload a file directly from the browser to Amazon S3.
I'm first of all enabling the root AWS account to use putObject. I don't have any additional accounts - I just want it to work for my root account to begin with. Here is the bucket policy:
{
"Version": "2012-10-17",
"Id": "XXXX",
"Statement": [
{
"Sid": "XXXXX",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::XXXX:root"
},
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::XXXXX/*"
}
]
}
This is my Node.js backend. Here I just generate the url and send it to the frontend. Some code for the backend:
const aws = require('aws-sdk');
aws.config.update({
region: "eu-north-1",
accessKeyId: "XXX",
secretAccessKey: "YYY"
});
const s3 = new aws.S3({ apiVersion: "2006-03-01" });
app.get('/geturl', (req,res) => {
const s3Params = {
Bucket: 'XXXXXXXXXXXXX',
Key: req.query.filename,
Expires: 500,
ContentType: req.query.type,
ACL: "public-read"
};
s3.getSignedUrl("putObject", s3Params, (err, data) => {
res.send(data);
});
})
In the frontend, i make a simple call using the URL with the file I wish to upload. When I perform the second fetch call, it will generate the error:
async function handleUpload(e) {
const file = e.target.files[0];
const res = await fetch('http://localhost:3001/geturl');
const url = await res.text();
const resUpload = await fetch(url, { method: 'PUT', body: file });
}
Any ideas what I did wrong?
Edit - Seems like it works if I uncheck the first checkbox - is this a big deal or should this always be blocked in a production env?

In your backend try changing ACL: "public-read" to ACL: "private". You should then be able to block all public access and still be able to successfully complete presigned puts/posts.

How to use react-s3-uploader with reactjs?

I am new to reactjs want to upload image on s3, But don't know how it would work... And don't know where will I get the image path come from aws (in which function)?
Here is my react code
import ApiClient from './ApiClient'; // where it comes from?
function getSignedUrl(file, callback) {
const client = new ApiClient();
const params = {
objectName: file.name,
contentType: file.type
};
client.get('/my/signing/server', { params }) // what's that url?
.then(data => {
callback(data); // what should I get in callback?
})
.catch(error => {
console.error(error);
});
}
My server file Server.js
AWS.config.setPromisesDependency(Bluebird)
AWS.config.update({
accessKeyId: 'accessKey',
secretAccessKey: 'secret',
region: 'ap-south-1'
})
const s3 = new AWS.S3({
params: {
Bucket: 'Bucketname'
},
signatureVersion: 'v4'
})
class S3Service {
static getPreSignedUrl(filename, fileType, acl = 'public-read') {
return new Bluebird(resolve => {
const signedUrlExpireSeconds = 30000
return resolve({
signedRequest : s3.getSignedUrl('putObject', {
Key: "wehab/"+filename,
ContentType:"multipart/form-data",
Bucket: config.get('/aws/bucket'),
Expires: signedUrlExpireSeconds
}),
url : `https://${config.get('/aws/bucket')}.s3.amezonaws.com/wehab/${filename}`
})
})
}
}

First You need to create s3 bucket and attach these policies
if bucket name is 'DROPZONEBUCKET' ( Bucket is globally unique )
Policy
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "PublicReadGetObject",
"Effect": "Allow",
"Principal": "*",
"Action": [
"s3:PutObject",
"s3:GetObject"
],
"Resource": "arn:aws:s3:::DROPZONEBUCKET/*"
}
]
}
CORS config
<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<CORSRule>
<AllowedOrigin>*</AllowedOrigin>
<AllowedMethod>GET</AllowedMethod>
<AllowedMethod>PUT</AllowedMethod>
<MaxAgeSeconds>9000</MaxAgeSeconds>
<AllowedHeader>*</AllowedHeader>
</CORSRule>
</CORSConfiguration>
Then run your node.js server and try to upload the file.
Once you select upload file it will call this getSignedUrl(file, callback) function and it returns url.
Once you successfully get this URL you can upload the file.
Then file path is
https://s3.amazonaws.com/{BUCKET_NAME}/{FILE_NAME}
ex.
https://s3.amazonaws.com/DROPZONEBUCKET/profile.jpeg
Modify API like this
var s3Bucket = new AWS.S3();
var s3 = new Router({ mergeParams: true });
var params = {
Bucket: 'BUCKET_NAME', // add your s3 bucket name here
Key: data.filename,
Expires: 60,
ContentType: data.filetype
};
s3Bucket.getSignedUrl('putObject', params, function (err, data) {
if (err) console.log(err, err.stack); // an error occurred
else console.log(data); // successful response
});

How do I delete an object on AWS S3 using JavaScript?

I want to delete a file from Amazon S3 using JavaScript. I have already uploaded the file using JavaScript. How can I delete it?

You can use the JS method from S3:
var AWS = require('aws-sdk');
AWS.config.loadFromPath('./credentials-ehl.json');
var s3 = new AWS.S3();
var params = { Bucket: 'your bucket', Key: 'your object' };
s3.deleteObject(params, function(err, data) {
if (err) console.log(err, err.stack); // error
else console.log(); // deleted
});
Be aware that S3 never returns the object if it has been deleted.
You have to check it before or after with getobject, headobject, waitfor, etc

You can use construction like this:
var params = {
Bucket: 'yourBucketName',
Key: 'fileName'
/*
where value for 'Key' equals 'pathName1/pathName2/.../pathNameN/fileName.ext'
- full path name to your file without '/' at the beginning
*/
};
s3.deleteObject(params, function(err, data) {
if (err) console.log(err, err.stack); // an error occurred
else console.log(data); // successful response
});
And don't forget to wrap it to the Promise.

Before deleting the file you have to check the 1) file whether it is in the bucket because If the file is not available in the bucket and using deleteObject API this doesn't throw any error 2)CORS Configuration of the bucket. By using headObject API gives the file status in the bucket.
AWS.config.update({
accessKeyId: "*****",
secretAccessKey: "****",
region: region,
version: "****"
});
const s3 = new AWS.S3();
const params = {
Bucket: s3BucketName,
Key: "filename" //if any sub folder-> path/of/the/folder.ext
}
try {
await s3.headObject(params).promise()
console.log("File Found in S3")
try {
await s3.deleteObject(params).promise()
console.log("file deleted Successfully")
}
catch (err) {
console.log("ERROR in file Deleting : " + JSON.stringify(err))
}
} catch (err) {
console.log("File not Found ERROR : " + err.code)
}
As params are constant, the best way to use it with const. If the file is not found in the s3 it throws the error NotFound : null.
If you want to apply any operations in the bucket, you have to change the permissions of CORS Configuration in the respective bucket in the AWS. For changing permissions Bucket->permission->CORS Configuration and Add this code.
<CORSConfiguration>
<CORSRule>
<AllowedOrigin>*</AllowedOrigin>
<AllowedMethod>PUT</AllowedMethod>
<AllowedMethod>POST</AllowedMethod>
<AllowedMethod>DELETE</AllowedMethod>
<AllowedMethod>GET</AllowedMethod>
<AllowedMethod>HEAD</AllowedMethod>
<AllowedHeader>*</AllowedHeader>
</CORSRule>
</CORSConfiguration>
for more information about CROS Configuration : https://docs.aws.amazon.com/AmazonS3/latest/dev/cors.html

You can use deleteObjects API to delete multiple objects at once instead of calling API for each key to delete. Helps save time and network bandwidth.
You can do following-
var deleteParam = {
Bucket: 'bucket-name',
Delete: {
Objects: [
{Key: 'a.txt'},
{Key: 'b.txt'},
{Key: 'c.txt'}
]
}
};
s3.deleteObjects(deleteParam, function(err, data) {
if (err) console.log(err, err.stack);
else console.log('delete', data);
});
For reference see - https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html#deleteObjects-property

You can follow this GitHub gist link https://gist.github.com/jeonghwan-kim/9597478.
delete-aws-s3.js:
var aws = require('aws-sdk');
var BUCKET = 'node-sdk-sample-7271';
aws.config.loadFromPath(require('path').join(__dirname, './aws-config.json'));
var s3 = new aws.S3();
var params = {
Bucket: 'node-sdk-sample-7271',
Delete: { // required
Objects: [ // required
{
Key: 'foo.jpg' // required
},
{
Key: 'sample-image--10.jpg'
}
],
},
};
s3.deleteObjects(params, function(err, data) {
if (err) console.log(err, err.stack); // an error occurred
else console.log(data); // successful response
});

Very straight forward
At first, create an instance of s3 and configure it with credentials
const S3 = require('aws-sdk').S3;
const s3 = new S3({
accessKeyId: process.env.AWS_ACCESS_KEY_ID,
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
region: process.env.AWS_REGION
});
Afterward, follow the docs
var params = {
Bucket: "ExampleBucket",
Key: "HappyFace.jpg"
};
s3.deleteObject(params, function(err, data) {
if (err) console.log(err, err.stack); // an error occurred
else console.log(data); // successful response
/*
data = {
}
*/
});