In my ReactJS amplify website I would like to get my S3 bucket items and get the metadata. I tried the following code, and get a blank metadata as a result:
s3.listObjects(params, function(err,data){
if(err){
console.log("There was an error getting bucket contents:", err);
}
else{
var bucketContents = data.Contents;
console.log(JSON.stringify(bucketContents));
this.setState(state => {
return { s3Data: bucketContents };
});
// get meta data
for (var i = 0; i < bucketContents.length; i++) {
var urlParams = { Bucket: bucketName, Key: bucketContents[i].Key };
imageTitles.push(bucketContents[i].Key);
s3.headObject(
{
Bucket : bucketName,
Key: bucketContents[i].Key
},
function(err,data){
if(err){
console.log(err.message);
}
else{
console.log("HEAD RESULTS:");
console.log(JSON.stringify(data));
}
});
I have a s3 bucket with 2 custom tags. Inside the s3 bucket I have added the following CORS policy for those tags:
Using amplify cli I have ran amplify pull to update. However I don't see it in my CORS policy locally on my code.
So trying to be useful, I tried manually adding it in the file. Still doesn't work.
Why is this happening, and how can I populate the metadata?
I can retrieve the metadata after enabling CORS headers as explained here: https://github.com/aws/aws-sdk-js/issues/232
export const get = (filename, level) => {
/*First enable CORS headers for the metadata you want to retrieve. Manual setup:
S3 console > select bucket > Click on Permissions > CORS Configuration > Add x-amz-meta-* keys:
<ExposeHeader>x-amz-meta-audience</ExposeHeader>
<ExposeHeader>x-amz-meta-location</ExposeHeader>
*/
Storage.get(filename, {
level,
download: true
})
.then(res => {
console.log(res);
})
.catch(err => console.log(err));
};
Related
I am using angular to create my application. Am trying to upload a picture to s3 bucket but everytime i try uploading it, this error shows in my console.
Here's my upload.service.ts file code
uploadBarcode(file: any) {
const uniqueId: number = Math.floor(Math.random() * Date.now());
const file_name: string = 'lab_consignments/'+uniqueId + "_" + file.name ;
const contentType = file.type;
console.log(contentType)
const bucket = new S3(
{
accessKeyId: myAccessKey,
secretAccessKey: secretAccessKey,
region: 'Asia Pacific (Mumbai) ap-south-1'
}
);
const params = {
Bucket: 'chc-uploads',
Key: file_name,
Body: file,
ACL: 'public-read',
ContentType: contentType
};
return bucket.upload(params, function (err, data) {
if (err) {
console.log('There was an error uploading your file: ', err);
localStorage.setItem('docDataStatus', 'false');
return false;
}
else {
console.log('Successfully uploaded file from service');
return true;
}
});
}
}
the access key and secret access key are statically typed in my code so don't worry about it. i just changed those to the variable names because of security issues while posting this question to stack overflow.
Any help would be appreciated.
Though this does not answer your initial question, as Marcin said with your AWS credentials, hard-coding them into your code is very bad practice and should be avoided at all costs. For frontend applications, this can be performed by having a simple API endpoint generate you a signed upload URL to the S3 bucket:
https://aws.amazon.com/blogs/developer/generate-presigned-url-modular-aws-sdk-javascript/
To actually answer your question, you are likely seeing the error because the region you are passing is incorrectly formatted.
const bucket = new S3(
{
region: 'ap-south-1'
}
);
https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Config.html#region-property
How to download a file with Node.js from google drive api
I don't need anything special. I only want to download a file from a GoogleDrive, and then save it to a given directory of client.
app.get("/download",function(req,res){
const p38290token = new google.auth.OAuth2(CLIENT_ID, CLIENT_SECRET, REDIRECT_URI);
p38290token.setCredentials({ refresh_token: token.acc });
const p38290Id = google.drive({
version: "v3",
auth: p38290token,
});
var dest = fs.createWriteStream("./test.png");
try {
p38290Id.files.get({
fileId: "1daaxy0ymKbMro-e-JnexmGvM4WzW-3Hn",
alt: "media"
}, { responseType: "stream" },
(err, res) => {
res.data
.on("end", () => {
console.log("Done");
})
.on("error", err => {
console.log("Error", err);
})
.pipe(dest); // i want to sent this file to client who request to "/download"
}
)
} catch (error) {
}
})
I want to do that just someone come to www.xyz.com/download and file will be download automatically
The issue seems to be with this line:
var dest = fs.createWriteStream("./test.png");
You are using a file system command which is meant to interact with files on the server. Your question makes it clear that you wish for express to deliver the contents of the file over to the client making the HTTP request.
For that you can just use the res parameter of the route callback function. You declare it on this line:
app.get("/download",function(req,res){
In your case I'd remove the dest variable completely and simply pipe the file to res like so:
.pipe(dest);
Have a look at this answer as well.
I am using AWS transcription API on Node JS with following code
const tClient = new TranscribeClient({
region: "us-east-1",
credentials: {
accessKeyId: AWS_ID,
secretAccessKey: SECRET,
}
});
const params = {
TranscriptionJobName: "firstjob",
LanguageCode: "en-US", // For example, 'en-US'
MediaFormat: "m4a", // For example, 'wav'
Media: {
MediaFileUri: "https://transcribe-demo.s3-REGION.amazonaws.com/hello_world.m4a",
},
};
const run = async () => {
try {
const data = await tClient.send(
new StartTranscriptionJobCommand(params)
);
console.log("Success - put", data);
return data; // For unit tests.
} catch (err) {
console.log("Error", err);
}
};
run();
But I am getting following error, I have checked all the permissions and access keys are correct . I am unable to understand error reason.
AccessDeniedException: User: arn:aws:iam::494240200407:user/demno_system is not authorized to perform: transcribe:StartTranscriptionJob on resource: arn:aws:transcribe:us-east-1:494240200407:transcription-job/firstjob because no permissions boundary allows the transcribe:StartTranscriptionJob action
Any inputs are appreciated.
As stated in the error message, you have a permission boundary that doesn't allow this action. The fact that you added the required policy to the user has no effect, since the access is limited by the permission boundary. You need to edit the user's permission boundary to allow this.
Refer to the documentation.
I am trying to upload a file from mobile to google bucket using ionic 4. Although a file can upload into the could. I am struggling to get the file properties out of file object.
Here is my method,
async selectAFile() {
const uploadFileDetails = {
name: '',
contentLength: '',
size: '',
type: '',
path: '',
};
this.fileChooser.open().then(uri => {
this.file.resolveLocalFilesystemUrl(uri).then(newUrl => {
let dirPath = newUrl.nativeURL;
const dirPathSegments = dirPath.split('/');
dirPathSegments.pop();
dirPath = dirPathSegments.join('/');
(<any>window).resolveLocalFileSystemURL(
newUrl.nativeURL,
function(fileEntry) {
uploadFileDetails.path = newUrl.nativeURL;
const file: any = getFileFromFileEntry(fileEntry);
//log 01
console.log({ file });
uploadFileDetails.size = file.size;
uploadFileDetails.name = `${newUrl.name
.split(':')
.pop()}.${file.type.split('/').pop()}`;
uploadFileDetails.type = file.type;
async function getFileFromFileEntry(fileEntry) {
try {
return await new Promise((resolve, reject) =>
fileEntry.file(resolve, reject)
);
} catch (err) {
console.log(err);
}
}
},
function(e) {
console.error(e);
}
);
});
});
// here uploadFileDetails is simller to what I declared at the top ;)
// I wan't this to be populated with file properties
// console.log(uploadFileDetails.name) --> //''
const uploadUrl = await this.getUploadUrl(uploadFileDetails);
const response: any = this.uploadFile(
uploadFileDetails,
uploadUrl
);
response
.then(function(success) {
console.log({ success });
this.presentToast('File uploaded successfully.');
this.loadFiles();
})
.catch(function(error) {
console.log({ error });
});
}
even though I can console.log the file in log 01. I am unable to get file properties like, size, name, type out of the resolveLocalFileSystemURL function. basically, I am unable to populate uploadFileDetails object. What am I doing wrong? Thank you in advance.
you actually need 4 Ionic Cordova plugins to upload a file after getting all the metadata of a file.
FileChooser
Opens the file picker on Android for the user to select a file, returns a file URI.
FilePath
This plugin allows you to resolve the native filesystem path for Android content URIs and is based on code in the aFileChooser library.
File
This plugin implements a File API allowing read/write access to files residing on the device.
File Trnafer
This plugin allows you to upload and download files.
getting the file's metadata.
file.resolveLocalFilesystemUrl with fileEntry.file give you all the metadata you need, except the file name. There is a property called name in the metadata but it always contains value content.
To get the human readable file name you need filePath. But remember you can't use returning file path to retrieve metadata. For that, you need the original url from fileChooser.
filePathUrl.substring(filePathUrl.lastIndexOf('/') + 1) is used to get only file name from filePath.
You need nativeURL of the file in order to upload it. Using file path returning from filePath is not going to work.
getFileInfo(): Promise<any> {
return this.fileChooser.open().then(fileURI => {
return this.filePath.resolveNativePath(fileURI).then(filePathUrl => {
return this.file
.resolveLocalFilesystemUrl(fileURI)
.then((fileEntry: any) => {
return new Promise((resolve, reject) => {
fileEntry.file(
meta =>
resolve({
nativeURL: fileEntry.nativeURL,
fileNameFromPath: filePathUrl.substring(filePathUrl.lastIndexOf('/') + 1),
...meta,
}),
error => reject(error)
);
});
});
});
});
}
select a file from the file system of the mobile.
async selectAFile() {
this.getFileInfo()
.then(async fileMeta => {
//get the upload
const uploadUrl = await this.getUploadUrl(fileMeta);
const response: Promise < any > = this.uploadFile(
fileMeta,
uploadUrl
);
response
.then(function(success) {
//upload success message
})
.catch(function(error) {
//upload error message
});
})
.catch(error => {
//something wrong with getting file infomation
});
}
uploading selected file.
This depends on your backend implementation. This is how to use File Transfer to upload a file.
uploadFile(fileMeta, uploadUrl) {
const options: FileUploadOptions = {
fileKey: 'file',
fileName: fileMeta.fileNameFromPath,
headers: {
'Content-Length': fileMeta.size,
'Content-Type': fileMeta.type,
},
httpMethod: 'PUT',
mimeType: fileMeta.type,
};
const fileTransfer: FileTransferObject = this.transfer.create();
return fileTransfer.upload(file.path, uploadUrl, options);
}
hope it helps. :)
I'm trying to upload an image to my AWS S3 bucket after downloading the image from another URL using Node (using request-promise-native & aws-sdk):
'use strict';
const config = require('../../../configs');
const AWS = require('aws-sdk');
const request = require('request-promise-native');
AWS.config.update(config.aws);
let s3 = new AWS.S3();
function uploadFile(req, res) {
function getContentTypeByFile(fileName) {
var rc = 'application/octet-stream';
var fn = fileName.toLowerCase();
if (fn.indexOf('.png') >= 0) rc = 'image/png';
else if (fn.indexOf('.jpg') >= 0) rc = 'image/jpg';
return rc;
}
let body = req.body,
params = {
"ACL": "bucket-owner-full-control",
"Bucket": 'testing-bucket',
"Content-Type": null,
"Key": null, // Name of the file
"Body": null // File body
};
// Grabs the filename from a URL
params.Key = body.url.substring(body.url.lastIndexOf('/') + 1);
// Setting the content type
params.ContentType = getContentTypeByFile(params.Key);
request.get(body.url)
.then(response => {
params.Body = response;
s3.putObject(params, (err, data) => {
if (err) { console.log(`Error uploading to S3 - ${err}`); }
if (data) { console.log("Success - Uploaded to S3: " + data.toString()); }
});
})
.catch(err => { console.log(`Error encountered: ${err}`); });
}
The upload succeeds when I test it out, however after trying to redownload it from my bucket the image is unable to display. Additionally, I notice after uploading the file with my function, the file listed in the bucket is much larger in filesize than the originally uploaded image. I'm trying to figure out where I've been going wrong but cannot find where. Any help is appreciated.
Try to open the faulty file with a text editor, you will see some errors written in it.
You can try using s3.upload instead of putObject, it works better with streams.