I sign the URL on my server and send it back to the client which works fine. This is how that function looks
const aws = require('aws-sdk'),
config = require('config'),
crypto = require('crypto');
module.exports = async function(file_type) {
aws.config.update({accessKeyId: config.AWS_ACCESS_KEY, secretAccessKey: config.AWS_SECRET_KEY})
const s3 = new aws.S3();
try {
if (!file_type === "image/png") {
return ({success: false, error: 'Please provide a valid video format'});
}
let buffer = await crypto.randomBytes(12);
let key = buffer.toString('hex');
let options = {
Bucket: config.AWS_S3_BUCKET,
Key: key,
Expires: 60,
ContentType: file_type,
ACL: 'public-read',
}
let data = await s3.getSignedUrl('putObject', options);
console.log('data was', data)
return ({
success: true,
signed_request: data,
url: ('https://s3.amazonaws.com/' + config.AWS_S3_BUCKET + '/' + key),
key,
});
} catch (error) {
console.log('the error was', error)
return ({
success: false,
error: error.message,
})
}
}
So this works fine and winds up getting me a url like
https://mybucket.s3.amazonaws.com/a33b4a43f23fc41de9ddck1k?AWSAccessKeyId=ADIFJDGPMRFRGLXSYWPQ&Content-Type=image%2Fpng&Expires=1496716543&Signature=0zcx%2BFzWUoeFD02RF2CQ2o0bLmo%3D&x-amz-acl=public-read
Then when I get that url back on the client.. i send a PUT request using axios with a function like -
function uploadToS3(file, signedRequest, callback){
var options = {
headers: {
'Content-Type': file.type
}
};
axios.put(signedRequest, file, options)
.then(result =>{
console.log('the result was', result)
callback(result)
})
.catch(err =>{
callback(err)
})
}
The only I'm getting back is (400) Bad Request
I faced the same issue and after searching for hours, I was able to solve it by adding the region of my bucket to the server side backend where I was requesting a signed URL using s3.getSignedUrl().
const s3 = new AWS.S3({
accessKeyId:"your accessKeyId",
secretAccessKey:"your secret access key",
region:"ap-south-1" // could be different in your case
})
const key = `${req.user.id}/${uuid()}.jpeg`
s3.getSignedUrl('putObject',{
Bucket:'your bucket name',
ContentType:'image/jpeg',
Key:key
}, (e,url)=>{
res.send({key,url})
})
After getting the signed URL, I used axios.put() at the client side to upload the image to my s3 bucket using the URL.
const uploadConf = await axios.get('/api/uploadFile');
await axios.put(uploadConf.data.url,file, {
headers:{
'Content-Type': file.type
}
});
Hope this solves your issue.
Guess bad header you provided
Works for me
function upload(file, signedRequest, done) {
const xhr = new XMLHttpRequest();
xhr.open('PUT', signedRequest);
xhr.setRequestHeader('x-amz-acl', 'public-read');
xhr.onload = () => {
if (xhr.status === 200) {
done();
}
};
xhr.send(file);
}
Related
I am running into an issue where when I want to upload an image to s3 bucket nothing goes through.
Basically the only message I get is
API resolved without sending a response for /api/upload/uploadPhoto, this may result in stalled requests.
In the front end, I have an input which can take multiple files ( mainly images ) and then those are stored in event.target.files.
I have a function that stores each file in a state array, and with the button submit it sends a post request to my next.js API.
Here's the logic on the front end:
This function handles the photos, so whenever I add a photo it will automatically add it to the listingPhotos state:
const handleListingPhotos = async (e: any) => {
setMessage(null);
let file = e.target.files;
console.log("hello", file);
for (let i = 0; i < file.length; i++) {
const fileType = file[i]["type"];
const validImageTypes = ["image/jpeg", "image/png"];
if (validImageTypes.includes(fileType)) {
setListingPhotos((prev: any) => {
return [...prev, file[i]];
});
} else {
setMessage("Only images are accepted");
}
}
};
Once the photos are stored in the state, I am able to see the data of the files in the browsers console.log. I run the onSubmit to call the POST API:
const handleSubmit = async (e: any) => {
e.preventDefault();
const formData = new FormData();
formData.append("files[]", listingPhotos);
await fetch(`/api/upload/uploadPhoto`, {
method: "POST",
headers: { "Content-Type": "multipart/form-data" },
body: formData,
}).then((res) => res.json());
};
console.log("listingphotos:", listingPhotos);
Which then uses this logic to upload to the S3 Bucket, but the issue is that when I log req.body I am getting this type of information:
req.body ------WebKitFormBoundarydsKofVokaJRIbco1
Content-Disposition: form-data; name="files[]"
[object File][object File][object File][object File]
------WebKitFormBoundarydsKofVokaJRIbco1--
api/upload/UploadPhoto logic:
import { NextApiRequest, NextApiResponse } from "next";
const AWS = require("aws-sdk");
const access = {
accessKeyId: process.env.AWS_ACCESS_KEY_ID as string,
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY as string,
};
// creates an S3 Client
const s3 = new AWS.S3({ region: "region", credentials: access });
export default async function uploadPhoto(
req: NextApiRequest,
res: NextApiResponse
) {
// take info from parent page
// console.log("req.body: ", req.body);
if (req.method === "POST") {
console.log("req.body", req.body);
let body = req.body;
let headers = req.headers;
let contentType = headers["Content-Type"] || headers["content-type"];
// check for correct content-type
if (!contentType.startsWith("multipart/form-data")) {
return { statusCode: 400, body: "Invalid content type" };
}
let boundary = contentType.replace("multipart/form-data; boundary=", "");
let parts = body.split(boundary);
for (let part of parts) {
if (part.startsWith("Content-Disposition")) {
let [fileData] = part.split("\r\n\r\n");
fileData = fileData.slice(0, -2);
let [fileName] = part.split("filename=");
fileName = fileName.slice(1, -1);
let params = {
Bucket: "RANDOM BUCKET NAME",
Key: fileName,
Body: fileData,
ContentType: { "image/png": "image/jpg" },
};
// Need to set the PARAMS for the upload
await s3.putObject(params);
console.log(
"Successfully uploaded object: " + params.Bucket + "/" + params.Key
);
}
}
return {
statusCode: 200,
body: "File uploaded",
};
// Uploads the files to S3
}
}
I was able to find a way to read if the files were correctly displayed.
req.body {
fileName: 'b699417375e46286e5a30fc252b9b5eb.png',
fileType: 'image/png'
}
POST request code was changed to the followng:
const s3Promises = Array.from(listingPhotos).map(async (file) => {
const signedUrlRes = await fetch(`/api/upload/uploadPhoto`, {
method: "POST",
body: JSON.stringify({
fileName: file.name,
fileType: file.type,
}),
headers: { "Content-Type": "application/json" },
});
Obviously, this is not the solution but it's part of it. The only problem I am running into right now is handling CORS in order to see if the files are sent to the bucket.
when I'm trying to call the request in front node, I'm getting error in my backend node " RequestError: Error: Invalid URI "undefined"" , it seems like backend node request is not getting the data form my frontend node request.
knowing that uploadLink already have a value and in my browser console the frontend request looks ok
my backend request code
const ThumbnailUpload = async (req, res) => {
const { Uploadlink } = req.body;
const { selectedFile } = req.body;
const clientServerOptions = {
uri: `${Uploadlink}`,
body: JSON.stringify({
name: selectedFile,
}),
method: 'PUT',
headers: {
'Content-Type': ' application/json',
Accept: 'application/vnd.vimeo.*+json;version=3.4',
Authorization: getVimeoAuthorization(),
},
};
request(clientServerOptions, function (error, response) {
if (error) {
res.send(error);
} else {
const body = JSON.parse(response.body);
res.send(body);
}
console.log(Uploadlink);
});
};
and my frontend code is
const handleSubmit = (event) => {
event.preventDefault();
const formData = new FormData();
formData.append(
'selectedFile',
new Blob([selectedFile], { type: 'image/jpg, image/png, or image/gif' }),
);
formData.append('uploadLink', uploadLink);
const headers = {
'Content-Type': 'image/jpg, image/png, or image/gif',
Accept: 'application/vnd.vimeo.*+json;version=3.4',
};
try {
axios
.post(`${backendPostPath}/thumbnail-upload`, formData, {
headers,
})
.then((response) => {
applyThumbnial();
console.log(response);
});
} catch (error) {
console.log(error);
}
};
any advise ?
change:
const { Uploadlink } = req.body;
to:
const { uploadlink } = req.body;
make variable consistent throughout the code
EDIT
also, since you're uploading a file, you need to use upload middleware before request handler, and file will be within req.file:
route.post('/thumbnail-upload', upload.single('selectedFile'), ThumbnailUpload);
//... handler..
const selectedFile = req.file;
presently I am attempting to make 2 different api calls one after the other within 1 java/nodejs script. It seems after my first function runs successfully, the second one errors with the following:
FetchError: request to failed, reason: socket hang up;
type: 'system',
errno: 'ECONNRESET',
code: 'ECONNRESET'
Below is a code snippet of what I have tried thus far:
const fetch = require("node-fetch");
const formData = require("form-data");
const fs = require("fs");
//const express = require("express");
var apiName = "<LOCAL SYSTEM FILE TO UPLOAD>";
var lookupName = "<LOCAL SYSTEM FILE TO UPLOAD>";
var accessToken = "Bearer <ACCESS TOKEN>";
var url = '<URL API #1>';
var url2 = '<URL API #2>;
var headers = {
'Accept': 'application/json',
'Authorization': accessToken,
};
const form = new formData();
const buffer2 = fs.readFileSync(lookupName);
const buffer = fs.readFileSync(apiName);
const uploadAPI = function uploadAPI() {
form.append("Content-Type", "application/octect-stream");
form.append('file', buffer);
fetch(url, {method: 'POST', headers: headers, body: form})
.then(data => {
console.log(data)
})
.catch(err => {
console.log(err)
});
};
const uploadLookup = function uploadLookup() {
form.append("Content-Type", "application/octect-stream");
form.append('file', buffer2);
fetch(url2, {method: 'PUT', headers: headers, body: form})
.then(data => {
console.log(data)
})
.catch(err => {
console.log(err)
});
};
if (!apiName !== true) {
uploadAPI()
} else {}
if (!lookupName !== true) {
console.log("Uploading Lookup File");
uploadLookup()
} else {}
I tried using a "setTimeout" function which does not seem to work as I would have liked it to. My best guess is each API call would need to be it's own separate socket connection? Any help with getting me in the right direction is appreciated.
Promise.all([
fetch(url),
fetch(url2)
]).then(function (res) {
// Get a JSON object from each of the responses
return res.json();
}).then(function (data) {
// do something with both sets of data here
console.log(data);
}).catch(function (error) {
// if there's an error, log it
});
I'm using node.js and would like to put on aws of the images I receive in base64. Everything goes well and loads them but when I open them it gives me an error, tells me that I am in the wrong form. And how do I get the image link to store it in the database?
The base64 form:
data:image/jpeg;base64,/9j/4QAYRXhpZgAASUkqAAgAAAAAAAAAAAAAAP/......
The function that uploads the image:
const s3 = new aws.S3({ params: { Bucket: process.env.S3_BUCKET } });
let data = this.createData(req.body.image);
s3.putObject(data, (err, response) => {
if (err) {
console.log(err);
}
else {
console.log(response)
/*tmp = task
.update(req.body)
.then(() => res.status(200).send(JSON.stringify(task.id_creator)))
.catch(error => res.status(400).send(error));*/
}
})
createData(image) {
//TODO NOME CARTELLA
let data = {
Key: 'test1',
Body: image,
ContentEncoding: 'base64',
ContentType: 'image/jpeg'
};
return data;
}
Everything goes well apparently, the response is:
{ ETag: '"20eaa681c71825d8f57472eb378be651"',
VersionId: 'kjQCDdfoq5H0Clhbs79SU4JiIUq8BgOn' }
But when I go in s3 console in my bucket if i download the image gives me an error ('format is wrong')
I figure out a solution:
I just added
let buf = new Buffer(req.body.image.replace(/^data:image\/\w+;base64,/, ""),'base64');
And sent buf as data.
And I added an ACL parameter:
createData(image) {
//TODO NOME CARTELLA
let data = {
Key: 'test1',
Body: image,
ContentEncoding: 'base64',
ContentType: 'image/jpeg',
ACL: 'public-read'
};
return data;
}
I'm trying to upload an image to slack using the slack api https://api.slack.com/methods/files.upload.
function uploadImage(file) {
var form = new FormData();
form.append('image', file);
var self = this;
const config = {
headers: {
'content-type': 'multipart/form-data'
}
}
var token = 'myToken'
axios.post('https://slack.com/api/files.upload?token=' + token, form, config)
.then(function (response) {
console.log(response);
self.imageUploaded("Image uploaded")
})
.catch(function (error) {
console.log(error);
self.error(error);
})
}
I'm getting response with "invalid form data". Any idea what might be going wrong?
Thanks and best regards!
PS: I was able to post an image to slack with python
def post_image(filename, token, channels):
f = {'file': (filename, open(filename, 'rb'))}
response = requests.post(url='https://slack.com/api/files.upload',
data={'token': token, 'channels': channels},
headers={'Accept': 'application/json'},
files=f)
return response.tex
Just need to make the same request with axioms
you can upload a file using axios with the form-data lib
const fs = require("fs");
const axios = require("axios");
const FormData = require("form-data");
const form = new FormData();
form.append("token", 'token here');
form.append("channels", channelId);
form.append("file", fs.createReadStream(__dirname + "/aapl.png"), 'file name');
try {
const res = await axios.post("https://slack.com/api/files.upload", form, {
headers: form.getHeaders(),
});
} catch (err) {
throw new Error(err);
}