We Keep Coding

How to read files after sending a post request in the req.body

I am running into an issue where when I want to upload an image to s3 bucket nothing goes through.
Basically the only message I get is
API resolved without sending a response for /api/upload/uploadPhoto, this may result in stalled requests.
In the front end, I have an input which can take multiple files ( mainly images ) and then those are stored in event.target.files.
I have a function that stores each file in a state array, and with the button submit it sends a post request to my next.js API.
Here's the logic on the front end:
This function handles the photos, so whenever I add a photo it will automatically add it to the listingPhotos state:
const handleListingPhotos = async (e: any) => {
setMessage(null);
let file = e.target.files;
console.log("hello", file);
for (let i = 0; i < file.length; i++) {
const fileType = file[i]["type"];
const validImageTypes = ["image/jpeg", "image/png"];
if (validImageTypes.includes(fileType)) {
setListingPhotos((prev: any) => {
return [...prev, file[i]];
});
} else {
setMessage("Only images are accepted");
}
}
};
Once the photos are stored in the state, I am able to see the data of the files in the browsers console.log. I run the onSubmit to call the POST API:
const handleSubmit = async (e: any) => {
e.preventDefault();
const formData = new FormData();
formData.append("files[]", listingPhotos);
await fetch(`/api/upload/uploadPhoto`, {
method: "POST",
headers: { "Content-Type": "multipart/form-data" },
body: formData,
}).then((res) => res.json());
};
console.log("listingphotos:", listingPhotos);
Which then uses this logic to upload to the S3 Bucket, but the issue is that when I log req.body I am getting this type of information:
req.body ------WebKitFormBoundarydsKofVokaJRIbco1
Content-Disposition: form-data; name="files[]"
[object File][object File][object File][object File]
------WebKitFormBoundarydsKofVokaJRIbco1--
api/upload/UploadPhoto logic:
import { NextApiRequest, NextApiResponse } from "next";
const AWS = require("aws-sdk");
const access = {
accessKeyId: process.env.AWS_ACCESS_KEY_ID as string,
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY as string,
};
// creates an S3 Client
const s3 = new AWS.S3({ region: "region", credentials: access });
export default async function uploadPhoto(
req: NextApiRequest,
res: NextApiResponse
) {
// take info from parent page
// console.log("req.body: ", req.body);
if (req.method === "POST") {
console.log("req.body", req.body);
let body = req.body;
let headers = req.headers;
let contentType = headers["Content-Type"] || headers["content-type"];
// check for correct content-type
if (!contentType.startsWith("multipart/form-data")) {
return { statusCode: 400, body: "Invalid content type" };
}
let boundary = contentType.replace("multipart/form-data; boundary=", "");
let parts = body.split(boundary);
for (let part of parts) {
if (part.startsWith("Content-Disposition")) {
let [fileData] = part.split("\r\n\r\n");
fileData = fileData.slice(0, -2);
let [fileName] = part.split("filename=");
fileName = fileName.slice(1, -1);
let params = {
Bucket: "RANDOM BUCKET NAME",
Key: fileName,
Body: fileData,
ContentType: { "image/png": "image/jpg" },
};
// Need to set the PARAMS for the upload
await s3.putObject(params);
console.log(
"Successfully uploaded object: " + params.Bucket + "/" + params.Key
);
}
}
return {
statusCode: 200,
body: "File uploaded",
};
// Uploads the files to S3
}
}

I was able to find a way to read if the files were correctly displayed.
req.body {
fileName: 'b699417375e46286e5a30fc252b9b5eb.png',
fileType: 'image/png'
}
POST request code was changed to the followng:
const s3Promises = Array.from(listingPhotos).map(async (file) => {
const signedUrlRes = await fetch(`/api/upload/uploadPhoto`, {
method: "POST",
body: JSON.stringify({
fileName: file.name,
fileType: file.type,
}),
headers: { "Content-Type": "application/json" },
});
Obviously, this is not the solution but it's part of it. The only problem I am running into right now is handling CORS in order to see if the files are sent to the bucket.

GLB file type changes when trying to dwonload it from s3 through a signedURL from my s3 bucket

I am using s3 for storing/accessing my files ".glb" files, when the file gets uploaded it is uploaded as a ".glb" file type, but when I am trying to access it through the signedURL it is coming as a "file" file type
exports.uploadGLB = async (req, res) => {
const file = req.file;
//console.log("skslsk", file);
let { _id } = req.params;
//console.log("ID", _id);
const imageName = generateFileName();
const result = await uploadFile(file, imageName, file.mimetype)
await unlinkFile(file.path);
console.log(">>>>>>>>>>>>>>>",result)
User.findOneAndUpdate(
{ _id: _id },
{ $push: { metaSpaces: result } },
function (error, success) {
if (error) {
console.log(error);
} else {
console.log(success);
}
});
res.status(201).json({
message: "Uploaded",
});
};
The above code snippet is to upload and to generate the signedURL from s3-config.js file.
The below mentioned code is the controller of that api responsible for uploading the file
exports.uploadFile = async (file, name, mimetype) => {
//console.log("Uploadingg,,,,,,")
//console.log("paramsss", fileBuffer, fileName, bucketName)
const uploadParams = {
Bucket: bucketName,
Body: fs.createReadStream(file.path),
Key: name,
ContentType: mimetype,
};
const signedUrlExpireSeconds = 60 * 5;
let url = s3Client.getSignedUrl("getObject", {
Bucket: bucketName,
Key: name,
Expires: signedUrlExpireSeconds,
});
// console.log("url", url);
return new Promise( (resolve, reject) => {
s3Client.upload(uploadParams,(err,data)=>{
if(data){
let res = {url:url}
resolve(Object.assign(res,data))
}else
{
reject(err)
}
})
});
};
This is the log result after the file is successfully uploaded in the S3, where you can see that it is clearly the mimetype.
{
fieldname: 'glb',
originalname: 'Tiger.glb',
encoding: '7bit',
mimetype: 'model/gltf-binary',
destination: 'files/',
filename: '534e864015afbad9724e6686e19c481b',
path: 'files\534e864015afbad9724e6686e19c481b',
size: 9201964
} f998b0a3efe7b9f0387580c9bc3dea0fc36e754839f029522cc46631c4fedc8f model/gltf-binary
but when I am trying to access the same file using the generated signedURL the type of the file is changing to only "file", it is showing when I am checking its properties.
Screenshot of the property of the file, which is showing the file type
but when I am following exact same steps for any image file like png, jpeg, etc it is working fine without any issue, I am able to access the exact file which one is uploaded.
any solution for the above problem will be appreciated, really hoping for a solution. Thank you in advance.

How to correctly send an image to the backend (nodejs) using multer in React Native

I'm having trouble sending a photo, in a PUT route, be it with Axios or Fetch, anyway, my Nodejs backend is configured and the whole upload process works normal testing by Insonmia, but in React Native It does not work properly.
BACKEND - CONTROLLER FOR UPDATING USER DATA
async updateUserData(req: Request, res: Response, next: NextFunction) {
const { id } = req.params;
const { username, status } = req.body as IBodyData;
try {
const userExists = await knex('tb_user')
.where('id', Number(id))
.first();
if (!userExists) {
return res.status(400).json({ error: 'User does not exist.' });
}
const serializedUserInfo = {
photo: `${process.env.BASE_URL}/uploads/${req.file.filename}`,
username,
status,
};
const updateInfo = await knex('tb_user')
.update(serializedUserInfo)
.where('id', Number(id));
if (!updateInfo) {
return res.status(400).json({ error: 'Error updating data.' });
}
return res.json(updateInfo);
} catch(err) {
return res.status(500).json({ error: 'Error on updating user.' });
}
}
BACKEND - ROUTE
routes.put(
'/updateuser/:id',
multer(multerConfig).single('userphoto'),
UserController.updateUserData
);
CONFIG MULTER
export default {
fileFilter: (req: Request, file, cb) => {
const allowedMimes = [
'image/jpeg',
'image/jpg',
'image/pjpeg',
'image/png',
];
if (!allowedMimes.includes(file.mimetype)) {
return cb(new Error('Invalid file type.'));
}
return cb(null, true);
},
limits: {
fileSize: 2 * 1024 * 1024,
},
storage: multer.diskStorage({
destination: resolve(__dirname, '..', '..', 'uploads'),
filename: (req: Request, file, cb) => {
const filename = `${randomBytes(6).toString('hex')}-${file.originalname}`;
return cb(null, filename);
}
})
} as Options;
REACT NATIVE - FUNCTION FOR GET DATA OF PHOTO AND SET ON STATE
function handleUploadImage(image: IImagePickerResponse) {
if (image.error) {
return;
}
if (image.didCancel) {
return;
}
if (!image.uri) {
return;
}
setSelectedImage({
fileName: image.fileName,
fileSize: image.fileSize,
type: image.type,
uri: image.uri,
});
}
REACT NATIVE - FUNCTION FOR SEND DATA IN API
async function handleSaveChange() {
try {
const formData = new FormData();
formData.append('userphoto', {
uri:
Platform.OS === 'android'
? selectedImage?.uri
: selectedImage?.uri?.replace('file://', ''),
type: selectedImage.type,
name: selectedImage.fileName,
});
formData.append('username', usernameInput);
formData.append('status', statusInput);
console.log(formData);
await api.put(`/updateuser/${userData?.id}`, formData, {
headers: {
'Content-Type': 'multipart/form-data',
},
});
// const updateUser = await fetch('http://192.168.2.8:3333/updateuser/1', {
// method: 'POST',
// headers: {
// 'Content-Type': 'multipart/form-data',
// },
// body: formData,
// })
// .then((response) => {
// response.json();
// })
// .then((response) => {
// console.log(response);
// })
// .catch((err) => console.log(err));
return Toast.success('Informações alteradas com sucesso.');
} catch (err) {
const {error} = err.response.data;
return Toast.error(error);
}
}
obs: A note to take into account is that in the formData.append ('userphoto', value) part if the value has sent a normal json object, the request is neither made nor the Network Error error even IP address being correct the port too, anyway, and if I use formData.append ('userphoto', JSON.stringfy(value)) the request is made normally but in the backend the photo arrives as undefined and the rest of fields are sent and received normal.
I really tried several things and I couldn't, I changed the type of the PUT method to POST without success, add 'Accept': 'application/json' without success, as I said, the normal sending by Insomnia is working.
Finally, here is the lib configuration I used to get the photo:
REACT NATIVE IMAGE PICKER
ImagePicker.showImagePicker(
{
title: 'Selecione uma imagem',
takePhotoButtonTitle: 'Tirar foto',
chooseFromLibraryButtonTitle: 'Escolher da galeria',
cancelButtonTitle: 'Cancelar',
noData: true,
// storageOptions: {
// skipBackup: true,
// path: 'images',
// cameraRoll: true,
// waitUntilSaved: true,
// },
},
handleUploadImage,
)
SOLVED
Apparently it was a problem with the version of React Native, I am currently using version 0.62.XX
To resolve, just comment line number 43 in the file:
android/app/src/debug/java/com/mobile/ReactNativeFlipper.java
The code is this:
builder.addNetworkInterceptor(new FlipperOkhttpInterceptor (networkFlipperPlugin));

Corrupted file from aws s3

The file I get from the backend says it's corrupted (pdf) or shows nothing (image/png).
This is how I upload to AWS:
const s3 = new AWS.S3();
try {
await s3.upload({
Bucket: bucket,
Key: filename,
Body: file,
ContentType: mimetype,
ContentDisposition: contentDisposition,
}).promise();
return { success: true, data: null };
} catch(e) {
return { success: false, data: null, message: e.code };
}
This is how I get the object from AWS
try {
const data = await s3.getObject({Bucket: bucket, Key: idAWSFile}).promise()
return { success: true, data }
} catch(e) {
return { success: false, data: null, message: e.code };
}
This is the object I get in the client side.
{
AcceptRanges: "bytes",
Body: <Buffer>,
ContentDisposition: "attachment; filename = "blablabla"",
ContentLength: 23361,
ContentType: "application/pdf",
ETag: <randomnumbers>,
LastModified: <DATE>,
Metadata: {},
}
This is how I'm trying to download it in the client side:
const blob = new Blob([data.Body], { type: data.ContentType });
const url = URL.createObjectURL(blob)
window.open(url)
What have I tried doing:
I tried with file-saver and same result -> corrupted file
I tried using toString(utf-8) on the Body in the s3.getObject function as I saw in other SO answers. But when I do this I get gibberish in the Body on the client side and don't know and haven't found what to do with it.

This change in the way I was getting the blob, in the client side, solved it.
const blob = new Blob([Buffer.from(data.Body, 'binary')], {type: data.ContentType})

getting 400 Bad Request when trying to upload to aws s3 bucket

I sign the URL on my server and send it back to the client which works fine. This is how that function looks
const aws = require('aws-sdk'),
config = require('config'),
crypto = require('crypto');
module.exports = async function(file_type) {
aws.config.update({accessKeyId: config.AWS_ACCESS_KEY, secretAccessKey: config.AWS_SECRET_KEY})
const s3 = new aws.S3();
try {
if (!file_type === "image/png") {
return ({success: false, error: 'Please provide a valid video format'});
}
let buffer = await crypto.randomBytes(12);
let key = buffer.toString('hex');
let options = {
Bucket: config.AWS_S3_BUCKET,
Key: key,
Expires: 60,
ContentType: file_type,
ACL: 'public-read',
}
let data = await s3.getSignedUrl('putObject', options);
console.log('data was', data)
return ({
success: true,
signed_request: data,
url: ('https://s3.amazonaws.com/' + config.AWS_S3_BUCKET + '/' + key),
key,
});
} catch (error) {
console.log('the error was', error)
return ({
success: false,
error: error.message,
})
}
}
So this works fine and winds up getting me a url like
https://mybucket.s3.amazonaws.com/a33b4a43f23fc41de9ddck1k?AWSAccessKeyId=ADIFJDGPMRFRGLXSYWPQ&Content-Type=image%2Fpng&Expires=1496716543&Signature=0zcx%2BFzWUoeFD02RF2CQ2o0bLmo%3D&x-amz-acl=public-read
Then when I get that url back on the client.. i send a PUT request using axios with a function like -
function uploadToS3(file, signedRequest, callback){
var options = {
headers: {
'Content-Type': file.type
}
};
axios.put(signedRequest, file, options)
.then(result =>{
console.log('the result was', result)
callback(result)
})
.catch(err =>{
callback(err)
})
}
The only I'm getting back is (400) Bad Request

I faced the same issue and after searching for hours, I was able to solve it by adding the region of my bucket to the server side backend where I was requesting a signed URL using s3.getSignedUrl().
const s3 = new AWS.S3({
accessKeyId:"your accessKeyId",
secretAccessKey:"your secret access key",
region:"ap-south-1" // could be different in your case
})
const key = `${req.user.id}/${uuid()}.jpeg`
s3.getSignedUrl('putObject',{
Bucket:'your bucket name',
ContentType:'image/jpeg',
Key:key
}, (e,url)=>{
res.send({key,url})
})
After getting the signed URL, I used axios.put() at the client side to upload the image to my s3 bucket using the URL.
const uploadConf = await axios.get('/api/uploadFile');
await axios.put(uploadConf.data.url,file, {
headers:{
'Content-Type': file.type
}
});
Hope this solves your issue.

Guess bad header you provided
Works for me
function upload(file, signedRequest, done) {
const xhr = new XMLHttpRequest();
xhr.open('PUT', signedRequest);
xhr.setRequestHeader('x-amz-acl', 'public-read');
xhr.onload = () => {
if (xhr.status === 200) {
done();
}
};
xhr.send(file);
}