We Keep Coding

How to send a file in array of object to express server?

I have an array of the object, each object contains some properties of type string and a file.
This is how my array looks like:
const args = [
{
name: 'presentation',
price: 9.99,
tags: 'invest, finance, business',
file: File,
thumbnail: File
},
{
name: 'presentation2',
price: 20.99,
tags: 'invest, finance, business',
file: File,
thumbnail: File
}
]
const headers = {
headers: {
Authorization: `Bearer ${token}`
}
};
My goal is to send this whole array of the object to the Express Server. there I will save information to the Mongo and upload the file to S3.
But Currently, I am unable to get the file stream on the server when I send this in a POST request,
even the whole req.body is an empty object when I console.log it
axios.post(`${slidesRoute}/createSlides`, args, headers);
Alternatively, I tried to put everything into FormData.
for example:
let formData = new FormData();
selectedFiles.forEach((selectedFile, i) => {
formData.append(`name_${i}`, selectedFile.name);
formData.append(`price_${i}`, selectedFile.price);
formData.append(`tags_${i}`, selectedFile.tags);
formData.append(`file_${i}`, selectedFile.file);
formData.append(`thumbnail_${i}`, selectedFile.thumbnail);
});
axios.post(`${slidesRoute}/createSlides`, formData, headers);
Then In the backend. I am unable to catch these files using multer. Because the filenames in form data are being generated dynamically like
file_1, file_2 file_3,...
But multer expects the exact filename to be passed
for example multer().array('file')
so in my second approach I am unable to catch files using Multer

You can't include file inside JSON. File is not JSON-serializable. Ideally you should have a separate endpoint to accept your files, send them with content-type: multipart/form-data, receive unique id for that file stored on the server, then send your JSON with ids instead of files.
As a chaper alternative you can Base64 encode your files into a string, insert this string in your request, and then decode it back on the receiving side

FormData key values are array by default
in your code Just change
formData.append(`file_${i}`, selectedFile.file);
to
formData.append(`file`, selectedFile.file);
You should be able to receive the file array in server side using multer().array('file')
Sample below demonstrating FormData key values array behavior
const formData = new FormData();
formData.append('key1', 'value1');
formData.append('key1', 'value2');
formData.append('key2', 'value1');
formData.forEach(function(value, key){
console.log(key, value);
});

I use FormData to send a number of files - and then on the node.js server I use Formidable.
I've pasted some partial code below that may help - sorry it's not a smaller sample.
Hope it helps
On the browser side I have:
let form_data = new FormData()
form_data.append('upload_data', file)
$.ajax({
url: '/file/upload' + remote_path + '/' + filename,
type: 'POST',
data: form_data,
processData: false,
contentType: false,
success: (res) => {
...
On the node.js side (apologies for the length..)I have:
const fs = require('fs')
const formidable = require('formidable')
const path = require('path')
...
app.post('/file/upload/*', (req, res) => {
let filename = req.params[0]
let media = path.basename(filename)
let folder = filename.substring(0, filename.length - media.length)
let form = new formidable.IncomingForm()
// form.encoding = 'utf-8'
form.multiples = true
form.uploadDir = path.join(media_folder, folder)
form.keepExtensions = true
form.parse(req, (err, fields, files) => {
if (err) {
fail(res, 'failed to upload')
} else {
success(res)
}
})
form.on('fileBegin', (name, file) => {
const [fileName, fileExt] = file.name.split('.')
file.path = path.join(form.uploadDir, `${fileName}_${new Date().getTime()}.${fileExt}`)
})
form.on('file', (field, file) => {
fs.rename(file.path, path.join(form.uploadDir, file.name), (err) => {
if (!res.headersSent) { // Fix since first response is received
fail(res, 'an error has occured with form upload' + err)
}
})
})
form.on('error', (err) => {
fail(res, 'an error has occured with form upload' + err)
})
form.on('aborted', (err) => {
fail(res, 'Upload cancelled by browser')
})
})

Header section has more than 10240 bytes (maybe it is not properly terminated)

I'm using NodeJs to try to upload an attachment to a Jira Issue via the Jira Rest API.
The api expects multipart/form-data so this is how I'm calling it in Node:
function uploadAttachments(supportFormData, callback) {
const url =
'https://somewhere.com/jira/rest/api/2/issue/' +
supportFormData.issueId +
'/attachments';
var options = {
url: url,
headers: {
Authorization: { user: username, password: password },
'X-Atlassian-Token': 'nocheck'
}
};
var r = request.post(options, function(err, res, body) {
if (err) {
console.error(err);
callback(false);
} else {
console.log('Upload successful! Server responded with:', body);
callback(false);
}
});
var form = r.form();
form.append('file', supportFormData.attachments[0].contents, {
filename: supportFormData.attachments[0].fileName,
contentType: supportFormData.attachments[0].contents
});
}
The error I'm receiving is:
org.apache.commons.fileupload.FileUploadException: Header section
has more than 10240 bytes (maybe it is not properly terminated)
The "supportFormData.attachments[0].contents" is ofType Buffer.
Any suggestions as to what could be causing this error?

I ran into this same issue and it turns out JIRA (or Java) requires \r\n as new line character. After I changed \n to \r\n my requests went through without problem.

If its a basic auth change options object to
let auth = new Buffer(`${username}:${password}`).toString('base64');
var options = {
url: url,
headers: {
Authorization: `Basic ${auth}`,
'X-Atlassian-Token': 'nocheck'
}
};

Can't get uploaded file from react js to express js

I want to upload the file from front end developed in React and get that uploaded file inside Express js.
Below is the code snippet of React side when I am uploading the file :
handleUploadFile(ev) {
ev.preventDefault();
var a6 = "File";
const data = new FormData();
data.append('file', this.uploadInput.files[0]);
data.append('fileName', a6);
fetch('http://localhost:4000/api/addDcuments/upload', {
method: 'POST',
body: {data},
json: true,
headers: { "Authorization": cookie.load('userToken') }
}).then((response) => {
response.json().then((body) => {
this.setState({ imageURL: `http://localhost:4000/${body.file}` });
});
});
}
In above code, I have taken the form and called handleUploadFile function on it's onSubmit event.
Now, below is my backend express js code on which I am getting the uploaded file:
export function uploadDocument(req, res, next) {
console.log(JSON.stringify(req.body));
let imageFile = req.files.file;
var ext = path.extname(imageFile.name)
if(imageFile.mimetype == "application/pdf" ||imageFile.mimetype == "application/octet-stream"){
imageFile.mv('D:/React Workspace/React-videos-example/file_path/'+req.body.filename+ext, function(err) {
if (err) {
return res.status(500).send(err);
}
res.json({file: `public/${req.body.filename}.pdf`});
});
}
}
In above code, when I am trying print the req.body, it is returning "{}" as well as I am getting one error:
TypeError: Cannot read property 'file' of undefined.
So, my backend function has been called, but not able to getting that uploaded file. So can anyone have any solution or any reference link for this issue?

When using formdata, you can't use type json in your POST, it needs to be:
contentType: 'multipart/form-data'
instead of "json: true".

How to convert the string type from an API response to an image file - ����\u0000\u0010JFIF\u0000\u0001\u0001\u0000\u0000\u0001 -

I have used https://graph.microsoft.com/beta/me/photo/$value API to get the profile picture of the outlook user. I get an image on running the above API in the rest-client. The content-type of the API is "image/jpg"
But, in Node.js, the response of the API is as follows:
����\u0000\u0010JFIF\u0000\u0001\u0001\u0000\u0000\u0001\u0000\u0001\u0000\u0000��\u0000�\u0000\u0005\u0005\u0005\u0005\u0005\u0005\u0006\u0006\u0006\u0006\b\t\b\t\b\f\u000b\n\n\u000b\f\u0012\r\u000e\r\u000e\r\u0012\u001b\u0011\u0014\u0011\u0011\u0014\u0011\u001b\u0018\u001d\u0018\u0016\u0018\u001d\u0018+"\u001e\u001e"+2*(*2<66<LHLdd�\u
I used 'fs' to create an image file. The code is as follows:
const options = {
url: "https://graph.microsoft.com/beta/me/photo/$value",
method: 'GET',
headers: {
'Accept': 'application/json',
'Authorization': `Bearer ${locals.access_token}`,
'Content-type': 'image/jpg',
}
};
request(options, (err, res, body) => {
if(err){
reject(err);
}
console.log(res);
const fs = require('fs');
const data = new Buffer(body).toString("base64");
// const data = new Buffer(body);
fs.writeFileSync('profile.jpg', data, (err) => {
if (err) {
console.log("There was an error writing the image")
}
else {
console.log("The file is written successfully");
}
});
});
The file is written successfully, but the .jpg image file generated is broken. I am unable to open the image.
The output of the image file is as follows:
77+977+977+977+9ABBKRklGAAEBAAABAAEAAO+/ve

You can do this by streaming the response like this,
request(options,(err,res,body)=>{
console.log('Done!');
}).pipe(fs.createWriteStream('./profile.jpg'));
https://www.npmjs.com/package/request#streaming
https://nodejs.org/api/fs.html#fs_class_fs_writestream

The reason for this is that by default, request will call .toString() on the response data. In case of binary data, like a RAW JPEG, this isn't what you want.
It's also explained in the request documentation (albeit vaguely):
(Note: if you expect binary data, you should set encoding: null.)
Which means that you can use this as well:
const options = {
encoding : null,
url : "https://graph.microsoft.com/beta/me/photo/$value",
method : 'GET',
headers : {
'Accept' : 'application/json',
'Authorization' : `Bearer ${locals.access_token}`,
'Content-type' : 'image/jpg',
}
};
However, streaming is probably still the better solution, because it won't require the entire response being read into memory first.

Request is deprecated. You can do this with axios;
// GET request for remote image in node.js
axios({
method: 'get',
url: 'http://example.com/file.jpg',
responseType: 'stream'
})
.then(function (response) {
response.data.pipe(fs.createWriteStream('ada_lovelace.jpg'))
});

How do I send a form with a request promise?

I am trying to use the request-promise library or anything similar to send files via a post request from node to another machine that is running Node. Using the normal request module I could so something like
var req = request.post(url, function (err, resp, body) {
if (err) {
console.log('Error!');
} else {
console.log('URL: ' + body);
}
});
var form = req.form();
form.append('file', '<FILE_DATA>', {
filename: 'myfile.txt',
contentType: 'text/plain'
});
This code is from the question:
Uploading file using POST request in Node.js however it is not using promises.
Can anyone explain how to do the same thing but with the request-promise library or if there is any other way to promisify this?

According to the docs that are linked from the answer you already found, you don't need to use a .form() method on the resulting request object, but can simply pass the form as the formData option to request. You'll be able to do the same with request-promise:
requestPromise.post({url: url, formData: {
file: {
value: '<FILE_DATA>',
options: {
filename: 'myfile.txt',
contentType: 'text/plain'
}
}
}).then(function(body) {
console.log('URL: ' + body);
}, function(err) {
console.log('Error!');
});
Alternatively, request-promise still seems to return request instances (just decorated with then/catch/promise methods), so the form function should still be available:
var req = requestPromise.post(url);
var form = req.form();
form.append('file', '<FILE_DATA>', {
filename: 'myfile.txt',
contentType: 'text/plain'
});
req.then(function(body) {
console.log('URL: ' + body);
}, function(err) {
console.log('Error!');
});