I have an array of the object, each object contains some properties of type string and a file.
This is how my array looks like:
const args = [
{
name: 'presentation',
price: 9.99,
tags: 'invest, finance, business',
file: File,
thumbnail: File
},
{
name: 'presentation2',
price: 20.99,
tags: 'invest, finance, business',
file: File,
thumbnail: File
}
]
const headers = {
headers: {
Authorization: `Bearer ${token}`
}
};
My goal is to send this whole array of the object to the Express Server. there I will save information to the Mongo and upload the file to S3.
But Currently, I am unable to get the file stream on the server when I send this in a POST request,
even the whole req.body is an empty object when I console.log it
axios.post(`${slidesRoute}/createSlides`, args, headers);
Alternatively, I tried to put everything into FormData.
for example:
let formData = new FormData();
selectedFiles.forEach((selectedFile, i) => {
formData.append(`name_${i}`, selectedFile.name);
formData.append(`price_${i}`, selectedFile.price);
formData.append(`tags_${i}`, selectedFile.tags);
formData.append(`file_${i}`, selectedFile.file);
formData.append(`thumbnail_${i}`, selectedFile.thumbnail);
});
axios.post(`${slidesRoute}/createSlides`, formData, headers);
Then In the backend. I am unable to catch these files using multer. Because the filenames in form data are being generated dynamically like
file_1, file_2 file_3,...
But multer expects the exact filename to be passed
for example multer().array('file')
so in my second approach I am unable to catch files using Multer
You can't include file inside JSON. File is not JSON-serializable. Ideally you should have a separate endpoint to accept your files, send them with content-type: multipart/form-data, receive unique id for that file stored on the server, then send your JSON with ids instead of files.
As a chaper alternative you can Base64 encode your files into a string, insert this string in your request, and then decode it back on the receiving side
FormData key values are array by default
in your code Just change
formData.append(`file_${i}`, selectedFile.file);
to
formData.append(`file`, selectedFile.file);
You should be able to receive the file array in server side using multer().array('file')
Sample below demonstrating FormData key values array behavior
const formData = new FormData();
formData.append('key1', 'value1');
formData.append('key1', 'value2');
formData.append('key2', 'value1');
formData.forEach(function(value, key){
console.log(key, value);
});
I use FormData to send a number of files - and then on the node.js server I use Formidable.
I've pasted some partial code below that may help - sorry it's not a smaller sample.
Hope it helps
On the browser side I have:
let form_data = new FormData()
form_data.append('upload_data', file)
$.ajax({
url: '/file/upload' + remote_path + '/' + filename,
type: 'POST',
data: form_data,
processData: false,
contentType: false,
success: (res) => {
...
On the node.js side (apologies for the length..)I have:
const fs = require('fs')
const formidable = require('formidable')
const path = require('path')
...
app.post('/file/upload/*', (req, res) => {
let filename = req.params[0]
let media = path.basename(filename)
let folder = filename.substring(0, filename.length - media.length)
let form = new formidable.IncomingForm()
// form.encoding = 'utf-8'
form.multiples = true
form.uploadDir = path.join(media_folder, folder)
form.keepExtensions = true
form.parse(req, (err, fields, files) => {
if (err) {
fail(res, 'failed to upload')
} else {
success(res)
}
})
form.on('fileBegin', (name, file) => {
const [fileName, fileExt] = file.name.split('.')
file.path = path.join(form.uploadDir, `${fileName}_${new Date().getTime()}.${fileExt}`)
})
form.on('file', (field, file) => {
fs.rename(file.path, path.join(form.uploadDir, file.name), (err) => {
if (!res.headersSent) { // Fix since first response is received
fail(res, 'an error has occured with form upload' + err)
}
})
})
form.on('error', (err) => {
fail(res, 'an error has occured with form upload' + err)
})
form.on('aborted', (err) => {
fail(res, 'Upload cancelled by browser')
})
})
Related
I'm wanting to allow users to select a folder(containing a .glb file) from their device, and then upload it to IPFS. The problem is that all documentation that I've seen for using a post method requires that you pass a local directory into the FormData (for example):
const pin = async (pinataApiKey, pinataSecretApiKey) => {
const url = `https://api.pinata.cloud/pinning/pinFileToIPFS`;
const src = "LOCAL/FILE/PATH";
var status = 0;
try {
const { dirs, files } = await rfs.read(src);
let data = new FormData();
for (const file of files) {
data.append(`file`, fs.createReadStream(file), {
filepath: basePathConverter(src, file),
});
}
return axios.post(url, data, {
maxBodyLength: "Infinity",
headers: {
"Content-Type": `multipart/form-data; boundary=${data._boundary}`,
pinata_api_key: pinataApiKey,
pinata_secret_api_key: pinataSecretApiKey,
},
});
} catch (error) {}
};
My question is, how do I get a user-inputted folder, then handle it so that it can be uploaded to Pinata IPFS through a POST method? I'm aware of <input type= "file" webkitdirectory>, but I'm not sure how to extract and "prep" the folder data for posting once a user has uploaded it. Any suggestions are welcome!
I am working on a project (using React.js Express.js and Node.js) to convert a user uploaded image into and NFT on Ethereum blockchain and for that, I need to upload the image on an IPFS (I am using Pinata) and then use the pinata URI in the metadata to mint a new NFT. (Do let me know if I am wrong here, I am still newbie to web3)
For uploading my image onto the Pinata IPFS, I am sending the base64 string of the image from the client side to the server side and then calling the pinFileToIPFS method. This is the code of my server side file
const axios = require('axios');
const fs = require('fs');
const FormData = require('form-data');
const router = require('express').Router();
const { Readable } = require('stream');
const pinFileToIPFS = (image) => {
const url = `https://api.pinata.cloud/pinning/pinJSONToIPFS`;
const buffer = Buffer.from(image);
const stream = Readable.from(buffer);
const filename = `an-awesome-nft_${Date.now()}.png`;
stream.path = filename;
const formData = new FormData();
formData.append("file", stream);
return axios.post(url,
formData,
{
headers: {
'Content-Type': `multipart/form-data; boundary= ${formData._boundary}`,
'pinata_api_key': "*******************",
'pinata_secret_api_key': "**********************************",
}
}
).then(function (response) {
console.log("Success: ", response);
}).catch(function (error) {
console.log("Fail! ", error.response.data);
});
};
router.route('/').post((req, res) => {
const image = req.body.image;
pinFileToIPFS(image);
});
module.exports = router;
Here req.body.image contains the base64 string of the user uploaded file.
I have tried to convert the base64 string into a buffer and then convert the buffer into a readable stream (as done in the official Pianata documentation but for a localy file) and then wrap it up in FormData(), but I keep getting the following error.
data: {
error: 'This API endpoint requires valid JSON, and a JSON content-type'
}
I know the problem is with the format my image/file is being sent to the API but I can't figure out. I am still a newbie to web3 and blockchains so please help!
The recommended way of interacting with Pinata, is by using their Node.JS SDK. This SDK has a pinFileToIPFS function, allows you to upload an image to their IPFS nodes in the form of a readableStream.
A sample of this would look like
const fs = require('fs');
const readableStreamForFile = fs.createReadStream('./yourfile.png');
const options = {
pinataMetadata: {
name: MyCustomName,
keyvalues: {
customKey: 'customValue',
customKey2: 'customValue2'
}
},
pinataOptions: {
cidVersion: 0
}
};
pinata.pinFileToIPFS(readableStreamForFile, options).then((result) => {
//handle results here
console.log(result);
}).catch((err) => {
//handle error here
console.log(err);
});
However, if you are deadset on using their API endpoints and simply posting to them via axios, there is a seperate API endpoint. /pinning/pinFileToIPFS. Examples of this method can be found in their API Docs.
You may want to consider changing the following two lines and using the https://api.pinata.cloud/pinning/pinFileToIPFS endpoint instead:
const buffer = Buffer.from(image); -> const buffer = Buffer.from(image, "base64");
and
formData.append("file", stream); -> formData.append("file", stream, "fileNameOfChoiche.png);
When you are uploading an image or file to Pinata IPFS with node js. These are the steps that even don't need Pinata Node.js SDK.
1- You can upload an image from the front end with React or Next.js. Code is given below.
const uploadAttachment = async (data, token) => {
try {
return await Api.post(`${ApiRoutes.upload_attachment}`, data, {
headers: {
Authorization: "Bearer " + token, //the token is a variable which holds the token
},
});
} catch (error) {
return {
status: 404,
};
}
};
export default uploadAttachment;
2- You need to install multer to upload an image.
const multer = require("multer");
global.uploadSingleFile = multer({ dest: "uploads/" });
3- Set up your route with multer middleware and action which you are going to call.
.post(
"/attachments/upload",
uploadSingleFile.single("file"),
actions.attachments.upload.pinFileToIPFSLocal
);
4- Last step where you will hit the Pinata endpoint with Pinata API & Secret key.
pinFileToIPFSLocal: async (req, res, next) => {
try {
const url = "https://api.pinata.cloud/pinning/pinFileToIPFS";
let formData = new FormData();
formData.append("file", JSON.stringify(req.file), req.file.originalname);
axios
.post(url, formData, {
maxContentLength: -1,
headers: {
"Content-Type": `multipart/form-data; boundary=${formData._boundary}`,
pinata_api_key: process.env.PINATA_API_KEY,
pinata_secret_api_key: process.env.PINATA_API_SECRET,
path: "somename",
},
})
.then((data) => {
console.log("Result...", data);
return utils.response.response(
res,
"Upload image to ipfs.",
true,
200,
data.data
);
})
.catch((err) => {
return utils.response.response(
res,
"Image not upload to ipfs",
false,
200,
err
);
});
} catch (error) {
next(error);
}
The error message is clear. You are using url that used for json file upload. this is the url you should use to upload image
const url = `https://api.pinata.cloud/pinning/pinFileToIPFS`;
you don't have to convert buffer to a readable stream.
I am not sure if this is correct ${formData._boundary}. should be
"Content-Type": `multipart/form-data: boundary=${formData.getBoundary()}`,
There must be an error on the image parameter. A simple buffer representation of the image should work. The readable stream is not necessary.
Instead of creating the buffer, you could use middleware like express-fileupload to access the buffer representation of the file uploaded on the client-side directly.
const file = req.files;
const url = "https://api.pinata.cloud/pinning/pinFileToIPFS";
const data = new FormData();
data.append("file", file.file.data, { filepath: "anyname" });
const result = await axios.post(url, data, {
maxContentLength: -1,
headers: {
"Content-Type": `multipart/form-data; boundary=${data._boundary}`,
pinata_api_key: process.env.PINATA_API_KEY,
pinata_secret_api_key: process.env.PINATA_API_SECRET,
path: "somename",
},
});
I'm trying upload multiple images with axios in React but i cannot figure out what is wrong. First I tried to upload single image and that work just fine. But with multiple images I'm out of options.
I'm creating FormData like so:
for (let i = 0; i < images.length; i++) {
formData.append('productPhotos[' + i + ']', images[i]);
}
The axios request looking like this
const config = { headers: { 'Content-Type': 'multipart/form-data' } };
axios
.post(endPoints.createProduct, formData, config)
.then(res => console.log(res))
.catch(err => console.log(err));
My back-end is written is node/express and I'm using multer for uploading. The signature is look like this:
app.post("/product", upload.array("productPhotos"), (req, res) => {
I tried this back-end end point in PostMan and uploading works for just fine, so the error must be on front-end. Thanks for help.
UPDATE
Right way to pass multiple files in formData:
images.forEach(img => {
formData.append("productPhotos", img)
})
Here is a full working set up (expanded version of the answer above)
Client side:
// silly note but make sure you're constructing files for these (if you're recording audio or video yourself)
// if you send it something other than file it will fail silently with this set-up
let arrayOfYourFiles=[image, audio, video]
// create formData object
const formData = new FormData();
arrayOfYourFiles.forEach(file=>{
formData.append("arrayOfFilesName", file);
});
axios({
method: "POST",
url: serverUrl + "/multiplefiles",
data: formData,
headers: {
"Content-Type": "multipart/form-data"
}
})
//some error handling
Server side (express, node - mutler)
const UPLOAD_FILES_DIR = "./uploads";
const storage = multer.diskStorage({
destination(req, file, cb) {
cb(null, UPLOAD_FILES_DIR);
},
// in case you want to change the names of your files)
filename(req, file = {}, cb) {
file.mimetype = "audio/webm";
// console.log(req)
const {originalname} = file;
const fileExtension = (originalname.match(/\.+[\S]+$/) || [])[0];
cb(null, `${file.fieldname}${Date.now()}${fileExtension}`);
}
});
const upload = multer({storage});
// post route that will be hit by your client (the name of the array has to match)
app.post("/multiplefiles", upload.array('arrayOfFilesName', 5), function (req, res) {
console.log(req.files, 'files')
//logs 3 files that have been sent from the client
}
This is not the right way to generate keys. You can try something like this:
let productimages = [];
for (let i = 0; i < images.length; i++) {
productimages.push(images[i]);
}
formData.append('productPhotos', productimages);
You may want to send the files as an array to the endpoint:
images.forEach( img => {
formData.append('productPhotos[]', img);
})
The way I have found examples are:
const file = fs.readFileSync(filePath)
const formData = new FormData()
formData.append('userId', userId)
formData.append('file', file)
const options = {
method: 'post',
url: 'http://localhost:5000/uploadFile',
headers: {
'content-type': 'multipart/form-data'
},
data: formData
}
await axios(options).then(res => { console.log(res) }).catch(err => { console.log(err) })
But this doesn't attach the file in request.files which is required for python. Since it doesn't attach it as file, the information about file type is also lost.
I have also tried using the following. It does attach the file to request.files but the contents are not of the proper file and all I get is a text string which assumingely is a buffer string output.
const file = new File(fs.readFileSync(filePath), fileName, { type: 'text/csv' })
The aim is to preserve the file type information so that server can save the file properly. What am I missing?
Note that the requests are not sent from nodejs (which has file access) directly.
I want to upload the file from front end developed in React and get that uploaded file inside Express js.
Below is the code snippet of React side when I am uploading the file :
handleUploadFile(ev) {
ev.preventDefault();
var a6 = "File";
const data = new FormData();
data.append('file', this.uploadInput.files[0]);
data.append('fileName', a6);
fetch('http://localhost:4000/api/addDcuments/upload', {
method: 'POST',
body: {data},
json: true,
headers: { "Authorization": cookie.load('userToken') }
}).then((response) => {
response.json().then((body) => {
this.setState({ imageURL: `http://localhost:4000/${body.file}` });
});
});
}
In above code, I have taken the form and called handleUploadFile function on it's onSubmit event.
Now, below is my backend express js code on which I am getting the uploaded file:
export function uploadDocument(req, res, next) {
console.log(JSON.stringify(req.body));
let imageFile = req.files.file;
var ext = path.extname(imageFile.name)
if(imageFile.mimetype == "application/pdf" ||imageFile.mimetype == "application/octet-stream"){
imageFile.mv('D:/React Workspace/React-videos-example/file_path/'+req.body.filename+ext, function(err) {
if (err) {
return res.status(500).send(err);
}
res.json({file: `public/${req.body.filename}.pdf`});
});
}
}
In above code, when I am trying print the req.body, it is returning "{}" as well as I am getting one error:
TypeError: Cannot read property 'file' of undefined.
So, my backend function has been called, but not able to getting that uploaded file. So can anyone have any solution or any reference link for this issue?
When using formdata, you can't use type json in your POST, it needs to be:
contentType: 'multipart/form-data'
instead of "json: true".