We Keep Coding

Multer validation with joi

I've created an endpoint to post and upload a csv file for processing in Node. It works fine but I'm trying to figure out how to validate a few things before uploading the file.
A sample request would be:
{
"test_doc": "/path/to/file/test.csv"
"offset": [0,1]
}
I want the form to require "test_doc" and only accept csv files and have "offset" be optional
The schema for the "offset" works but I'm unsure how to validate the file with multer as well, especially before uploading it.
Sample code below
const upload = multer({ dest: "/tmp" });
router.post("/", upload.single("test_doc"), async (req, res) => {
const schema = joi.object().keys({
offset: joi.array().items(joi.number().min(-60).max(60)).min(1).max(2)
});
});

You can use the fileFilter method to validate file types in Multer. The fileFilter method is a built-in method which comes with Multer middleware:
const upload = multer({
dest: "/tmp",
fileFilter: (req, file, cb) => {
if (file.mimetype == "text/csv" && file.fieldname === "test_doc") {
cb(null, true);
} else {
cb(null, false);
return cb(new Error('Invalid upload: fieldname should be test_doc and .csv format '));
}
}
});

set multer destination in api.js in nodejs and call another function?

i am using a api in server.js file.
app.post('/api/uploadalumniresume', api.UploadAlumniResume);
my api.js file code is:
exports.UploadAlumniResume = (req, res) => {
const storage = multer.diskStorage({
destination(req, file, cb) {
// cb(null, `${__dirname}public/images/uploads`);
cb(null, `${__dirname}/../public/imports`);
},
filename(req, file, cb) {
cb(null, `${Date.now()}-${file.originalname}`);
},
});
const importupload = multer({
storage,
});
const importmulter = importupload.single('file');
};
The server.js as call the api but the exports api not saving the uploading file to the destination folder
And i need to store the upload file on destination follder.
And if the file as uploaded then function need to work on create fuction in the same api.
please give a any solution to me!

NodeJS - multer - change filename depending on request attributes

I know that I can change the filename with multer by means of the storage object like following:
const storage = multer.diskStorage({
destination: (req, file, cb) => {
cb(null, process.env.UPLOAD_DIR);
},
filename: (req, file, cb) => {
cb(null, 'bla.png');
}
});
const upload = multer({ storage: storage } );
My request, besides having the file, also contains some text attributes such as name: myPic.png.
Is it possible to dynamically change the filename dependent on other request attributes or within the controller like following:
filename: (req, file, cb) => {
cb(null, `${req.body.name}.png`);
}
or
router.post('/upload', upload.single('pic'), myController.upload);
/* in controller */
upload = async (req: Request, res: Response) => {
try {
/* change the filename of multer here? */
} catch (err) {
winston.error(`Error while uploading: ${err.message}`);
winston.error(`Stack trace: ${err.stack}`);
sendJSONResponse(res, err, HttpStatus.INTERNAL_SERVER_ERROR);
}
}

Multer is the middleware which both populates req.body AND stores the file.
Also, when it reaches the filename() function, there is no guarantee that the text fields will be populated in req.body because it depends on which order the client sends them in (see last note).
From what I see, you have two options:
1) Rename the uploaded file after the multer upload middleware does its thing and populates req.body as well as req.file. So in your controller upload middleware, you'd do something like:
if (req.file) {
fs.renameSync(req.file.path, req.file.destination + req.body.name);
}
2) Change the request body text field into a query parameter. Then, inside filename() you can do a req.query.name.
Con: Not a very RESTful design, but maybe that is not so important to you.

According to the multer documentation it does not have access to req.body for other additional fields, if you test it it receives an undefined value, then a not so perfect but functional solution is the following, once the file is uploaded you can rename it as follows.
Add the native class fs for access to files option
const fs = require('fs');
In diskStorage configuration add the name you want, for example bla.png
var storage = multer.diskStorage({
destination: path.join('public/images/'),
filename: function ( req, file, cb ) {
cb(null, 'bla.png');
}
});
The form with the text field for the custom name
<form action="/upload" enctype="multipart/form-data" method="POST">
<input type="file" accept="image/*" name="photo" >
<br><!--here is the custom file name-->
<input type="text" name="file_name">
<br>
<button type="submit">Send</button>
</form>
Within the post path, once you have sent the file whose name will be bla.png, you can replace that name with the one in a field of the form by accessing req.body.field_name
router.post('/upload', upload.single('photo'), (req, res) => {
//Here change the file name bla.png for the new value in req.body.field_name + original ext of file
fs.renameSync(req.file.path, req.file.path.replace('bla.png',
req.body.field_name + path.extname(req.file.originalname)));
if(req.file) {
res.json(req.file);
}
else throw 'error';
});

Sending file through HTTP request

I tried to receive the file and store it in the multer storage
Node js code
enter code here
app.post('/createLicence', upload.single('photo'),function(req, res ,next) {
// any logic goes here
console.log("filename" ,req.body.name)
if (!req.file) {
console.log("No file received");
return res.send({
success: false
});
} else {
console.log('file received');
var function_name = 'createLicence'
var arguments_array = [req.file.path,'Raghav','Mumbai','Approved']
invoke = require('/Users/sanjeev.natarajan/fabric-samples/fabcar/invoke.js');
invoke.invokechaincode(function_name,arguments_array)
return res.send({
success: true
})
}
});
but i am receiving no file is receivedi have send the request through postman

-
From : https://www.npmjs.com/package/multer
In order to use the multer package, you have first to define a few parameters so that it can work on your fileDirectory.
In your server.js :
let multer = require('multer');
let storage = multer.diskStorage({
destination: function(req, file, cb) {
cb(null, '/path/to/storage/')
},
filename: function(req, file, callback) {
callback(null, file.originalname + '-' + Date.now());
}
});
let upload = multer({
storage: storage
});
Now, configure your route
router.route('/your/payload')
.post(authController.isAuthenticated, upload.any(), albumController.postFile)
Note that upload.any() will allow you to upload multiple different formatted files at once. Feel free to use any other kind of upload.method() depending on your needs.
From this point, multer already is doing its job, however you might want to keep track of the files uploaded on your server.
So, in your own module, the logic is pretty much straight forward :
(I'm assuming that you're using mongoose models since you're not giving much information, but that's not the relevant part anyway)
exports.postFile = async (req, res) => {
if (!req || !req.files || !req.files[0]) return res.status(400).send("Bad request.");
for (let i = 0; req.files[i]; i++) {
await File.create({
path: req.files[i],
originalName: req.files[i].originalName,
mimetype: req.files[i].mimetype,
owner: req.user.userId
}, (err, file) => {
if (err) console.log("Something went wrong: " + err); else {
// Do something with file
}
});
}
return res.status(418).send("I'm a teapot.");
}
This configuration and middleware use is ONLY for testing purpose, never ever let anyone upload something to your server without carefully handle that uploading process (file integrity, resource management, ...). An open uploading system can become a very wide backdoor getting straight to your server.
Hope this helps,
regards.

nodejs multer diskstorage to delete file after saving to disk

I am using multer diskstorage to save a file to disk.
I first save it to the disk and do some operations with the file and then i upload it to remote bucket using another function and lib.
Once the upload is finished, i would like to delete it from the disk.
var storage = multer.diskStorage({
destination: function (req, file, cb) {
cb(null, '/tmp/my-uploads')
},
filename: function (req, file, cb) {
cb(null, file.fieldname + '-' + Date.now())
}
})
var upload = multer({ storage: storage }).single('file')
and here is how i use it:
app.post('/api/photo', function (req, res) {
upload(req, res, function (err) {
uploadToRemoteBucket(req.file.path)
.then(data => {
// delete from disk first
res.end("UPLOAD COMPLETED!");
})
})
});
how can i use the diskStorage remove function to remove the files in the temp folder?
https://github.com/expressjs/multer/blob/master/storage/disk.js#L54
update:
I have decided to make it modular and put it in another file:
const fileUpload = function(req, res, cb) {
upload(req, res, function (err) {
uploadToRemoteBucket(req.file.path)
.then(data => {
// delete from disk first
res.end("UPLOAD COMPLETED!");
})
})
}
module.exports = { fileUpload };

You don't need to use multer to delete the file and besides _removeFile is a private function that you should not use.
You'd delete the file as you normally would via fs.unlink. So wherever you have access to req.file, you can do the following:
const fs = require('fs')
const { promisify } = require('util')
const unlinkAsync = promisify(fs.unlink)
// ...
const storage = multer.diskStorage({
destination(req, file, cb) {
cb(null, '/tmp/my-uploads')
},
filename(req, file, cb) {
cb(null, `${file.fieldname}-${Date.now()}`)
}
})
const upload = multer({ storage: storage }).single('file')
app.post('/api/photo', upload, async (req, res) =>{
// You aren't doing anything with data so no need for the return value
await uploadToRemoteBucket(req.file.path)
// Delete the file like normal
await unlinkAsync(req.file.path)
res.end("UPLOAD COMPLETED!")
})

Multer isn't needed. Just use this code.
const fs = require('fs')
const path = './file.txt'
fs.unlink(path, (err) => {
if (err) {
console.error(err)
return
}
//file removed
})

You may also consider using MemoryStorage for this purpose, with this storage the file is never stored in the disk but in memory and is deleted from the memory automatically after execution comes out of controller block, i.e., after you serve the response in most of the cases.
When you will use this storage option, you won't get the fields file.destination, file.path and file.filename, instead you will get a field file.buffer which as name suggests is a buffer, you can convert this buffer to desired format to do operations on and then upload using a stream object.
Most of the popular libraries support streams so you should be able to use stream to upload your file directly, code for converting buffer to stream:
const Readable = require('stream').Readable;
var stream = new Readable();
stream._read = () => { }
stream.push(file.buffer);
stream.push(null);
// now you can pass this stream object to your upload function
This approach would be more efficient as files will be stored in memory which will result in faster access, but it does have a con as mentioned in multer documentation:
WARNING: Uploading very large files, or relatively small files in
large numbers very quickly, can cause your application to run out of
memory when memory storage is used.

To do it truly automatically across all routes I used this strategy :
when the request ends, we delete all the uploaded files (req.files). Before that, if you want to keep the files on the server, you need to save them in another path.
var express = require('express');
var app = express();
var http = require('http');
var server = http.Server(app);
// classic multer instantiation
var multer = require('multer');
var upload = multer({
storage: multer.diskStorage({
destination: function (req, file, cb) {
cb(null, `${__dirname}/web/uploads/tmp/`);
},
filename: function (req, file, cb) {
cb(null, uniqid() + path.extname(file.originalname));
},
}),
});
app.use(upload.any());
// automatically deletes uploaded files when express finishes the request
app.use(function(req, res, next) {
var writeHead = res.writeHead;
var writeHeadbound = writeHead.bind(res);
res.writeHead = function (statusCode, statusMessage, headers) {
if (req.files) {
for (var file of req.files) {
fs.unlink(file.path, function (err) {
if (err) console.error(err);
});
}
}
writeHeadbound(statusCode, statusMessage, headers);
};
next();
});
// route to upload a file
router.post('/profile/edit', access.isLogged(), async function (req, res, next) {
try {
// we copy uploaded files to a custom folder or the middleware will delete them
for (let file of req.files)
if (file.fieldname == 'picture')
await fs.promises.copy(file.path, `${__dirname}/../uploads/user/photo.jpg`);
} catch (err) {
next(err);
}
});

I have removed directory after file uploaded using fs-extra
const fs = require('fs-extra');
// after you uploaded to bucket
await fs.remove('uploads/abc.png'); // remove upload dir when uploaded bucket