We Keep Coding

How to manage request data on entry file in node js

In Node Js, on the entry file e.g. index.js, How can I get requested data either as Form-data or Form-URL-encoded or Raw JSON data in middleware?
In my project, I am handling various API request,
Few requests contain file type so requesting as form-data.
Few requests do not contain file type so requests are coming as Form-URL-encoded.
Now the main problem is before routing, I need a specific field from req.body in the middleware.
But I am getting req.body as undefined.
For reference here is my index.js file:
const express = require('express');
const app = express();
app.use(express.raw());
app.use(express.urlencoded({ extended: false }));
app.use((req, res, next) => {
const routes_handler = require('./routes/index.js')(app, express, req);
next();
})
app.listen(3000, () => {
console.log("Server running at Port " + 3000);
});
and the routes/index.js file as follows:
module.exports = function (app, express, req) {
console.log(req.body);
//I need here data of req.body for all requests type (form data, URL-encoded, raw JSON)
app.post('/', function (req, res) {
console.log("Here I can get the requested body easily", req.body)
res.send('Hello World');
});
app.post('*', function (req, res) {
res.send({
code: 0,
message: 'No Content',
status_code: 204,
data: {
error: 'API not found!'
}
});
});
}
Also, I know for file type data, POSTMAN will send the request as Form-data, not as Form-url-encoded. So which I should choose Formidable or Multer?

The way you get all the data in index.js is by creating middlewares for your application, every time any routes that go into your application will be passed through this middleware.
Middleware functions are functions that have access to the request object (req), the response object (res), and the next function in the application’s request-response cycle. The next function is a function in the Express router which, when invoked, executes the middleware succeeding the current middleware.
The below middleware will simply listen to all routes & adds up request time to request time, here goes the code
let express = require('express')
let app = express()
let bodyParser = require("body-parser")
app.use(bodyParser.json())
let requestTime = function (req, res, next) { // simply updating add the requestBody using the middleware
req.requestTime = Date.now();
req.json_body = req.body;
next()
}
app.use(requestTime) // requestTime is the middleware here
app.get('/', function (req, res) {
var responseText = 'Hello World!<br>'
responseText += '<small>Requested at: ' + req.requestTime + '</small>'
res.send(responseText)
})
app.listen(3000)
Few things to note here
Always add interceptor above all routes
Don't forget to add next() inside the middleware, else it will not go to the next route.
Now, coming to the second part of your question which is accessing body,formdata, etc
You can use body-parser npm module to achieve that, something like this
Starting from Express 4, body-parser comes inbuilt with it, so you can try out something
app.use(express.json());
app.use(
bodyParser.urlencoded({
extended: false
})
);
Now, the last bit, you don't need Multer for formdata, but for file upload like multipart/form-data you will need this. Both are good in their own ways, I would go for Multer.
Hope this will help you :)

I believe the body-parser module is your answer.
https://www.npmjs.com/package/body-parser
Add the following line before the routes in your index.js after installing the body-parser package.
app.use(bodyParser.json())

req.files return undefined data while req.body is populated

Setting:
Node.JS(v8.9.4)
Express(v4.16.2)
Superagent v3.8.2(https://github.com/visionmedia/superagent) for POST from client
multer for file upload(v1.3.0)
React/Babel/Webpack
CentOS 7,Vultr VPS
File size limit is expaned to 50mb from nginx & multer
Deployed by private git(the version is consistent all across the platform)
Symptom:
When a mix up of stringified JSONdata and image are sent via http POST call, the server receives random amount of image via multer(multer.any()). The file is uploaded and exists in the server's public folder. req.body is also OK but somehow I cannot access the req.files. it returns undefined. From heroku and my desktop localhost, the app runs fine. The image is uploaded OK, accessing req.files data has no problem as well. Only the VPS/CentOS7 server has got the problem.
Client Side(React / Superagent / Babel / Webpack)
import request from 'superagent'
request.post('/modwimg')
.query({
//some token and other info goes here
})
.accept('application/json')
.field('data',JSON.stringify(jsonData))
.attach('image',this.state.imagedata)
.attach('bannerimage',this.state.bannerimagedata)
.then((res)=>{
console.log('upload finished')
this.setState({goback:true})
})
.catch((err)=>{
})
Server Side
const bodyParser = require('body-parser')
const multer = require('multer')
const pubDir = path.join(__dirname, 'pub')
const storage = multer.diskStorage({
destination: (req,file,cb)=>{
cb(null,'pub/')
},
filename: (req,file,cb)=>{
cb(null,Date.now() + file.originalname)
}
})
const upload = multer({storage:storage})
//allowing public access, image goes to public dir
app.use('/pub', express.static(pubDir))
/* initialize bodyparser to build up RESTful app */
//file size limit expansion
app.use(bodyParser.urlencoded({limit:'50mb', extended:true}))
app.use(bodyParser.json({limit:'50mb'}))
app.post('/imageupload',upload.any(),(req,res,next)=>{
//some token approval goes here
console.log(req.files) // <----this returns undefined data, but image is still uploaded
console.log(req.body) // <---- this is fine!!
//putting data(JSON.parse(req.body)) to db
db.any( //....
)
//then respond to client
res.json({result:true})
})

Problem solved. I've changed extended option to extended:false from app.use(bodyParser.urlencoded({limit:'50mb', extended:true})), now it works like magic. I read from somewhere the extended option of bodyParser makes the rest of the http request data to some other format, so I've changed it and now it works.

req.body is empty on express POST call

I am trying ti write a sample POST call in express.
var express = require("express")
, app = express()
, server = require('http').createServer(app)
, bodyParser = require('body-parser');
app.listen(80, function() {
console.log("server started");
});
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({extended: false}));
app.post("/test", function(req, res) {
try {
console.log(req.body);
res.send("working " + req.body.name);
} catch(e) {
console.log("error in test API" + e.message);
res.end();
}
});
But I am unable to access body data on the server. It is empty. Below is postman query.

Select raw under body in Postman, not form-data if you want JSON data.
Then enter the data in JSON format. Postman should format it if you have the header set: Content-Type: application/json
Edit:
If you want to parse form-data you can't use body-parser as stated in the readme:
This does not handle multipart bodies, due to their complex and typically large nature. For multipart bodies, you may be interested in the following modules:
busboy and connect-busboy
multiparty and connect-multiparty
formidable
multer
Github link

To read as req.body you're missing the Content-Type : application/json header I think, add it and make the request into a raw json. Or else youre getting a string and cannot directly access it as req.body
eg :
{"name" : "abcd"}
Update :
If you need Form data use the bodyParser to convert text to json
app.use(bodyParser.urlencoded({ extended: true }));
app.use(bodyParser.json());
Update 2 :
It seems the issue is with multipart/form-data you're using. To handle it you would need a specific framework like multer due to security issues. The below should work.
var multer = require('multer');
var upload = multer() ;
app.post('/test', upload.array(), function (req, res, next) {
console.log(req.body);
});

Error: incorrect header check when running post

I need to get zip from rest call (for simulation I use postman with binary option for post and add a little zip file with folder and html file),during the simulation I want to get the data with express and extract the zip and put in some folder under C drive.
Currently when I run the following program(this is all the code which i've tried) but im getting error
events.js:85
throw er; // Unhandled 'error' event
^ Error: incorrect header check
at Zlib._handle.onerror (zlib.js:366:17)
var express = require('express'),
fs = require('fs'),
zlib = require('zlib'),
app = express();
app.post('/', function (req, res) {
var writeStream = fs.createWriteStream('C://myFolder', {flags: 'w'});
req.pipe(zlib.createInflate()).pipe(writeStream);
});
var server = app.listen(3000, function () {
console.log("Running on port" + 3000)
}
)
in postman header i've added the following
content-Type ----> application/zip
How should I overcome this issue and save the zip ? there is other recommended (zlib)library to get extract and save zip?

zlib is meant to extract gzipped or deflated data, not .ZIP files.
You can use the node-unzip module for those:
var unzip = require('unzip');
...
app.post('/', function(req, res) {
var extractor = unzip.Extract({ path : 'C://myFolder' }).on('close', function() {
res.sendStatus(200);
}).on('error', function(err) {
res.sendStatus(500);
});
req.pipe(extractor);
});
If Postman can't handle uploads like this (as suggested in the comments), you can test using cURL:
$ curl -XPOST localhost:3000 --data-binary #test.zip

Express js form data

Can someone please tell me the recommended (up to date) way to get POSTed form data in express.
So many tutorials/ posts etc talk about bodyParser but this is no longer bundled with Express and other blogs etc recommend using urlencoded directly, but now this is not available either.
Trying to find accurate information on these frameworks or technologies is doing my head in.
BTW what I am intrerested in is very simple and small form data

You should install body-parser through npm-install. Now it comes as a separate middleware.
After that add following line in your app.js
var bodyParser = require('body-parser');
app.use(bodyParser.json());
app.use(bodyParser.urlencoded());
// in latest body-parser use like below.
app.use(bodyParser.urlencoded({ extended: true }));
It parses the post request as an object. You will get your variables in req.body.
In your post request handler.
app.post('/post',function(request,response){
console.log(request.body) //you will get your data in this as object.
})
Edit 1
The answer above was for the question specifically asked, the OP was looking for the bodyParser(deprecated) which was not part of express anymore.
Since the title of the question is very generic and the answer doesn't include all aspects of form-data, I will put #StLia's answer as an edit.
Body-Parser Readme
This does not handle multipart bodies, due to their complex and typically large nature. For multipart bodies, you may be interested in the following modules:
busboy and
connect-busboy
multiparty and
connect-multiparty
formidable
multer

You can make use of express-formidable module to that.
install 'express-formidable' by the following command
npm install express-formidable
the simple example is as follows
const express = require('express');
const formidable = require('express-formidable');
var app = express();
app.use(formidable());
app.post('/upload', (req, res) => {
//req.fields contains non-file fields
//req.files contains files
res.send(JSON.stringify(req.fields));
});
Click here for further description

From the README of body-parser:
This does not handle multipart bodies, due to their complex and
typically large nature.
The above is going to work with x-www-form-urlencoded and json but it will NOT work with any multipart. form-data is also multipart with the header multipart/form-data.
In case of form-data, your best solution would be to use express-formidable.

As stated in this StackOverflow answer:
Express 4.16+ has implemented their own version of body-parser so you do not need to add the dependency to your project. You can run it natively in express
app.use(express.json()); // Used to parse JSON bodies
app.use(express.urlencoded()); // Parse URL-encoded bodies using query-string library
// or
app.use(express.urlencoded({ extended: true })); // Parse URL-encoded bodies using qs library
See also: query-string vs qs

I noticed #HubballiHuli answer was to use a package called express-formidable.
You don't need to use this unnecessary package, it provide one (small) file of code.
Instead you can do it yourself (now removing the dependency).
Here is the formidableMiddleware file:
'use strict';
const formidable = require('formidable');
function parse(opts, events) {
return (req, res, next) => {
if (req.express_formidable && req.express_formidable.parsed) {
next();
return;
}
const form = new formidable.IncomingForm();
Object.assign(form, opts);
let manageOnError = false;
if (events) {
events.forEach((e) => {
manageOnError = manageOnError || e.event === 'error';
form.on(e.event, (...parameters) => { e.action(req, res, next, ...parameters); });
});
}
if (!manageOnError) {
form.on('error', (err) => {
next(err);
});
}
form.parse(req, (err, fields, files) => {
if (err) {
next(err);
return;
}
Object.assign(req, { fields, files, express_formidable: { parsed: true } });
next();
});
};
}
module.exports = parse;
exports.parse = parse;
Now on how to use it:
const express = require('express');
const formidableMiddleware = require('./formidableMiddleware.js');
var app = express();
app.use(formidableMiddleware());
app.post('/upload', (req, res) => {
//req.fields contains non-file fields
//req.files contains files
res.send(JSON.stringify(req.fields));
});
I wrote an article on unnecessary packages a while ago and why not to use them:
https://medium.com/#alexjamesdunlop/unnecessary-packages-b3623219d86

Besides the solutions with formidable, there is another module which I have been using in my recent projects since 2019. The module express-form-data can be easily declared in your server file like:
const express = require('express');
const formData = require('express-form-data');
app.use(formData.parse());
app.post('/image-upload', (req, res) => {
console.log(req.files);
})
...
In case of image uploading, for instance, req.files will provide all relevant data you need for handling the files such as path, size, filename, etc.

If you would like to apply the formidableMiddleware to only one API route and not globally, this is how you would pass the value.
This would be useful if u want to mix between different headers to pass to the API for other API's which you which u do not want the formidableMiddleware API to be applied.
const express = require('express');
const formidable = require('express-formidable');
var app = express();
app.post('/mypath', formidableMiddleware(), (req, res) => {
// rest of the code
})