I'm facing this weird issue in NodeJS when using with Passport.js, Express and Mongoose. Basically, I get an error saying "Cannot set headers after they are sent to the client" even though I don't send more than one header.
I've read other posts and tried them out as well, and none of them worked.
app.get - is there any difference between res.send vs return res.send
Error [ERR_HTTP_HEADERS_SENT]: Cannot set headers after they are sent to the client
Cannot set headers after they are sent to the client
I've dug through github issues and I can't seem to find a solution. I get the problem that this error is triggered when I send multiple response headers, but the fact is that I am not sending multiple headers. It seems just weird.
This is my stack trace:
(node:9236) DeprecationWarning: current URL string parser is deprecated, and will be removed in a future version. To use the new parser, pass option { useNewUrlParser: true } to MongoClient.connect.
Server Running on port 5000
MongoDB Connected Error
[ERR_HTTP_HEADERS_SENT]: Cannot set headers after they are sent to the
client
at validateHeader (_http_outgoing.js:503:11)
at ServerResponse.setHeader (_http_outgoing.js:510:3)
at ServerResponse.header (/Users/lourdesroashan/code/github/devlog/node_modules/express/lib/response.js:767:10)
at ServerResponse.json (/Users/lourdesroashan/code/github/devlog/node_modules/express/lib/response.js:264:10)
at Profile.findOne.then.profile (/Users/lourdesroashan/code/github/devlog/routes/api/profile.js:27:30)
at <anonymous>
This is my server code:
router.get("/userprofile", passport.authenticate('jwt', { session: false }), (req, res) => {
Profile.findOne({ user: req.user.id }).then(profile => {
if (!profile) {
return res.status(404).json({ error: "No Profile Found" });
}
else {
res.json(profile);
}
}).catch(err => {
console.log(err);
})
});
I understand what the error means, but from what I know, I don't think I am sending multiple headers, I even checked by console.log that only one of the blocks is run.
Thank you so much in advance! :)
Full Code at: https://github.com/lourdesr/devlog
EDIT:
I figured it out. It was a problem in my passport.js while trying to get the authenticated user. I forgot to use 'return' on the 'done' method, which had caused it. Just added the return statement and it worked!
That particular error occurs whenever your code attempts to send more than one response to the same request. There are a number of different coding mistakes that can lead to this:
Improperly written asynchronous code that allows multiple branches to send a response.
Not returning from the request handler to stop further code in the request handler from running after you've sent a response.
Calling next() when you've already sent a response.
Improper logic branching that allows multiple code paths to execute attempt to send a response.
The code you show in your question does not appear like it would cause that error, but I do see code in a different route here that would cause that error.
Where you have this:
if (!user) {
errors.email = "User not found";
res.status(404).json({ errors });
}
You need to change it to:
if (!user) {
errors.email = "User not found";
res.status(404).json({ errors });
// stop further execution in this callback
return;
}
You don't want the code to continue after you've done res.status(404).json({ errors }); because it will then try to send another response.
In addition, everywhere you have this:
if (err) throw err;
inside an async callback, you need to replace that with something that actually sends an error response such as:
if (err) {
console.log(err);
res.sendStatus(500);
return;
}
throwing inside an async callback just goes back into the node.js event system and isn't thrown to anywhere that you can actually catch it. Further, it doesn't send a response to the http request. In otherwords, it doesn't really do what the server is supposed to do. So, do yourself a favor and never write that code in your server. When you get an error, send an error response.
Since it looks like you may be new here, I wanted to compliment you on including a link to your full source code at https://github.com/lourdesr/devlog because it's only by looking at that that I was able to see this place where the error is occuring.
I was receiving this error because of a foolish mistake on my part. I need to be more careful when referencing my other working code. The truly embarrassing part is how long I spent trying to figure out the cause of the error. Ouf!
Bad:
return res
.send(C.Status.OK)
.json({ item });
Good:
return res
.status(C.Status.OK)
.json({ item });
Use ctrl + F hotkey and find all 'res.' keywords
then replace them with 'return res.',
change all 'res.' to 'return res.'
something like this:
res.send() change to --> return res.send()
maybe you have 'res.' in some block, like if() statement
Sorry for the Late response,
As per the mongoose documentation "Mongoose queries are not promises. They have a .then() function for co and async/await as a convenience. However, unlike promises, calling a query's .then() can execute the query multiple time"
so to use promises
mongoose.Promise = global.Promise //To use the native js promises
Then
var promise = Profile.findOne({ user: req.user.id }).exec()
promise.then(function (profile){
if (!profile) {
throw new Error("User profile not found") //reject promise with error
}
return res.status(200).json(profile) //return user profile
}).catch(function (err){
console.log(err); //User profile not found
return res.status(404).json({ err.message }) //return your error msg
})
here is an nice article about switching out callbacks with promises in Mongoose
and this answer on mongooses promise rejection handling Mongoose right promise rejection handling
There is a simple fix for the node error [ERR_HTTP_HEADERS_SET]. You need to add a return statement in front of your responses to make sure your router exits correctly on error:
router.post("/", async (req, res) => {
let user = await User.findOne({email: req.body.email});
if (!user) **return** res.status(400).send("Wrong user");
});
Because of multiple response sending in your request. if you use return key word in your else condition your code will run properly
if (!profile) {
return res.status(404).json({ error: "No Profile Found" });
}
else {
**return** res.json(profile);
}
This also happen when you tries to send the multiple response for a same request !!
So make sure you always use return keyword to send response to client inorder to stop the further processing !!
Where you have this:
if (!user) { errors.email = "User not found"; res.status(404).json({ errors }); }
You need to change it to:
if (!user) { errors.email = "User not found"; return res.status(404).json({ errors }); }
I got the same error using express and mongoose with HBS template engine. I went to Expressjs and read the docs for res.render, and it says // if a callback is specified, the rendered HTML string has to be sent explicitly. So I wasnt originally sending my html explicitly in the callback,. This is only for a contact form btw, not login info, albeit GET
//Original
let { username, email } = req.query; //My get query data easier to read
res.status(200).render('index', { username, email });
//Solution without error. Second param sending data to views, Third param callback
res.status(200).render('index', { username, email }, (err, html)=>{
res.send(html);
});
In react, if your are calling the function in useEffect hook, make sure to add a dependency to the dependency Array.
I had this error from an if statement not having an else block.
if(someCondition) {
await () => { }
}
await () => { }
I changed the above to this below and solved my issue
if(someCondition) {
await () => { }
} else {
await () => { }
}
For me, I accidentally put a res.status inside of a for loop. So my server would trigger the error the second time a res.status was returned. I needed to put the res.status outside of the for loop so it would only trigger once within the function.
First of all : make sure you didn't miss any asynchronous action without an async/await or use promises/callbacks.
Then attach any res with the return keyword : return res.status(..).json({});
And finally which was my problem: don't use return res.sendStatus if you always have some return res... inside a callback function, but you can always do a retun res.status();
in my case it was :
users.save((err,savedDoc){
if(err) return res.status(501).json({})
res.status(200).json({});
});
return res.status(500); // instead ofdoing return res.sendStatus(500)
you have to enable Promises in your programm, in my case i enabled it in my mongoose schema by using mongoose.Promise = global.Promise .
This enables using native js promises.
other alternatives to this soloution is :
var mongoose = require('mongoose');
// set Promise provider to bluebird
mongoose.Promise = require('bluebird');
and
// q
mongoose.Promise = require('q').Promise;
but you need to install these packages first.
My problem besides not returning, i was forgetting to await an asynchronous function in the handler. So handler was returning and after a bit the async function did its thing. 🤦🏻♀️
Before:
req.session.set('x', {...});
req.session.save();
return req.status(200).end();
When i needed to await:
req.session.set('x', {...});
await req.session.save();
return req.status(200).end();
I'm putting this here for anyone else who has the same problem as me- this happened to me because I'm using the next() function without a return preceding it. Just like a lot of the other answers state, not using return with your response will / can cause / allow other code in the function to execute. In my case, I had this:
app.get("/customerDetails", async (req, res, next) => {
// check that our custom header from the app is present
if (req.get('App-Source') !== 'A Customer Header') next();
var customerID = req.query.CustomerID
var rows = await get_customer_details(customerID);
return res.json(rows);
});
In my case, I forgot to include the header in my request, so the conditional statement failed and next() was called. Another middleware function must have then been executed. After the middleware finishes, without a return, the rest of the code in the original middleware function is then executed. So I simply added a return before my next() call:
// serve customer details payload
app.get("/customerDetails", async (req, res, next) => {
// check that our custom header from the app is present
if (req.get('App-Source') !== 'A Customer Header') return next();
var customerID = req.query.CustomerID
var rows = await get_customer_details(customerID);
return res.json(rows);
});
_
MY CHALLENGE:
I would like to access a third party Rest API from within my Lambda function. (e.g."http://www.mocky.io/v2/5c62a4523000004a00019907").
This will provide back a JSON file which I will then use for data extraction
_
MY CURRENT CODE:
var http = require('http');
exports.handler = function(event, context, callback) {
console.log('start request to Mocky');
http.get('http://www.mocky.io/v2/5c62a4523000004a00019907', function(res) {
console.log(res);
})
.on('error', function(e) {
console.log("Got error: " + e.message);
});
};
This does not throw an error but also does not seem to provide back the JSON
_
MY OPEN QUESTIONS:
1) How can I extract the JSON so that I can work on it
2) I will probably need to also send through an Authentification in the request header (Bearer) in the future. Will this also be possible with this method?
The problem is likely that your lambda function is exiting before logging the response.
We use Authorization headers all the time to call our lambdas. The issue of if you can use one to call the third party API is up to them, not you, so check the documentation.
Since your HTTP call is executed asynchronously, the execution of the lambda continues while that call is being resolved. Since there are no more commands in the lambda, it exits before your response returns and can be logged.
EDIT: the http.get module is difficult to use cleanly with async/await. I usually use superagent, axios, or request for that reason, or even node-fetch. I'll use request in my answer. If you must use the native module, then see EG this answer. Otherwise, npm install request request-promise and use my answer below.
The scheme that many people use these days for this kind of call uses async/await, for example (Requires Node 8+):
var request = require('request-promise')
exports.handler = async function(event, context, callback) {
console.log('start request to Mocky');
try {
const res = await request.get('http://www.mocky.io/v2/5c62a4523000004a00019907')
console.log(res)
callback(null, { statusCode: 200, body: JSON.stringify(res) })
}
catch(err) {
console.error(err.message)
callback('Got error ' + err.message)
}
};
The async/await version is much easier to follow IMO.
Everything inside an async function that is marked with await with be resolved before the execution continues. There are lots of articles about this around, try this one.
There are a lot of guys having an equal problem already solved... Look at that
or that
I want to make the implementation of this https.getSync the wrapper method, so that it calls the api synchronously, same like the readFileSync method which we use for reading file synchronously,
const https = require('https');
How should i implement this method -
https.getSync = (url) => {
let data = '';
https.get(url, resp => {
resp.on('data', (chunk) => {
data += chunk;
});
resp.on('end', () => {
console.log(JSON.parse(data));
});
}).on("error", (err) => {
console.log("Error: " + err.message);
});
return data;
}
I want the below two calls to be made synchronously, without changing the below code where we are calling the getSync method. Here for this calling i don't want to use promises or callback.
let api1 = https.getSync('https://api.nasa.gov/planetary/apod?api_key=DEMO_KEY');
let api2 = https.getSync('https://api.nasa.gov/planetary/apod?api_key=NNKOjkoul8n1CH18TWA9gwngW1s1SmjESPjNoUFo');
You can use npm package sync-request.
It's quite simple.
var request = require('sync-request');
var res = request('GET', 'http://example.com');
console.log(res.getBody());
Here is the link: sync-request
Read this also: An announcement from the package, in case readers would think using it is a good idea. You should not be using this in a production application. In a node.js application you will find that you are completely unable to scale your server. In a client application you will find that sync-request causes the app to hang/freeze. Synchronous web requests are the number one cause of browser crashes.
According to me also you should avoid making http sync request. Instead clear your concepts of using callback, promise, async/await.
I'm trying provide the response of an external API call when I perform a local GET request but struggling with how to get this to work.
My code at the moment is:
app.get('/', function(req, res){
request.post('http://data.fixer.io/api/latest?access_key=' + apikey +
'&symbols=gbp', function(err, res, body) {
console.log(body)
})
res.render('index')
})
My knowledge and experience with callbacks and async programming is limited, but how do I pass the response of the request POST into the GET request to then pass it to the index?
Thanks!
Callbacks can be difficult to understand, and the problem you describe isn't uncommon (it even has a name - Callback Hell). Partly the reason why Node introduced the async / await syntax - here's the equivalent of your code in that style
app.get('/', async (req, res, next) => {
try {
const uri = `http://data.fixer.io/api/latest?access_key=${apikey}&symbols=gbp`;
const data = await request.post(uri);
return res.render('index', { data }); // or pass whatever you need from `data` into the view
} catch (e) {
return next(e);
}
}
Notice the one big difference? No callbacks and you get all the same benefits of asynchronous code with bonus of writing code in a synchronous style.
You can chain calls in Express, so it's very easy to call an external service within a GET request, e.g.
app.get('/', function(req, res){
request.post('http://data.fixer.io/api/latest?access_key=' + apikey + '&symbols=gbp', function(err, response, body) {
console.log(body)
res.send(body);
})
})
In this case we're sending back the raw response from the POST, it is easy to wrap this in another object, e.g.
res.send( { status: 'ok', post_result: body });
Here is my scenario:
I want to get some external resource (binary file) using request library and pipe it to the client of my own application. If response code is != 200 or there are problems reaching remote server, I want to intercept and provide custom error message instead. Ideally, if response is fine, I want original headers to be preserved.
I was able to achieve that with the first piece of code I've pasted below. However, my whole application is based on Promise API so I wanted to make it consistent and wrap it in promise too. And when I do that, it no longer works. Firstly, I tried to achieve that with request-promise, without success. Then I tried to prepare very simple example on my own, still no luck.
Working example
var r = request.get('http://foo.bar');
r.on('response', result => {
if (result.statusCode === 200) {
r.pipe(res);
} else {
res.status(500).send('custom error message')
}
});
r.on('error', error => res.status(500).send('custom error message');
Not working example
var r;
return new Promise((resolve, reject) => {
r = request.get('http://foo.bar');
r.on('response', result => {
if (result.statusCode === 200) {
resolve();
} else {
reject()
}
});
r.on('error', reject);
}).then(() => {
r.pipe(res);
}).catch(() => {
res.status(500).json('custom error message');
});
By 'not working' I mean - no response is delivered, request is pending until timeout.
I've changed the code to call .pipe() on result passed to resolve instead of r. It responds to client, but response is empty then.
At the end, I've tried replacing request lib with simply http.get(). And with that, server returns file to the client, but headers (like Content-Type or Content-Length) are missing.
I've googled a lot, tried several request versions... and nothing is working.
The problem is that when "response" is triggered, you create a new promise that resolves immeadiately, but the then callback is always executed asynchronously, and when it gets called the file has arrived at the server, and there is no data flowing through the stream anymore. Instead you could just use the body parameter of the callback:
request.get('http://foo.bar', function(request, response, body) {
if(response.statusCode === 200) {
res.end(body);
} else {
res.status(500).end();
}
});
For working with streams request seems a bit buggy, axios seems to do it better:
axios.get("http://foo.bar"', {
validateStatus: status => status === 200,
responseType: "stream"
}).then(({data: stream}) => {
stream.pipe(res);
}).catch(error => {
res.status(500).json(error);
});