Asynchronous Requests before rendering Page in Express.JS - javascript

I'm trying to implement a Sequence of HTTP-Requests for training purposes in express.js. In order to do that i worked with async/await to wait for my requests and wrapped them in a try catch block. Everything works as aspected, since i added some console.log to check the flow. When everything is done, i want to render my page with the built in res.render() function but it throws an error when calling res.render()[res.render is not a function].
I could load the data via AJAX but is this really necessary?
/* Express Functions */
router
.get('/', async function (res,req,next){
try{
doSynchronousThings();
const ParticipantIds = await xingRequests.getParticipantIds();
console.log('...getParticipantIds response:');
console.log(ParticipantIds);
finalLog();
const ParticipantData = await xingRequests.getParticipantData(ParticipantIds[0]);
console.log('...getParticipantData response:');
console.log(ParticipantData);
console.log('...all done');
}catch(err){
console.error(err);
next(err);
}
res.render('test', {
page: {
title:'Xing Event API',
message: 'Happy Hacking!'
}
});
});
The xingRequest methods just fire a request-promise based request and return the response objects.

Related

Error [ERR_HTTP_HEADERS_SENT]: Cannot set headers after they are sent to client [duplicate]

I'm facing this weird issue in NodeJS when using with Passport.js, Express and Mongoose. Basically, I get an error saying "Cannot set headers after they are sent to the client" even though I don't send more than one header.
I've read other posts and tried them out as well, and none of them worked.
app.get - is there any difference between res.send vs return res.send
Error [ERR_HTTP_HEADERS_SENT]: Cannot set headers after they are sent to the client
Cannot set headers after they are sent to the client
I've dug through github issues and I can't seem to find a solution. I get the problem that this error is triggered when I send multiple response headers, but the fact is that I am not sending multiple headers. It seems just weird.
This is my stack trace:
(node:9236) DeprecationWarning: current URL string parser is deprecated, and will be removed in a future version. To use the new parser, pass option { useNewUrlParser: true } to MongoClient.connect.
Server Running on port 5000
MongoDB Connected Error
[ERR_HTTP_HEADERS_SENT]: Cannot set headers after they are sent to the
client
at validateHeader (_http_outgoing.js:503:11)
at ServerResponse.setHeader (_http_outgoing.js:510:3)
at ServerResponse.header (/Users/lourdesroashan/code/github/devlog/node_modules/express/lib/response.js:767:10)
at ServerResponse.json (/Users/lourdesroashan/code/github/devlog/node_modules/express/lib/response.js:264:10)
at Profile.findOne.then.profile (/Users/lourdesroashan/code/github/devlog/routes/api/profile.js:27:30)
at <anonymous>
This is my server code:
router.get("/userprofile", passport.authenticate('jwt', { session: false }), (req, res) => {
Profile.findOne({ user: req.user.id }).then(profile => {
if (!profile) {
return res.status(404).json({ error: "No Profile Found" });
}
else {
res.json(profile);
}
}).catch(err => {
console.log(err);
})
});
I understand what the error means, but from what I know, I don't think I am sending multiple headers, I even checked by console.log that only one of the blocks is run.
Thank you so much in advance! :)
Full Code at: https://github.com/lourdesr/devlog
EDIT:
I figured it out. It was a problem in my passport.js while trying to get the authenticated user. I forgot to use 'return' on the 'done' method, which had caused it. Just added the return statement and it worked!
That particular error occurs whenever your code attempts to send more than one response to the same request. There are a number of different coding mistakes that can lead to this:
Improperly written asynchronous code that allows multiple branches to send a response.
Not returning from the request handler to stop further code in the request handler from running after you've sent a response.
Calling next() when you've already sent a response.
Improper logic branching that allows multiple code paths to execute attempt to send a response.
The code you show in your question does not appear like it would cause that error, but I do see code in a different route here that would cause that error.
Where you have this:
if (!user) {
errors.email = "User not found";
res.status(404).json({ errors });
}
You need to change it to:
if (!user) {
errors.email = "User not found";
res.status(404).json({ errors });
// stop further execution in this callback
return;
}
You don't want the code to continue after you've done res.status(404).json({ errors }); because it will then try to send another response.
In addition, everywhere you have this:
if (err) throw err;
inside an async callback, you need to replace that with something that actually sends an error response such as:
if (err) {
console.log(err);
res.sendStatus(500);
return;
}
throwing inside an async callback just goes back into the node.js event system and isn't thrown to anywhere that you can actually catch it. Further, it doesn't send a response to the http request. In otherwords, it doesn't really do what the server is supposed to do. So, do yourself a favor and never write that code in your server. When you get an error, send an error response.
Since it looks like you may be new here, I wanted to compliment you on including a link to your full source code at https://github.com/lourdesr/devlog because it's only by looking at that that I was able to see this place where the error is occuring.
I was receiving this error because of a foolish mistake on my part. I need to be more careful when referencing my other working code. The truly embarrassing part is how long I spent trying to figure out the cause of the error. Ouf!
Bad:
return res
.send(C.Status.OK)
.json({ item });
Good:
return res
.status(C.Status.OK)
.json({ item });
Use ctrl + F hotkey and find all 'res.' keywords
then replace them with 'return res.',
change all 'res.' to 'return res.'
something like this:
res.send() change to --> return res.send()
maybe you have 'res.' in some block, like if() statement
Sorry for the Late response,
As per the mongoose documentation "Mongoose queries are not promises. They have a .then() function for co and async/await as a convenience. However, unlike promises, calling a query's .then() can execute the query multiple time"
so to use promises
mongoose.Promise = global.Promise //To use the native js promises
Then
var promise = Profile.findOne({ user: req.user.id }).exec()
promise.then(function (profile){
if (!profile) {
throw new Error("User profile not found") //reject promise with error
}
return res.status(200).json(profile) //return user profile
}).catch(function (err){
console.log(err); //User profile not found
return res.status(404).json({ err.message }) //return your error msg
})
here is an nice article about switching out callbacks with promises in Mongoose
and this answer on mongooses promise rejection handling Mongoose right promise rejection handling
There is a simple fix for the node error [ERR_HTTP_HEADERS_SET]. You need to add a return statement in front of your responses to make sure your router exits correctly on error:
router.post("/", async (req, res) => {
let user = await User.findOne({email: req.body.email});
if (!user) **return** res.status(400).send("Wrong user");
});
Because of multiple response sending in your request. if you use return key word in your else condition your code will run properly
if (!profile) {
return res.status(404).json({ error: "No Profile Found" });
}
else {
**return** res.json(profile);
}
This also happen when you tries to send the multiple response for a same request !!
So make sure you always use return keyword to send response to client inorder to stop the further processing !!
Where you have this:
if (!user) { errors.email = "User not found"; res.status(404).json({ errors }); }
You need to change it to:
if (!user) { errors.email = "User not found"; return res.status(404).json({ errors }); }
I got the same error using express and mongoose with HBS template engine. I went to Expressjs and read the docs for res.render, and it says // if a callback is specified, the rendered HTML string has to be sent explicitly. So I wasnt originally sending my html explicitly in the callback,. This is only for a contact form btw, not login info, albeit GET
//Original
let { username, email } = req.query; //My get query data easier to read
res.status(200).render('index', { username, email });
//Solution without error. Second param sending data to views, Third param callback
res.status(200).render('index', { username, email }, (err, html)=>{
res.send(html);
});
In react, if your are calling the function in useEffect hook, make sure to add a dependency to the dependency Array.
I had this error from an if statement not having an else block.
if(someCondition) {
await () => { }
}
await () => { }
I changed the above to this below and solved my issue
if(someCondition) {
await () => { }
} else {
await () => { }
}
For me, I accidentally put a res.status inside of a for loop. So my server would trigger the error the second time a res.status was returned. I needed to put the res.status outside of the for loop so it would only trigger once within the function.
First of all : make sure you didn't miss any asynchronous action without an async/await or use promises/callbacks.
Then attach any res with the return keyword : return res.status(..).json({});
And finally which was my problem: don't use return res.sendStatus if you always have some return res... inside a callback function, but you can always do a retun res.status();
in my case it was :
users.save((err,savedDoc){
if(err) return res.status(501).json({})
res.status(200).json({});
});
return res.status(500); // instead ofdoing return res.sendStatus(500)
you have to enable Promises in your programm, in my case i enabled it in my mongoose schema by using mongoose.Promise = global.Promise .
This enables using native js promises.
other alternatives to this soloution is :
var mongoose = require('mongoose');
// set Promise provider to bluebird
mongoose.Promise = require('bluebird');
and
// q
mongoose.Promise = require('q').Promise;
but you need to install these packages first.
My problem besides not returning, i was forgetting to await an asynchronous function in the handler. So handler was returning and after a bit the async function did its thing. 🤦🏻‍♀️
Before:
req.session.set('x', {...});
req.session.save();
return req.status(200).end();
When i needed to await:
req.session.set('x', {...});
await req.session.save();
return req.status(200).end();
I'm putting this here for anyone else who has the same problem as me- this happened to me because I'm using the next() function without a return preceding it. Just like a lot of the other answers state, not using return with your response will / can cause / allow other code in the function to execute. In my case, I had this:
app.get("/customerDetails", async (req, res, next) => {
// check that our custom header from the app is present
if (req.get('App-Source') !== 'A Customer Header') next();
var customerID = req.query.CustomerID
var rows = await get_customer_details(customerID);
return res.json(rows);
});
In my case, I forgot to include the header in my request, so the conditional statement failed and next() was called. Another middleware function must have then been executed. After the middleware finishes, without a return, the rest of the code in the original middleware function is then executed. So I simply added a return before my next() call:
// serve customer details payload
app.get("/customerDetails", async (req, res, next) => {
// check that our custom header from the app is present
if (req.get('App-Source') !== 'A Customer Header') return next();
var customerID = req.query.CustomerID
var rows = await get_customer_details(customerID);
return res.json(rows);
});

nodejs retrieve body from inside request scope

I'm new to nodejs and javascript in general. I believe this is an issue with the scope that I'm not understanding.
Given this example:
...
...
if (url == '/'){
var request = require('request');
var body_text = "";
request('http://www.google.com', function (error, response, body) {
console.log('error:', error);
console.log('statusCode:', response && response.statusCode);
console.log('body:', body);
body_text=body;
});
console.log('This is the body:', body_text)
//I need the value of body returned from the request here..
}
//OUTPUT
This is the body: undefined
I need to be able to get the body from response back and then do some manipulation and I do not want to do all the implementation within the request function. Of course, if I move the log line into:
request( function { //here })
It works. But I need to return the body in some way outside the request. Any help would be appreciated.
You can't do that with callbacks because this will works asynchronously.
Work with callbacks is kind of normal in JS. But you can do better with Promises.
You can use the request-promise-native to do what you want with async/await.
async function requestFromClient(req, res) {
const request = require('request-promise-native');
const body_text = await request('http://www.google.com').catch((err) => {
// always use catches to log errors or you will be lost
})
if (!body_text) {
// sometimes you won't have a body and one of this case is when you get a request error
}
console.log('This is the body:', body_text)
//I need the value of body returned from the request here..
}
As you see, you always must be in a function scope to use the async/await in promises.
Recommendations:
JS the right way
ES6 Fetures
JS clean coding
More best practices...
Using promises

How do I make a MongoDB query throw an error if there is no database connection? [duplicate]

I'm new to Mongo. I needed a database for a simple project and ended up following a tutorial using Mongo with Monk but I have problems understanding how to handle errors.
Background: I have a registration form on the client side. When the user clicks a button, the data is sent via AJAX to the controller that (upon validation, but this is not relevant now) inserts such data into the database and sends back either success or error. When the db is up all seems to work fine.
The problem: If I don't start the db and try to send the request anyway, no error is returned. Simply nothing happens. After some time on the console I get: POST /members/addmember - - ms - -.
I think some error should be returned to the user in this case, so how could I do this?
The post request is below (pretty much as from the tutorial):
// app.js
var db = monk('localhost:27017/dbname')
[...]
// I realize it might be not optimal here
app.use(function(req,res,next){
req.db = db;
next();
});
// members.js
router.post('/addmember', function(req, res) {
var db = req.db;
var collection = db.get('memberstest');
collection.insert(req.body, function(err, result){
res.json(
(err === null) ? { msg: 'success' } : { msg: err }
);
});
});
If the db is down I guess the problem is actually even earlier than the insert, that is in that "db.get()". So how to check if that get can actually be done? I suppose that given the asynchronous nature of node something like a try/catch would be pointless here. Correct?
EDIT: After Neil's answer and a bit of trying, I put together the following that seems to do the job. However, given my scarce degree of confidence on this, I'd appreciate a comment if the code below works because it makes sense or by chance. I added the bufferMaxEntries: 0 options and modified the controller as follows. In the ajax callback I simply have an alert for now that shows the error message thrown (if any).
router.post('/addmember', async (req,res) => {
try {
let db = req.db;
let collection = db.get('memberstest');
collection.insert(req.body, function(err, result){
res.json(
(err === null) ? { msg: 'success' } : { msg: err }
);
});
await db.then(() => 1);
} catch(e) {
res.json({msg: e.message})
}
});
Well you can actually set the bufferMaxEntries option ( documented under Db but deprecated for that object usage, use at "top level as demonstrated instead" ) on the connection, which essentially stops "queuing" requests on the driver when no connection is actually present.
As a minimal example:
index.js
const express = require('express'),
morgan = require('morgan'),
db = require('monk')('localhost/test',{ bufferMaxEntries: 0 }),
app = express();
const routes = require('./routes');
app.use(morgan('combined'));
app.use((req,res,next) => {
req.db = db;
next();
});
app.use('/', routes);
(async function() {
try {
await db.then(() => 1);
let collection = db.get('test');
await collection.remove({});
await collection.insert(Array(5).fill(1).map((e,i) => ({ a: i+1 })));
console.log('inserted test data');
await app.listen(3000,'0.0.0.0');
console.log('App waiting');
} catch(e) {
console.error(e);
}
})();
routes.js
var router = require('express').Router();
router.get('/', async (req,res) => {
try {
let db = req.db,
collection = db.get('test');
let response = await collection.find();
res.json(response);
} catch(e) {
res.status(500).json(e);
}
});
module.exports = router;
So I am actually awaiting the database connection to at least be present on "start up" here, but really only for example since I want to insert some data to actually retrieve. It's not required, but the basic concept is to wait for the Promise to resolve:
await db.then(() => 1);
Kind of trivial, and not really required for your actual code. But I still think it's good practice.
The real test is done by stopping mongod or otherwise making the server unreachable and then issuing a request.
Since we set the connection options to { bufferMaxEntries: 0 } this means that immediately as you attempt to issue a command to the database, the failure will be returned if there is no actual connection present.
Of course when the database becomes available again, you won't get the error and the instructions will happen normally.
Without the option the default is to "en-queue" the operations until a connection is resolved and then the "buffer" is essentially "played".
You can simulate this ( as I did ) by "stopping" the mongod daemon and issuing requests. Then "starting" the daemon and issuing requests. It should simply return the caught error response.
NOTE: Not required, but in fact the whole purpose of async/await syntax is to make things like try..catch valid again, since you can actually scope as blocks rather than using Promise.catch() or err callback arguments to trap the errors. Same principles apply when either of those structures are actually in use though.

Node server not handling async post requests from client properly

We have a node server which doesn't handle post requests properly when they are made asynchronously. When the requests are made synchronously, it handles them fine.
There is a node api server and to mimic a client, there is node script which makes a post request to the server.
While making a single post request or post requests in a loop synchronously, everything works as expected.
While making asynchronous post requests in a loop to the server, the code doesn't execute properly.
Here is the code on the server side. This method is called from router.post() method.
async insert(params) {
let account = new Account();
try {
let totalLicenses = await this.getLicenses(params);
if (totalLicenses === 0) throw new AccountError('NO_AVAILABLE_LICENSE');
let accountResponse = await account.insert(params);
let useLicense = await license.use(accountResponse, params);
/*
Do other account setup stuff here
*/
return accountResponse;
} catch(err) {
throw err;
}
}
getLicenses(params) is a async function that prepares the sql query and awaits the response from a queryDb method which wraps the callback in a promise. Here is the code:
getLicense(params) {
let vals = [...arguments],
sql = 'my sql query';
try {
return await queryDb(sql, val);
} catch (err) {
throw new Error()
}
}
We are using mysql package to talk to db. This is what queryDb method looks like.
queryDb(query, vals) {
return new Promise( (resolve, reject) => {
connection.query(query, vals, (err, results, fields) => {
if (err) reject(err);
resolve(results);
}
}
}
While making asynchronous post requests in a loop to the server, the insert method above executes this.getLicenses(params) method for the requests and then calls the account.insert(params) for the requests and then license.use(accountResponse, params) for the requests.
This becomes a problem when let's say a client has 3 licenses available and they send 5 asynchronous post requests. Technically, it should throw error for the last 2 requests but that is not the case. What ends up happening is, it inserts all 5 accounts since it calls this.getLicenses(params) for all 5 requests before proceeding to insert the account.
Any help is appreciated!

Send response and continue to perform tasks Express | Node.js

In Node.js (which I'm new to) I am trying to perform a series of tasks after receiving a response. However, I want to make the response time as fast as possible. I don't need to return the results of these tasks to the client, so I'm trying to return the response immediately.
My current implementation is roughly:
var requestTime = Date.now;
app.post('/messages', function (req, res) {
console.log("received request");
// handle the response
var body = res.body;
res.send('Success');
res.end();
console.log("sent response");
performComplexTasks(body)
})
function performComplexTasks(body){
// perform data with body data here;
console.log("finished tasks:", Date.now()-requestTime, "ms");
}
// -------LOG-----------
// received request
// POST /api/messages 200 3.685 ms - 59
// sent response
// finished tasks: 2500ms
The client making the request seems to hang until performComplexTasks() is finished. (The POST finishes in 3.685ms, but the response takes 2500ms to finish.)
Is there a way to send the response immediately and complete other tasks without having the client wait/hang? (In my case, the client cannot make multiple API calls.)
If your job is not super-CPU-intense and you can tolerate some work on the main server thread, then just use await to break the execution so that the request can be properly sent. You can use setTimeout or await.
// This line is wrong too - ask a separate question if needed
var requestTime = Date.now;
app.post('/messages', async function (req, res) {
console.log("received request");
// handle the response
var body = res.body;
res.status(200).send({ success: true });
console.log("sent response");
// Method 1:
await performComplexTasks(body)
// Method 2:
setTimeout(() => performComplexTasks(body), 0);
})
async function performComplexTasks(body){
// The line below is required if using the `await` method -
// it breaks execution and allows your previous function to
// continue, otherwise we will only resume the other function after
// this function is completed.
await new Promise(resolve => setTimeout(resolve, 1));
// perform data with body data here;
console.log("finished tasks:", Date.now()-requestTime, "ms");
}
This isn't really a fantastic solution and you'd need to use worker threads for long operations.
Am I right that you're trying to execute a CPU-intensive job in performComplexTasks? If so, then event loop is being locked by that task and new requests are waiting until the job is finished.
It's a bad practice in node.js to execute such 'complex' tasks in the same process as http server. Consider using background workers, queues or something like that.
See this topic for details: Node.js and CPU intensive requests

Categories