Is this async or blocking code? - javascript

Is this blocking code if I have JSON.stringify inside writeFile.Lets say it is big file. Is this why I am getting error conection refused when I push button multiple times in second which triggers this api endpoint?
app.patch('/', function(req, res) {
...some query
fs.writeFile(path.join(__dirname, "../../") + 'Data/deals.json', JSON.stringify(tickets), function (err) {
if (err) {
console.log(err);
return res.status(400).json({error: err});
}
return res.status(200).json({success: true, message: 'Deal Updated successfully'});
});
})

Edit #1:
JSON.stringify parsing object to string synchronously. but you missing that you called JSON.stringify inside the patch function and not inside the writeFIle.
Original:
It's Async. if you want to write a file synchronously you have to use writeFileSync.
For more information visit Node.js fs Docs
Example
// Async callback will called after finish
// Non-Blocking
fs.writeFile(file, data[, options], callback)
// write to file with return the result, may throw exception
// Blocking
fs.writeFileSync(file, data[, options])

Related

Error [ERR_HTTP_HEADERS_SENT]: Cannot set headers after they are sent to client [duplicate]

I'm facing this weird issue in NodeJS when using with Passport.js, Express and Mongoose. Basically, I get an error saying "Cannot set headers after they are sent to the client" even though I don't send more than one header.
I've read other posts and tried them out as well, and none of them worked.
app.get - is there any difference between res.send vs return res.send
Error [ERR_HTTP_HEADERS_SENT]: Cannot set headers after they are sent to the client
Cannot set headers after they are sent to the client
I've dug through github issues and I can't seem to find a solution. I get the problem that this error is triggered when I send multiple response headers, but the fact is that I am not sending multiple headers. It seems just weird.
This is my stack trace:
(node:9236) DeprecationWarning: current URL string parser is deprecated, and will be removed in a future version. To use the new parser, pass option { useNewUrlParser: true } to MongoClient.connect.
Server Running on port 5000
MongoDB Connected Error
[ERR_HTTP_HEADERS_SENT]: Cannot set headers after they are sent to the
client
at validateHeader (_http_outgoing.js:503:11)
at ServerResponse.setHeader (_http_outgoing.js:510:3)
at ServerResponse.header (/Users/lourdesroashan/code/github/devlog/node_modules/express/lib/response.js:767:10)
at ServerResponse.json (/Users/lourdesroashan/code/github/devlog/node_modules/express/lib/response.js:264:10)
at Profile.findOne.then.profile (/Users/lourdesroashan/code/github/devlog/routes/api/profile.js:27:30)
at <anonymous>
This is my server code:
router.get("/userprofile", passport.authenticate('jwt', { session: false }), (req, res) => {
Profile.findOne({ user: req.user.id }).then(profile => {
if (!profile) {
return res.status(404).json({ error: "No Profile Found" });
}
else {
res.json(profile);
}
}).catch(err => {
console.log(err);
})
});
I understand what the error means, but from what I know, I don't think I am sending multiple headers, I even checked by console.log that only one of the blocks is run.
Thank you so much in advance! :)
Full Code at: https://github.com/lourdesr/devlog
EDIT:
I figured it out. It was a problem in my passport.js while trying to get the authenticated user. I forgot to use 'return' on the 'done' method, which had caused it. Just added the return statement and it worked!
That particular error occurs whenever your code attempts to send more than one response to the same request. There are a number of different coding mistakes that can lead to this:
Improperly written asynchronous code that allows multiple branches to send a response.
Not returning from the request handler to stop further code in the request handler from running after you've sent a response.
Calling next() when you've already sent a response.
Improper logic branching that allows multiple code paths to execute attempt to send a response.
The code you show in your question does not appear like it would cause that error, but I do see code in a different route here that would cause that error.
Where you have this:
if (!user) {
errors.email = "User not found";
res.status(404).json({ errors });
}
You need to change it to:
if (!user) {
errors.email = "User not found";
res.status(404).json({ errors });
// stop further execution in this callback
return;
}
You don't want the code to continue after you've done res.status(404).json({ errors }); because it will then try to send another response.
In addition, everywhere you have this:
if (err) throw err;
inside an async callback, you need to replace that with something that actually sends an error response such as:
if (err) {
console.log(err);
res.sendStatus(500);
return;
}
throwing inside an async callback just goes back into the node.js event system and isn't thrown to anywhere that you can actually catch it. Further, it doesn't send a response to the http request. In otherwords, it doesn't really do what the server is supposed to do. So, do yourself a favor and never write that code in your server. When you get an error, send an error response.
Since it looks like you may be new here, I wanted to compliment you on including a link to your full source code at https://github.com/lourdesr/devlog because it's only by looking at that that I was able to see this place where the error is occuring.
I was receiving this error because of a foolish mistake on my part. I need to be more careful when referencing my other working code. The truly embarrassing part is how long I spent trying to figure out the cause of the error. Ouf!
Bad:
return res
.send(C.Status.OK)
.json({ item });
Good:
return res
.status(C.Status.OK)
.json({ item });
Use ctrl + F hotkey and find all 'res.' keywords
then replace them with 'return res.',
change all 'res.' to 'return res.'
something like this:
res.send() change to --> return res.send()
maybe you have 'res.' in some block, like if() statement
Sorry for the Late response,
As per the mongoose documentation "Mongoose queries are not promises. They have a .then() function for co and async/await as a convenience. However, unlike promises, calling a query's .then() can execute the query multiple time"
so to use promises
mongoose.Promise = global.Promise //To use the native js promises
Then
var promise = Profile.findOne({ user: req.user.id }).exec()
promise.then(function (profile){
if (!profile) {
throw new Error("User profile not found") //reject promise with error
}
return res.status(200).json(profile) //return user profile
}).catch(function (err){
console.log(err); //User profile not found
return res.status(404).json({ err.message }) //return your error msg
})
here is an nice article about switching out callbacks with promises in Mongoose
and this answer on mongooses promise rejection handling Mongoose right promise rejection handling
There is a simple fix for the node error [ERR_HTTP_HEADERS_SET]. You need to add a return statement in front of your responses to make sure your router exits correctly on error:
router.post("/", async (req, res) => {
let user = await User.findOne({email: req.body.email});
if (!user) **return** res.status(400).send("Wrong user");
});
Because of multiple response sending in your request. if you use return key word in your else condition your code will run properly
if (!profile) {
return res.status(404).json({ error: "No Profile Found" });
}
else {
**return** res.json(profile);
}
This also happen when you tries to send the multiple response for a same request !!
So make sure you always use return keyword to send response to client inorder to stop the further processing !!
Where you have this:
if (!user) { errors.email = "User not found"; res.status(404).json({ errors }); }
You need to change it to:
if (!user) { errors.email = "User not found"; return res.status(404).json({ errors }); }
I got the same error using express and mongoose with HBS template engine. I went to Expressjs and read the docs for res.render, and it says // if a callback is specified, the rendered HTML string has to be sent explicitly. So I wasnt originally sending my html explicitly in the callback,. This is only for a contact form btw, not login info, albeit GET
//Original
let { username, email } = req.query; //My get query data easier to read
res.status(200).render('index', { username, email });
//Solution without error. Second param sending data to views, Third param callback
res.status(200).render('index', { username, email }, (err, html)=>{
res.send(html);
});
In react, if your are calling the function in useEffect hook, make sure to add a dependency to the dependency Array.
I had this error from an if statement not having an else block.
if(someCondition) {
await () => { }
}
await () => { }
I changed the above to this below and solved my issue
if(someCondition) {
await () => { }
} else {
await () => { }
}
For me, I accidentally put a res.status inside of a for loop. So my server would trigger the error the second time a res.status was returned. I needed to put the res.status outside of the for loop so it would only trigger once within the function.
First of all : make sure you didn't miss any asynchronous action without an async/await or use promises/callbacks.
Then attach any res with the return keyword : return res.status(..).json({});
And finally which was my problem: don't use return res.sendStatus if you always have some return res... inside a callback function, but you can always do a retun res.status();
in my case it was :
users.save((err,savedDoc){
if(err) return res.status(501).json({})
res.status(200).json({});
});
return res.status(500); // instead ofdoing return res.sendStatus(500)
you have to enable Promises in your programm, in my case i enabled it in my mongoose schema by using mongoose.Promise = global.Promise .
This enables using native js promises.
other alternatives to this soloution is :
var mongoose = require('mongoose');
// set Promise provider to bluebird
mongoose.Promise = require('bluebird');
and
// q
mongoose.Promise = require('q').Promise;
but you need to install these packages first.
My problem besides not returning, i was forgetting to await an asynchronous function in the handler. So handler was returning and after a bit the async function did its thing. 🤦🏻‍♀️
Before:
req.session.set('x', {...});
req.session.save();
return req.status(200).end();
When i needed to await:
req.session.set('x', {...});
await req.session.save();
return req.status(200).end();
I'm putting this here for anyone else who has the same problem as me- this happened to me because I'm using the next() function without a return preceding it. Just like a lot of the other answers state, not using return with your response will / can cause / allow other code in the function to execute. In my case, I had this:
app.get("/customerDetails", async (req, res, next) => {
// check that our custom header from the app is present
if (req.get('App-Source') !== 'A Customer Header') next();
var customerID = req.query.CustomerID
var rows = await get_customer_details(customerID);
return res.json(rows);
});
In my case, I forgot to include the header in my request, so the conditional statement failed and next() was called. Another middleware function must have then been executed. After the middleware finishes, without a return, the rest of the code in the original middleware function is then executed. So I simply added a return before my next() call:
// serve customer details payload
app.get("/customerDetails", async (req, res, next) => {
// check that our custom header from the app is present
if (req.get('App-Source') !== 'A Customer Header') return next();
var customerID = req.query.CustomerID
var rows = await get_customer_details(customerID);
return res.json(rows);
});

Node: Wait for python script to run

I have the following code. Where i upload the file first and then i read the file and console the output like console.log(obj). But the response comes first and the python scripts runs behind the scene. How can i make code to wait for the python script to run then proceed?
router.post(`${basePath}/file`, (req, res) => {
//Upload file first
PythonShell.run('calculations.py', { scriptPath: '/Path/to/python/script' }, function (err) {
console.log(err);
let obj = fs.readFileSync('Path/to/file', 'utf8');
console.log(obj);
});
return res.status(200).send({
message : 'Success',
});
});
I cannot get console.log(obj); output because it runs after the response. How can i make it wait for the python script to run and get console.log(obj) output on console.
To return the result after some async operation, you should call res.send inside the done-callback.
router.post(`${basePath}/file`, (req, res) => {
//Upload file first
PythonShell.run('calculations.py', { scriptPath: '/Path/to/python/script' }, function (err) {
console.log('The script work has been finished.'); // (*)
if(err) {
res.status(500).send({
error: err,
});
console.log(err);
return;
}
let obj = fs.readFileSync('Path/to/file', 'utf8');
console.log(obj); // (**)
res.status(200).send({
message : 'Success',
});
});
});
Then if you will not see the log (*) in the console, then it would mean that the script does not work or works improperly. The callback is not being called. First of all, you need to be sure that the script (PythonShell.run) works and the callback is being called. The POST handler will wait until you call res.send (with no matter of delay value), so that callback is the main point.
Also readFileSync could fail. In case of readFileSync failure you should see an exception. If it's ok then you'll see the next log (**) and the response will be sent.
I see PythonShell in your code. I have no experience with it, but after some reading I think that the problem could be in how you are using it. It seems the python-shell npm package, so following it's documentation you may try to to instantiate a python shell for your script and then to use listeners:
let pyshell = new PythonShell('calculations.py');
router.post(`${basePath}/file`, (req, res) => {
pyshell.send(settings); // path, args etc
pyshell.end(function (err) {
console.log('The script work has been finished.');
if(err) { res.status(200).send({ error: err }); }
else { res.status(200).send({ message : 'Success' }); }
});
});
This approach could be more appropriate because the pyton shell is kept open between different POST requests. This depends on your needs. But I guess it does not solve the problem of script running. If you are sure that the script itself is fine, then you need just to run it properly in the Node environment. There are some points:
path to script
arguments
other settings
Try to remove all arguments (create some new test script), cleanup settings object (keep only path) and execute it from Node. Handle its result in Node. You should be able to run the simplest script by correct path! Research how to setup correct scriptPath. Then add an argument to your script and run it with an argument. Hanlde the result again. There are not so many options, but each of them could be the cause of improper call.

NodeJS - stat doesn't always run

Here's my code:
console.log("Path 1: " + fullName);
fs.stat(fullName, function(err, stats) {
console.log("Path 2: " + fullName);
}, function(err) { // I don't know if this part actually does something
console.log("An error occurred: " + err); // but I saw it in a different SO answer
});
The code simply doesn't run for some files. As in, my logs will show Path 1 with the file but not path 2 with the file, and also none of the "an error occurred". I was thinking maybe the files have an invalid character, because they all have equal signs in them. They look like this:
...file.device_type=mobile.jsx
Even if that's the case, why no error or anything? And if so, how can I actually stat these files?
You aren't logging or checking for an error. fs.stat() accepts two parameters only, the filename and a callback function. You are passing three parameters, the filename and two separate callbacks. So that second callback is just wrong. Then, in the first callback, you need to check the err variable to see if an error occurred.
This is the correct usage:
fs.stat(fullName, function(err, stats) {
if (err) {
console.log("Error in fs.stat(): ", err);
} else {
console.log("Got stats: ", stats);
}
});
If you're using this proper form and you still don't see either message in the console, then I'd suggest putting an exception handler around it to see if something else is going on:
try {
console.log("about to call fs.stat()");
fs.stat(fullName, function(err, stats) {
if (err) {
console.log("Error in fs.stat(): ", err);
} else {
console.log("Got stats: ", stats);
}
});
} catch(e) {
console.log("fs.stat() exception: ", e);
}
In looking at the source code for fs.stat(), there are several ways it could throw a synchronous exception, particularly if it detects invalid arguments passed to it. As usual, the node.js documentation does not describe that behavior, but you can see it in the code. This is the code for fs.stat():
fs.stat = function(path, callback) {
callback = makeStatsCallback(callback);
if (handleError((path = getPathFromURL(path)), callback))
return;
if (!nullCheck(path, callback)) return;
var req = new FSReqWrap();
req.oncomplete = callback;
binding.stat(pathModule._makeLong(path), req);
};
Both makeStatsCallback() and handleError() can throw an exception (when you look at their implementations in that same file).
I do not know where you got the notion of passing two callbacks to fs.stat(). As documented here, it does not accept two callback functions. It looks remotely like a promisified version of the fs library where every async operation returns a promise and then you can pass two callbacks to fs.statPromise.then(fn1, fn2), but I have no idea if that's where you saw this or not.
https://nodejs.org/api/fs.html#fs_fs_stat_path_callback
As per the documentation it shouldn't have the 3rd param function

Meteor.call does not wait for the result

The Async callback in Meteor.call does not wait for the result from the Meteor.method.This is the code.
Meteor.call("fetchData",function (err,res) {
if (err){
console.log("error ", err);
}else {
console.log("success ", res);
return res;
}
});//calling this from onRendered of client/somejs.js
Here is the method
fetchData :function(){
HTTP.call("POST","http://localhost:8080",{
data:'{"apple":"grape"}'
},function (err,res) {
if (err){
console.log("error ", err);
}else {
console.log("success ", res);
return res;
}
})
}//Server/methods.js
When Meteor.call is triggered,i get a log on the Server as success with its result.
On the client i get success undefined .
The call on the client does not wait for the result.Also i tried Fibers and Synchronous execution on the server.It does not work for me.In this case a publish is blocked(i guess due to the API call).
Another thing is that i tried the same with a DB query instead of API call.That works fine.I get the result from the Method.
Where am i going wrong.Help.
Thanks
Sanjith.
You were on the right path with futures. By default, Meteor's methods are async, so some "waiting" mechanism is needed on the client. For this, I'd recommend either using Meteor.wrapAsync or Promises. Here are two detailed explanations on implementing both:
https://themeteorchef.com/snippets/synchronous-methods/#tmc-using-wrapasync
https://themeteorchef.com/snippets/promise-based-modules/#tmc-calling-our-promise-based-module-from-the-client
The second link is more focused on structuring your code using promises, but gives a nice demo of how to call a method that relies on a Promise's response.

Express | Making multiple http requests asynchronously

I am currently building a small node application that makes a few api calls and renders a webpage with charts on it. I'm using express and jade as the render engine.
The problem is that I'm quite new to javascript and I don't know how to scheme out my http requests so I can pass an object of variables I got from the api (http get) when there is more than one request. I don't know how to map it out to make a single object and send it to the jade rendering engine.
Here is what I have so far :
app.get('/test', function(req, res) {
apiRequestGoesHere(name, function(error, profile) {
//Get some data here
});
anotherApiRequest(tvshow, function(error, list) {
//Get some data here
});
res.render('test', data);
});
As it is right now, the page renders and the requests are not done yet, and if I place res.render inside one of the request, I can't access the other's data.
So what I want is a way to set it up so I can have multiple api calls, then make an object out of some elements of what is returned to me from the rest api and send it to Jade so I can use the data on the page.
You probably want to use async to help with this. async.parallel is a good choice for something simple like this:
app.get('/test', function(req, res) {
async.parallel([
function(next) {
apiRequestGoesHere(name, function(error, profile) {
//Get some data here
next(null, firstData);
});
},
function(next) {
anotherApiRequest(tvshow, function(error, list) {
//Get some data here
next(null, secondData);
});
}], function(err, results) {
// results is [firstData, secondData]
res.render('test', ...);
});
});
The first argument to those functions next should be an error if relevant (I put null) - as soon as one is called with an error, the final function will be called with that same error and the rest of the callbacks will be ignored.
You can async parallel.
async.parallel([
function(callback){
// Make http requests
// Invoke callback(err, result) after http request success or failure
},
function(callback){
// Make http requests
// Invoke callback(err, result) after http request success or failure
}
],
// optional callback
function(err, results){
// the results array will be array of result from the callback
});
The reason your page renders is the callbacks haven't "called back" yet. To do what you want, you would need to do something like:
app.get('/test', function(req, res) {
apiRequestGoesHere(name, function(error, profile) {
//Get some data here
anotherApiRequest(tvshow, function(error, list) {
//Get some data here
res.render('test', data);
});
});
});
This strategy leads to what is known as "pyramid code" because your nested callback functions end up deeper and deeper.
I would also recommend the step library by Tim Caswell. It would make your code look something like:
var step = require('step');
app.get('/test', function(req, res) {
step(
function () {
apiRequestGoesHere(name, this)
},
function (error, profile) {
if error throw error;
anotherApiRequest(tvshow, this)
},
function done(error, list) {
if error throw error;
res.render('test', list)
}
)
});
You could also use the group method to make the calls in parallel and still maintain the sequence of your callbacks.
Gl,
Aaron

Categories