We Keep Coding

Node.js res.render() with Request not working

I am trying to render an EJS template and pass in data to it with the node package Request. I have got this to work with no problem using node-fetch on my last project.
Here's a quick snippet of that:
const fetchCurrentWeather = (url, res) => {
fetch(url)
.then(res => res.json())
.then(data => res.render('pages/weather', {
currentId: data.list
}))
.catch(err => {
console.log(err);
res.sendStatus(500);
});
}
And this will render the EJS weather page template with the data from the API response.
With this latest project however, I am trying to use the Node Package Request to do the same thing and I am failing. Here's the code for that:
app.post('/', (req, res, next) => {
const paramOptions = {
url: 'https://api.foursquare.com/v2/venues/search',
method: 'GET',
qs: {
client_id: 'W033PDIYFI2TSAUO4L5ANUFOAUMZV32NUWNOF0NL0JS2E5W4',
client_secret: 'SM1ZI11XOMPVMEGMBZXSN3LGTLOBQCVGIM1SN4QAO0QTCSM1',
near: 'xxxxxx',
intent: 'browse',
radius: '15000',
query: 'pizza',
v: '20170801',
limit: 1
}
};
request(paramOptions, function(err, res, body) {
if(err) {
console.log(err)
} else {
res.render('pages/search'); //THIS WILL NOT WORK
console.log(body) // RETURNS DATA FROM API ENDPOINT
}
});
res.render('pages/search'); // THIS WILL WORK
});
The res.render() that does work is outside of the scope of the Request function, so I can not access the returned data from this Request function. When I console.log the body of the Request function, I do get JSON data returned in my terminal view, so I know Request is working, but I am unsure of how to pass this data into an EJS template as I did with my fetch() example.

There are two separate res parameters and the inner one is hiding the outer one. Change the name of the one in this line:
request(paramOptions, function(err, res, body) {
to something else like this:
request(paramOptions, function(err, response, body) {
And, then when you call res.render(), it will use the higher scoped one you want.
You probably also need to pass some data to res.render(filename, data) as the second argument so you can feed the results of the request() to your render operation.
And, in your if (err) condition, you should send an error status, perhaps res.sendStatus(500).

Nodejs respond after data has finished processing in request

I would like a response to differ depending on the finished request I recieve. I am sending a POST request and receive an xml file. The result is either a success or error. I use xml2json to convert the xml into a json object, then depending on the response I want to output json.
The problem is that I can't have a response inside a response. I also can't save the value of the callback for later usage (since its asynchronous).
I have thought about using Promises but I'm not sure. What should I do?
The order of operations should be
1) Send request
2) Get buffer response
3) Join Buffers. Process xml into JSON
4) Depending on the type of JSON entry, output either res.json('success') or res.json('error') if the xml responds with an error.
app.post('/api/submit', (req, res) => {
...
const request = https.request(options, (res) => {
let chunks = [];
res.on("data", function(chunk) {
chunks.push(chunk);
});
res.on("end", function(err) {
if (err) throw err;
let body = Buffer.concat(chunks);
xmlConverter(body, function(err, result) {
console.dir(result);
if (result.entry) {
console.log('✅ Success')
//Respond with json here --> res.json('success')
} else if (result.error) {
console.log('There was an error processing your request');
//or here if there was an error --> res.json('error')
}
});
});
});
request.end()

You can respond inside the callback. The problem is that you have two variable, both named res, so one shadows the other. You just need to change one of the res variable names so your not shadowing it. For example, you can change:
const request = https.request(options, (http_res) // <--change argument name
Then later:
if (result.entry) {
console.log('✅ Success')
http_res.json('success') // <-- use the response object from request
The problem of not being able to save the result for later is a different problem, but easy to solve. The solution though really depends one what you are trying to do. If, for example, you want to further process the data, you can set up a function to call and pass the response data in. Something like:
function process_data(response){
// use the response here
}
Then you can simply call it when you get the data:
if (result.entry) {
console.log('✅ Success')
http_res.json('success') // <-- use the response object from request
process_data(result)
Of course maybe your use case is more complicated but without more details its hard to give a specific answer.

Don't use the same name for both res, because they are different variables. And simply use the out res variable to respond the request with the value you want.
I think it would be something like this:
app.post('/
api/submit', (req, res) => {
...
const request = https.request(options, (resValue) => {
let chunks = [];
resValue.on("data", function(chunk) {
chunks.push(chunk);
});
resValue.on("end", function(err) {
if (err) throw err;
let body = Buffer.concat(chunks);
xmlConverter(body, function(err, result) {
console.dir(result);
if (result.entry) {
console.log('✅ Success')
res.json('success')
} else if (result.error) {
console.log('There was an error processing your request');
res.json('error')
}
});
});
});
request.end()

What exactly is the issue? You are perfectly able to rename the argument of the callback function supplied to https.request(options, callbackFunction) -- it is not important what this variable is named.
app.post('/api/submit', (req, res) => {
const request = https.request(options, (potato) => {
let chunks = [];
potato.on("data", function(chunk) {
chunks.push(chunk);
});
potato.on("end", function(err) {
if (err) throw err; // TODO res.status(500).json({}); ??
let body = Buffer.concat(chunks);
xmlConverter(body, function(err, result) {
console.dir(result);
if (result.entry) {
console.log('✅ Success')
res.status(200).json({});
} else if (result.error) {
console.log('There was an error processing your request');
res.status(500).json({});
}
request.end()
});
});
});
});

Socket.io-emit in callback gives SyntaxError: Unexpected end of JSON input

So I'm exporting a callback-function in a module like this:
(function() {
let request = require("request");
module.exports = function GithubApi(url, callback) {
let options = {
uri: url,
headers: {
"User-Agent": "Me",
"Content-Type": "application/x-www-form-urlencoded"
}
};
request(options, function(err, body) {
let context = {
issues: JSON.parse(body.body).map(function(issue) {
return {
title: issue.title,
comments: issue.comments,
};
})
};
callback(context) // The callback
});
};
}());
And this callback works perfectly fine when I'm using it in my GET-request with express.js:
app.get("/", (req, res) => {
let url = "some url";
GithubApi(url, (data) => {
res.render("../some-views", data);
});
});
But when I add a socket-emit, the callback-function returns SyntaxError: Unexpected end of JSON input
app.get("/", (req, res) => {
let url = "some url";
GithubApi(url, (data) => {
io.socket.emit("update", {message: data}); // adding this
res.render("../some-views", data);
});
});
Can't understand why the socket would interfere with the request and return an error with JSON. Can anybody help?

The probablem must be caused by the fact that body.body doesn't contain a valid JSON string.
When you run code like this:
JSON.parse(body.body)
you should always use try/catch because JSON.parse throws exceptions on bad JSON.
See those answers for more details:
Calling a JSON API with Node.js
Node JS ignores undefined check
How to extract inner string from the array string
Node server crashes in parsing JSON

So the problem was with the io.sockets.emit("update", {message: data});. For some reason, that interfered with the request(still don't know why tough). I guess it has something to do with the socket broadcasting to all channels, and that causes some kind of error, read something about it here.
So I changed the call to the callback-function to this:
GithubApi(orgs, repo, token, (data) => {
io.of("/").emit("update", {message: data}); // This line made it work
res.render("../views/home", data);
});

returning data to a middleware from a promise

i am new to nodejs, this might seem pretty trivial but i am having trouble retrieving data from a function that returns a promise. The response from the middleware is sent back to the front end. Here is my code
// middleware
app.get('/player', function(req, res) {
//data i want to return
res.send(getPlayerStats.getPlayerId(req.query.name)));
});
//getPlayerStats.js
var getPlayerId = function(name) {
return start(name)
.then(getPlayerInGame)
.then(getPlayerStats)
.then(getPlayers);
//.then(sendToSingular)
}
//getplayers function
var getPlayers = function(data) {
return data;
}
I'm i sending the data back the wrong way? The response i see at the front end is an object with prototype as the only property. I can print out the data in getPlayers() and i see it is working fine.

That's because you're passing the promise itself into res.send.
res.send(/* You are passing a promise here */);
What you should do is wait for the promise to resolve with the data and then send that data:
getPlayerStats.getPlayerId(req.query.name).then(function(data) {
res.send(data);
});

And I always recommend to finish your promise chain with a catch() to make sure errors are handled:
getPlayerStats.getPlayerId(req.query.name)
.then(function(data) {
res.send(data);
})
.catch(function(error){
res.status(500).send('Some error text');
});

When you write middleware it sounds like you want the player id to be available for other actions so in addition to the other comments here:
app.get('/player/*', function(req, res, next) {
//data i want to return
getPlayerStats.getPlayerId(req.query.name))).then(function(id){
res.locals.playerId = id;
next();
});
});
app.get('/player/action', function(req, res){
res.send(res.locals.playerId); //or use it for further processing
});

Node script needs to return success/failure to Angular service that called it

I'm trying to make a request to a node function that will write to a file, but I'm not sure the best way to get the success or failure back to angular. The setup looks like this:
//controller (in an ng-click handler)
writeFileRoute.writeFile(file)
.then(function(response){
console.log('success', response);
}, function(error){
console.log('error', error);
});
//service
app.service('WriteFileService', function($http) {
this.writeFile = function(data) {
return $http.post('/writeFile', data)
.then(function(response) {
return {
'success': response
};
}, function(response) {
return {
'error': response
};
});
};
})
//server.js
app.post('/writeFile', function(req, res){
components.writefile(req.body, function(err){
//do something with error here?
});
//by this point I have become increasingly confused by what is going on
//node write script
var writefile = function(data, callback){
//data = JSON.stringify(data).message;
fs.writeFile('./testFile.txt', data.message, function(err){
callback(err);
});
};
module.exports = exports = writefile;
So the file is actually writing. I just don't know how to combine these separately working components into something that can notify angular of success or failure. I thought about using q, however I don't know if I should use angular $q or node Q. I would really like to use Q and/or $q, but I don't know which is the right solution or where to plug them in.
Edit (3 Feb)
This is the code that I am currently working with:
Angular:
angular.module('test', [])
.controller('ctrl', function($scope, WriteService){
$scope.testMessage = "test"
$scope.writeTheFile = function(){
WriteService.write()
.then(function(err){
if (err){
console.log(err);
}else{
console.log('no error');
}
});
}
})
.service('WriteService', function($http){
this.write = function(){
$http.get('./write/', function(response){
return {'success': response};
}, function(error){
return {'error': error};
});
};
});
Node write module
var writes = {
nfcall: function(data){
data = data.repeat(5);
return Q.nfcall(fs.writeFile, dest, data);
}
}
Node Server
app.get('/write/', function(req, res, next){
var data = 'some text string ' + +new Date() + '\n';
writeModule.nfcall(data)
.then(function(response){
return {'response': response};
})
.catch(function(err){
return {'index.js error': err};
});
});

Promises are a very good alternative to callbacks, and Q is an excellent Promise library to use on your node.js backend:
var Q = require('q');
Your server will look like something like this:
app.post('/writeFile', function(req, res, next) {
components.writefile(req.body)
.then(function() {
// Success! 'Writefile' worked fine, just send some 200 OK response.
res.send('OK');
})
.catch(function(err) {
// If 'Writefile' fails, this will be called!
// Just pass the error to the Error Handling middleware
next(err);
});
});
var writefile = function(data){
return Q.nfcall(fs.writefile, './testFile.txt', data.message);
};
module.exports = exports = writefile;
Note that nfcall() method is used for calling Node.js-style functions and getting back a promise.
Your components.writefile now returns a Promise, and what you do in app.post could be two things (have a look at the Promise Core Methods):
Catching any errors if your promise is rejected (catch method is called)
Istead, if the promise was resolved, then method is called.
I don't see anything wrong on your fronted, in fact, it already uses Promises: note that $http returns a Promise (have a look at it's Documentation).

In app.post, you must return something back to the client through res. Use res.send(data or error)
See: http://expressjs.com/en/api.html
This also mean you do not need a promise from the server to the client. The promise for writeFileRoute will be resolved with the res return.