I have a request that needs to be made upon app initialization in which part of the JSON response is parsed into JS and has to be written to a file – the rest of the response also needs to be received by the front end. Currently I have it setup like this:
function getAds(req, res) {
console.log('GET Advertisements');
let url = 'http://example.com/json/advertisements/v5/all';
request(url, (err, response, body) => {
let advertisements = JSON.parse(body);
let script = advertisements.header;
fs.writeFile('client/ads/dfpAds.js', script, function(err) {
if (err) {
console.error(err);
}
});
res.send(advertisements);
});
}
This successfully writes to the JavaScript file, but it does not send the response to my front end and also blocks all other HTTP requests I'm making in other controllers.
Is it possible to do this? Do I need to separate the request that will handle the WriteFile and the other that will be sent to the front end?
I've tried a variety of combinations but I can never get the WriteFile to not block all other HTTP requests regardless. Any help is very much appreciated.
Related
I want upload the content of an excel file into the server in order to get its data and do some stuff...
I came up with the following code, however it seems like it is not working properly as the following error displays in the console Error: Can't set headers after they are sent.
The file is getting uploaded into the folder and the json message is being displayed... However I do not know if I am going to face any issue in the future...
Actually I just need the excel data no need for the excel being uploaded... Maybe you could give me a workaround, guys...
const router = express.Router();
const storage = multer.diskStorage({
destination(req, file, cb) {
cb(null, 'uploads/');
},
filename(req, file, cb) {
cb(
null,
`${file.fieldname}-${Date.now()}${path
.extname(file.originalname)
.toLowerCase()}`
);
},
});
const excelFilter = (req, file, cb) => {
if (
file.mimetype.includes('excel') ||
file.mimetype.includes('spreadsheetml')
) {
cb(null, true);
} else {
cb('Please upload only excel file.', false);
}
};
const upload = multer({
storage,
fileFilter: excelFilter,
});
router.post('/', upload.single('file'), (req, res) => {
var workbook = XLSX.readFile(req.file.path);
var sheet_name_list = workbook.SheetNames;
var xlData = XLSX.utils.sheet_to_json(workbook.Sheets[sheet_name_list[0]]);
res.json(xlData).sendFile(`/${req.file.path}`, { root: path.resolve() });
});
May I have a res.json and res.sendFile together in the same api endpoint in express?
No, you cannot. Each of those methods, sends a complete http response (including calling res.end() which terminates the http request) and you can only send one http response to each incoming request. The particular error you're getting has to do with the res.sendFile() trying to configure the response that it's getting ready to send and finding that the http response object has already been used for sending a response and can't be used again.
Ordinarily, if you wanted to sent two different pieces of data, you would just combine them into a single Javascript object and just call res.json() on the object that contains both pieces of data.
But, sending a binary file is not something you can easily put in a JSON package. You could construct a multipart response where one part was the JSON and one part was the file. You could JSON encode binary data (though that's inefficient). I presume there are probably some modules that would help you do that, but for most clients, that isn't what they are really expecting or equipped to handle.
The only way to a proper solution is for us to understand what client/server workflow you're trying to implement here and why you're trying to send back the same file that was just uploaded. There would normally not be a reason to do that since the client already has that data (they just uploaded it).
I am currently sending a request from NodeJS to get some data, as such :
In Angular :
$http.post(API_ENDPOINT.url + '/annonce', {'link': url}).then(function (result) {
...
}
And in Node :
apiRoutes.post('/annonce', function (req, res) {
const url = req.body.link
request.get({
uri: url,
encoding: null
}, function (error, response, html) {
if (!error) {
res.send(parser(url, html))
}
})
return res
})
I would like to send this request from my front-end (Angular). I guess I could simply do the request like this :
$http.get(url).then(function(result)) {
// send another post request to the back end with the result
}
but I've heard it was easier to use pipes in this case.
Thing is, I really don't understand how to make it work. Can anyone help ?
If the page you are trying to retrieve does not have the CORS headers then angular will fail to make the request.
If you expand on your usecase it should be easier to get you some help.
I'm new to node.js so I'll try my best to explain the problem here. Let me know if any clerification is needed.
In my node.js application I'm trying to take a code (which was received from the response of the 1st call to an API), and use that a code to make a 2nd request(GET request) to another API service. The callback url of the 1st call is /pass. However I got an empty response from the service for this 2nd call.
My understanding is that after the call back from the 1st call, the function in app.get('/pass', function (req, res).. gets invoked and it sends a GET request. What am I doing wrong here? Many thanks in advance!
Here is the part where I try to make a GET request from node.js server and receive an empty response:
app.get('/pass', function (req, res){
var options = {
url: 'https://the url that I make GET request to',
method: 'GET',
headers: {
'authorization_code': code,
'Customer-Id':'someID',
'Customer-Secret':'somePassword'
}
};
request(options, function(err, res, body) {
console.log(res);
});
});
Im a little confused by what you are asking so ill just try to cover what i think you're looking for.
app.get('/pass', (req, res) => {
res.send("hello!"); // localhost:port/pass will return hello
})
Now, if you are trying to call a get request from the request library when the /pass endpoint is called things are still similar. First, i think you can remove the 'method' : 'GET' keys and values as they are not necessary. Now the code will be mostly the same as before except for the response.
app.get('/pass', (req, res) => {
var options = {
url: 'https://the url that I make GET request to',
headers: {
'authorization_code': code,
'Customer-Id':'someID',
'Customer-Secret':'somePassword'
}
};
request(options, function(err, res, body) {
// may need to JSONparse the body before sending depending on what is to be expected.
res.send(body); // this sends the data back
});
});
I should preface my post by saying that I am a beginner and this is my first time using Node.js and Express in a real project.
I have a simple Node.js/Express project and I want to read a JSON object from a URL. Afterwards, I intend to build another url that displays html from an external website using iframe.
I read about the 'request' module online and know that I need to do something along these lines:
var express = require('express');
var router = express.Router();
var request = require('request');
// Urls for App Center REST functions
var url = 'https://someserver.com/appserver/portal/api/1.0/results/recent';
/* GET list of recent reports */
router.get('/testapi', function(req, res, next) {
res.render('testapi', { title: 'List Recent Reports' });
});
/* TEST: function to GET report list */
router.get('/recentreports', function(req, res){
request({
url: url,
json: true
}, function (error, response, body) {
if (!error && response.statusCode === 200) {
console.log(body) // Print the json response
}
})
});
I have tried to define a function /recentreports which is called in the testapi.jade view, however nothing is printed in the console when I load the page and I suspect I am doing something horribly wrong.
My questions are:
How do I read the JSON into my app and where does this code go (index.js, app.js, testview.jade etc...?)
How do I export the URL I construct from wherever that code lives to my .jade view?
There was nothing logged to the browser console because no response was sent from your server. The response was only logged to the server's console.
You'll need to refactor the code for the 'recentreports' route to send data. You could use a simple res.send call:
...
function (error, response, body) {
if (!error && response.statusCode === 200) {
res.send(body) // Send the response to the client
}
}
...
This response will be received by testapi.jade via an AJAX call to the '/recentreports' route. The AJAX call can be defined in a Javascript file sourced by the testapi.jade file.
The constructed URL would not need to be exported as it exists within the same testapi.jade file (after you've formed it from the results from the AJAX call).
I might be out of depth but I really need something to work. I think a write/read stream will solve both my issues but I dont quite understand the syntax or whats required for it to work.
I read the stream handbook and thought i understood some of the basics but when I try to apply it to my situation, it seems to break down.
Currently I have this as the crux of my information.
function readDataTop (x) {
console.log("Read "+x[6]+" and Sent Cached Top Half");
jf.readFile( "loadedreports/top"+x[6], 'utf8', function (err, data) {
resT = data
});
};
Im using Jsonfile plugin for node which basically shortens the fs.write and makes it easier to write instead of constantly writing catch and try blocks for the fs.write and read.
Anyways, I want to implement a stream here but I am unsure of what would happen to my express end and how the object will be received.
I assume since its a stream express wont do anything to the object until it receives it? Or would I have to write a callback to also make sure when my function is called, the stream is complete before express sends the object off to fullfill the ajax request?
app.get('/:report/top', function(req, res) {
readDataTop(global[req.params.report]);
res.header("Content-Type", "application/json; charset=utf-8");
res.header("Cache-Control", "max-age=3600");
res.json(resT);
resT = 0;
});
I am hoping if I change the read part to a stream it will allievate two problems. The issue of sometimes receiving impartial json files when the browser makes the ajax call due to the read speed of larger json objects. (This might be the callback issue i need to solve but a stream should make it more consistent).
Then secondly when I load this node app, it needs to run 30+ write files while it gets the data from my DB. The goal was to disconnect the browser from the db side so node acts as the db by reading and writing. This due to an old SQL server that is being bombarded by a lot of requests already (stale data isnt an issue).
Any help on the syntax here?
Is there a tutorial I can see in code of someone piping an response into a write stream? (the mssql node I use puts the SQL response into an object and I need in JSON format).
function getDataTop (x) {
var connection = new sql.Connection(config, function(err) {
var request = new sql.Request(connection);
request.query(x[0], function(err, topres) {
jf.writeFile( "loadedreports/top"+x[6], topres, function(err) {
if(err) {
console.log(err);
} else {
console.log(x[6]+" top half was saved!");
}
});
});
});
};
Your problem is that you're not waiting for the file to load before sending the response. Use a callback:
function readDataTop(x, cb) {
console.log('Read ' + x[6] + ' and Sent Cached Top Half');
jf.readFile('loadedreports/top' + x[6], 'utf8', cb);
};
// ...
app.get('/:report/top', function(req, res) {
// you should really avoid using globals like this ...
readDataTop(global[req.params.report], function(err, obj) {
// setting the content-type is automatically done by `res.json()`
// cache the data here in-memory if you need to and check for its existence
// before `readDataTop`
res.header('Cache-Control', 'max-age=3600');
res.json(obj);
});
});