Node.js Express: loading JSON file only on server startup - javascript

I'm working on an express.js app which should load an API definition file (most likely the swagger file in JSON format). Currently I've created a middleware, which should parse the JSON file via fs.readFile() and JSON.parse() in order to check user's permissions on accessing some resource. So basically each time the request is performed, my middleware gets the same JSON file and parses it, which is obviously a piece of extra work. Is it possible to load this JSON file, parse and store it to some internal object in a sort of global configuration and reload it in case it was modified so as not to perform the same operation on each request?

Of course, you could create a function like this (pseudo code):
var jsonData=null;
function getConfiguration() {
if (!jsonData) {
jsonData= readFileSync(...);
}
return jsonData;
}
module.exports.getConfiguration=getConfiguration;
Or, as #AleksandrM commented, you can just "import" it using require:

This is what worked for me, very much aligned with the answer here
let config_file = path.join(__dirname, 'config.json')
let configJSON = {}
function getConfiguration() {
jsonfile.readFile(config_file, function (err, obj) {
if (err) console.error(err)
console.dir(obj)
configJSON = obj
})}
Then on app startup
app.listen(port, () => {
getConfiguration()
console.log(`App listening at http://localhost:${port}`)
})

Related

Javascript object is devoid of property after axios post request

I am making a web app with VueJS that makes axios calls to a NodeJS API that uses express. I'm currently trying to send files to my NodeJS so it can save them. However, even though my browser displays all the properties of my array of files, my NodeJS reads it as empty. I've read all the questions previously asked on this topic but I haven't made any progress whatsoever. Note that I can POST any other data just fine, all my SQL Insert requests do well, mind you they don't involve sending arrays.
Example of promising solution that did not work : https://stackoverflow.com/a/66541165/12498040
Here is my client side JS, console.log() prints an array of N files object in the browser console :
console.log(data)
await axios.post('/api/fichiers', {fichiers:data});
Here is my NodeJS, with console.log(element) it prints N number of "{}" in the browser console and console.log(element.name) prints N number of "undefined" in the server terminal :
app.post('/api/fichiers', (req, res) => {
req.body.fichiers.forEach((element) => {
console.log(element.name);
});
});
Thank you for any help you could provide :)
You are getting an array of empty objects because File objects are not serializable to JSON.
If you want to send files, you need to replace the object containing a plain array you are passing to axios (which it will try to serialize to JSON) with a FormData object.
Then you need body parsing middleware in your Express application which can handle the multipart/form-data request body that a FormData object will generate. The body-parser module can't do that, but its documentation links to a number of options.
Thank you !! It's now working as intended. Here is how I made it if anyone is interested :
Client-side JS :
var formData = new FormData();
for (let i = 0; i < data.length; i++) {
formData.append("files", data[i]);
}
await axios.post('/api/fichiers', formData);
NodeJS :
const multer = require('multer')
var storage = multer.diskStorage(
{
destination: function (req, file, cb) {
cb(null, './tempDir/')
},
filename: function (req, file, cb) {
cb(null, Date.now() + '_' + file.originalname);
}
}
);
var upload = multer({ storage: storage });
app.post('/api/fichiers', upload.array('files'));
I then installed npm fs.extra to move the files to the desired directory

How to write correct app.get()? (node.js / express / boby-parser)

I create server (node.js / express / boby-parser)
and I need to get array of objects 'users'.
its part of code from server.js file:
let users = [{
name: 'user1',
}];
app.get('/users/', (req, res) => {
const filePath = path.join(pth.dir, 'build', 'index.html');
res.json(users);
res.sendFile(filePath);
});
Its my code from frontend:
const handleResponse = (response) => {
return response.text().then(text => {
const data = text && JSON.parse(text);
if (!response.ok) {
const error = (data && data.message) || response.statusText;
return Promise.reject(error);
}
return data;
});
};
const getAll = (baseUrl) => {
const requestOptions = {
method: 'GET'
};
return fetch(baseUrl, requestOptions).then(handleResponse);
};
Something wrong with my code at server. (I just dodnt know how to use express server).
when I use getAll function I got JSON text replace my page. Can anyone help? How should I write app.get() in server.js. Or do I need write in server part one app.get() to get page or another app.get() to get JSON data?
Why are you trying to send a file in the response?:
res.sendFile(filePath);
For starters, the response content can either be JSON or a file (or any of a variety of other things of course), but not both. With data like JSON or XML it's possible to combine multiple objects into one larger object for a single response, but this won't work if the content types are entirely different as it is with a file.
Looking at your client-side code, you're not even doing anything with that file. You only read the JSON data:
return response.text().then(text => {
const data = text && JSON.parse(text);
if (!response.ok) {
const error = (data && data.message) || response.statusText;
return Promise.reject(error);
}
return data;
});
So the simplest approach here would just be to not try to send back the file:
app.get('/users/', (req, res) => {
res.json(users);
});
Edit: Based on comments below, you seem to be struggling with the different requests the client makes to the server. The browser loading the page is one request with one response. If that page includes JavaScript that needs to fetch data, that would be a separate AJAX request with its own response containing just that data.
It’s possible to use JSON (or any data) server-side to populate a page template and return a whole page with the data. For that you’d need to use (or build) some kind of templating engine in the server-side code to populate the page before returning it.
The res.json() represents the HTTP response that an Express app sends when it gets an HTTP request. On the other hand, res.sendFile() transfers the file at the given path.
In both cases, the flow is essentially transferred to client who might have made the request.
So no, you cannot use res.sendFile and res.json together.
var options = {
headers: {
'name': 'user1',
}
};
res.sendFile(path.join(__dirname, 'build', 'index.html'), options);
Thats really the closest you can do to achieve the desired task.

Send data with Javascript function in Express response

I want to send an application/javascript response from my Express server, passing the data which I have got from MongoDB.
This is a response to a call for loading some in a third party website.
I have created all the different parts of the process, now just need to pass on the data into the Javascript response.
server.js
app.get('/', (req, res) => {
productInfo(param1, param2, res)
}
productInfo.js - MongoDB call
function productInfo(param1, param2, res){
Product.find({key1: param1}, (err, docs) => {
let idList = docs.idList;
res.set('Content-Type', 'application/javascript');
res.sendFile(__dirname + '/script.js', (err) => {
if (err) { console.log(err) }
else { console.log('file sent') }
});
}
module.exports = productInfo;
script.js - sending a self executing anonymous function
(function(){
// function - load jQuery & Bootstrap in 3rd party website
$masterDiv = $(`
<div>
...
... *data required*
</div>
`)
$('body').append($masterDiv);
// function - jquery event handlers where *data is required*
})();
When some event happens on the third party website page, the event handlers update the right data (id).
How do I pass along data (idList) to script.js?
If I set dummy global variables data before the (function(){})(); line in script.js then I can access it within the function.
I tried res.render but it says "Cannot find module 'js'".
res.render(__dirname + '/scriptproduct.js', (err) => {});
Can I somehow set params to script.js function and call the function with res.send(functionName(idList))?
I have seen answers with templates being sent in html views with res.render but how do I use such a solution in my case where the data is required both in JS and HTML?
I have lots of other routes which are not using a template engine. Can I use it for just one route if that is the solution?
I am very new to all this and basically hacking forward to a solution. So some of my questions above might be elementary.
Using ejs you can pass a string to a EJS template or a .js file. However, you can only pass strings. What you could do is pass the object as a string using JSON.stringify(obj) and then use JSON.parse(obj) to convert it back.
Here's another answer that has some code and may help: How to include external .js file to ejs Node template page

express node server return a value to async client call

Sorry, I tend to be a bad writer when I have not fully woken up, let me revise.
I am using expressjs with passportjs (local strategy) to manage my server and using connect-busboy to manage file uploading. I do not think passport will play a role in this.
Here is the server code for managing file uploads:
app.post('/upload', isLoggedIn, (req, res) => {
if(req.busboy){
req.pipe(req.busboy);
req.busboy.on('file', (fieldname, file, filename, encoding, mimetype) => {
if(mimetype.match(/^image\//)){
var root = path.join(__dirname, "../public/images/");
if(fs.existsSync(path.join(root, filename))){
var name = getUnique(path.join(root, filename));
} else {
var name = filename;
}
var ws = fs.createWriteStream(path.join(root, name), { flags: "a" });
file.pipe(ws);
}
});
}
});
As for my client page, it is used to change a JSON object which will get re-uploaded to the server as a configuration tool. When I upload a new image asynchronously I need to get the filename to update this JSON object while working on it. For uploading from the clients end I am using dropzonejs, which did not require any configuration on my part to work.
So, in summary I upload a number of images via dropzone asynchronously, busboy and fs on my server save the file, and I would like to get the filename returned to my javascript to modify the existing JSON object.
Edit solution:
Thanks to Elliot Blackburn for pointing me in the right direction.
By calling:
ws.on('close', () => {
res.send({filename: name});
});
after file.pipe(ws); to send the response back to the client. On the client side modify dropzone to handle the response like so:
dropzone.on('success', (file, res) => {
console.log(res);
});
Just send it in the normal http response. It'll depend what library you're using but most will allow you to trigger a normal req, res, next express call. From that you can access the file object, and return anything you want.
Something like:
req.send({filename: name}); // name is the filename var set earlier in the code.
Once you've finished editing the file and such, you can get the name and put it into that returned object and your client will receive that as object as the response which you can act upon.

writestream and express for json object?

I might be out of depth but I really need something to work. I think a write/read stream will solve both my issues but I dont quite understand the syntax or whats required for it to work.
I read the stream handbook and thought i understood some of the basics but when I try to apply it to my situation, it seems to break down.
Currently I have this as the crux of my information.
function readDataTop (x) {
console.log("Read "+x[6]+" and Sent Cached Top Half");
jf.readFile( "loadedreports/top"+x[6], 'utf8', function (err, data) {
resT = data
});
};
Im using Jsonfile plugin for node which basically shortens the fs.write and makes it easier to write instead of constantly writing catch and try blocks for the fs.write and read.
Anyways, I want to implement a stream here but I am unsure of what would happen to my express end and how the object will be received.
I assume since its a stream express wont do anything to the object until it receives it? Or would I have to write a callback to also make sure when my function is called, the stream is complete before express sends the object off to fullfill the ajax request?
app.get('/:report/top', function(req, res) {
readDataTop(global[req.params.report]);
res.header("Content-Type", "application/json; charset=utf-8");
res.header("Cache-Control", "max-age=3600");
res.json(resT);
resT = 0;
});
I am hoping if I change the read part to a stream it will allievate two problems. The issue of sometimes receiving impartial json files when the browser makes the ajax call due to the read speed of larger json objects. (This might be the callback issue i need to solve but a stream should make it more consistent).
Then secondly when I load this node app, it needs to run 30+ write files while it gets the data from my DB. The goal was to disconnect the browser from the db side so node acts as the db by reading and writing. This due to an old SQL server that is being bombarded by a lot of requests already (stale data isnt an issue).
Any help on the syntax here?
Is there a tutorial I can see in code of someone piping an response into a write stream? (the mssql node I use puts the SQL response into an object and I need in JSON format).
function getDataTop (x) {
var connection = new sql.Connection(config, function(err) {
var request = new sql.Request(connection);
request.query(x[0], function(err, topres) {
jf.writeFile( "loadedreports/top"+x[6], topres, function(err) {
if(err) {
console.log(err);
} else {
console.log(x[6]+" top half was saved!");
}
});
});
});
};
Your problem is that you're not waiting for the file to load before sending the response. Use a callback:
function readDataTop(x, cb) {
console.log('Read ' + x[6] + ' and Sent Cached Top Half');
jf.readFile('loadedreports/top' + x[6], 'utf8', cb);
};
// ...
app.get('/:report/top', function(req, res) {
// you should really avoid using globals like this ...
readDataTop(global[req.params.report], function(err, obj) {
// setting the content-type is automatically done by `res.json()`
// cache the data here in-memory if you need to and check for its existence
// before `readDataTop`
res.header('Cache-Control', 'max-age=3600');
res.json(obj);
});
});

Categories