this is my first question here. I need some help with my code structure. My api in node with express have to do the following:
receipt GET /api/file/{filename}
return the file content, its could be a big one (a few GB).
For now, I could get the files with streams, but I don't the best practice to handle error in this case.
'use strict';
const fs = require('fs');
const express = require('express');
const app = express();
const path = require('path');
const filePath = path.join(__dirname, `../file`);
console.log(filePath)
app.get('/api/:filename', (req, res) => {
let filename = req.params.filename
const streamFile = fs.createReadStream(`${filePath}/${filename}`);
streamFile.pipe(res);
} );
module.exports = app;
Should I make another dir, maybe 'modules', and there code an async function to read and pipe the files, and call the function from app.get in routes dir ?
Remember that Express is an "un-opinionated, minimalist web framework for Node.js applications", unopinionated means that it doesn't decide for you a lot of aspects of what tool you use for each specific task, and that is the main difference with another frameworks like Rails. Said that, you could use the classical and and old try and catch, in this case around your I/O operation. A module is a way to mantain separation of concerns and it's a way to organize your code so you can fastly identify what is the part of your code that is causing a malfunction. So in this case i don't consider it necessary because your router's callback is doing one thing and that is ok.
app.get('/api/:filename', (req, res) => {
let filename = req.params.filename
try{
const path = `${filePath}/${filename}`;
if (!fs.existsSync(path)) return res.status(404).send('You could send any message here...');
const streamFile = fs.createReadStream(path);
streamFile.pipe(res);
} catch {
res.status(500).send();
};
});
Related
const jsonServer = require('json-server')
const cors = require('cors')
const path = require('path')
const server = jsonServer.create()
const router = jsonServer.router(path.join(__dirname, 'db.json'))
const middlewares = jsonServer.defaults()
server.use(cors())
server.use(jsonServer.bodyParser)
server.use(middlewares)
server.use(router)
const PORT = 8000
server.listen(PORT, () => {
console.log(`JSON Server is running on http://localhost:${PORT}`)
})
I am using cyclic.sh for deployment. How to solve this error?
I was trying to deploy a json-server. And while doing a post request I got this error.
Taking as a starting point that you shouldn't use a tool like json-server in production environments, I could understand you might be using it for demo purposes. That case, there are some serverless solutions which prevents your code from open files in write mode, so that's why the jsonServer.router(...) line fails its execution.
If you don't matter about that db.json file being updated by the requests (because anyway, your deployment solution doesn't seem to allow it), you could just use instead a js object, that can be modified in memory (but, of course, the json file will still keep intact). So, instead of:
const router = jsonServer.router(path.join(__dirname, 'db.json'))
try using something like:
const fs = require('fs')
const db = JSON.parse(fs.readFileSync(path.join(__dirname, 'db.json')))
const router = jsonServer.router(db)
I want to enter path to the folder in browser and display all file names which located inside, i found that it will possible with node fs, but i also have code which runs at browser, and it need vars, located in file, which will run outside of the browser with node. I need to create server with node and runs all code from it? Or what you can reccomend me to reach this goal? PS: By the way i use webpack
There are differences between javascript in browser end and server end. You can't access the directory directly from the browser. You need some sort of backend technology like PHP, Java, node, python, etc in order to get the file list. You can use node server and below code for the reading directory. Then make a simple HTTP request to your backend server from the frontend.
const path = require('path');
const express = require('express');
const fs = require('fs');
const PORT = 3000;
const app = express();
app.get('/getfiles', async (req, res) => {
const directoryPath = path.join(__dirname, 'Documents');
let data = [];
await fs.readdir(directoryPath, function (err, files) {
//handling error
if (err) {
return console.log('Unable to scan directory: ' + err);
}
//listing all files using forEach
files.forEach(function (file) {
// Do whatever you want to do with the file
data.push(file)
});
});
res.send(data);
});
app.listen(PORT, ()=>{
console.log(`server running on port ${PORT}`);
});
I'm developing my first Node.js App with Socket.IO and everything is fine but now the app is slowly getting bigger and I'd like to divide the app-code into different files for better maintenance.
For example I'm defining all my mongoose schemas and the routings in the main file. Underneath are all the functions for the socket.IO connection. But now I want to have an extra file for the schemas, an extra file for routing and one for the functions.
Of course, I'm aware of the possibility to write my own module or load a file with require. That just does not make sense for me, because I can't work with the vars like app, io or db without making them global. And if I pass them to a function in my module, I can't change them. What am I missing? I'd like to see an example how this is done in practice without using global vars..
It sounds like you have a highly coupled application; it's difficult for you to split out your code into modules because pieces of the application that should not depend on each other do. Looking into the principles of OO design may help out here.
For example, if you were to split your dataabse logic out of the main application, you should be able to do so, as the database logic should not depend on app or io--it should be able to work on its own, and you require it into other pieces of your application to use it.
Here's a fairly basic example--it's more pseudocode than actual code, as the point is to demonstrate modularity by example, not to write a working application. It's also only one of many, many ways you may decide to structure your application.
// =============================
// db.js
var mongoose = require('mongoose');
mongoose.connect(/* ... */);
module.exports = {
User: require('./models/user');
OtherModel: require('./models/other_model');
};
// =============================
// models/user.js (similar for models/other_model.js)
var mongoose = require('mongoose');
var User = new mongoose.Schema({ /* ... */ });
module.exports = mongoose.model('User', User);
// =============================
// routes.js
var db = require('./db');
var User = db.User;
var OtherModel = db.OtherModel;
// This module exports a function, which we call call with
// our Express application and Socket.IO server as arguments
// so that we can access them if we need them.
module.exports = function(app, io) {
app.get('/', function(req, res) {
// home page logic ...
});
app.post('/users/:id', function(req, res) {
User.create(/* ... */);
});
};
// =============================
// realtime.js
var db = require('./db');
var OtherModel = db.OtherModel;
module.exports = function(io) {
io.sockets.on('connection', function(socket) {
socket.on('someEvent', function() {
OtherModel.find(/* ... */);
});
});
};
// =============================
// application.js
var express = require('express');
var sio = require('socket.io');
var routes = require('./routes');
var realtime = require('./realtime');
var app = express();
var server = http.createServer(app);
var io = sio.listen(server);
// all your app.use() and app.configure() here...
// Load in the routes by calling the function we
// exported in routes.js
routes(app, io);
// Similarly with our realtime module.
realtime(io);
server.listen(8080);
This was all written off the top of my head with minimal checking of the documentation for various APIs, but I hope it plants the seeds of how you might go about extracting modules from your application.
Say I have post.js with the following.
var functions = require('firebase-functions');
const express = require('express');
exports.post = functions.https.onRequest((req, res) => {
//stuff.
});
Then I just want to include this function into the main file, as it it, so that when running index.js which requires post.js, have the post function, already exported.
Which in the case of firebase functions would run the https function, but now it doesn't unless I explicitly do exposts.post again in the requiring file.
I tried this.
index.js
// here
exports.post = require("./post");
//Another functions ...
exports.user = functions.https.onRequest((req, res) => {
//stuff
});
But because of this, exports.post = require("./post");, I get http://localhost:5000/project-id/us-central1/post-post, which should just be ...us-central1/post.
Also, is it possible to have the required module reference its variables from the requiring file, so that I don't have to do require in post.js, for variables that already exist in index.js, things like "fs" from the file system.
Thanks.
Soo it seems like you are exporting post as a property. You will need to change post.js to:
const functions = require('firebase-functions');
const express = require('express');
module.exports = functions.https.onRequest((req, res) => {
//stuff.
});
Regarding your question. This is usually a bad practice. Each module needs to have its own scope. So, you should require every dependency you need in each file by using require. If you still want to do this, you could use global variables. More info
Can someone please tell me the recommended (up to date) way to get POSTed form data in express.
So many tutorials/ posts etc talk about bodyParser but this is no longer bundled with Express and other blogs etc recommend using urlencoded directly, but now this is not available either.
Trying to find accurate information on these frameworks or technologies is doing my head in.
BTW what I am intrerested in is very simple and small form data
You should install body-parser through npm-install. Now it comes as a separate middleware.
After that add following line in your app.js
var bodyParser = require('body-parser');
app.use(bodyParser.json());
app.use(bodyParser.urlencoded());
// in latest body-parser use like below.
app.use(bodyParser.urlencoded({ extended: true }));
It parses the post request as an object. You will get your variables in req.body.
In your post request handler.
app.post('/post',function(request,response){
console.log(request.body) //you will get your data in this as object.
})
Edit 1
The answer above was for the question specifically asked, the OP was looking for the bodyParser(deprecated) which was not part of express anymore.
Since the title of the question is very generic and the answer doesn't include all aspects of form-data, I will put #StLia's answer as an edit.
Body-Parser Readme
This does not handle multipart bodies, due to their complex and typically large nature. For multipart bodies, you may be interested in the following modules:
busboy and
connect-busboy
multiparty and
connect-multiparty
formidable
multer
You can make use of express-formidable module to that.
install 'express-formidable' by the following command
npm install express-formidable
the simple example is as follows
const express = require('express');
const formidable = require('express-formidable');
var app = express();
app.use(formidable());
app.post('/upload', (req, res) => {
//req.fields contains non-file fields
//req.files contains files
res.send(JSON.stringify(req.fields));
});
Click here for further description
From the README of body-parser:
This does not handle multipart bodies, due to their complex and
typically large nature.
The above is going to work with x-www-form-urlencoded and json but it will NOT work with any multipart. form-data is also multipart with the header multipart/form-data.
In case of form-data, your best solution would be to use express-formidable.
As stated in this StackOverflow answer:
Express 4.16+ has implemented their own version of body-parser so you do not need to add the dependency to your project. You can run it natively in express
app.use(express.json()); // Used to parse JSON bodies
app.use(express.urlencoded()); // Parse URL-encoded bodies using query-string library
// or
app.use(express.urlencoded({ extended: true })); // Parse URL-encoded bodies using qs library
See also: query-string vs qs
I noticed #HubballiHuli answer was to use a package called express-formidable.
You don't need to use this unnecessary package, it provide one (small) file of code.
Instead you can do it yourself (now removing the dependency).
Here is the formidableMiddleware file:
'use strict';
const formidable = require('formidable');
function parse(opts, events) {
return (req, res, next) => {
if (req.express_formidable && req.express_formidable.parsed) {
next();
return;
}
const form = new formidable.IncomingForm();
Object.assign(form, opts);
let manageOnError = false;
if (events) {
events.forEach((e) => {
manageOnError = manageOnError || e.event === 'error';
form.on(e.event, (...parameters) => { e.action(req, res, next, ...parameters); });
});
}
if (!manageOnError) {
form.on('error', (err) => {
next(err);
});
}
form.parse(req, (err, fields, files) => {
if (err) {
next(err);
return;
}
Object.assign(req, { fields, files, express_formidable: { parsed: true } });
next();
});
};
}
module.exports = parse;
exports.parse = parse;
Now on how to use it:
const express = require('express');
const formidableMiddleware = require('./formidableMiddleware.js');
var app = express();
app.use(formidableMiddleware());
app.post('/upload', (req, res) => {
//req.fields contains non-file fields
//req.files contains files
res.send(JSON.stringify(req.fields));
});
I wrote an article on unnecessary packages a while ago and why not to use them:
https://medium.com/#alexjamesdunlop/unnecessary-packages-b3623219d86
Besides the solutions with formidable, there is another module which I have been using in my recent projects since 2019. The module express-form-data can be easily declared in your server file like:
const express = require('express');
const formData = require('express-form-data');
app.use(formData.parse());
app.post('/image-upload', (req, res) => {
console.log(req.files);
})
...
In case of image uploading, for instance, req.files will provide all relevant data you need for handling the files such as path, size, filename, etc.
If you would like to apply the formidableMiddleware to only one API route and not globally, this is how you would pass the value.
This would be useful if u want to mix between different headers to pass to the API for other API's which you which u do not want the formidableMiddleware API to be applied.
const express = require('express');
const formidable = require('express-formidable');
var app = express();
app.post('/mypath', formidableMiddleware(), (req, res) => {
// rest of the code
})