We Keep Coding

Error with my server file - Invalid URL: index.js

I am getting the following error when trying to run my server (index.js):
Error: TypeError [ERR_INVALID_URL]: Invalid URL: index.js
The code block is here:
app.get('/:url', async function (req, res) {
try {
return res.status(200).json(data);
} catch (ex) {
console.log(ex);
return res.status(500).json({ message : "Oops." });
}
With the specific line of code it is referring to is:
const site = await .open(decodeURIComponent(req.params.url));
Has anyone ever encountered this error before, and have any idea how to fix it in this context? Not sure how it is throwing an error for my entire index.js server
Note: This is an express app

The value of req.params.url is index.js.
A web browser can expand index.js into an absolute URL because it knows that it can use the URL of the current HTML document as the base URL.
You're running your code in Node.js. It doesn't run inside a document. There is no base URL.
You need to provide an absolute URL (e.g. http://example.com/index.js) yourself.
That said, I suspect wappalyzer may require that you give it the URL to an HTML document not a JS file.

Well, since there isn't much that i can see about your usage, I'm just going to assume you went to something like http://localhost/index.js.
The problem now is, that wappalyzer does not actually get a valid url. It gets index.js without any form of protocol (http/https/ws/wss...). This means it doesn't know what to do and tells you that the url you provided is invalid.
In order to fix this go to http://localhost/https%3A%2F%2Flocalhost%2Findex.js or append https:// or something similar to your parameter.

Race condition when fetching request.url in Node / Express

I have a Node webapp which supports URLs in multiple languages. I have some middleware to generate a Welsh version of the current URL (eg. for the URL /foo, the Welsh version would be /welsh/foo), which is presented on-page as a link.
This function takes a request object and returns the updated URL (accessed via request.url). Unfortunately, when I open several pages at once on my site, some pages get the wrong URL in the <a> link (eg. they get the Welsh URL for one of the other pages I've opened). If I reload the page, the link is re-generated, this time correctly.
My middleware looks like this:
res.locals.getCurrentUrl = (req, locale) => {
// snip: some logic to check if the URL already contains the locale
return "://" + req.get('host') + req.originalUrl;
};
... then I call it in templates like getCurrentUrl(request, 'welsh').
I only see this behaviour when I open a dozen tabs at once (via JavaScript – I have a status page with a button that opens a bunch of site URLs at once). Obviously this isn't a real-life use case, but at moments of high traffic, this race condition might kick in.
Is there a better, more reliable way here to associate per-request variables like url with the rendered output? Am I doing something wrong?

Fixed this by making the request object available to my templates in a middleware function (beforehand I set it as a global on the template engine which clearly introduced this race condition):
app.use(path, (req, res, next) => {
res.locals.request = req;
return next();
});

It might be that you're using a Lambda function, and those retain the outer scope.
Try
res.locals.getCurrentUrl = function() {...}
instead.

Dynamic configuration file in Nodejs and Express

I am working with NodeJs with expressjs and I would like to use a middleware that will read a variable from dynamic configuration file before each request (some js file that holds an object that I can change manually while the server is running).
I would like to store a flag that will close down all requests (i.e send response without proceeding to next routes) until I change the variable value to true and then the middleware will call next() so all routes will work.
I prefer to avoid shutting down the server for that, and I also don't want to store this variable in a database...
Is there something similar to what I need?
Thanks

You will read config.json file each time, if you just put it inside function. If you will use require('config.json), then you will read this file only once on start.
var fs = require("fs");
function (req,res,next) {
var config = fs.readFileSync('./config.json');
config = JSON.parse(config.toString());
if (config.registrationStatus === "open") {
next();
} else {
res.send("registration closed");
}
}

Check supervisor: https://github.com/petruisfan/node-supervisor
It will watch for file changes and crashes and will restart your application each time.
You will get much better performance.

NodeJS Modulization

So, I was told that passing around the request and or response variable in nodeJS is "bad practice". But this means that most of your code has to be in the server.js file, making it cluttered and kind of ugly.
How can you modularize your nodejs server, passing around req/res appropriately and be able to organize your code into separate files?
For example, I would like to split my socket routing, .get and .post into different files, but still be able to use the callback parameters like so:
app.io.route("disconnect", function(req,res) { <--- these params
db.query("UPDATE player_data SET online=0 WHERE id="+mysql.escape(req.session.user));
req.io.broadcast("event", {msg:req.session.username+" has logged out!"});
app.io.broadcast("reloadXY");
});
As of right now they're all in one file and I don't like that.

I think what the person meant by 'passing around' was something like this (in plain express):
app.get('/kittens', function(req, res) {
db.doAthing(req);
updateSomethingElse(res);
upvoteThisAnswer(res);
});
That is, passing around the two variables beyond the first function. This is bad because it becomes increasingly difficult to figure out where the call actually ends. One little res.end(500) in updateSomethingElse can cause the whole house of cards to come tumbling down.
It's perfectly ok (in fact, standard to the point of being the default in express) to declare that callback elsewhere (usually the /routes directory of your project.)
// app.js
var user = require('./routes/user')
, kittens = require('./routes/kittens');
// express stuff...
app.get('/settings', user.getSettings);
app.get('/fur', kittens.shed);
Then, in routes/user.js:
exports.getSettings = function(req, res) {
// Note how we're passing around properties of req/res, not the objects themselves.
db.getUserSettings(req.user.id).then(function(settings) {
res.render('settings', settings);
});
};

This video from TJ Holowaychuk (the guy who wrote Express and a ton of other Node infrastructure that we all use) helped me take Express modularization to the next level. Basically you can make individual apps in their own folders and consume them as middleware very easily. I have managed to extend this technique to socket.io with some tricks.
http://vimeo.com/56166857

You should not pass req and res to another modules but pass callbacks from another modules to route.
It should look like.
var someModule = require("./someModule")
app.get("/someAction", someModule.handleSomeAction) ;
If You want to have post and get in another modules You should pass reference to app (from express()) once to that module and operate on that.
For example :
var express = require("express") ;
var app = express();
var get_handler = require("./get_handler ")
var post_handler = require("./post_handler ")
get_handler.init(app);
post_handler.init(app);
and in post/get_handler :
var app;
exports.init = function( eApp){
app = eApp;
// operate on app
}

MeteorJS what to put under /server folder?

I'm following the Discover Meteor book and the book suggets put js file under /collections folder. The js file defines server side methods:
Meteor.methods({
post: function(postAttributes) {
var user = Meteor.user();
var postWithSameLink = Posts.findOne({url: postAttributes.url});
// ensure the user is logged in
if (!user)
throw new Meteor.Error(401, "You need to login to post new stories");
// ensure the post has a title
if (!postAttributes.title)
throw new Meteor.Error(422, 'Please fill in a headline');
// check that there are no previous posts with the same link
if (postAttributes.url && postWithSameLink) {
throw new Meteor.Error(302,
'This link has already been posted',
postWithSameLink._id);
}
// pick out the whitelisted keys
var post = _.extend(_.pick(postAttributes, 'url', 'title', 'message'), {
userId: user._id,
author: user.username,
submitted: new Date().getTime(),
commentsCount: 0,
upvoters: [],
votes: 0
});
var postId = Posts.insert(post);
return postId;
},
});
Well, in that case, isn't the whole logic accessible to public since Meteor gathers all JavaScript files in your tree, with the exception of the server, public, and private subdirectories, for the client.?
Is this a concern?
What should I put to server folder?

There are many server-only things that might go in that folder – subscriptions, allow rules, cron jobs etc.
The server folder is the only place from which your code is not visible to general public. Naturally, it's the place where you should put your security-related code: allow / deny rules, accounts configuration etc. If you talk to an external API and want to place your access keys in code, /server folder is the only acceptable place to do so.
Putting your server-side logic in an accessible folder is not a critical issue, as server-side code cannot be altered from the client. The only security concern here is that someone might study your code and find a breech if you did a shortcut somewhere. Also, you're clogging the connection with code that's not needed in the client.
I'd also say that most methods should go in /server, but that depends on what you need. Making methods code accessible on the client allows to take advantage of latency compensation (here, third paragraph below the example), but you need to make sure yourself that the client-side simulation doesn't produce side effects that would interfere with the actual (server-side) modifications.
In your case, you can put this method in /collections or /model folder and don't worry about this. You can also put it in /server and things will work as well.