We Keep Coding

Passing webpage form data to Python script on my location machine

I have a Python script that I have written that an utilizes an API to retrieve weather information, its just a simple terminal print script for right now. I am starting to learn more about HTML/JS and am wondering where I could start to learn how to pass information from my web pages to scripts.
Eventually what I am trying to work towards is passing a zip code string input from an HTML form over to my Python script on my local machine, and then have the script return data to my webpage.
The problem is, I have no idea where to start, or where even to start looking for information. For example, I understand that you can pass values to a server side application & that is kinda what I'm simulating here.

If you want to run your python script from your websites, you could use a server for that. Given that you are already into Javascript, I suggest that you create a server using a popular JS framework called Express. Express is designed for NodeJS, a JS runtime.
Once you setup your Express server, you can start creating routes and integrate them into your websites by making asynchronous calls with utilities like fetch or axios. For example, you can create a sample app like this:
var express = require('express');
var app = express();
app.get('/', function (req, res) {
res.send('Hello World!');
});
app.use('/run-script', function (req, res) {
res.send('Script run!');
};
app.listen(3000, function () {
console.log('Example app listening on port 3000!');
});
If you look closer, app.use() allows you to define routes. When a user or a JS script calls this route, the function gets executed. For example, inside the route run-script you could execute yours:
app.use('/run-script', function(){
const spawn = require("child_process").spawn;
const pythonProcess = spawn('python',["path/to/script.py", arg1, arg2, ...]);
});
As you see, there are numerous possibilities. For more info on calling python scripts from node, see this stackoverflow question.
You could start digging into NodeJS in general. For that, a good place to start is the official guides.
Please let me know if this answer was useful to you.

setting up a lot of routes in node.js

I've created a webpage to use it locally as a way to save information about random topics and college stuff. I have a ton of routes like the ones shown below:
//app.js - using node and express
app.get('/page1', function(req, res){
res.render('page1');
});
app.get('/page2', function(req, res){
res.sendFile('views/page2.html', { root: __dirname });
});
Each one of these routes has a .ejs file or a .html file, and they are all quite small.
I don't think I've made any major mistakes, but I gotta a feeling I'm not using "best practices", or at least could be doing some differently.
Is there anything wrong with using a lot a routes like this? Should change something to optimize the rendering of my pages?
As I said before, I'm using .ejs on most of the pages, and most of them have the same header.ejs and footer.ejs. Every time I change pages that have the same header/footer, do they get loaded again, or since they are using the same header/footer files the server only requests the content in between?
what's the difference between using res.render and res.send?
Should I be using a different syntax: const express = require('express'); & const app = express(); instead of var express = require('express'); & var app = express();
``` or this
app.get('/page1', (req, res) => {
res.render('page1');
});
```
Instead of the first block of code above.
Thanks! :D

Is there anything wrong with using a lot a routes like this? Should change something to optimize the rendering of my pages?
Nothing technically wrong with it. Many plain res.sendFile() routes can probably be replaced with a single express.static() middleware statement to simplify your code.
Lots of res.render() routes that don't pass any customized data to EJS can also probably be replaced by a single middleware that handles either a whole directory of template files (and their corresponding routes) or a list of files. That would be a lot more DRY than spelling out each route separately.
As I said before, I'm using .ejs on most of the pages, and most of them have the same header.ejs and footer.ejs. Every time I change pages that have the same header/footer, do they get loaded again, or since they are using the same header/footer files the server only requests the content in between?
EJS caches templates in memory (unless you disable caching) so the header and footer templates won't get loaded over and over again from disk.
what's the difference between using res.render and res.send?
These are fully covered in the documentation. In a nutshell, res.render() supports your template engine and local data to feed to the template engine to allow it to add data to the template. res.send() just sends the raw content you give it.
Should I be using a different syntax: const express = require('express'); & const app = express(); instead of var express = require('express'); & var app = express();
It is considered a good practice to use const whenever the variable you are declaring should get its initial value and not be assigned to again. In addition to some coding safety, this also can sometimes allow the interpreter to do more optimizations when using the variable (since its value can't be changed).

Running a NodeJS cron from cron-job.org

I am trying to run a NodeJS cron at a set interval using cron-job.org, but they don't have any documentation about how to actually run the script.
Basically the service visits a URL that you provide at a set interval, but I am specifically confused about what kind of code I can put on the endpoint (specifically what type of code will actually run). Can someone provide an example of what I would put at the endpoint URL?

You can do something really simple using either the HTTP module in Node.js or the popular Express module. Using express you can do something really simple like:
var express = require('express');
var app = express();
app.get("/test", function(req, res, next){
res.setHeader('Content-Type', 'application/json');
res.send(JSON.stringify({ status: 'OK', timeStamp: new Date().toISOString() }));
});
console.log('Express listening.. on 3000');
app.listen(3000);
You can really run anything you like in the /test endpoint, though when it's being called from cron-job.org they'll probably stop if you keep throwing back 400 errors at them or the script takes really long to execute.
You'd call this using the url
http://yourdomain:3000/test
And of course you might well want to change the port number and path!

cron-job.org only allows you to call an endpoint at a set time interval.
If you want to have some code run at a set interval without worrying about HTTP server, deploying, etc... Check out services like chunk.run
Here's an example code:
https://chunk.run/c/scheduled-run-example
Then you can just select the trigger "Scheduled" like so:

Express.js: reverse proxying different web app along with assets

I want to allow an authenticated client in Express to access to other web applications that are running on the server, but on different ports.
For example, I have express running on http://myDomain and I have another application running on say port 9000. I want to be able to reach the other app through http://myDomain/proxy/9000.
I had a little bit of success using node-http-proxy, for example:
function(req, res) {
var stripped = req.url.split('/proxy')[1];
var path = stripped.split('/');
var port = path.shift();
var url = path.join('/');
req.url = url;
proxy.web(req, res, {
target: 'http://127.0.0.1:' + port
});
}
However, the big problem is that when the web app makes GET requests, such as for /js/lib.js, it resolves to http://myDomain/js/lib.js, which is problematic because express is not aware of those assets. The correct request would be to http://myDomain/proxy/9000/js/lib.js. How do I route all these additional requests?

What you need to do is to replace URLs in the initial page with the new URL pattern. What is happening is that the initial page that your reverse proxy returns has a reference to:
/js/lib.js or http://myDomain/js/lib.js
so when the browser makes a second request it has the wrong pattern for your reverse proxy.
Based on the incoming request you know what the pattern should look like. In your example it's http://myDomain/proxy/9000. You then fetch the appropriate page from the other server running on http://127.0.0.1:9000/. You do a string replace on any resources in that file. You'll need to experiment with the pattern but you might look for 'script src="/' or 'href="/' and you might find regex helps with the pattern if, for example, the src attribute isn't the first listed in a script tag.
For example you might find 'scr="/' and then you replace it with 'src="/proxy/9000/' that way when the browser asks for that local resource it will come through with the port that you're looking for. This is going to need experimentation and it's a great algorithm to write unit testing around to get perfect.
Once you've done the replacement you just stream that page to the client. res.send() will do this for you.
Something else that you might find useful is that ExpressJS gives you a way to pull out the port number with a little less hassle than you're doing. Take a look at this example:
app.get('/proxy/:port', function(req, res){
console.log('port is ' + req.params.port);
});

I don't think http://myDomain/proxy/9000 is the correct way to do it. Web pages are going to assume the site's domain to be just myDomain and not myDomain/proxy/9000, because that is what the standard says.
Your use case would be better served by using subdomains like 9000.proxy.myDomain.

Node.js Programming Pattern for getting Execution Context

I am writing a web app in node.js. Now every processing on the server is always in the context of a session which is either retrieved or created at the very first stage when the request hits the server. After this the execution flows through multiple modules and callbacks within them. What I am struggling with is in creating a programming pattern so that at any point in the code the session object is available without the programmer requiring it to pass it as an argument in each function call.
If all of the code was in one single file I could have had a closure but if there are function calls to other modules in other files how do I program so that the session object is available in the called function without passing it as an argument. I feel there should be some link between the two functions in the two files but how to arrange that is where I am getting stuck.
In general I would like to say there is always a execution context which could be a session or a network request whose processing is spread across multiple files and the execution context object is to be made available at all points. There can actually be multiple use cases like having one Log object for each network request or one Log object per session. And the plumbing required to make this work should be fitted sideways without the application programmer bothering about it. He just knows that that execution context is available at all places.
I think it should fairly common problem faced by everyone so please give me some ideas.
Following is the problem
MainServer.js
app = require('express').createServer();
app_module1 = require('AppModule1');
var session = get_session();
app.get('/my/page', app_module1.func1);
AppModule1.js
app_module2 = require('AppModule2');
exports.func1 = function(req,res){
// I want to know which the session context this code is running for
app_module2.func2(req,res);
}
AppModule2.js
exports.func2 = function(req,res){
// I want to know where the session context in which this code is running
}

You can achieve this using Domains -- a new node 0.8 feature. The idea is to run each request in it's own domain, providing a space for per-request data. You can get to the current request's domain without having to pass it all over via process.domain.
Here is an example of getting it setup to work with express:
How to use Node.js 0.8.x domains with express?
Note that domains in general are somewhat experimental and process.domain in particular is undocumented (though apparently not going away in 0.8 and there is some discussion on making it permanent). I suggest following their recommendation and adding an app-specific property to process.domain.data.
https://github.com/joyent/node/issues/3733
https://groups.google.com/d/msg/nodejs-dev/gBpJeQr0fWM/-y7fzzRMYBcJ

Since you are using Express, you can get session attached to every request. The implementation is following:
var express = require('express');
var app = express.createServer();
app.configure('development', function() {
app.use(express.cookieParser());
app.use(express.session({secret: 'foo', key: 'express.sid'}));
});
Then upon every request, you can access session like this:
app.get('/your/path', function(req, res) {
console.log(req.session);
});
I assume you want to have some kind of unique identifier for every session so that you can trace its context. SessionID can be found in the 'express.sid' cookie that we are setting for each session.
app.get('/your/path', function(req, res) {
console.log(req.cookies['express.sid']);
});
So basically, you don't have to do anything else but add cookie parser and enable sessions for your express app and then when you pass the request to these functions, you can recognize the session ID. You MUST pass the request though, you cannot build a system where it just knows the session because you are writing a server and session is available upon request.

What express does, and the common practice for building an http stack on node.js is use http middleware to "enhance" or add functionality to the request and response objects coming into the callback from your server. It's very simple and straight-forward.
module.exports = function(req, res, next) {
req.session = require('my-session-lib');
next();
};
req and res are automatically passed into your handler, and from their you'll need to keep them available to the appropriate layers of your architecture. In your example, it's available like so:
AppModule2.js
exports.func2 = function(req,res){
// I want to know where the session context in which this code is running
req.session; // <== right here
}

Nodetime is a profiling tool that does internally what you're trying to do. It provides a function that instruments your code in such a way that calls resulting from a particular HTTP request are associated with that request. For example, it understands how much time a request spent in Mongo, Redis or MySQL. Take a look at the video on the site to see what I mean http://vimeo.com/39524802.
The library adds probes to various modules. However, I have not been able to see how exactly the context (url) is passed between them. Hopefully someone can figure this out and post an explanation.
EDIT: Sorry, I think this was a red-herring. Nodetime is using the stack trace to associate calls with one another. The results it presents are aggregates across potentially many calls to the same URL, so this is not a solution for OP's problem.