We Keep Coding

External JS file engine -- manipulating db using node.js? PHP?

Admittedly, I'm new to some of this...
Building a website on a local server. It has a ton of JS function in an external JS file.
Site has a MYSQL DB. (I am still learning this).
As part of my calculations from functions in that external JS file, I want to update and/or read from that DB.
I have been trying to read up on node.js and trying to read up on PHP (still learning both), but I'm not sure if I'm sniffing in the right direction.
Do I somehow invoke functions from node.js from the external JS file? Do I somehow invoke the PHP (in the form of a function, I suppose) from the external JS file?
How does one typically do this?
I have definitely learned that this in the external JS file does not do the trick. First window appears, but second doesn't:
// Activate the node.js library for MYSQL access
alert("got here 1");
var mysql = require('./mysql');
alert("got here 2"); // nope, this never pops up
Higher-level advice might be more useful than detailed in-the-weeds advice...? Still very new to this.
Thank you kindly!
-=-=-=-=-
self-muttering thoughts... I am using the external JS file to hold a bunch of functions that do all kinds of manipulation and conformation to the data that I collect on the front end:
<button class="ButtonOperation" onclick="DataLog(document.getElementById('DataWindow').value,'NE_AssembleOrder')">Log Data</button>
Am I eventually going to discover that I should instead port all of these functions over to a big PHP file instead?
-=-=-=-=-

Okay, took a while before I better understood this. So, this is the answer that would have gotten me moving in the right direction (for any future reference):
The thing to understand is that for this project, you want to manipulate data to and from a database, which means that (at least for now, for the sake of simplicity), the key is to get your data into a package and send it up to the server, and then have a function running on the server take up the yoke from there.
The way to do that (old school), is with a form.
When you SUBMIT a form, all that data on the form is bundled up and sent to the server.
In this instance you have an index.html page, and that page will open a new page for each of the functions you're trying to track. Use JavaScript to pop open the window and then when you include the URL for the window, pop in a Popup_SpecificFunction.php file. (change SpecificFunction as needed)
So far, so good. ;)
Now, in that Popup_SpecificFunction.php, you will collect all your data under a single form. A good ol' HTML form, with a [SUBMIT] button. That very same Popup_SpecificFunction.php file also has a reference in the header, referring to the big main library of PHP functions -- which is a file sitting on the server.
That [SUBMIT] button invokes a ProcessAllThisData function -- which is on the server-side PHP file. In doing this, it sends all the data from the form -- including a lot of data you include in hidden controls -- to the serverside function.
And at that point, you are basically "on the server" with all your data, and then you can just code that function in PHP and manipulate the database and other stuff as needed.
Using the form was the thought-jump you needed, because prior to this, you've generally thought of forms as standalone data, but they can have actions associated with the entire forms.
You can still use JavaScript to do client-side stuff, but here's another thing that can trip a person up:
There is a difference between these two HTML items as far as whether or not you should use them to send data to and from the server, or whether or not you are just going to JavaScript something on that button:
<button></button>
and
<input type="button"></input>
You might have to experiment a bit to figure out which is which.
This was everything you needed to get you moving in the right direction.
Sincerely,
Future Me. :)

One data.json accessed and changed by multiple sources

My goal is the use the data in a JSON file ("data.json") for both my Java application and my node.js application.
Java application reads the JSON file and rewrites the file, only with additional or edited JSON objects.
I would like also the node.js application to read the JSON file and rewrite the file with additional or edited JSON objects.
What is the best way to do this? What are my options for collaborating, syncing and queuing changes from each source?
Thanks

Obviously to protect the file from corruption on concurrent writes and reads you need to implement some lock system. Between the threads in a single process it's a trivial task, however between processes it's not that easy.
There's one trick that could work on UNIX systems and it's based on the fact that file rename procedure is atomic in unix. Based on the file renaming you could implement a simple mutex, or even a more complicated read-write lock semantics. For a simple mutex use the following strategy:
When an app want's to use the file, check the data* file name. If it's data_locked_<some_pid>, wait (or subscribe to FS notifications so that you don't need to poll FS and spend CPU time) and repeat step 1.
If the file name is data attempt to rename the file to data_locked_<pid>. If the attempt succeed - use the file.
After you finish working with the file rename it back to data
This simple mutex semantics will keep the file locked for use by a single process, however, reading operations be safely performed concurrently, so you could optimise the solution further by making a ReadWriteLock:
For reads:
When the app whats to read the file, check if the file name is data or data_readlocked_<some_pid_list>. Attempt to rename the file to data_readlocked_<original_pid_list + pid>. If attempt failed, repeat step 1.
Read the file.
Rename the file so that it's name doesn't contain this process pid any more. If this process pid is the only in the list - rename to data. If rename attempt failed repeat step 3 until attempt is succeed.
For write:
Check if the file name if data. If it is, rename to data_writelocked_<pid>, if failed repeat step 1. If the file name is anything other then data - wait and repeat step 1.
Write to the file.
Rename it back to data.

Webpage with Javascript executing Python script with arguments

I have three files in the same directory. One is a python script, which takes argumenet. One is a html page with javascript. And the last one is a source .wav file.
./myfolder/sound_manipulation.py
./myfolder/volume_slider.html
./myfolder/the_song.wav
The sound_manipulation.py file can be executed like:
python sound_manipulation.py -v 50
and it generates a new wav file, new_song.wav, based on the_song.wav, but only has 50% of the original volume level.
On the other hand, the volume_slider.html contains a slider goes from 0 to 100%, and a button calling an onclick javascript function, update_vol();
So far, the update_vol() alerts the value of the slider, and that's all.
function update_vol() {
var vol = document.getElementById('vol_slider').value;
alert(vol);
}
But I want the update_vol() to actually execute the python script using the vol.
How can I make that happen?
Also, when the "python sound_manipulation.py -v 50" is executed, how can I return the location of the new_song.wav back to volume_slider.html?
Please help. Thanks!

the simplest and crudest one-time cgi script might solve your problem.
set up a cgi script/environment to just get volume value from user / then use subprocess module to process the .wav file and send it back to user. if you need anything more than that, build your own web app.
import cgi
import subprocess
import sys
form = cgi.FieldStorage()
volume = form.getfirst('volume') #read from form 'volume'
subprocess.call(['python', 'sound_manipulation.py', '-v', volume])
with open('new_song.wav', 'rb') as wav_file:
print("Content-Type: audio/wav\n")
sys.stdout.write(wav_file.read())

Hm, well, I guess you're out of luck. Browser scripts can't execute anything (so no Python script) on your system. As I'm sure you can imagine, this could present a huge security risk -- that's exactly how much trouble Microsoft's ActiveX got into.
I'm assuming you want to create a graphical interface for that script, so alternatively you could :
Do a JavaScript equivalent of your Python script if you absolutely need HTML
Or create an interface in QT, GTK or the like. Python can easily do that, by the way!

Does Apache use file timestamps to check whether a resource has been modified?

I am developing an html page referencing a large Javascript file (1MB+) which is seldom modified. From here, I get that the js file will not be resent if not modified.
My question is: how does Apache checks whether an ftp uploaded javascript file has been modified? Is it from its file timestamp? If not, where does it get this information? I want to understand the process to control performance issues.

For static files, a call to stat() is typically used to check if the file size or modification time has changed.
The Caching Guide goes into detail and also contains the above reference in the section A Brief Guide to Conditional Requests.

How to deny direct access to an xml file in the server

I have an html file index.html (in my server say abc.com) which accesses xyz.js like
<script type="text/javascript" src="xyz.js"></script>
The javascript file in turn accesses data.xml file.
The files index.html,xyz.js and data.xml are in the same folder.
How can I deny direct access to xyz.js and data.xml if a user types
abc.com/xyz.js and abc.com/data.xml in the browser.
Needless to say index.html must be able to access these files.
How can I do this(preferably with .htaccess)

I'm assuming you mean index.html refers to the .js file via a script tag, and then the js reads in the xml using XMLHttpRequest or something similar. ie: the js and xml both need to be readable by the browser, but you want to restrict this to only be in an "approved" way.
If that's right, then you can't. You could try looking at the Referer, but it's unreliable and easily spoofable. Even without spoofing, many browsers have debugging tools that make it easy to see the result of every GET that has been performed.
It's better to just get used to the fact that anything you send to the browser is potentially viewable by the user if they work hard enough at it.
I suppose for JavaScript you could use an obfuscator tool if you feel so inclined. For XML, there isn't much you can do. I suppose you could encrypt it, but that would be easy to break as your js code will necessarily contain the decryption routine and key.

If you truly need to protect the data, you need to implement the sensitive part of your program to run on the server and not in the client. Then you can keep your datasource out of the public web space completely. If the client (browser) can access the raw data, then so can the user (even if you force them to go through multiple steps to get at it).
To acheive your goal you need to split your program architecture in two:
the non-sensitive parts run, in javascript, in the browser
the sensitive parts run, in .net/java/php/ruby/python etc etc, on the server