Our workplace has a shared network drive that everyone in our department has access to without explicitly requiring credentials. The link is something like
myworkplace.com/mydept/archive/etc
Anyone from my department can access this link via file explorer or even their browser. Currently, I use it by calling python scripts that exist on the drive by using
> pushd myworkplace.com/.../python/
> ./python.exe scriptPath/script.py arg1 arg2
The result is that it copies a set of files (arg1) from an SFTP server (arg2) onto this drive for everyone in the department to review via a new link/directory path. Now, I want to create a very basic webpage that others in the department can use to essentially pass arg1 and arg2 as input to this script and by the click of a button run this script themselves.
My research suggests that I should use node.js as client-side webpages cannot interact with the server-side scripts/tools without it. This requires serious network configs which is entirely out of the question as that is not our responsibility and the last thing I would want to do is get an angry email from the networking team. However, if me and others can easily pushd into the drive and run scripts from it, shouldn't there be something just as easy that I can do from a webpage? Maybe I can leverage the network config of the network drive itself?
I also believe that I may be misunderstanding the issue entirely, and that maybe I'm calling the script from the drive but actually running it locally, but my knowledge on networking is very slim so I'm not exactly sure what is really possible here. Any suggestions?
So I was able to solve this rather simply
When I pushd into the network drive I can run the command
ipconfig
which returns the IP address of the drive. I used that as the host name and used port 8080. When running the node.js script I made sure to use those details as below
const express = require("express");
const app = express();
const host = '11.111.111.111';
const port = 8080;
app.get('/Home', (req,res) => {
res.sendFile(<UNC Network path to HTML landing page >)
})
app.listen(port, host, () => console.log(`App listening on port ${port}`))
When my colleagues and I typed '11.111.111.111:8080/Home' into our browser we were able to access the page and it's contents. I think this is the standard protocol for creating a website and I may have simply misunderstood the requirements.
The rendered HTML page contained a button that would call a different function in my node.js script which returned the output from my python script. See below
function myFunc(arg1, arg2){
const spawner = require('child_process');
const path = require('path');
var dataToRead;
//declare path to python executable
const python_exe = path.resolve(<UNC path> + 'python/python.exe');
//declare path to python script
const python_py = path.resolve(<UNC path> + 'script.py');
//create a process on server side calling the python executable to initiate python
//and pass the script and arguments as a list of paramaters
const python_call = spawner(python_exe, [python_py, arg1, arg2]);
//read the output from python script
python_call.stdout.on('data', (data) => {
dataToRead = data.toString();
}
}
And that's how I used a network drive to create a webapp that calls a python script and retrieves the results
P.S. I could not change the website name to be a string as that would require configuring the DNS to point to this IP and port, which is not something folks outside the network team have access to.
Related
I want to implement socket.io room to room file sharing system so users can send images to their respective rooms to all user can see it and I have tried using base64 encoding method to emit sender image file to specific room but it can send only approximately 700kb to 800kb file.
Is there any easier way of doing this and can support larger files above 1mb and it should be able to load images progressively?
I am using ejs template engine, nodejs, socket.io, javascript.
Console.log("please help me guys if you any idea about this, I tried many things but none of them are working and I have read the socket.io documentation but didn't get any clue about it
I have also tried binary streaming but got no luck please help me guys with some codes samples
You will probably find it easier for the client to upload the file to your http server with a room name and then have your http server send a message to all the other clients in the room via socket.io with a URL where the client can download the file using http. socket.io is just not a streaming protocol, it's a packet or message-based protocol so to send large things, it has to be broken up into messages and then reassembled on the client. This can be done, but it's just extra non-standard work that http uploads and downloads already know how to do.
Here would be the steps:
Client uploads file to server via http post with the room name as a field in the form.
Server receives uploaded file, assigns it a unique ID and stores it in a temporary location on the server on disk.
When file upload completes server notifies all other clients in the room via socket.io that the file is uploaded and ready for download and sends them the URL for download that has the uniqueID in it.
Each client sends request to download the file via http using the unique URL they received.
Server serves the file to each client as requested over http.
Server either keeps track of whether all clients have now finished downloading or just removes the file after some period of time based on timestamp of the file (to just clean up disk space) with some regular cleanup function on a recurring timer.
You can create a single route that handles all the downloads:
const downloadRoot = "/temp/filexfer";
app.get("/download/:id", (req, res) => {
const fullPath = path.resolve(path.join(downloadRoot, req.params.id));
// detect any leading . or any double .. that might jump outside
// the downloadRoot and get to other parts of the server
if (!fullPath.startsWith(downloadRoot)) {
console.log(`Unsafe download request ${fullPath}`);
res.sendStatus(500);
return;
}
res.download(fullPath);
});
A cleanup algorithm could look like this:
const fsp = require('fs').promises;
const path = require('path');
const oneHour = 1000 * 60 * 60;
// run cleanup once per hour
let cleanupTimer = setInterval(async () => {
let oneHourOld = Date.now() - oneHour;
try {
let files = await fsp.readdir(downloadRoot, {withFileTypes: true});
for (let f of files) {
if (f.isFile()) {
let fullName = path.join(downloadRoot, f.name);
let info = await fsp.stat(fullName);
// if file modification time is older than one hour, remove it
if (info.mtimeMs <= oneHourOld) {
fsp.unlink(fullName).catch(err => {
// log error, but continue
console.log(`Can't remove temp download file ${fullName}`, err);
});
}
}
}
} catch(e) {
console.log(e);
}
}, oneHour);
// unref the timer so it doesn't stop node.js from exiting naturally
cleanupTimer.unref();
There are a lot of ways to do this sort of thing and it depends heavily on what sort of architecture you want to support.
Sending large files through socket.io or any other web-socket is fine. it does require a bunch of chopping and reassembling on your web app but it will work.
WebRTC is another way to share files of any description, and it will not burden your server in anyway which is good. (here is a tutorial on it https://ably.com/tutorials/web-rtc-file-transfer)
The issue with either of these methods is that they are transient shares, are new user to the room will not get the image, unless your server re-transmits the data again.
My suggestion would be to upload the file to s3 directly and then share a link to it that can be resolved on each of the clients. This will keep the server burden down and reduce your storage requirements in the backend server
I have a Python script that I have written that an utilizes an API to retrieve weather information, its just a simple terminal print script for right now. I am starting to learn more about HTML/JS and am wondering where I could start to learn how to pass information from my web pages to scripts.
Eventually what I am trying to work towards is passing a zip code string input from an HTML form over to my Python script on my local machine, and then have the script return data to my webpage.
The problem is, I have no idea where to start, or where even to start looking for information. For example, I understand that you can pass values to a server side application & that is kinda what I'm simulating here.
If you want to run your python script from your websites, you could use a server for that. Given that you are already into Javascript, I suggest that you create a server using a popular JS framework called Express. Express is designed for NodeJS, a JS runtime.
Once you setup your Express server, you can start creating routes and integrate them into your websites by making asynchronous calls with utilities like fetch or axios. For example, you can create a sample app like this:
var express = require('express');
var app = express();
app.get('/', function (req, res) {
res.send('Hello World!');
});
app.use('/run-script', function (req, res) {
res.send('Script run!');
};
app.listen(3000, function () {
console.log('Example app listening on port 3000!');
});
If you look closer, app.use() allows you to define routes. When a user or a JS script calls this route, the function gets executed. For example, inside the route run-script you could execute yours:
app.use('/run-script', function(){
const spawn = require("child_process").spawn;
const pythonProcess = spawn('python',["path/to/script.py", arg1, arg2, ...]);
});
As you see, there are numerous possibilities. For more info on calling python scripts from node, see this stackoverflow question.
You could start digging into NodeJS in general. For that, a good place to start is the official guides.
Please let me know if this answer was useful to you.
I have been developing a vscode extension that consits of client and server using the language server protocol.
At the moment, I am trying to do the following thing: when the server detects a certain condition, he requests the client to load a certain number of files into the workspace.
I am having serious problems doing this. Since the language server protocol does not have a specific request to do this I thought about sending a message from the server to the client and once the client detects this message he would proceed to execute this command.
The problem is, I also do not know how to do this. Can anyone please help me?
As long as you're sure the name doesn't collide with existing LSP methods, you can define arbitrary methods of your own. For instance, in the official lsp-sample, you could do this:
(at the end of client/src/extension.ts)
let client = new LanguageClient('lspSample', 'Language Server Example', serverOptions, clientOptions);
client.onReady().then(() => {
client.onNotification("custom/loadFiles", (files: Array<String>) => {
console.log("loading files " + files);
});
});
context.subscriptions.push(client.start());
(in the documents.onDidChangeContent listener of server/src/server.ts)
var files = ["path/to/file/a.txt", "path/to/file/b.txt"];
connection.sendNotification("custom/loadFiles", [files]);
This will output the following to the dev console whenever you change the contents of a .txt file (since the sample uses plaintext as its document selector):
loading files path/to/file/a.txt,path/to/file/b.txt
You pretty much have complete flexibility here when it comes to the names of custom methods, their parameters or when you invoke them. It's quite common for language servers to use custom methods like this that are not part of the protocol for various purposes (advanced functionality, internal debugging/development features, etc...).
I am diving into Node.js and have had success installing express and socket. I was able to make the basic chat program easily. This requires these steps:
In app.js (my Node server)
var server = require('http').Server(app);
var io = require('socket.io')(server);
I included minified socket.io.js file in public/javascripts
In public/javascripts/testchat.js simply call IO like this:
var socket = io();
Then do whatever you want with IO stuff.
I've made my own Node.js module but I don't understand how to expose it.
In node_modules/myModule/index.js
module.exports = require('./lib');
In node_modules/myModule/lib/index.js
module.exports = myModule;
function myModule(){
this.name = "hello";
}
myModule.prototype.test = function(){
console.log(this.name);
}
I can call this stuff easily in app.js
var myModule = require('myModule');
var myMod = new myModule.test(); //hello
But I don't want this thing to run every time I start the server. I want to to only run when a user accesses a specific page. How do I get from here to there? I know I am missing something... I was expecting to be able to do this:
In public/javascripts/myModule.js
var myModule = myModule(); //myModule is undefined
myModule.test();
Thank you.
You can request info through socket.io. It tends to be a bit resource intensive to hold the network connection open. Typically, most of the games and apps that we write, either for a mobile app or a web site, call REST APIs that we have written in Express, which is a node.js framework. Essentially you create endpoints that respond to HTTP calls. They send back whatever you want, we normally use JSON to respond with the data. So, you would need to create a RESTful api app, and behind one of the APIs would your module. You call the API with whatever parameters from your web page, and the app responds with your data.
Then of course you need to get into authentication, so only the people you want to have access get it. You've got security and scaling issues. It's a new world.
Problem
I need a way to store and collect JSON data in an entirely offline(!) web application, hosted on a local (shared) machine. Several people will access the app but it will never actually be online.
I'd like the app to:
Read and write JSON data continuously and programmatically (i.e. not using a file-upload type schema)
Preferably not require any software installation other than the browser, specifically I'd like not to use local server. (edit: I may be willing to learn a bit of Python if that helps)
The amount of data I need to store is small so it is very much overkill to use some sort of database.
Solution?
My first thought was to let the html5 file API, just read/parse and write my JSON object to a local txt file, but this appears not to be possible?!
Local storage is not applicable here right, when several people - each with their own browser - need to access the html?
Any ideas?
note
I know this topic is not entirely novel, but I think my situation may be slightly different than in other threads. And I've spent the better part of the last couple hours googling this and I'm none the wiser..
Have you considered Python's Json module? http://docs.python.org/2/library/json.html
Using this you can convert python objects to and from json strings. You can store the strings however you want to
You can't use Localstorage to enable such features because every client will have its own dataset stored.
Have you ever considered using a java applet to handle such informations ?
You could start a java applet using it as a bridge between browser clients and as a store of informations.
Browsers could share such information using websockets.
Some times ago I build demo with such solution.
Check it at: https://github.com/KingRial/SBrower
In this demo I open a browser/tab which starts a java Applet to create a websocket server.
All the browsers/tabs are just clients connecting to the websocket server and sharing informations.
Since python is one of the question tags, I am giving a python solution:
import json
#reading
file_json = open("json.txt")
print file_json
python_json_object = json.loads(file_json)
print python_json_object
file_json.close()
#writing
file_json = open("json.txt", 'w')
file_json.write(json.dumps(python_json_object))
My suggestion would be something like WampServer (Windows, Apache, MySQL, PHP). I've seen a few tutorials about adding Python to that mix.
You would have access to reading and writing JSON data to the local storage or placing your data in a local database.
I know you said you don't want to opt for local server, but nodejs could be the solution. If you know JavaScript, then it's very simple to set one server up and let everybody access to the server from any browser. Since it's entirely JavaScript you don't even have conversion issues with the JSON format.
For storing the JSON you can use the FileSystem built-in library of nodejs which lets you read and write from a file, so you don't even need a database.
This is using Node.js and Express.
const express = require('express');
const http = require("http");
const app = express();
const server = http.createServer(app);
const fs = require('fs');
const newestData = {
"id":123,
"title":"spoon"
}
app.get("/",async(req,res,next)=> {
res.status(200).send('Hello World!');
fs.readFile("Metadata.json", 'utf8', (err,content)=>{
if(err) throw err
let data = JSON.parse(content)
data.push(newestData)
fs.writeFile("Metadata.json",JSON.stringify(data), (err)=>{
if(err) throw err
})
})
})
const port = process.env.PORT || 5000;
require("dotenv").config();
server.listen(port, function() {
console.log('Express server listening on port ' + port);
});