I'm writing a Java library, actually, a Clojure library, but for this question, what matters is that it runs on the JVM. This library needs to execute some JavaScript. I tried with Nashorn but I encounter some limitations that might be too hard to overcome. As an alternative, I want to try NodeJS.
I want my library to be self contained, to not depend on the system running NodeJS independently and thus requiring a particular deployment mechanism to place the Java and NodeJS artifacts in the right places to be picked up by the two different network servers. This approach, though, brings some issues.
I will be talking to NodeJS over HTTP but I don't want NodeJS to open a specific port. I want to find a random unused one so there's no collisions. I also want to control where the logs from NodeJS go, as to keep them with the rest of my application. Lastly, my app should be able to detect when NodeJS crashed and re-run it or report an error with information.
What's the best way to approach this? Are there any Java libraries to help manage child process in this fashion? Anything in particular I should do from the NodeJS side (I'm very new to NodeJS, I never used it before).
My solution in the end was to use ProcessBuilder like this:
(defn create-process-builder [js-engine]
(doto (ProcessBuilder. ["node" (:path js-engine)
"--port-file" (:port-file js-engine)
"--default-ajax-host" (:default-ajax-host js-engine)
"--default-ajax-port" (str (:default-ajax-port js-engine))])
.inheritIO))
and then call start in it. inheritIO causes the output of it to go to the output of the current process which effectively merges stdout and stderr.
On top of that NodeJS opens a random port by specifying 0 as the port number and writes it to a file:
(let [app (-> (express)
(.use (cookie-parser))
(.get "/" (fn [_req res] (.send res "Universal JavaScript engine for server side pre-rendering single page applications.")))
(.get "/render" render))
server (.createServer http app)]
(.listen server 0 (fn [] (.writeFile fs (:port-file options) (.-port (.address server)))))))))
which then is opened by the Java side (waiting for it to appear):
(defn get-port-number [js-engine]
(or (with-timeout
(:start-timeout js-engine)
(loop [port-number (read-port-file js-engine)]
(if (string/blank? port-number)
(if (is-running? js-engine)
(do (Thread/sleep 100)
(recur (read-port-file js-engine)))
(throw (Exception. (str "While waiting for port number, process died: " (:path js-engine)))))
port-number)))
(throw (Exception. (str "Waited for " (:start-timeout js-engine) " for " (:path js-engine) " to start and report its port number but it timed out.")))))
There is a pretty good answer here on how to run javascript in java. Would something like that be doable for your case? If not here are some resources:
Random port in nodejs You could hit yet another service to find an open port during the build, or have your node app fire an http request to your java server based on the port it grabs.
Winston is the best logging library I've found, you shouldn't have any issues logging to the same path.
Forever and PM2 are the common node process managers which keep node running. I currently prefer forever (not sure why)
It sounds like you will be using a lot of cpu within node. If that is the case you will probably want to use the cluster module (so nodejs can utilize multiple cores). If you block the event loop (which cpu based processing will, then you will only be able to perform 1 concurrent request per forked process).
I have been in a similar position where I had to run fortran from a python script, so here is my two cents. Run your node.js script with a terminal command in Java like this:
Runtime rt = Runtime.getRuntime();
String[] commands = {"node example.js", "args"};
Process proc = rt.exec(commands);
BufferedReader stdInput = new BufferedReader(new
InputStreamReader(proc.getInputStream()));
BufferedReader stdError = new BufferedReader(new
InputStreamReader(proc.getErrorStream()));
// read the output from the command
System.out.println("Here is the standard output of the command:\n");
String s = null;
while ((s = stdInput.readLine()) != null) {
System.out.println(s);
}
// read any errors from the attempted command
System.out.println("Here is the standard error of the command (if any):\n");
while ((s = stdError.readLine()) != null) {
System.out.println(s);
}
With setup you can pass in parameters to your node program and get responses. Although I am not sure what you are using your node.js program for, so not sure if this is helpful.
Nashorn does have some issues that I've run into as well, such as finding information about some of their APIs (documentation leaves a lot to be desired), and the slow boot up.
What I would recommend instead: treat your server-side rendering as a service and not a child process.
In other words, you could run a Node.js instance on your internal net on say port 10015 and only allow local connections to it (you could also send logs wherever you want). Then you can make a request to the service with the pages that you want to render, such as localhost:10015/posts/ and have that Node.js app render the page inside of a headless browser (using something like Phantom or Slimer).
To keep your Node server up, you can use Forever or supervisord, and to help you gain traction faster, you could look at what the Prerender team has made:
https://github.com/prerender/prerender-node
Related
Ok, so I'm creating a webpage using socket.io in node.js.
This works great, and data is going to each device as it should.
Now i want to expand, and use node.js to control my pc.
I've read this: Child Process
and gotten to this code wich runs an executable, and prints its output in the console.
var spawn = require('child_process').spawn;
var ahk = spawn('outsideNode.exe');
ahk.stdout.on('data', function(data){
console.log(data.toString());
});
Here is the code, or script for outsideNode.exe that gets launched.
I'm using AutoHotKey for this.
Loop,10
{
FileAppend,External program spawned by Node.js says hi!,*
Sleep,1000
}
ExitApp
This works one way. Node.js captures AHK's output. Now, the problem i'm having is, how do i SEND data from node.js? And how do i recieve it in AHK?
I do belive i need to read about stdio. But i'm new to the std's.
My current solution is using Node.js to write a .txt file to disk with the command i want to give, and a seperate .exe file that reads the .txt and executes the command and exits. This works, but it is slow-ish, and just feels wrong to do it this way.
Also, I want it to be asynchronous, or synchronous? The one where i can do other stuff with node.js while the executable does its thing. Now i have to wait to be sure the file can be overwritten.
Any input on how to this is greatly appreciated.
There is the option to pass arguments to the spawned app. Maybe that's already what you are looking for. How to handle the arguments in your exe is another question related to some other programming languages. In node, child processes run asynchronous and non-blocking, that's the one you are looking for.
However, if you want to communicate between the programs, you can write to the stdin of the childprocess like so:
child.stdin.write("here you go");
Your exe must be able to wait for stdin in order to process the incoming data.
Despite my extensive experience with nginx, apache, jboss etc. servers, I am very new to nodejs server (It took my interest for socket.io features). I find it strange that unexpected data like say object.MyProperty (which is undefined) etc - which are trivial in my opinion, cause the entire server to crash (it doesnt crash for that client request, but the entire server crashes!) and you need to restart the server.
I am not sure if it is because I am on development mode or it will be like that on live mode as well. When it crashes, all runtime data get lost.
My question is that what should I do make sure that the server doesnt crash but it can write issues to the log file like in other servers.
Any assistance is much appreciated.
You can use try catch blocks for this. And log them under catch.
Updated:
Also, Node being single process based, uncaught exceptions leads to crash (of that process). One of the methods suggested is to use domains.
Example:
domain = require('domain'),
d = domain.create();
d.on('error', function(err) {
console.error(err);
});
d.run(function() {
...
Ref. is a good article and explains in short various approaches to handle the problem.
process.on('uncaughtException', function (err) {
console.error((new Date).toUTCString() + ' uncaughtException:', err.message)
console.error(err.stack)
})
Since nodejs is a single process server, you can place this code anywhere in js script for this to act as a catch all for unhandled/unforeseen errors.
I think you are looking for https://github.com/foreverjs/forever. Forever runs a script forever and restarts it in the event of a crash without user intervention.
You can install forever using npm. Once installed, you can run your nodejs scripts using forever start <path/to/script>. forever list will list all the currently running scripts and forever stop <script-id> will stop a given script.
Keep in mind that using this doesn't mean you don't have to do proper exception handling. Implement try...catch statements as needed and handle all exceptions in code.
The best way to do this would be for you to validate all your incoming post request bodies using a json schema validator. There are several available that are really fast and will add very minimal overhead (less than 1 ms) to your overall response time. I'm the author of one such validator called Themis which we use for REST APIs. There's also a comprehensive benchmark which compares all the available JSON schema validators.
This would prevent your sever from crashing and becoming unavailable simply because of a bad request.
I have a set of database administration tasks that need to run after some other rake tasks. So I read the Ruby Mongo driver source and a few blog posts and I ended up with this code:
scr = open("./scripts/update_stats.js", &:read)
db = Mongoid::Config::master
result = db.command({:$eval => scr})
logger.debug result
logger.warn "Finished updating stats"
In the script I have several functions and then one final one:
r = update_stats();
print("update result:");
print (r);
Ok, so it's all running fine when I run it directly via the command line. Well, it seems to be. But I'd like to take the result value and stow it into my logs. I could store the value into the database and then pull it out I suppose. Seems like there should be a better way. What I do get it my logs is:
DEBUG 2012-01-03 22:27:03 -0800 (21392) {"retval"=>nil, "ok"=>1.0}
That doesn't tell me much other than that it didn't blow up. So, how can I get the returned value of update_stats and log it in my Ruby code?
OK, so there are two potential issues here.
The first issue is that you are running a JS file against the database, but I'm not really clear how this works. It looks like some special feature of Mongoid? What result are you expecting?
The second issue is that it's not clear why you're using a JS file at all if the rake machine has driver access. What commands are you running that cannot be simply run as commands via the Ruby driver directly?
In general, the only time you need to bring in an arbitrary JS file is when you're calling jobs directly via the mongo program. So you would run mongo my_map_reduce.js from a cron job and this would make sense.
If the commands you're running are accessible via the driver (this is true 99% of the time). Then just write the whole admin script in Ruby.
I want to run an external program in XPCOM but I need to get the actual return string from the program, not just a return code. How would I do this?
All I've found is nsICommandLine, nsICommandLineHandler, nsICommandLineRunner and nsIProcess none of which return anything beyond an exit code (from what I can see).
Gecko doesn't have any meaningful IPC capabilities. While the process is running you can communicate with it via TCP sockets. If you want to return something when the process finishes - write to a file and your Firefox extension can read out that file then.
I usually type too much, so read bold copy if in a hurry.
I'm trying to develop a little app in a very restrictive environment (at work)... I want to read data from a database, but I cant install stuff on my machine (so my usual choice of using python or visual studio is a no-no). Basically I will have to do with whatever I've got at hand...
What solution can you come up with to access an odbc connection and read the records of a table in an environment where you can't install any software? feel free to suggest any language, as long as you don't need to install anything.
My best idea so far is trying to use the web-browser (since i only need notepad to code), so... basically using only HTML and javascript to try to access it (although I have no clue how to acomplish that task, as I've never done it before)...
I know it is not a good idea, but since I won't post this on internet (I only I would have access to this from my desk, and the DB is on my local network), I don't think security is an issue.
Even if I don't get a solution, I would like to hear what would you guys try if the need arose. But any ideas or links pointing me in the right direction would be appreciated.
Edit: For clarity's sake, it is a Windows environment.
You could use Portable Python, and Portable Notepad++. That way, you'll have nothing to install, and you'll still be able to use your preferred language.
If you like Django, you can have all that in a portable bundle : Instant Django
You don't mention OS or why the environment is so locked down, but if it's a managed Windows environment, you probably have Office installed. You may find that you can connect using Excel. (In 2003, it's under data -> import external data -> new database query, which brings up a list of ODBC connections.) Heck, if Office is installed, maybe you'll even have the Ultimate Root of All Evil program (a.k.a. MS Access).
If you have a web server on the db machine, you could write a Java (not javascript) application to access the db using java connector (or odbc). That's assuming Java is already installed on your machine.
Another possibility is to write an AJAX application to access the data server-side.
Not sure if you are on windows or not, but if you are: cscript.exe
If you have access to notepad you should be able to do it:
Something along these lines:
option explicit
dim conn : set conn = wscript.createobject("ADODB.connection")
conn.open("Driver={SQL Server};Server=127.0.0.1\sqlexpress;Database=tinker;Trusted_Connection=Yes;")
dim sql : sql = "select * from demos"
dim rs : set rs = conn.execute(SQL)
dim line
dim cnt
line = ""
for cnt = 0 to rs.fields.count-1
line = line & CHR(9) & rs.fields(cnt).name
next
wscript.echo line
while not rs.eof
line = ""
for cnt = 0 to rs.fields.count-1
line = line & CHR(9) & rs.fields(cnt).value
next
wscript.echo line
rs.movenext
wend
Save that to a text file with a .vbs extension and run it with cscript