Whats the difference here? I want to create a small API for queueing jobs, but I am not sure which should I be using. I'm leaning towards just using Node, but I do understand what is the point of having a web server module for PhantomJS.
From the PhantomJS docs, the Phantom webserver is still experimental and intended to manage other phantom scripts and provide an interface to those scripts from the web. It currently only supports up to 10 concurrent requests. I'd recommend using node if you are looking for a general purpose web server.
Related
I seem to be confused by conflicting sources, yesterday I was reading the node docs and was sure Node's 'net' and 'http' modules had web socket capabilities, but maybe I misunderstood the documentation because today an article said that node has no built in web socket support.
Can you create a node server that can handle web socket connections with just node and javascript, no external libraries?
Node does not have native support for websockets like it does for http or tcp (net) connections.
It's been discussed a few times, and rejected for various reasons of the last few years. The current discussion is going on here: https://github.com/nodejs/node/issues/19308
You can, of course, implement a websocket server yourself using the native modules, but you'll need to do a lot of boilerplate work.
To see both an example of what you'd need to do to implement your own websocket server using node and a good pre-built library you can use to work with websockets in node, I would reccomend taking a look at: https://github.com/websockets/ws
Sure, you could... if you re-implemented the functionality from those Web Socket libraries yourself.
The Node.js core libraries are minimal. They're only intended to cover the basics needed to function, and the most common use cases. As much functionality as possible is left to modules outside of the Node.js core. This is a very intentional design.
It would be incredibly irresponsible to implement your own Web Socket library without a very good reason. You should rethink why you don't want to use existing libraries.
I have a little question here. Is there any way to build a server side JS without any use of external software. The reason for this is I am not able to install any software on the webspace but I wanted to experiment with realtime communication with the server and another user for a game or something like this.
If someone knows a way to establish something like this I would be quite happy to hear about it.
EDIT: Guys NOT NodeJS I AM NOT ALLOWED TO INSTALL ANY OTHER SOFTWARE!
https://cloud.google.com/nodejs/
or
https://devcenter.heroku.com/articles/getting-started-with-nodejs#introduction
Those $15 a year webhosts typically don't allow you advanced customization. You need a better web hosting service that allows you CLI access to the server back end.
Running JavaScript on backend requires NodeJS runtime environment. Download it here https://nodejs.org/en/
Alternatively you can can configure an instance on Heroku and run your application there. But I would suggest to try the first one.
No. Javascript is a scripting language that was designed to work within the browser. There's no out of the box solution for you, as Javascript doesn't do server-side out of the box
However, Node.Js (and others) use Chrome's V8 engine to create an event-driven runtime environment which can allow writing server-side code with Javascript. That's your best option
I work with a team whose only way to get a user in their company's database, is to navigate through and fill out ~5 or so pages of web forms in their browser. Truly brutal stuff. I've developed web automation scripts in VBScript, Java (w/ Selenium WebDriver) and iMacro, but all of these solutions are slow. They also depend on the browser, which I'm trying to move away from.
I'm looking for a new platform, possibly some scripting technique/language that will allow me to issue HTTP requests and read HTTP responses, then build my script around there. The script would perform calculations on the HTTP responses, use File I/O and use this data to issue further HTTP requests. Again, I'm just spitballing here. If anyone else has a better solution, I'm all ears!
My question for you is: Accepting the team's limitations (read-only DB access), how would you approach a solution and what tools/languages/platforms would you use to do so?
Broad and ambiguous answers are welcome. Thank you for your time.
I agree with #Grisk on using NodeJS/ioJS as a platform. It is a powerful tool designed from the ground up for I/O, making it perfect for solving your problem. Additionally, the node community is extraordinarily vibrant, with npm, the nodejs package manager, hosting thousands of easily accessible modules. To avoid any future confusion: don't mistake NodeJS for a language or a backend framework; it is a native javascript interpreter built atop Google's V8 engine as well as a set of built in modules to build powerful I/O applications. Read up about node online.
As for your specific problem, I'd say you have two options:
To feign being a browser using phantom cookies
By programmatically navigating through the website as you have been doing.
As for the former option, you'd need to manually determine which cookies are sent to the server when forms are submitted on each page and then in your script generate these cookies and include them in the http request. Check out the nodejs http documentation for more information on customizing the headers of requests.
You're header will need to look something like this:
var headers = {
'host': < website host address here > ,
'origin' : <website origin here>
'referer' : <website origin here>
'User-Agent': 'Opera/9.52 (X11; Linux i686; U; en)',
'Cookie': <cookie sent over by server here>
}
I recently came across the node-icloud library, which uses the first method I describe above to provide programmatic access to one's icloud account. I strongly suggest reading through its code to see how it works here.
Additionally, I'd suggest reading up about http headers here
For the second option, check out phantomjs and zombiejs. Phantom is nice because it works without a browser. I'm not sure how the speed of these two libraries compare to what you have already been doing, but they are worth testing out.
One last thing: I would recommend building a custom (JSON)DSL for automating interaction with webpages so you can very easily redesign your browser interaction workflows.
Additionally, if you choose to use nodejs, an understanding of node streams and the details behind its event loop would be beneficial.
Best of luck!
I would start looking into NodeJS as a platform. The HTTP library is an incredibly powerful method for writing applications that need to make multiple http requests with unusual structure and it can communicate easily with a browser or basically anything else you could possibly need. Look at using the FileSystem class if you need to do file I/O.
If you wanted to get really fancy and use websockets to build a dynamic webapp that you can use as a front-end for your tool, you could even do that, so there's a lot of flexibility.
I'm looking into using PhantomJS to generate static html from a dynamic AngularJS app that can be indexed by google. What I want to do is to start a PhantomJS server that sits behind a proxy and gets the ?escaped_fragment requests. PhantomJS appears to be (mainly) a command line tool (I have read the FAQ explaining why it's not a regular node module), and although I have found a couple of nodejs bridges for it, they appear to be a little bit unreliable.
Therefore, I'm looking into running PhantomJS with an embedded HTTP server. I have seen some examples of a built in webserver in PhantomJS, but I'm not sure if it's meant to be used this way? If not, is it possible to have PhantomJS use regular node modules, like e.g. expressjs, so I can use the PhantomJS runtime to also host a simple webserver?
The bridge node-phantom isn't unreliable (phantom-node is unreliable, and overcomplicated, so don't use that one).
Phantom itself can sometimes be a bit unreliable, but it tends to be with specific web sites.
I'm not convinced Phantom is the right solution for you though - you might want to check out JSDom instead, and just have your code run in-process.
I have been using Jasmine to write BDD tests for a web app. I am interested in running these same tests from the command line using Rhino, and I found this blog entry which has been very helpful. However; my app uses pusher, which offers a simple API for passing events between clients using web sockets.
My tests run fine from a browser, but running them via Rhino fails to connect to pusher. Pusher offers a debug stream, and it would show authentication errors or other failures, but it doesn't register any activity when I run my tests from the command line.
This may be a simple yes or no answer, but I haven't been able to locate any relevant documentation, and I'm brand new to Rhino so I apologize if this is a total n00b question.
My question
As the title says, does anyone know if Rhino supports the websockets API? Am I doing something else wrong here, or will scripts that rely on websockets simply not work via Rhino?
My overall goal was to use my already written jasmine tests as stress tests by running them on several EC2 instances at once; is there a good (simple?) way to do this while reusing my existing javascript tests, or should I suck it up and just write my stress tests in a server side scripting language? I am aware of selenium grid, but was hoping to avoid having to spawn new browsers to run these tests, if possible.
Thanks so much!
Ringo, a Rhino-based CommonJS runtime supports them: http://ringojs.org/api/v0.6/ringo/webapp/websocket/