automating web activity which includes form submission, javascript, and SSL - javascript

I'm attempting to automate login into my small community bank, navigating through several links, and then downloading the .OFX.
I can't seem to find a good open source tool which is robust enough to handle this case. Automation frameworks like mechanize can't handle the Javascript.
I need this to be run in a graphic-less environment (via cron), so I don't think I can depend on tools such as watir which control standard browsers.
Any language is fine, although a scripting language is preferred.
Suggestions? Can I tap into the browser engines without displaying the browser?

Take a look at WWW::HtmlUnit. It is a Perl module that uses a Java library (included in the Perl module) that makes it easy to automate web pages, even those with JavaScript.

Use webdriver . The API page says you can toggle the visibility.
If webdriver doesn't fit you, you can use COM directly to automate Internet Explorer, you'd just have to read the documentation page on MSDN.
I'm not really sure how they will react to the graphic-less environment though.
Another option would be to use HtmlUnit, which can handle JavaScript, and emulates a browser, so you don't need to worry about interfacing with a real browser.
As for the scripting part, you can use groovy which compiles to bytecode. If you know Ruby, you'll have no trouble picking up groovy. Here's a link that will help PLEAC groovy

This may do what you want: Test Run: Web UI Automation with Windows PowerShell:
Dr. James McCaffrey - March 2008 Here
we show you how to use Windows
PowerShell to create quick and easy UI
test automation for ASP.NET and
classic ASP Web applications.

Related

Is it possible writing a cross-platform desktop application with XHTML, CSS and JavaScript?

Is it possible writing a cross-platform desktop application with XHTML (or HTML), CSS and JavaScript?
If the answer is yes, how to run this application? Should I run it with a browser like Firefox?
Note: Currently Windows supports HTML Application.
It seems that what you're looking for is the Open Web Apps
HTA is really Old technology.
Try tools like Titanium (Appcelerator). They are tools, which render HTML content inside an APP. So, from outside, you will be running an executable but inside , it is loading a HTML file.
Same principle is being used in many apps now a days. Facebook for android is a best example
If client-side JavaScript, and/or HTML/CSS hackery accomplishes the task you wish it to, yes, and with a user agent like Firefox, indeed.
Yes. With HTML5.
The other answer's solutions are great, but generally they assume that the user is connected to the internet to have a server-side that handles the data.
But it's possible to use Open Web Apps (as ZER0 suggested) with HTML5, which has a lot of support for offline web apps.
Pay attention to it, though. Even HTML5 assumes that eventually the user will have some sort of internet connection, and offline functionalities are seen as a "cache". So, I'm not sure that this solution is enough for every requirement, because I don't know how much space HTML5 provides to offline applications

What is the simple environment setup to learn javascript in Mac

I am new to apple and would like to learn to program web sites in mac. I am interested in learning Backbone/Knockout js. Is there any write up available that I can use to kick start my learning experience? All my development on the two technologies were using Windows ASP.Net under IIS. What are the things I need to setup a simple development environment to learn and test above mentioned technologies in Mac?
(Your question is pretty broad, and I'm assuming that you know relatively little about JavaScript and are just looking to start writing JavaScript applications on your Mac. Note that this is not going to be much different from writing them on a Windows machine, but I have little experience with developing web applications on Windows machines. Hopefully my answer is within the scope of your question.)
For JavaScript development, all you need is a text editor and some sort of JavaScript engine. As far as text editors go, TextMate is the most popular Mac app of that sort (although it requires a license). You could also use the built-in TextEdit app (although it's horrible IMO), a command-line text editor (such as Vi or Vim), or a different editor such as Sublime Text. But in a nutshell: JavaScript does not require any specific IDE or anything like that. Use whatever you like.
As far as a JavaScript engine goes, since you're looking to do web design, you really just need a web browser. The major browsers for Mac (Chrome, Safari, and Firefox) all have advanced web design tools (the built-in inspectors in Chrome and Safari, and the Firebug add-on for Firefox) that will allow you to see the JavaScript console, network requests, etc. You can even fire those up and write JavaScript statements directly into them. You should explore those sooner rather than later, as they'll prove immensely valuable.
I can add Jetbrains WebStorm with awesome javascript/coffeescript intellisense. And node.js as server.

How to change registry settings on a system through web browser

I am trying to develop a web page that will allow user to edit registry settings in windows system. Can i achieve it with client side scripting language.? If yes please suggest me language to do.
Can we do it with jQuery or any other type of library.
Due to obvious security concerns, this is only possible in Internet Explorer(!). This is not a jQuery library, but an activeX control; so it's quite unpleasant to use.
You have been warned, so here is the documentation :
http://technet.microsoft.com/en-us/library/ee156602.aspx
Fortunately is impossible to access the registry from a web app: the only way you have is through an ActiveX control but I would not go down this road.
have a look at the below
Access registry from a web aplication
Far from ideal but ...
If you serve up a ".hta" file (HTml Application) from your web server, Windows will run it as a program outside of IE and give it the privileges of the PC user. It will be in a separate window and there won't be any browser features (Back/Refresh/Address bar etc).
Even then, modern versions of Windows will prompt the user with security warnings if a HTA is launched from anywhere other than a local drive.
I know this thread is old, but I am not sure I like any answers for this problem. Instead of trying to access the Registry directly through Javascript, try writing a Java Applet and talk to the java applet using Javascript. Then in the JavaApplet you can write some JNI code to write a native dll to do what you need. It isn't a direct solution to your problem, but it will allow you to do what you need across multiple browsers. The downside is that you can't use it on browsers that do not support running a Java Applet, such as a mobile platform.
This method will also require you to sign your Java Applet. This is how you get around the security issues. The user will have to accept the applet the first time to give the security access.

Building a Windows app w/ embedded browser & JavaScript 'hooks'

I built a PHP / JavaScript website for a customer. Then they asked me to replicate it except as a standalone Mac application. I did this with an app that combined an embedded web server, PHP, and 'WebView' - a Cocoa-ish version of the WebKit web browser that I can embed in a standard app window. This all worked great - I got to reuse 10,000+ lines of PHP/JS code, which saved months off of re-implementing it all again in 'native' code.
Now they want a Windows equivalent. I'm reasonably confident I can get PHP and the web server to work. And I know embedding basic IE functionality is pretty easy.
However...in my Mac setup, WebView (via the windowScriptObject stuff) gave me the ability to call JavaScript methods from C++. For instance, I might call a JavaScript method from C++ to update the screen. Likewise I could set things up so that a JavaScript call in the browser could trigger a C++ method - I used this, for instance, to let a user click 'BROWSE' and pick a file path using a real, standard file browser.
So my question is, is there a Windows-based embedded browser setup that would let me interact with JavaScript in this same way?
(the JavaScript <--> WebKit interface is described in much better detail here: http://lipidity.com/apple/javascript-cocoa-webkit/)
Maybe try using something like Appcelerator Titanium so you'll be ready when your client says they want it to work on Linux, or iPhone, or Android.
Quoting Wikipedia: "Appcelerator Titanium is a platform
for developing mobile and desktop
applications using web technologies.
[...] Support for standards-based web
technologies: HTML, CSS and Javascript
on all platforms along with PHP,
Python and Ruby for desktop platforms.
Integrated support for popular
JavaScript and AJAX Frameworks
including jQuery, YUI, MooTools,
Scriptaculous and others."
Sounds like a perfect tool for the job.
When you embed the Web Browser Control (IE), your application code can simply call execScript (http://msdn.microsoft.com/en-us/library/ms536420(v=vs.85).aspx) on the window object. You can have your script call out to the application by using the window.external object from the script, and by using the ObjectForScripting (or C++ equivalent) from the application.
maybe Qt will be good for your case, also you have QtScript and can inject javascript with evaluateJavaScript
I found a great example on the web for invoking JS in my embedded browser from C...basically using COM-ish methods that let you get a DISPID from a script object, and then using the Invoke() method with that. This works great.
But it turns out I need to also call C++ funcs from my JS code. It appears this is possible, and after hours of messing around I think I almost had it - it's like the above in reverse - you create a COM object, then hook it to the browser's script object - but in the end I could not close the deal - I kept getting "library not registered" errors. Honestly I don't know COM well enough to do this right.
So then I, for the heck of it, tried building my first C# app. In about 20 minutes I had an app running with a browser where I could both invoke JS inside of it and have the browser invoke C# methods. Geesh. I'm a believer in .NET after this experience, and a confirmed non-believer in 90's Microsoft technology.
In the interest of completeness, I'd like to mention my Windows port of WebKit, which includes the various cross-layer features of WebKit on the Mac.
I posted some example code showing how to embed WebKit in a native WinAPI application, complete with JavaScript->C++ and C++->JavaScript examples.
The example is a tiny test case for a much larger application using embedded WebViews for major UI components. I can confirm that what you are doing is not only possible, but a great way to build an application.

Javascript (and HTML rendering) engine without a GUI for automation?

Are there any libraries or frameworks that provide the functionality of a browser, but do not need to actually render physically onto the screen?
I want to automate navigation on web pages (Mechanize does this, for example), but I want the full browser experience, including Javascript. Thus, I'd like to have a virtual browser of some sort, that I can use to "click on links" programmatically, have DOM elements and JS scripts render within it, and manipulate these elements.
Solution preferably in Python, but I can manage others.
PhantomJS and PyPhantomJS are what I use for tasks like these.
What it is, is a headless WebKit based browser which is fully controllable via JavaScript. There's a C++ implementation (PhantomJS) and a Python one (PyPhantomJS). I prefer the Python one though, because it has a plugin system which allows you to add functionality to the core without actually modifying any code, unlike the C++ one. :)
There is an absolute ton of free software technology now available: take your pick at http://wiki.python.org/moin/WebBrowserProgramming but if you have specific questions join pyjamas-dev on google groups and i'll be happy to give further details, there. brief answer: you can run pywebkitgtk "headless", or you can use xulrunner (via python-hulahop) again using pygtk without actually doing "browserwidget.show()", and there's also pykhtml. also you could use python COM to connect to MSHTML.DLL.
these are all "cheat" methods: using python bindings to a graphical web browser engine without actually firing up the graphical bit. if you really wanted to put some serious hard-core programming in, you could create a "port" of webkit which was not connected to a GUI toolkit: as an experienced webkit programmer i'd put it as around... 2 weeks of full-time effort to make such a "headless" version of webkit.
l.
Looks like http://watin.sourceforge.net/ might be a good way to go.
If you don't have to go pure Python, you could do IronPython since it's a C# project.
take a look at this little doosy on ajaxian
http://ajaxian.com/archives/server-side-rendering-with-yui-on-node-js
It also talks about Aptana Jaxer which I think runs on a headless firefox so is basically the Mozilla browser engine in all it's glory.
There is Kapow. Its pure Java and costs money:
http://kapowtech.com/
And there is Lixto: Its Eclipse based and uses Mozilla Gecko as rendering engine (unless they already changed it to WebKit, as they said they'll do years ago). Its very nice and also costs money:
http://www.lixto.com/?page_id=50
They are both graphical tools where you define the site navigation and what should be extracted by point and click. But you can also write xpath and regular expressions and even JavaScript that runs in the sites context.
I used them both in the lectures web data extraction and applied web data extraction at the technical university Vienna (Lixto is written by the Professor who held the lecture).
HTMLUnit in Java is very good. I think it's only the Java implementations of headless browsers that manage to provide Javascript support.
MaxQ, I read about here, sounds like it might be interesting: "written in Java, generates Jython scripts"
Try HtmlUnit !!!

Categories