I've realized I need a full-fledged browser automation tool for testing user interactions with our JavaScript widget library. I was using qunit, starting with unit testing and then I unwisely started incorporating more and more functional tests. That was a bad idea: trying to simulate a lot of user actions with JavaScript. The timing issues have gotten out of control and have made the suite too brittle. Now I spend more time fixing the tests, then I do developing.
Is it possible to find a browser automation tool that works in:
Windows XP: IE6,7,8, FF3
OSX: Safari, FF3
?
I've looked into SeleniumIDE and RC, but there seems to be some IE8 problems.
I've also seen some things about Google's WebDriver, which confusingly seems to work with Selenium.
Our organziation has licenses for IBM's Rational Functional Tester, but I don' think that will work on the MAC.
The idea is to try to run tests on all the browsers our organization supports. Doable? Are my requirements unrealistic? Any recommendations as far as software to try?
Thanks!
I would recommend using Selenium but I say that as a Selenium Committer.
Selenium works on any browser that supports JavaScript since the framework has been written in JavaScript. This means if your browser on any OS supports JavaScript it will run in Selenium. That documentation it out of date, you can see that since it is talking about IE8b1 and IE9 preview is out now.
Selenium and WebDriver (which isn't a Google thing since it started at ThoughtWorks) are currently being merged as they both have their strengths and weaknesses. The current merged work will be called Selenium 2 and you can start using the alpha release now at http://code.google.com/p/selenium/. It will still work on any OS as that is still the main driving force behind the work being done.
Selenium IDE only works on Firefox because it is a Firefox add on.
I personally would avoid Rational Functional Tester because it has a lot of weaknesses that its not even worth contemplating.
If you start with Selenium there are some tutorials on my site at http://www.theautomatedtester.co.uk
Try Sahi (http://sahi.co.in/) It works across browsers and operating systems. It has a powerful recorder, and great APIs for object identification. It supports HTTPS, proxy tunneling etc. and has drivers in sahi script, java and ruby. It also has parallel playback inbuilt. It is 5 yr old mature project hosted on SourceForge, with releases almost every month.
It automatically waits for AJAX and page loads, and does not use XPaths for object identification. It also handles sites with dynamic ids.
Selenium is probably your best bet out of the tools you mentioned. What are the issues it has with IE8? You might want to check out HttpUnit to see if that meets your needs, also.
Selenium RC is a great tool if you invest the time to use it. With significant modifications to the existing library I've gotten it to fulfill all of my front end testing needs.
The confusion you are having about Webdriver is understandable. Selenium 2 is in development and will be a merge of Webdriver and Selenium. Check out: http://www.youtube.com/watch?v=RQD4EzWI4qk to get more detail.
The only browser that I have found to be unusable with Selenium is IE6. IE7 and IE8 work fine as does Firefox (which I have modified to include firebug for debugging purposes).
I'm in the same boat. It is a difficult problem to solve. Windmill and Selenium are the 2 best I've found. Though they both have issues. Selenium can only record scripts in Firefox and I haven't managed to get the proxy chaining to work as advertised. Windmill you can record in any browser and you can supposedly tweak the proxy to put extra logic in there, but the js mechanism for recording across page loads has been in my experience very brittle at least on the app I have to test.
I don't think anyone can get it quite right as long as there is more than one browser that needs to be supported.
Maybe have a look at SIKULI. It's a different paradigm but, depending on what you want to test exactly, it may do the job and will work with any browser, on any platform.
Have a look at their official blog for some examples of interactions with web applications.
So I wrote some of my more problematic tests in Selenium RC, using the Python driver. It was a better experience than writing the same tests in pure JavaScript, but I still had some of the same issues.
Testing something like an ajax autocomplete widget, meant forking some of the code depending on IE, or Firefox, and I still can't get typeKeys or a combination of type with typeKeys to work in Safari.
So, I am not sure if having cross browser clean, extensive ui tests is a bit unrealistic.
Should I try webdriver/Selenium 2? Would that make things better, or is that product not ready for prime time yet? How's the Python binding for that? I don't know Java, but I would learn some if need be.
Related
I'm making a game with HTML/CSS/Javascript because it's the code I'm most comfortable with at the moment. I'm only really doing this as an exercise in game development and plan to learn C# later. But for now that's what I'm using, and I have a question about it.
Obviously when making a website, you want your website to be compatible with all web browsers equally. Right now, I'm using Chrome to test/debug my game, and I've decided to develop this game with Chrome in mind. But not everybody has Chrome, and not everyone would want to download it in order to play my game.
Is there a way to run an HTML/CSS/Javascript file in a Google Chrome "environment" without having the actual browser installed? Just it's code engine and none of the rest of the browser.
I've been reading about their V8 Javascript engine that they use in Chrome, and am wondering if that is part of the answer I'm looking for. What I'd like to do is include this "engine" in an installer with my game files and have it install like any other game.
Hopefully this makes sense. This may not be possible/exist, but if someone knows something I don't or an point me in the right direction, that'd be amazing. Thanks!
You could look into Node-webkit which essentially allows you to write desktop applications in html/css/js. When you distribute your game along with your node webkit executable, it is always run in the same environment. You can see some cool examples on their demos and examples page.
Usually a common path to convert web application to native desktop applications is to use a "thin" browser as app container and ship it.
A lot of current applications out there are using this trick (Spotify, Slack, etc...) and it works pretty well.
I've read of some people using the CocoonJS game engine framework and successfully ship it with this techniques.
To most famous wrappers, that I know are node-webkit or electron (AKA atom-shell).
Once you include your game in either one of those you can just "compile" it (it is not a real compile, but just to give you an idea) and ship it: with some tricks it is also possible to publish it in the Mac/Win app stores.
In case you want to focus on mobile instead, there are similar frameworks but I don't really know which are the most common.
Note: if you're using fancy WebGL or very advanced stuff these tools may have some issues sometimes.
So essentially you want to install the JS engine to use it with any browser? In this case, the answer is: nope. Browsers act different, they don't have a standard interface, nor have this "swapping" capability in mind.
In case you'd be asking for an embedded browser inside an app... well, isn't that worse than installing Chrome? You may embed webkit/V8, but it's a hard way and you'd know programming.
So simple answer is: you'd make it compatible for at least the evergreen browsers (Chrome / Firefox mainly). Or reduce your target to webkit based only browsers (or in your case V8, i.e. Chrome, Chromium and the forks).
If you want that your game is only for chrome, because you read V8 documentation, you can create it as an extension. There'sn't other way to install your JS, because browser interpret javascript, not compile it. And the docs you need is found at: https://developer.chrome.com/extensions/getstarted
Currently, I am using PhantomJS for running Javascript unit tests in QUnit and Sinon framework on our build server.
But, PhantomJS uses JavaScriptCore with JIT compiler as its Javascript engine. Instead, I want to use the V8 engine, which is used in Google Chrome, or Chakra, which is used in IE. I want to do this because I want to check platform compatibility for the code.
Are there any popular test runners like PhantomJS, which use these engines?
The closest I can think of is Zombie.js, which is a headless browser written in Javascript that runs under Node.js.
It's not a genuine browser in the way that Phantom is, so there are things you won't be able to do with it that you can do with Phantom, but since it uses Node.js, it obviously does use the V8 engine, so it fulfils your criteria.
But if you really want to test in all the browser's various engines, your other option is, of course, to use a real browser. You don't have to have a visible UI for it; use a tool like Selenium or Sahi, which can launch and run the browser from a script, and have it run in a VM; you don't ever need to even look at it. It may not be as quick as using Phantom, but it will be a genuine test, which is clearly what you're really interested in.
[EDIT]
Worth adding a note to this answer because I recently found out about SlimerJS, which is an open source project aiming to produce a PhantomJS-compatible browser that uses the Gecko engine. Again, this isn't exactly what was asked for in the question, but it is in the spirit of it; it's great to have another tool available to make cross-platform testing easier.
Some code in my page is making my browser slow after 20-30 min. I need to know which one. What tools can i use to debug this out.
Following js files are being loaded
Jquery
Jquery ui
History
Mustache
Yes i had settimeout and thought that to be the culprit but alas after removing it too, it's still a bit slow.
You should use Google's Speed Tracer. It's a Chrome extension.
Speed Tracer is a tool to help you identify and fix performance
problems in your web applications. It visualizes metrics that are
taken from low level instrumentation points inside of the browser and
analyzes them as your application runs. Speed Tracer is available as a
Chrome extension and works on all platforms where extensions are
currently supported (Windows and Linux).
Alternatively, you have Yahoo!'s YUI 2: Profiler.
The YUI Profiler is a simple, non-visual code profiler for JavaScript.
Unlike most code profilers, this one allows you to specify exactly
what parts of your application to profile. You can also
programmatically retrieve profiling information as the application is
running, allowing you to create performance tests YUI Test or other
unit testing frameworks.
An addendum to #Julio Santos ' answer
You can use Dynatrace Ajax which has a good free version of their product
Is there anything that provides a console, error logging, etc. for testing Windows Gadgets? I've searched but I'm unable to find anything.
Thanks in advance!
A simple and easy start is to just use any web developing toolkit. I found out myself that for developing simple Windows Gadgets, Notepad++ and Firefox with its Web Developer toolbar was more than enough. You can implement all the business logic and just test the results in a browser. Actually starting it as a Gadget was only needed to fine-tune some graphical glitches that always occur because of different browsers still interpreting CSS a little differently.
Firefox has some nice tools for debugging Javascript.
You can use firebug lite for IE (i.e put a script tag and include the latest firebug lite version).It does look a bit crude but it works nonetheless and its seamless integration for testing.
http://getfirebug.com/lite/ie.html
Though your gadget might seem a bit mutilated when using it.
:)
We have a web application that makes extensive use of AJAXy Javascript in the UI. We have nearly complete code coverage of our backend using Shoulda and Webrat, and would like to extend our test suite to include full integration testing through the Javascript UI.
We tried Selenium but found it brittle and temperamental. Are there more reliable options?
UPDATE
For those still checking this out, we ended up using Xvfb so we can run Firefox without a screen. Allows us to run the test on a headless Jenkins CI server. We still have to run tests "live" locally occasionally to debug, but it works pretty well.
One of the JavaScript gurus where I work recently pointed out PhantomJS as an interesting tool for testing our JavaScript-heavy web applications. We haven't tried it out yet but the idea of a headless WebKit for DOM testing sounds promising to me.
This is something I have been wrestling with for a while, as I am doing some work with ExtJS (a very powerful JavaScript UI builder for the browser) and Rails.
After having researched quite a few different options. I still haven't found a perfect solution for it. Ideally, I would be able to run them headless and just report on the output. Unfortunately, none of the emulators out there seem to be able to run JavaScript with full DOM support seamlessly (at least, none of the options I've found are). So that pretty much means that you have to run your full-powered JavaScript code in a real interpreter (such as a browser). Webrat with Selenium works acceptably well, assuming you're willing to deal with the pain of trying to path out your requests to the UI properly. If it's your own JavaScript that you're implementing it against, that may be easier. But when it comes to a third party UI library that you don't have much control over, it can certainly get, shall we say, interesting.
Probably not the most helpful response, but that has been my findings up to now!
Hmm I would give Capybara a look, it can use selenium-webdriver (not to be confused with selenium-RC, they are different) for javascript testing. I haven't found it very brittle when compared with Webrat... it seems to be fairly consistent.
As Chris Rueber says, there aren't really any headless DOM interpreters that support JS as well - for now it's fire up a web browser for your automation or write unit tests in the javascript itself (Which isn't really integration testing either).
When you have a lot of selenium-webdriver-backed tests they can take awhile to run sometimes, but it's surely better than no tests at all.
check out the gem jasminerice to test your js logic.
https://github.com/bradphelan/jasminerice
for the integration test I would recommend to use rspec with capybara as acceptance tests. distinguish request specs and acceptance specs!
another possibility is to use turnip as an alternative to cucumber.
https://github.com/jnicklas/turnip
to speed up your tests test headless. You could use capybara-webkit (depends on qt) or poltergeist (which depends on phantomjs).
both are easily to set up. I prefer poltergeist.
There are a couple of gems you could use if you didn't like Selenium.
The one I recommend is Jasmine: https://github.com/pivotal/jasmine
You can also check out Culerity: https://github.com/langalex/culerity