I've never used Selenium but I guess it's for simulating user interaction in all browsers.
That's like integration tests.
But how do you test your js libraries/frameworks (unit testing) on all the browsers in an automated way?
For unit-testing you can try http://code.google.com/p/js-test-driver/
JsTestDriver consist of a single JAR file which contains everything you need to get started. For in depth discussion of command line option see GettingStarted.
Here is an overview of how JsTestDriver works at runtime...
You can have a look at TestSwarm:
project that I’m working on: TestSwarm...
Its construction is very simple. It’s a dumb JavaScript client that continually pings a central server looking for more tests to run. The server collects test suites and sends them out to the respective clients.
All the test suites are collected. For example, 1 “commit” can have 10 test suites associated with it (and be distributed to a selection of browsers)...
The nice thing about this construction is that it’s able to work in a fault-tolerant manner. Clients can come-and-go. At any given time there might be no Firefox 2s connected, at another time there could be thirty. The jobs are queued and divvied out as the load requires it. Additionally, the client is simple enough to be able to run on mobile devices (while being completely test framework agnostic)...
The best one imo is the one from YUI : http://developer.yahoo.com/yui/3/test/
But doing unit-testing in every browser is kind of hard... Most people just test with it during development and just use node.js to test later on in case they broke something.
As the referenced post in a previous article suggests, you could use js-test-driver.
Its specifically for JavaScript unit testing across multiple browsers, exactly what you want. I have messed around with it and it is pretty good. I haven't done any serious commercial testing in it though.
For unit testing you should look at solutions that do not load up a browser to do the tests.
You can look at something like RhinoUnit for that - http://code.google.com/p/rhinounit
Also have a look at Dojo Object Harness (DOH) unit test framework http://dojotoolkit.org/reference-guide/util/doh.html
Look at a similar question here which can give you an idea on how to TDD js - JavaScript unit test tools for TDD
It's no longer actively maintained, but I've still been happy with JSUnit for Javascript unit testing: https://github.com/pivotal/jsunit
It includes both an HTML/Javascript framework you can run in the browser, and a java-based test runner that you can invoke from ant.
To test multiple browsers in parallel you would Selenium Grid. Please take a look here: http://selenium-grid.seleniumhq.org/step_by_step_installation_instructions_for_windows.html for step by step instructions on how to use it.
I haven't used it so far, and it is still in beta, but FuncUnit declares itself as "A functional test suite based of qUnit, Selenium and jQuery".
There is also an infographic explaining how it works
Maybe it is something what you want? The github repo seems to be quite active.
You should check http://saucelabs.com its a cloud base selenium environment that allow you to build your test and then upload them, run them in as many browsers as you want.
You should consider the capabilities offered by crossbrowsertesting.com
(It is not for free). For js testing try JsUnit (http://www.jsunit.net/).
Quoted from its homepage
JsUnit is a Unit Testing framework for client-side (in-browser) JavaScript. It is essentially a port of JUnit to JavaScript. Also included is a platform for automating the execution of tests on multiple browsers and mutiple machines running different OSs.
As people have already said so, you should use JsTestDriver. It has it's own test system, but you can use other test libraries with it, for example Jasmine ( http://pivotal.github.com/jasmine/ ). You can find a list of adaptors for JsTestDriver here: http://code.google.com/p/js-test-driver/wiki/XUnitCompatibility
I'd go for QUnit, which is what jQuery uses. I've ran it on lots of desktop browsers as well as iPhones and Android phones.
There's some great tutorials and it can be integrated easily with things like js-test-driver. QUnit is modeled after JUnit and all the other xUnit testing frameworks (like PHPUnit) so it's easy to pick up the API.
The basic syntax is as follows:
test("my first test", function() {
var str = "hello";
equals( "hello", str, "Should be hello" );
});
It also looks quite nice:
There are few companies that specialize in cross browser testing.
http://browserling.com/
http://www.browserstack.com/
http://saucelabs.com/ (already mentioned here)
http://browsershots.org/
http://www.browsercam.com/
http://browserseal.com/
Use whatever testing lib you want.
Selenium or SauceLabs etc are not unit testing. They are functional/integration testing solutions.
You need to abstract your external usages like DOM to unit test javascript.
Write your tests so that they can use any external library like jquery by configuration. So that, you can unit test your logic without touching any externality and you can also both test cross-browser testing.
Related
I'd like to know if there is anything underlying jasmine-jquery like a browser engine or something that it uses to simulate HTML stuff?
Of is it just a self written DOM Model? Is it even a DOM Model?
The reason for that question is that i need to know how reliable tests are written with this fixtures. Can they / should they be used with a whole application loaded into the fixtures for unit testing the application?
How can they be compared to the behaviour of different browsers? So does it behave more like Chrome, Firefox, or even IE6 or is it a custom implementation of the W3C Standards?
Unfortunately i wasn't able to find anything about this. Maybe it's to obvious?
Check out the Karma project. It runs your jasmine tests in real browsers, so you don't have to worry about it.
I'm not sure what engine jasmine uses to power its DOM, but you are better running your front end tests in real browsers.
For node script, use jasmine_node.
Good luck.
Tl;Dr: The fixtures are simply appended into a div with an id of jasmine-fixtures as defined in the jasmine.Fixtures constructor.
You can check for yourself:
fit("See fixtures",function(){
setFixtures("<h1 id='ahoy'>Hello World</h1>");
expect($("#ahoy")).toExist();
alert($("html").html());
});
In your jasmine.jquery.js dependency the code is really clean and easy to follow, check the following methods - Fixtures,addToContainer_,createContainer_
As a sidenote - we currently successfully adopted Jasmine which allows us to follow TDD in a front-end ES5 + jQuery project I am working on. It is a very similar experience to what the test setup of the create-react-app ships with.
Currently, I am using PhantomJS for running Javascript unit tests in QUnit and Sinon framework on our build server.
But, PhantomJS uses JavaScriptCore with JIT compiler as its Javascript engine. Instead, I want to use the V8 engine, which is used in Google Chrome, or Chakra, which is used in IE. I want to do this because I want to check platform compatibility for the code.
Are there any popular test runners like PhantomJS, which use these engines?
The closest I can think of is Zombie.js, which is a headless browser written in Javascript that runs under Node.js.
It's not a genuine browser in the way that Phantom is, so there are things you won't be able to do with it that you can do with Phantom, but since it uses Node.js, it obviously does use the V8 engine, so it fulfils your criteria.
But if you really want to test in all the browser's various engines, your other option is, of course, to use a real browser. You don't have to have a visible UI for it; use a tool like Selenium or Sahi, which can launch and run the browser from a script, and have it run in a VM; you don't ever need to even look at it. It may not be as quick as using Phantom, but it will be a genuine test, which is clearly what you're really interested in.
[EDIT]
Worth adding a note to this answer because I recently found out about SlimerJS, which is an open source project aiming to produce a PhantomJS-compatible browser that uses the Gecko engine. Again, this isn't exactly what was asked for in the question, but it is in the spirit of it; it's great to have another tool available to make cross-platform testing easier.
We have a web application that makes extensive use of AJAXy Javascript in the UI. We have nearly complete code coverage of our backend using Shoulda and Webrat, and would like to extend our test suite to include full integration testing through the Javascript UI.
We tried Selenium but found it brittle and temperamental. Are there more reliable options?
UPDATE
For those still checking this out, we ended up using Xvfb so we can run Firefox without a screen. Allows us to run the test on a headless Jenkins CI server. We still have to run tests "live" locally occasionally to debug, but it works pretty well.
One of the JavaScript gurus where I work recently pointed out PhantomJS as an interesting tool for testing our JavaScript-heavy web applications. We haven't tried it out yet but the idea of a headless WebKit for DOM testing sounds promising to me.
This is something I have been wrestling with for a while, as I am doing some work with ExtJS (a very powerful JavaScript UI builder for the browser) and Rails.
After having researched quite a few different options. I still haven't found a perfect solution for it. Ideally, I would be able to run them headless and just report on the output. Unfortunately, none of the emulators out there seem to be able to run JavaScript with full DOM support seamlessly (at least, none of the options I've found are). So that pretty much means that you have to run your full-powered JavaScript code in a real interpreter (such as a browser). Webrat with Selenium works acceptably well, assuming you're willing to deal with the pain of trying to path out your requests to the UI properly. If it's your own JavaScript that you're implementing it against, that may be easier. But when it comes to a third party UI library that you don't have much control over, it can certainly get, shall we say, interesting.
Probably not the most helpful response, but that has been my findings up to now!
Hmm I would give Capybara a look, it can use selenium-webdriver (not to be confused with selenium-RC, they are different) for javascript testing. I haven't found it very brittle when compared with Webrat... it seems to be fairly consistent.
As Chris Rueber says, there aren't really any headless DOM interpreters that support JS as well - for now it's fire up a web browser for your automation or write unit tests in the javascript itself (Which isn't really integration testing either).
When you have a lot of selenium-webdriver-backed tests they can take awhile to run sometimes, but it's surely better than no tests at all.
check out the gem jasminerice to test your js logic.
https://github.com/bradphelan/jasminerice
for the integration test I would recommend to use rspec with capybara as acceptance tests. distinguish request specs and acceptance specs!
another possibility is to use turnip as an alternative to cucumber.
https://github.com/jnicklas/turnip
to speed up your tests test headless. You could use capybara-webkit (depends on qt) or poltergeist (which depends on phantomjs).
both are easily to set up. I prefer poltergeist.
There are a couple of gems you could use if you didn't like Selenium.
The one I recommend is Jasmine: https://github.com/pivotal/jasmine
You can also check out Culerity: https://github.com/langalex/culerity
I want to unit test the javascript I have embedded in the webapp portion of my liftweb project. Liftweb is a subset of the maven webapp archetype, so my question applies to that framework as well.
By 'good', I mean that the tests can be integrated into the maven automated testing.
I understand that different browsers support different versions of ecmascript, so I am okay with a testing solution that restricts itself to one specific version.
JSUnit might help with JavaScript testing.
I like QUnit for testing javascript. I have no idea how well it fits in with the maven test automation tools. I do know that you can extract the test results in a format that is more friendly to automated builds.
Typical solutions I've seen for including javascript with other forms of automated testing utilize a tool like Selenium or WATiR/WATiN to fire up a browser and execute the tests. Of course there is also TestSwarm if you want a way to test javascript against multiple browsers in an automated fashion, but again I am not sure what the capabilities are as far as integrating with other automated testing systems.
Some things that you may find helpful in testing your javascript applications:
QUnit http://docs.jquery.com/QUnit
Env.js http://github.com/thatcher/env-js
I'm developing large application using ExtJS framework. Because it grows too fast, I realized that this might be the time to start doing tests.
I want to go for BDD technique, I found several BDD frameworks for JavaScript around (Screw.Unit, JSpec, JSSpec), but I'm still not sure which one to choose. There are some articles about this topic, but I'm more interested in your own experiences/suggestions.
So my questions are:
Which one do you use and why?
Any other hints/tips are welcome.
Do you use Selenium alongside a BDD testing?
Do you use any other technique?
We use Selenium, yes.
I wrote a Unit Test framework (well, most of one, functional, not entirely pluggable) that I have used a few times. These sorts of topics keep coming up so maybe I should finish it... the url is http://code.google.com/p/jasproject/
I use buster.js because I develop in node.js as well as client-side javascript. It copes with both scenarios using a single API. The documentation is still lacking, but I really like the support for asynchronous testing.
Buster can also be used in a similar way to Selenium's WebDriver - you can slave various browsers to a buster "server" and run your tests in all those browsers at the same time.
Functional web testing has several challenges. Tests tend to be...
Slow to run (http requests are slow, dom traversal can be slow too)
Slow to write (write a test, start the app, run the test, realise you made a mistake, start again)
Hard to read (xpaths, css selectors etc)
Brittle (when tightly coupled to your HTML)
Expensive to maintain (if you don't use an abstraction layer such as the page object pattern)
Unrealistic (when run in a fake browser)
For these reason my preferred stack is
JavaScript - development time is fast since there's no compilation time
CasperJS or Zombie JS - very fast, webkit based (Safari & Chrome [for now])
Yadda for true BDD - Makes the test easy to read and provides an abstraction layer, mitigating the brittleness and maintenance
Downsides of this stack is that you're only testing in webkit, not firefox, IE or Chrome (when Google move to Blink)