Question
In terms of e2e testing, what can't we do with Karma and JQuery that is possible with Protractor?
Explanation
I'm currently building a testing framework for my JavaScript application.
I'm using Karma for Unit Testing and Protractor for E2E Testing as suggested by many people.
I'm aware of the conceptual differences between unit testing and e2e testing, but, in the context of JavaScript, I don't clearly understand why we need framework like Protractor.
As far as I know, the point of e2e testing is to use the application as a simple end-user. For that, tools like Protractor use a webdriver to interact with a browser and let us simulate some user events (clicking on element, filling forms...).
The thing is, why can't we simply do this by using Karma and JQuery?
Indeed, JQuery comes with lots of methods to interact with a DOM element (trigger event, get/set element's property, set value to an input...). Furthermore, it provides selectors which make the selection of DOM element very easy.
From my point of view, Karma and JQuery has (almost) everything needed for e2e testing (if we don't take care of the browser's window's operations available in Protractor which enable for example to set the window size or location).
I'm obviously missing something, any clarification will be helpful.
Using Jquery and Karma the way you would, is a behavior-driven-test but not an end-to-end test.
With Jquery and Karma, you can't detect cross-browser issues like you would in protractor. Protractor on the other hand runs your tests as if it was an end-user. It literally opens the browser you want and tests your application in that browser.
Related
I kind of think that answer of this question would be No by going through Protractor issue 1798 and Protractor vs Webdriver-IO comparisions.
I want to write automated tests cases for a hybrid mobile app in JavaScript which will run on Appium.
Currently I have both Protractor and Appium configured in same project to run those End-to-End test cases, and they work.
Problems are that,
I have to write separate test cases for each of them.
Cases should also work on mobile devices(Android and iOS) and protractor doesn't support native apps tests.
Situation is, I am more comfortable with Protractor's settings in my current project than Webdriver-IO which Appium uses (just a personal preference).
Question-
Is there a way of using only Protractor while writing test cases which would work using Appium and work perfectly on Devices/Emulators?
I am also open for any suggestion(s).
If answer remains No, I will change my codes to only use WebdriverIO in order to keep my test cases reusable and only once.
The answer is still NO protractor currently does not have support for mobile native apps but you can use it for automating browsers in your mobile.
Better use WebdriverIO but it also has its own limitations, please do check its changelog and github issues before deciding it as your webdriver framework.
What are the differences? What are the advantages of using one over the other for an Angular project?
Nightwatch.js vs Protractor
If you are working on an AngularJS project, the choice is simple - Protractor:
it is being made specifically for angular apps (though it can be used for non-angular apps also)
supports angular-specific locator strategies (like by.model, by.repeater etc)
waits for angular to start up during the page load (sync)
it is being actively maintained and improved mostly by google developers with a close cooperation with an angular team - this means that protractor follows the angular release cycle and new changes
There are a few more differences that weren't laid out in the top answer.
The big one for me is, as of now, protractor doesn't support phantomJS (http://www.protractortest.org/#/browser-setup#setting-up-phantomjs). I planned on integrating automation in to the CI system, but with protractor I would have to set up a windows VM to run the tests.
Other benefits of nightwatch:
Asserts are done automatically.
Clean console output, you could hand
off to anyone.
Test reports are automatically generated and saved to a /reports directory.
Protractor pros:
The angular-specific locators that protractor supports is very powerful and could help immensely with creating stable tests.
WaitForAngular is also very helpful.
EDIT:
Since the time I wrote this comment and present day our team has happily switched over to using cypress.io. If you are deciding test frameworks I highly recommend checking it out: https://www.cypress.io/
I'd like to know if there is anything underlying jasmine-jquery like a browser engine or something that it uses to simulate HTML stuff?
Of is it just a self written DOM Model? Is it even a DOM Model?
The reason for that question is that i need to know how reliable tests are written with this fixtures. Can they / should they be used with a whole application loaded into the fixtures for unit testing the application?
How can they be compared to the behaviour of different browsers? So does it behave more like Chrome, Firefox, or even IE6 or is it a custom implementation of the W3C Standards?
Unfortunately i wasn't able to find anything about this. Maybe it's to obvious?
Check out the Karma project. It runs your jasmine tests in real browsers, so you don't have to worry about it.
I'm not sure what engine jasmine uses to power its DOM, but you are better running your front end tests in real browsers.
For node script, use jasmine_node.
Good luck.
Tl;Dr: The fixtures are simply appended into a div with an id of jasmine-fixtures as defined in the jasmine.Fixtures constructor.
You can check for yourself:
fit("See fixtures",function(){
setFixtures("<h1 id='ahoy'>Hello World</h1>");
expect($("#ahoy")).toExist();
alert($("html").html());
});
In your jasmine.jquery.js dependency the code is really clean and easy to follow, check the following methods - Fixtures,addToContainer_,createContainer_
As a sidenote - we currently successfully adopted Jasmine which allows us to follow TDD in a front-end ES5 + jQuery project I am working on. It is a very similar experience to what the test setup of the create-react-app ships with.
I've never used Selenium but I guess it's for simulating user interaction in all browsers.
That's like integration tests.
But how do you test your js libraries/frameworks (unit testing) on all the browsers in an automated way?
For unit-testing you can try http://code.google.com/p/js-test-driver/
JsTestDriver consist of a single JAR file which contains everything you need to get started. For in depth discussion of command line option see GettingStarted.
Here is an overview of how JsTestDriver works at runtime...
You can have a look at TestSwarm:
project that I’m working on: TestSwarm...
Its construction is very simple. It’s a dumb JavaScript client that continually pings a central server looking for more tests to run. The server collects test suites and sends them out to the respective clients.
All the test suites are collected. For example, 1 “commit” can have 10 test suites associated with it (and be distributed to a selection of browsers)...
The nice thing about this construction is that it’s able to work in a fault-tolerant manner. Clients can come-and-go. At any given time there might be no Firefox 2s connected, at another time there could be thirty. The jobs are queued and divvied out as the load requires it. Additionally, the client is simple enough to be able to run on mobile devices (while being completely test framework agnostic)...
The best one imo is the one from YUI : http://developer.yahoo.com/yui/3/test/
But doing unit-testing in every browser is kind of hard... Most people just test with it during development and just use node.js to test later on in case they broke something.
As the referenced post in a previous article suggests, you could use js-test-driver.
Its specifically for JavaScript unit testing across multiple browsers, exactly what you want. I have messed around with it and it is pretty good. I haven't done any serious commercial testing in it though.
For unit testing you should look at solutions that do not load up a browser to do the tests.
You can look at something like RhinoUnit for that - http://code.google.com/p/rhinounit
Also have a look at Dojo Object Harness (DOH) unit test framework http://dojotoolkit.org/reference-guide/util/doh.html
Look at a similar question here which can give you an idea on how to TDD js - JavaScript unit test tools for TDD
It's no longer actively maintained, but I've still been happy with JSUnit for Javascript unit testing: https://github.com/pivotal/jsunit
It includes both an HTML/Javascript framework you can run in the browser, and a java-based test runner that you can invoke from ant.
To test multiple browsers in parallel you would Selenium Grid. Please take a look here: http://selenium-grid.seleniumhq.org/step_by_step_installation_instructions_for_windows.html for step by step instructions on how to use it.
I haven't used it so far, and it is still in beta, but FuncUnit declares itself as "A functional test suite based of qUnit, Selenium and jQuery".
There is also an infographic explaining how it works
Maybe it is something what you want? The github repo seems to be quite active.
You should check http://saucelabs.com its a cloud base selenium environment that allow you to build your test and then upload them, run them in as many browsers as you want.
You should consider the capabilities offered by crossbrowsertesting.com
(It is not for free). For js testing try JsUnit (http://www.jsunit.net/).
Quoted from its homepage
JsUnit is a Unit Testing framework for client-side (in-browser) JavaScript. It is essentially a port of JUnit to JavaScript. Also included is a platform for automating the execution of tests on multiple browsers and mutiple machines running different OSs.
As people have already said so, you should use JsTestDriver. It has it's own test system, but you can use other test libraries with it, for example Jasmine ( http://pivotal.github.com/jasmine/ ). You can find a list of adaptors for JsTestDriver here: http://code.google.com/p/js-test-driver/wiki/XUnitCompatibility
I'd go for QUnit, which is what jQuery uses. I've ran it on lots of desktop browsers as well as iPhones and Android phones.
There's some great tutorials and it can be integrated easily with things like js-test-driver. QUnit is modeled after JUnit and all the other xUnit testing frameworks (like PHPUnit) so it's easy to pick up the API.
The basic syntax is as follows:
test("my first test", function() {
var str = "hello";
equals( "hello", str, "Should be hello" );
});
It also looks quite nice:
There are few companies that specialize in cross browser testing.
http://browserling.com/
http://www.browserstack.com/
http://saucelabs.com/ (already mentioned here)
http://browsershots.org/
http://www.browsercam.com/
http://browserseal.com/
Use whatever testing lib you want.
Selenium or SauceLabs etc are not unit testing. They are functional/integration testing solutions.
You need to abstract your external usages like DOM to unit test javascript.
Write your tests so that they can use any external library like jquery by configuration. So that, you can unit test your logic without touching any externality and you can also both test cross-browser testing.
We have a web application that makes extensive use of AJAXy Javascript in the UI. We have nearly complete code coverage of our backend using Shoulda and Webrat, and would like to extend our test suite to include full integration testing through the Javascript UI.
We tried Selenium but found it brittle and temperamental. Are there more reliable options?
UPDATE
For those still checking this out, we ended up using Xvfb so we can run Firefox without a screen. Allows us to run the test on a headless Jenkins CI server. We still have to run tests "live" locally occasionally to debug, but it works pretty well.
One of the JavaScript gurus where I work recently pointed out PhantomJS as an interesting tool for testing our JavaScript-heavy web applications. We haven't tried it out yet but the idea of a headless WebKit for DOM testing sounds promising to me.
This is something I have been wrestling with for a while, as I am doing some work with ExtJS (a very powerful JavaScript UI builder for the browser) and Rails.
After having researched quite a few different options. I still haven't found a perfect solution for it. Ideally, I would be able to run them headless and just report on the output. Unfortunately, none of the emulators out there seem to be able to run JavaScript with full DOM support seamlessly (at least, none of the options I've found are). So that pretty much means that you have to run your full-powered JavaScript code in a real interpreter (such as a browser). Webrat with Selenium works acceptably well, assuming you're willing to deal with the pain of trying to path out your requests to the UI properly. If it's your own JavaScript that you're implementing it against, that may be easier. But when it comes to a third party UI library that you don't have much control over, it can certainly get, shall we say, interesting.
Probably not the most helpful response, but that has been my findings up to now!
Hmm I would give Capybara a look, it can use selenium-webdriver (not to be confused with selenium-RC, they are different) for javascript testing. I haven't found it very brittle when compared with Webrat... it seems to be fairly consistent.
As Chris Rueber says, there aren't really any headless DOM interpreters that support JS as well - for now it's fire up a web browser for your automation or write unit tests in the javascript itself (Which isn't really integration testing either).
When you have a lot of selenium-webdriver-backed tests they can take awhile to run sometimes, but it's surely better than no tests at all.
check out the gem jasminerice to test your js logic.
https://github.com/bradphelan/jasminerice
for the integration test I would recommend to use rspec with capybara as acceptance tests. distinguish request specs and acceptance specs!
another possibility is to use turnip as an alternative to cucumber.
https://github.com/jnicklas/turnip
to speed up your tests test headless. You could use capybara-webkit (depends on qt) or poltergeist (which depends on phantomjs).
both are easily to set up. I prefer poltergeist.
There are a couple of gems you could use if you didn't like Selenium.
The one I recommend is Jasmine: https://github.com/pivotal/jasmine
You can also check out Culerity: https://github.com/langalex/culerity