I've written some functional tests for the Intern, which supposedly should work on SauceLabs, BrowserStack, TestingBot, or my own Selenium grid.
The same code doesn't seem to work on all services though. I initially got the functional tests working on SauceLabs, so I'm using that service as my "base", so to speak.
On BrowserStack, the tests appeared to fail because the commands were executing too quickly. For example, I am using .pressKeys('TEST\nIN\nPROGRESS\n'), where \n is supposed to execute javascript on the page to turn the previous text into a tag (like the SO tags for this question: [intern] [javascript] [testing]).
That command should result in the following:
[TEST] [IN] [PROGRESS]
but instead results in
[TESTIN] [PROGRESS]
causing my assertions to fail. Changing the pressKeys command to
.pressKeys('TEST\n')
.sleep(500)
.pressKeys('IN\n')
.sleep(500)
.pressKeys('PROGRESS\n')
did not solve the issue. The test would pass / fail inconsistently, with the tags sometimes coming out as [TEST] [IN] [PROGRESS], and sometimes as [TESTIN] [PROGRESS].
Another example is that it wouldn't always wait for the next page to load when I .click() on a link, even with a .sleep() command after.
With regards to TestingBot, the application flat-out failed to upload files, and I could not for the life of me figure out how to enable the file_watcher service required to do so. They have a file upload example here, but I don't know how to configure the Intern to do this for me.
Isn't the Intern supposed to take care of these differences in the cloud providers for the tests?
Is there some standardized way of writing my tests in the Intern so that I can change my cloud testing provider without changing the tests themselves?
It should be possible to run the same test suite against any cloud-hosted Selenium providers and have them execute successfully, but there are some things you must do:
You need to make sure you’ve correctly configured providers so they all run the same version of Selenium. There is no standard for this; each provider uses a different key to decide which Selenium version to run. Check each provider’s documentation for the correct key to use.
You need to write tests that don’t have race conditions. What you’re describing here sounds like a classic race condition where you are performing some action that completes asynchronously, and so only happens in environments that execute operations within a certain period of time. Modifying this specific test so it has a find timeout and then tries to find the element you expect to be generated when the return key is hit should be a good solution, since this will allow you to wait as long as necessary without making your test slow.
Unfortunately, even with this advice, all of the cloud-hosted providers for Web browser testing are garbage and mess with things in a way that randomly causes tests to break. BrowserStack is the best by far at avoiding this, but even they do things to break tests from time to time that work perfectly well in a locally hosted Selenium installation.
For file uploads, Intern will automatically upload files if has detected that the remote provider supports it, and you type a valid path to a file on the server where intern-runner is running. You can check whether the server supports uploads by looking at this.remote.session.capabilities.remoteFiles. Feature detection must be turned on for this to work, and you should run Intern 3.0.6 or newer if you are trying to upload files to a Selenium server on the same machine as intern-runner.
I want to premise that I'm aware of Ember QUnit (recently covered at EmberConf) as well as using PhantomJS so please read my points in question closely if you're thinking of marking as a duplicate.
My goal is to run unit tests from the command line, similar to a mocha test might run
mocha simple_test.js
and see the results in the form of a command line reporter.
testing ember modules in isolation. I would like to be able to new-up an ember object, route, or controller without the context of a running ember app (perhaps some kind of ember test harness) and run assertions against that module.
testing ember modules in the command line (avoiding browser reporters like QUnit or headless browsers like PhantomJS)
I already have integration and acceptance tests using a combination of karma and phantomjs, I would like to see if I can compliment with more unit tests. Has anybody come across a unit test setup similar to to what I listed above or is it not really possible and/or productive?
Update
The ember guides list unit testing strategies here:
http://emberjs.com/guides/testing/unit/
In my opinion, these seem more like integration tests.
Yeah I do this with my application. You might like to look at the new testing guides in the ember site's documenation if you haven't already seen it (it went live last week sometime). I helped edit it. It's pretty good! :-)
Good luck and let me know if you need any more help, like I say, I do unit tests all the time on all parts of Ember. The hardest so far for me has been components because they're neither integration nor unit, really... they're like a hybrid: isolated integration unit tests that still require large parts of ember and rendering in the view.
I run headless using guard, jasmine and qunit. Jasmine's my preference and I've been moving over from qunit slowly.
http://emberjs.com/guides/testing/
Also I noticed that what you seem to want is to isolate the units outside of even ember itself. To do that, I'd put your code in separate javascript libraries... otherwise you'll have troubles: afterall how are you going to unit test a piece of code without Ember present if it uses Ember?
One of the great things about being a web developer in recent history is all the sharing that's going on, especially with JavaScript libraries. There's all these awesome tools to use: jQuery, jQuery-UI, Lightbox, bxslider, underscore.js, Backbone.js, the list goes on. Then there comes a time when one or more of these libraries need to be updated. But JavaScript runs on the client, it doesn't compile, and it's difficult or impossible to be notified when a problem occurs. What is the best technique right now to assure that after you update one or more JavaScript libraries, your web application will not start throwing JavaScript errors?
There's no way the best response to this is to just test. Especially with a complicated application it can be too difficult to go through every possibility and make sure no errors are thrown. What are other web application developers out there doing to make sure they don't have a deployment with an embarassing and crippling JavaScript bug caused by updating?
The best answer is to "just test". What you are asking, essentially, is "How do I test to make sure my software still works?". You can do all the homework you want to see what changed, but eventually you just need to test your application.
That being said, there are generic testing tools like JSLint and Selenium, but ultimately your application is going to be unique enough that you will need to have unit tests to cover the business logic and standard QA for non-standard processes.
One way to ensure that things still work functionally is to have a suite of automated browser tests (utilizing a tool like Selenium) that you run on your development environment.
I wouldn't really call this "just testing" but unit testing will help do the job. Write tests once for your app looking for the expected results (good practice either way), then when you update the plugins run these tests again.
http://qunitjs.com/
Nothing beats a good QA and browser testing though.
Plenty of other answers already include "test", so hopefully that's obvious at this point.
The other thing that you should ALWAYS do is read the release notes for every single version up to the version that you're upgrading to. I can't speak intelligently about bxslider or Lightbox, but the other major libraries that you referenced are extremely good at releasing detailed changelogs which will notify you of breaking changes. You can decide for yourself if any of these changes will have a negative effect on your app ( in addition to testing of course!).
I have been trying out different JavaScript unit testing libraries, but I still don't know what's the right way of doing this.
Is there a consensus on this subject?
For instance, I tried to write some tests for a modal window library with QUnit (the unit testing library used by jQuery), but it has to be run in the browser, and it doesn't hide the effects I'm testing, or other annoying things like the alerts (I don't even know if that is possible in JavaScript). In a server-side language, any output generated by a test suite would be sent to a buffer and get discarded after each test.
If alerts, DOM manipulations, and that kind of things are not encapsulable in JavaScript unit testing, is writing tests worth the effort in this language?
Testing frontend code (HTML, CSS, and JavaScript) is not an easy task mostly thanks to cross-browser issues which require visual testing eg. making sure an element is correctly positioned in all browsers.
In order to test your JavaScript code you need to write it in a "testable" way. I personally found this blogpost very helpful: Writing Testable JavaScript
As for the actual testing, try to combine QUnit (which you are already using) with http://funcunit.com/. FuncUnit lets you open a HTML page and run tests on its DOM. There's an example on the home showing how it's done.
If you are really annoyed by the DOM alterations and other effects (alert and prompt) you could use mocking and spying to ensure that those alterations are intended.
See Jasmine BDD that includes mocking and spying or SinonJS for a standalone library.
Then you can just write a test like that (using Sinon):
var mock = sinon.mock(window);
mock.expects("alert").once().withExactArgs("foo");
alert("foo");
mock.verify();
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 9 years ago.
My main JavaScript framework is jQuery, so I would like my unit test and mocking frameworks to be compatible with that. I'd rather not have to introduce another JavaScript framework.
I am currently using QUnit for unit testing and Jack for mocking, but I am pretty new to the whole unit testing of JavaScript.
Is there a better tool to suggest? What has worked for you?
I think that Jack is the best mocking framework for JavaScript as of the time of this writing. The main reason is that what's right for JavaScript is not likely what is right for a strongly typed language such as Java.
Many JavaScript mocking frameworks are inspired by Java mock frameworks (such as the excellent JsMockito, for example). But the problem with these is that they require dependency injection, because that's about the only reasonable way to use mocking in Java. But in JavaScript, there are many ways to use mocking, and you are not forced into using dependency injection everywhere.
For example, with JsMockito, you have to make mocks and then pass those mocks into your software-under-test (SUT). The SUT has to directly call the mocks. Therefore, you're forced to code the SUT as a constructor or function that takes in all its dependencies as parameters. (Sometimes, that's a fine way to implement it, but not in every case. The tail is wagging the dog if your mocking framework's design forces your implementation approach.)
In JavaScript, it's very easy to "hijack" any function. Therefore, there are tons of ways to build something such that you can mock parts of it without explicitly injecting its dependencies into it. For example, Jack lets you mock any function, whether it is public or on a local object. From there you can spy on it, stub it, or express expectations on it. The key point is this: once you've mocked a function, any calls to that original function will instead be directed to your mock. In other words, your mocks will still get used even though the original, un-mocked function was called. As a result, you are not forced to inject dependencies, although you certainly can do so in those cases which call for it.
JavaScript is a different language than Java (and C#, etc.). It allows for different implementation idioms. Dependency injection is still one valuable tool in the toolbox in JavaScript, but it is not the only game in town any more. Your mocking framework needs to know and respect that fact. Jack and a couple of others do, but of the ones that do, Jack appears to be the most mature and feature-rich.
QUnit
jqUnit
Writing JavaScript tests with QUnit and jqUnit
QUnit is the unit testing framework for the jQuery JavaScript framework. The testing framework itself uses the jQuery library, but the tests can be written for any JavaScript and do not require the code to use jQuery.
jqUnit is a modified version of QUnit that adds in the setup, teardown, and assert functions that are more typical of an xUnit framework, and encapsulates everything in one global variable.
The visual interface of the testrunner page is nice, allowing you to drill down and see each assert in every test method. Writing tests is fairly easy, and you can run the test code directly on the testRunner page [8]. This allows for easy and visible DOM testing.
QUnit: MIT or GPL (choose) jqUnit: MIT License
Pros
Asynchronous support
Good for DOM testing
Tests always run sequentially in the order they are added to a suite
Debug on test page using firebug
Syntax is similar to JUnit if using jqUnit, but simple to learn if using QUnit
Cons
Automation would be difficult to implement
I'm not sure why no one has mentioned JsTestDriver! It has to be the one of the only JavaScript testing tools that actually work like you'd expect them to if you've used unit testing tools in other languages.
Running tests can be done without touching a browser, you can integrate it with IDE's, and you can integrate it with Continuous integration systems... Oh, and it's fast, and can run tests in multiple browsers at the same time.
You can also use other testing frameworks like YUITest with it, making it even better.
YUI Test
TDD With YUI Test
YUI Test is the test framework for Yahoo’s User Interface (YUI) library. It is used by Yahoo to test its own library, and has syntax similar to JUnit.
Like jsUnit, YUI Test comes with its own logging console that can output information, warnings and errors in addition to the results of each test.
YUI also provides the ability to send reports on the results in either JSON or XML format.
YUI Test is BSD licensed.
Pros
Really good documentation
Active community
Regular releases
Syntax is similar to JUnit (test suites, asserts and setup/teardown)
Asynchronous support
Good for DOM testing
Tests always run sequentially in the order they are added to a suite
Cons
Automation not trivial to implement, but less difficult than other frameworks
Also check out
http://sinonjs.org/
It has test spies, test stubs, mocks, fake timers, fake XMLHttpRequest (XHR), fake server, sandboxing, and assertions
It does work along with QUnit and that has been a plus so far.
This is a pretty good review of mocking frameworks available for JavaScript:
http://testdrivenwebsites.com/2010/05/06/java-script-mock-frameworks-comparison
I use the Screw Unit test framework and I've written my own mocking library called jsMocha which has been in heavy use in the company I work at for over 6 months.
For mocking in JavaScript, take a look at qMock, a framework a colleague and I wrote to complement our use of QUnit. Although the latter is great for unit tests, it doesn't allow for very effective async/business logic testing. We haven't 'tagged' any release as stable, but there's some decent documentation on there, and if you checkout the SVN repository you'll see qmock itself has unit tests behind it which are fairly self-explanatory.
Oh, and to automate testing as part of the build, we used a simple Selenium script to navigate through our testsuite (one testing page per JavaScript file), and 'listened' for a pass or fail CSS class (added by QUnit). This works headless as well for Internet Explorer and Firefox 2, AFAIK.
For Firefox development, I have fallen in love with UXU, based on MozUnit, but it is still active. It has nice features, like a mock server and sleep / yield methods.
CrossCheck seemed extremely powerful when I looked at it, but we've not incorporated it into our build process at this time. It has the advantage of being browserless, and thus should work well in an automated build-and-test scenario.
http://thefrontside.net/crosscheck
I know you are asking for jQuery-compatible frameworks, but I want to throw script.aculo.us into the mix for completeness. They have a unit test suite that isn't bad.
JsUnit is run from either the browser, through its Eclipse plug-in, or automatically through an Ant task. You create an HTML page with a bunch of test functions, which must be named with the prefix ‘test’, include the JavaScript file you are testing. When any assert within a function fails, the entire function fails and stops executing. There is no guaranteed order in which these tests are run. You can create setup() and teardown() functions.
License: GPL, GLPL, and MPL
Pros
Automation is relatively easy to implement
A lot of functionality
Syntax is similar to JUnit
Cons
Not great for DOM testing since it runs tests inside an iFrame.
No guarantee that tests will be run in the order they are written.
Can’t use Firebug on the testrunner page. Need to have another tab open with the actual test code.
We've been using jsspec. It's very nice if you like rspec and BDD. I just saw an article by Justin Gehtland on using it "headless" as well.
You could try HtmlUnit which had a jQuery compatible release over a year ago.
The advantage of HtmlUnit is that it isn't driving a browser, so it is fast.
The downside is that it isn't driving a browser so there are some JavaScript things that won't work. But offsetting that they can run the jQuery tests so the JavaScript support might be good enough for what you need.