I want to be able to have all my describe statements in Mocha get kicked off in parallel. Can someone help me figure out how to do that?
You can't do this directly with mocha because it creates a list of it() callbacks and invokes them in order.
mocha-parallel-tests can do this if you're willing to move your describes into separate .js files. To convince yourself, install it somewhere and invoke it with a short --slow so it reports each time:
laptop:/tmp/delme$ npm install mocha-parallel-tests
laptop:/tmp/delme$ cd node_modules/mocha-parallel-tests
laptop:/tmp/delme/node_modules/mocha-parallel-tests$ ./bin/mocha-parallel-tests test/parallel/tests --timeout 10000 --slow 100
You will see that it ran three (very simple) tests suites in the time it took to run the longest.
If your tests don't depend on side-effects of earlier tests, you can make them all asynchronous.
A simple way to do this is to initiate the stuff that takes a while before the describe and use the regular mocha apparatus to evaluate it. Here, I create a bunch of promises which take a while to resolve and then iterate through the tests again, examining their results in a .then() function:
var expect = require("chai").expect;
var SlowTests = [
{ name: "a" , time: 250 },
{ name: "b" , time: 500 },
{ name: "c" , time: 750 },
{ name: "d" , time:1000 },
{ name: "e" , time:1250 },
{ name: "f" , time:1500 }
];
SlowTests.forEach(function (test) {
test.promise = takeAWhile(test.time);
});
describe("SlowTests", function () {
// mocha defaults to 2s timeout. change to 5s with: this.timeout(5000);
SlowTests.forEach(function (test) {
it("should pass '" + test.name + "' in around "+ test.time +" mseconds.",
function (done) {
test.promise.then(function (res) {
expect(res).to.be.equal(test.time);
done();
}).catch(function (err) {
done(err);
});
});
});
});
function takeAWhile (time) {
return new Promise(function (resolve, reject) {
setTimeout(function () {
resolve(time);
}, time);
});
}
(Save this as foo.js and invoke with mocha foo.js.)
Meta I disagree with the assertion that tests should be primarily be synchronous. before and after pragmas are easier but it's rare that one test invalidates all remaining tests. All discouraging asynchronous tests does is discourage extensive testing of network tasks.
Mocha does not support what you are trying to do out of the box. It runs tests sequentially. This has a big advantage when dealing with an unhandled exception: Mocha can be sure that it happened in the test that it is currently running. So it ascribes the exception to the current test. It is certainly possible to support parallel testing but it would complicate Mocha quite a bit.
And I tend to agree with David's comment. I would not do it. At the level at which Mocha usually operates, parallelism does not seem to me particularly desirable. Where I have used test parallelism before is at the level of running end-to-end suites. For instance, run a suite against Firefox in Windows 8.1 while at the same time running the same suite against Chrome in Linux.
Just to update this question, Mocha version 8+ now natively supports parallel runs. You can use the --parallel flag to run your tests in parallel.
Parallel tests should work out-of-the box for many use cases. However, you must be aware of some important implications of the behavior
1 thing to note, some reporters don't currently support this execution (mocha-junit-reporter for example)
If you are using karma to start your tests, you can use karma-parallel to split up your tests across multiple browser instances. It runs specs in different browser instances in parallel and is very simple and easy to install:
npm i karma-parallel
and then add the 'parallel' to the frameworks list in karma.conf.js
module.exports = function(config) {
config.set({
frameworks: ['parallel', 'mocha']
});
};
karma-parallel
Disclosure: I am the author
Related
I am working with a Jasmine testing suite that includes both "vanilla" Jasmine tests along with Jasmine tests for some Angular 2 components. Because of Angular 2's inclusion, zone.js gets loaded. This creates a conflict with Jasmine's clock. For example, the following test fails with the error, Error: Jasmine Clock was unable to install over custom global timer functions. Is the clock already installed?
describe('an async test with zone.js present', function() {
beforeEach(function() {
jasmine.clock().install();
});
afterEach(function() {
jasmine.clock().uninstall();
});
it('cannot install jasmine\'s mock clock', function() {
var callback = jasmine.createSpy('setTimeoutCallback')
setTimeout(callback, 55);
jasmine.clock().tick(56);
expect(callback).toHaveBeenCalled();
});
})
Here is plunker for above code.
Short of delivering the Angular 2 tests separately from the "vanilla" tests, I am wondering what options might be available. For example, is it possible to perform the Jasmine clock's job with the zone? For example, is it possible to simulate the tick with the zone or flush all of the scheduled tasks before the assertion?
For me it works if you uninstall the clock in beforeEach. Its not recommended by jasmine and a bit strange because for uninstall it makes more sense to use afterEach. But calling uninstall before the first install call happens fixed it for me.
As stated on Angular documentation you should use tick function in a fakeAsync body which is part of the #angular/core/testing module.
Using your example and TypeScript it would look something like this...
import { fakeAsync, tick } from '#angular/core/testing';
...
it('cannot install jasmine\'s mock clock', fakeAsync(() => {
var callback = jasmine.createSpy('setTimeoutCallback')
setTimeout(callback, 55);
tick(56);
expect(callback).toHaveBeenCalled();
}));
The code which throws this here.
It implies that jasmine was loaded before Zone.js. Switch the loading order. Zone always needs to be loaded first.
Here's the fixed fork of your plunker:
<script data-require="zone.js#*" data-semver="0.4.1" src="https://cdn.rawgit.com/angular/zone.js/v0.4.1/zone.js"></script>
<script data-require="zone.js#*" data-semver="0.4.1" src="https://cdn.rawgit.com/angular/zone.js/v0.4.1/long-stack-trace-zone.js"></script>
<script data-require="jasmine#*" data-semver="2.4.1" src="https://cdnjs.cloudflare.com/ajax/libs/jasmine/2.4.1/boot.js"></script>
https://plnkr.co/edit/6iuvWFOZLqHWJIo4
This have been resolved by https://github.com/angular/zone.js/pull/1009.
zone.js and future angular will support jasmine.clock().
I'm trying to get webdriver.io and Jasmine working.
Following their example, my script is at test/specs/first/test2.js (in accordance with the configuration) and contains:
var webdriverio = require('webdriverio');
describe('my webdriverio tests', function() {
var client = {};
jasmine.DEFAULT_TIMEOUT_INTERVAL = 9999999;
beforeEach(function() {
client = webdriverio.remote({ desiredCapabilities: {browserName: 'firefox'} });
client.init();
});
it('test it', function(done) {
client
.url("http://localhost:3000/")
.waitForVisible("h2.btn.btn-primary")
.click("h2.btn.btn-primary")
.waitForVisible("h2.btn.btn-primary")
.call(done);
});
afterEach(function(done) {
client.end(done);
});
});
I'm using wdio as the test runner, and set it up using the interactive setup. That config is automatically-generated and all pretty straightforward, so I don't see a need to post it.
In another terminal window, I am running selenium-server-andalone-2.47.1.jar with Java 7. I do have Firefox installed on my computer (it blankly starts when the test is run), and my computer is running OS 10.10.5.
This is what happens when I start the test runner:
$ wdio wdio.conf.js
=======================================================================================
Selenium 2.0/webdriver protocol bindings implementation with helper commands in nodejs.
For a complete list of commands, visit http://webdriver.io/docs.html.
=======================================================================================
[18:17:22]: SET SESSION ID 46731149-79aa-412e-b9b5-3d32e75dbc8d
[18:17:22]: RESULT {"platform":"MAC","javascriptEnabled":true,"acceptSslCerts":true,"browserName":"firefox","rotatable":false,"locationContextEnabled":true,"webdriver.remote.sessionid":"46731149-79aa-412e-b9b5-3d32e75dbc8d","version":"40.0.3","databaseEnabled":true,"cssSelectorsEnabled":true,"handlesAlerts":true,"webStorageEnabled":true,"nativeEvents":false,"applicationCacheEnabled":true,"takesScreenshot":true}
NoSessionIdError: A session id is required for this command but wasn't found in the response payload
at waitForVisible("h2.btn.btn-primary") - test2.js:21:14
/usr/local/lib/node_modules/webdriverio/node_modules/q/q.js:141
throw e;
^
NoSessionIdError: A session id is required for this command but wasn't found in the response payload
0 passing (3.90s)
$
I find this very strange and inexplicable, especially considering that it even prints the session ID.
Any ideas?
Please check out the docs on the wdio test runner. You don't need to create an instance using init on your own. The wdio test runner takes care on creating and ending the session for you.
Your example covers the standalone WebdriverIO usage (without testrunner). You can find examples which use wdio here.
To clarify that: there are two ways of using WebdriverIO. You can embed it in your test system by yourself (using it as standalone / or as a scraper ). Then you need to take care of things like create and end an instance or run those in parallel. The other way to use WebdriverIO is using its test runner called wdio. The testrunner takes a config file with a bunch of information on your test setup and spawns instances updates job information on Sauce Labs and so on.
Every Webdriver command gets executed asynchronously.
You properly called the done callback in afterEach and in your test it test, but forgot to do it in beforeEach:
beforeEach(function(done) {
client = webdriverio.remote({ desiredCapabilities: {browserName: 'firefox'} });
client.init(done);
});
I have two javascript files which contain mocha test cases.
//----------abc.js -------------
describe("abc file", function(){
it("test 1" , function(){
assert.equal(20 , 20);
});
});
//---------xyz.js--------------
describe("xyz file", function(){
it("test 1" , function(){
assert.equal(10 , 10);
});
});
I have put them in a folder called test and when I execute the mocha command the first file(abc.js) is always executed before xyz.js.
I thought this might be due to alphabetical ordering and renamed the files as
abc.js => xyz.js
xyz.js => abc.js
but still, the content of the xyz.js (previously abc.js) is executed first. How can I change the execution order of these test files?
In the second file, require the first one:
--- two.js ---
require("./one")
or if you are using ES modules:
--- two.js ---
import "./one"
Mocha will run the tests in the order the describe calls execute.
I follow a totally seperate solution for this.
Put all your tests in a folder named test/ and
Create a file tests.js in the root directory in the order of execution
--- tests.js ---
require('./test/one.js')
require('./test/two.js')
require('./test/three.js')
And in the tests files one.js, two.js and so on write your simple mocha tests
this way if you want to run them in the order you have defined then just run mocha tests.js
Mocha has a --sort (short -S) option that sorts test files:
$ mocha --help
[...]
-S, --sort sort test files
[...]
Since mocha sorts files in alphabetical order, I usually prefix my test files names with numbers, like:
0 - util.js
1 - something low level.js
2 - something more interesting.js
etc.
In addition to being really easy to maintain (no gulp grunt or any of that nonsense, no editing your package.json...), it provides the benefit that:
people reading your source code get an idea of the structure of your program, starting from the less interesting parts and moving up to the business layer
when a test fails, you have some indication of causality (if something failed in 1 - something.js but there are no failures in 0 - base.js then it's probably the fault of the layer covered by 1 - something.js
If you're doing real unit tests of course order should not matter, but I'm rarely able to go with unit tests all the way.
If you prefer a particular order, you can list the files (in order) as command-line arguments to mocha, e.g.:
$ mocha test/test-file-1.js test/test-file-2.js
To avoid a lot of typing every time you want to run it, you could turn this into an npm script in your package.json:
{
// ...
"scripts": {
"test": "mocha test/test-file-1.js test/test-file-2.js"
}
// ...
}
Then run your suite from the command line:
$ npm test
Or if you're using Gulp, you could create a task in your gulpfile.js:
var gulp = require('gulp');
var mocha = require("gulp-mocha");
gulp.task("test", function() {
return gulp.src([
"./test/test-file-1.js",
"./test/test-file-2.js"
])
.pipe(mocha());
});
Then run $ gulp test.
The way it worked for my tests to be executed in a specific order was to create a separate test.js file and then added a describe for each mocha test file I'd wanted to execute.
test.js:
describe('test file 1', function() {
require('./test1.js')
})
describe('test file 2', function() {
require('./test2.js')
})
Then simply run mocha test.js
I am exporting an array with all required files and that is the way I tell mocha the order of execution through index.js file in the folder with all my test files:
const Login = require('../login');
const ChangeBudgetUnit = require('./changeBudgetUnit');
const AddItemsInCart = require('./addItemsInCart');
// if the order matters should export array, not object
module.exports = [
Login,
ChangeBudgetUnit,
AddItemsInCart
];
mocha-steps allows you to write tests that run in a specific sequence, aborting the run at the first failure. It provides a drop-in replacement for it, called steps.
Example usage:
describe('my smoke test', async () => {
step('login', async () => {})
step('buy an item', async () => throw new Error('failed'))
step('check my balance', async () => {})
xstep('temporarily ignored', async () => {})
})
The repo hasn't seen much activity in three years, but it works fine with Mocha 9.
I'm currently running my test suite on AngularJS using Grunt, Karma, Jasmine and Protractor. The database library I'm using is hood.ie, which is a library on top of CouchDB. I start hood.ie using the following code in my Gruntfile:
hoodie: {
start: {
options: {
callback: function(config) {
grunt.config.set('connect.proxies.0.port', config.stack.couch.port);
}
}
}
},
However, I would like to have a separate database for running tests, which automatically resets afterwards. This way, the production data won't conflict with the tests.
How should I approach this? I would assume there's some kind of standard way of doing this, as I can imagine other people have come across the same problem, but I'm unable to find anything on the internet.
Currently, this seems to be impossible as the hoodie server does not support it. The best way to go about this is to modify it yourself at Hood.ie server Github repository by adding a parameter to define the folder in which the data will be stored, which is at the moment hardcoded to 'data' (https://github.com/hoodiehq/hoodie-server/blob/master/lib/core/environment.js#L48)
Something similar to this should work:
app_path: path.resolve(project_dir, argv.folder || 'data')
As the hoodie task is a 'multitask' you could have a test target in your hood.ie grunt task specific to testing, and then reference this in a grunt command used to run tests e.g:
hoodie: {
start: {
options: {
callback: function(config) {
grunt.config.set('connect.proxies.0.port', config.stack.couch.port);
}
}
},
test: {
options: {
callback: function(config) {
// Make test specific changes here.
}
}
}
}
// The task that runs tests first starting test deps. 'runtests' can be anything you want.
grunt.registerTask('test', 'Run unit tests', ['hoodie:test', 'runtests']);
Note: this will mean any other times you're referencing the hoodie task you'll need to be explicit as otherwise all the specified targets will be run. See this documentation on multitasks for more info. In this example you'd change hoodie to hoodie:start to run the 'start' task as previously defined.
If we have a unit test file my-spec.js and running with mocha:
mocha my-spec.js
The default timeout will be 2000 ms. It can be overwritten for partial test with a command line parameter:
mocha my-spec.js --timeout 5000
Is it possible to change the default timeout globally for all tests?
i.e. the default timeout value will be different from 2000 ms when you call:
mocha my-spec.js
By default Mocha will read a file named test/mocha.opts that can contain command line arguments. So you could create such a file that contains:
--timeout 5000
Whenever you run Mocha at the command line, it will read this file and set a timeout of 5 seconds by default.
Another way which may be better depending on your situation is to set it like this in a top level describe call in your test file:
describe("something", function () {
this.timeout(5000);
// tests...
});
This would allow you to set a timeout only on a per-file basis.
You could use both methods if you want a global default of 5000 but set something different for some files.
Note that you cannot generally use an arrow function if you are going to call this.timeout (or access any other member of this that Mocha sets for you). For instance, this will usually not work:
describe("something", () => {
this.timeout(5000); //will not work
// tests...
});
This is because an arrow function takes this from the scope the function appears in. Mocha will call the function with a good value for this but that value is not passed inside the arrow function. The documentation for Mocha says on this topic:
Passing arrow functions (“lambdas”) to Mocha is discouraged. Due to the lexical binding of this, such functions are unable to access the Mocha context.
Just adding to the correct answer you can set the timeout with the arrow function like this:
it('Some test', () => {
}).timeout(5000)
Adding this for completeness. If you (like me) use a script in your package.json file, just add the --timeout option to mocha:
"scripts": {
"test": "mocha 'test/**/*.js' --timeout 10000",
"test-debug": "mocha --debug 'test/**/*.js' --timeout 10000"
},
Then you can run npm run test to run your test suite with the timeout set to 10,000 milliseconds.
In current versions of Mocha, the timeout can be changed globally like this:
mocha.timeout(5000);
Just add the line above anywhere in your test suite, preferably at the top of your spec or in a separate test helper.
In older versions, and only in a browser, you could change the global configuration using mocha.setup.
mocha.setup({ timeout: 5000 });
The documentation does not cover the global timeout setting, but offers a few examples on how to change the timeout in other common scenarios.