I have a bunch of describes that test different parts of an API. In one section, all the tests are dependent on one test succeeding. I want to make Mocha run the first test, and if it fails, skip all following tests and run the next test suite for the next section of the API.
mocha --bail would stop testing altogether after the first fail, and won't continue to the next section.
mocha-steps is a viable solution, but I prefer not to use any external libraries. In addition, it doesn't skip steps after the failure, it doesn't print them altogether. Like I said, it's a viable solution, but not ideal.
What would be the best way to implement this behavior in vanilla Mocha?
Put what you call your "first test" in a before hook inside a describe block that contains all the other tests:
describe("bunch of related tests", function () {
before(function () {
// "first test" here
});
it("test 1", function () { ... });
it("test 2", function () { ... });
// ...
});
This is the proper way in "vanilla Mocha" to set a dependency between the code in the before hook and each of the tests. If the before hook fails, Mocha will report it, and it will skip all the tests in the describe block. If you have other tests elsewhere, they will still run.
Although I up-voted the accepted answer, I wasn't able to get a Mocha it test to run inside a before function. Instead I had to separate the first test into its own describe and set a variable if the test passed, then check the variable in the before of the describe containing all the other tests.
let passed = false
describe('first test', function() {
it('run the first test', function(done) {
if (theTestPassed)
passed = true
done()
})
})
describe('rest of the tests', function() {
before(function() {
if (!passed)
throw new Error('skip rest of the tests')
});
it('second test', ...)
it('third test', ...)
});
Related
jest provides afterEach, beforeEach, afterAll and beforeAll to complete setup and teardown logic. What I would like to do, is to clear up after one particular test. Consider the following:
describe("a family of tests it makes sense to group together", () => {
...
test("something I want to test", () => {
// some setup needed for just this test
global.foo = "bar"
// the test
expect(myTest()).toBe(true)
// clear up
delete global.foo
}
...
}
The problem with the above...
If the test above fails for some reason, then delete global.foo is never run. This means that potentially all of the tests following it will fail. Rather than seeing 1 test fail, I see a whole load of tests fail, which could be confusing.
Potential (non-ideal) solutions
One solution is just to add delete global.foo into my afterEach. It doesn't really need to be run after every test, but it doesn't do any harm either. Another solution would be to put the particular test by itself so that afterEach would only apply to it. But this doesn't seem ideal either - if that test belongs with other tests, it aught to be possible for it to remain with them.
My question:
Is there a way to run teardown logic for just a specific test (without running it inside the actual test). In my particular use-case the first outlined solution is fine, but I can imagine there might be situations where finer grained control is needed. If my teardown method took a long time for example I wouldn't want to repeat it lots, as this would slow down the whole test-suite.
In many cases tests can share a common afterEach clean-up even if it's needed for one of them, as long as it doesn't affect others.
Otherwise, this is what block structure is responsible for. One or several tests can be grouped with nested describe just to have their own afterEach, etc blocks, and the only downside is that it makes the report less pretty:
describe("a family of tests it makes sense to group together", () => {
...
describe("something I want to test", () => {
beforeEach(() => {
global.foo = "bar"
});
test("something I want to test", () => {
expect(myTest()).toBe(true)
}
afterEach(() => {
delete global.foo
});
});
beforeEach and afterEach can be desugared to try..finally:
test("something I want to test", () => {
try {
global.foo = "bar"
expect(myTest()).toBe(true)
} finally {
delete global.foo
}
})
This also allows for asynchronous tests but requires them to be written with async instead of done.
I know that this is an old question, but for anyone who stumbles upon this challenge in the future, here's a small library I wrote way back when, called jest-after-this that does just that:
import { afterThis } from 'jest-after-this';
it('should do something that requires a cleanup', () => {
global.foo = 'something';
afterThis(() => {
delete global.foo;
});
// ... rest of test here can fail, the cleanup method will run anyways
});
Hope this helps :)
My understanding of jest from observation is that it provides concurrent execution of tests by spawning helper processes and distributes test files to the workers to execute as they finish their current test files.
That suggests to me that jest won't attempt to execute tests in an individual test file concurrently. So I would expect that the following test would always pass (without needing to pass --runInBand):
describe('counting test', () => {
let variable = 0;
it('should start as 1', () => {
variable += 1;
expect(variable).toEqual(1);
});
it('should change to 2', () => {
variable += 1;
expect(variable).toEqual(2);
});
});
I.e. the second test is always run after the first test has finished. Is that safe, and is there an official document somewhere that specifies this behaviour? I couldn't find one.
Since this didn't have an official answer, I added one to the jest documentation after some further research / experimentation (and it was signed off by one of their moderators).
So, yes, jest runs each test in a file sequentially, waiting for each to finish before moving onto the next. This is now described in Setup and Teardown.
Further note that describe blocks are all executed before any of the test blocks.
For reference, the code that implements this is mostly in jest-circus/src/run.ts and eventHandler.ts.
I have a following function:
function prompt_mandatory_field_completion(){
$("#mandatory_fail").show(150, function() {
setTimeout(function() {
$("#mandatory_fail").fadeOut(500)
}, 2000);
});
window.scrollTo(0,0)
}
That I would like to test with jasmine but regardless to what I put in my spec file the test seems to pass.
The spec file contains the following code :
it(' NEED TO FIX THAT FADE OUT Should prompt user to fill in mandatory questions via prompt_mandatory_field_completion function', function() {
prompt_mandatory_field_completion();
setTimeout(2000, function(){
expect($('#mandatory_fail').css('display').toEqual('random thing'));
});
In my SpecRunner.html I am using the following function that I run in before each test in this description block:
function setupFixtures(){
setFixtures('<div id="mandatory_fail" style="display:none;"></div>');
prompt_mandatory_field_completion();
};
Any idea how to make this into a meaningful test? I guess I have been staring at it way too long and poking it from all the directions.
Best,
Adam
You're trying to write a functional test of asynchronous behavior. You might have a lot better experience trying to use protractor for this sort of test. It's tuned more toward asserting things that will eventually be true.
However, jasmine does have an asynchronous facility since about 2.0, known as done(), that will insist that all of the asynchronous code has run before the test passes or fails.
You have to pass the done function to get asynchronous tests :
it(' NEED TO FIX THAT FADE OUT Should prompt user to fill in mandatory questions via prompt_mandatory_field_completion function', function(done) {
prompt_mandatory_field_completion();
setTimeout(2000, function(){
expect($('#mandatory_fail').css('display').toEqual('random thing'));
done();
});
}, 3000);
You can also pass a timeout as a last parameter, depending on what you've set in your jasmine's settings.
Otherwise, Jasmine will consider this test to fail if its execution exceed its timeout.
Mocha has several 'hooks' to run assistive functionality in a test separate from the test cases themselves (clearing databases, creating mock files, etc).
However, in the case of before() (not beforeEach(), that one I get), it seems pretty redundant.
before() runs the logic once before all of the tests in the current suite, so why do I need to even wrap it in a function?
There's no observable difference between the two following:
describe('Something', function() {
before(doSomePreTestLogic);
//Tests ahoy
});
and
describe('Something', function() {
doSomePreTestLogic();
//Tests ahoy
});
What's the point of wrapping my pre-test logic with before()?
There's a couple different reasons why you might opt for the before() hook in your tests:
Semantics
This is probably the biggest one. Sometimes you may need to perform a task, such as populate some dummy data in your database, prior to running an actual test. You can certainly do that in the test itself, but that throws off your reporting if you encounter an error while pre-populating, and before having run the actual test case at all. By having a before() hook, you have a logical, semantically valid place to insert this kind of logic.
Cleaner for Async Tests
Just like mocha tests can be asynchronous, so can your before() hook logic. Going back to the pre-population example, this is handy since that means all of your async pre-test logic won't force another level of indentation on all of your actual testing logic.
Consistency with other hooks:
Mocha also provides several other hooks, namely: after(), beforeEach(), and afterEach(). You could probably ask this same question about all these other hooks as well, but if you buy into the notion that any of them have a validated existence, then before() really needs to be included to round out the API.
The Upshot
Put directly inside a describe callback code that will build the test suite. I'm talking about calls to it but also functions that may loop over tables or files to declare a bunch of tests (by calling it in the loop).
Put inside hooks the code that actually initialize the state that the tests depend on.
All other considerations are quite secondary.
Let me explain...
The Background
Mocha executes a test suite in two phases:
It discovers what tests exist. During this phase it will immediately execute the callbacks passed to describe and record for future invocation the callbacks passed to the functions that declare tests (it and the like) and functions that declare hooks (before, beforeEach, after, etc.).
It runs the tests. In this phase it will run the callbacks that it recorded earlier.
The Difference
So consider this example:
function dump () { console.log("running:", this.test.fullTitle()); }
describe("top", function () {
before(dump);
it("test 1", dump);
it("test 2", dump);
describe("level 1", function () {
before(dump);
it("test 1", dump);
it("test 2", dump);
});
});
Note that fullTitle gives the whole name of a test starting from the top level describe, going through any nested describe down to the it or hook that contains the call. Run with the spec reporter and keeping only the running: lines, you get:
running: top "before all" hook: dump
running: top test 1
running: top test 2
running: top level 1 "before all" hook: dump
running: top level 1 test 1
running: top level 1 test 2
Note the order of the hooks, and how each executes immediately before the tests declared in its respective describe callback.
Consider then this suite:
function dump () { console.log("running:", this.test.fullTitle()); }
function directDump() { console.log("running (direct):", this.fullTitle()); }
describe("top", function () {
directDump.call(this);
it("test 1", dump);
it("test 2", dump);
describe("level 1", function () {
directDump.call(this);
it("test 1", dump);
it("test 2", dump);
});
});
Run with the spec reporter and keeping only the running: lines, you get:
running (direct): top
running (direct): top level 1
running: top test 1
running: top test 2
running: top level 1 test 1
running: top level 1 test 2
Note how both calls to directDump are run before anything else.
The Consequences
If any initialization code that you put directly inside a callback to describe fails, the entire run fails right away. No test will be executed. End of story.
If any initialization code that you put inside a before hook fails, the consequences are contained. For one thing, because a before hook is run just at the moment is needed, there is an opportunity any tests that are scheduled earlier can run. Also, Mocha will only skip those tests that depend on the before hook. For instance, let's assume this suite:
function dump () { console.log("running:", this.test.fullTitle()); }
describe("top", function () {
before(dump);
it("test 1", dump);
it("test 2", dump);
describe("level 1", function () {
before(function () { throw new Error("foo"); });
it("test 1", dump);
it("test 2", dump);
});
describe("level 1 (second)", function () {
before(dump);
it("test 1", dump);
it("test 2", dump);
});
});
If you run it with the spec reporter, the entire output (minus the stack trace) will be something like:
top
running: top "before all" hook: dump
running: top test 1
✓ test 1
running: top test 2
✓ test 2
level 1
1) "before all" hook
level 1 (second)
running: top level 1 (second) "before all" hook: dump
running: top level 1 (second) test 1
✓ test 1
running: top level 1 (second) test 2
✓ test 2
4 passing (5ms)
1 failing
1) top level 1 "before all" hook:
Error: foo
[stack trace]
Note how a) some tests ran before the failing hook and b) Mocha still ran the tests that do not depend on the hooks.
Semantics, at both the machine and human level.
Also, it keeps test code in line with the "exports" interface, e.g.,
module.exports = {
before: function(){
// ...
},
'Array': {
'#indexOf()': {
'should return -1 when not present': function(){
[1,2,3].indexOf(4).should.equal(-1);
}
}
}
};
I use mocha for some integration testing and have many test sets.
Each set has initialization tests. When such tests fail, the rest of the set should not run at all, because if one fails then each will fail.
The thing is that I can't avoid such initialization tests, because part of the code/environment is generated by some tool which does not guarantee any correct result.
Is it possible to implement this using mocha ?
Using the BDD interface, the normal way to do this with Mocha is to put anything that sets up the testing environment into before or beforeEach:
describe("foo", function () {
describe("first", function () {
before(function () {
// Stuff to be performed before all tests in the current `describe`.
});
beforeEach(function () {
// Stuff to perform once per test, before the test.
});
it("blah", ...
// etc...
});
describe("second", function () {
before(function () {
// Stuff to be performed before all tests in the current `describe`.
});
beforeEach(function () {
// Stuff to perform once per test, before the test.
});
it("blah", ...
// etc...
});
});
If the before or beforeEach that a test depends on fails, then the test is not run. Other tests that don't depend on it will still run. So in the example above if the callback passed to before in the describe named first fails, the tests in the describe named second won't be affected at all and will run, provided that their own before and beforeEach callbacks don't fail.
Other than this, Mocha is designed to run tests that are independent from each other. So if one it fails, then the others are still run.
I found mocha-steps which basically allow you to write a "chain" of it()s (called step()) and mocha will abort the suite if one of them breaks, thus avoiding the cascade of inevitable failures, and I found pull request 8 marks subsequent steps and subsuites as pending. So I can write:
describe("businessCode()", function() {
step("should be not null", function() {
assert(businessCode() != null)
});
step("should be a number", function() {
assert(typeof businessCode() === 'number');
});
step("should be greater than 10", function() {
assert(businessCode() > 10);
});
describe("thingThatCallsBusinessCode()", function() {
step("should be greater than 10", function() {
assert(thingThatCallsBusinessCode() != null);
});
});
});
If e.g. businessCode() returns a boolean, only the should be a number test will fail; the subsequent ones (and the subsuite will be marked as pending).