We have a large test-suite based on Mocha (BDD), and run by the Mocha command-line client. Some of our designers / product managers would like to contribute test scenarios written in the Gherkin language (plus World / Step definitions and Hooks in JavaScript).
If we want to integrate those new test scenarios into our existing Mocha test-tree, what would be the recommended way to do this without sacrificing any basic Cucumber functionality? (Ideally, those same tests should also still be runnable with the Cucumber client.)
I don't yet understand Cucumber / Gherkin well enough to answer this myself, though I suspect it shouldn't be too difficult. Both Mocha and Cucumber seem to be quite modular, and I've seen one or two plugins that look promising. But I can't quite see how the pieces fit together.
For example, I imagine something like this should be possible with the right setup:
import gherkinToMocha from 'gherkin-to-mocha'
describe('test-tree', () => {
describe('Mocha test', () => {
it('can run', () => {})
})
describe('Cucumber / Gherkin tests', () => {
return gherkinToMocha('./test-scenario.feature')
})
})
The most direct way I see of quickly allowing product and design people to define feature files and then integrating the scenarios with your current test suite would probably be using the mocha-gherkin tool you mentioned.
You could make a folder available in your repository where product and design people are free to push feature files. Then you can have a script to generate "step-definition" files for all feature files in that folder, based on mocha-gherkin's usage:
cat file.feature | mocha-gherkin > test.js
I'd still make sure before the feature files are merged, they are properly reviewed by engineers so that steps that are similar are made uniform throughout the feature files, and the scenarios are valid.
Related
I am brand new to creating unit tests, and attempting to create tests for a project I did not create. The app I'm working with uses python/flask as a web container and loads various data stored from js files into the UI. I'm using pytest to run my tests, and I've created a few very simple tests so far, but I'm not sure if what I'm doing is even relevant.
Basically I've created something extremely simple to check if the needed files are available for the app to run properly. I have some functions put together that look for critical files, below are 2 examples:
import pytest
import requests as req
def test_check_jsitems
url = 'https://private_app_url/DB-exports_check_file.js'
r = req.get(url)
print(r.status_code)
assert r.status_code == req.codes.ok
def test_analysis_html
url = 'https://private_app_url/example_page.html'
r = req.get(url)
print(r.status_code)
assert r.status_code == req.codes.ok
My tests do work - if I remove one of the files and the page doesn't load properly - my basic tests will show what file is missing. Does it matter that the app must be running for the tests to execute properly? This is my first attempt at unit testing, so kindly cut me some slack
While testing is such a big topic, and does not fit in a single answer here, a couple of thoughts.
It's great that you started testing! These tests at least show that some part of your application is working.
While it is ok, that your tests require a running server, getting rid of that requirement, would have some advantages.
you don't need to start the server :-)
you know how to run your tests, even in a year from now (it's just pytest)
your colleagues can run the tests more easily
you could run your test in CI (continuous integration)
they are probably faster
You could check out the official Flask documentation on how to run tests without a running server.
Currently, you only test whether your files are available - that is a good start.
What about testing, whether the files (e.g the html file) have the correct content?
If it is a form, whether you can submit it?
Think about how the app is used - which problems does it solve?
And then try to test these requirements.
If you are into learning more about testing, I'd recommend the testandcode.com podcast by Brian Okken - especially the first episodes teach a lot about testing.
Jest uses describe and it and expect without you having to require them. This is okay because if you have a test file called test.spec.js, you'll never execute it directly by issuing the command node test.spec.js.
I want to execute it using node as a standard js file, without having to use jest's cli or npm test. Is it possible to do that?
For instance, I'd convert the following file:
// taken from documentation
const user = require('./users.js')
it('works with promises', () => {
expect.assertions(1);
return user.getUserName(4).then(data => expect(data).toEqual('Mark'));
});
To something like
var {describe, it, expect} = require('jest-primitives')
const user = require('./users.js')
it('works with promises', () => {
expect.assertions(1);
return user.getUserName(4).then(data => expect(data).toEqual('Mark'));
});
So that it's a self-contained js file, which can be run using just node.
A complete list of globals that jest provides is given here.
Is it technically possible? Yes.
Should you? Probably not. (But there are better ways - tl;dr use tape)
Background
Jest is a test harness.
Jest is not the only test harness to use describe() and it(). These are typical test keywords for behavior-driven development (BDD). You'll also find them used with Mocha and others.
describe() and it() are functions that interface with the test harness, telling it to add a test suite and a test case, respectively. The test harness then runs the test cases, collects the results, formats the results and outputs them.
Why you shouldn't do this with Jest
Generally, you should use technology as idiomatically as possible. This makes it easier for others to read, understand, and work with your technology.
In particular, using Jest in this way will be self-implemented, hacky, buggy and generally incomprehensible to anyone not familiar with your code. That being said, it should be possible.
How you might attempt that with Jest
Jest defines runner packages in their Jest monorepo. One is Circus, the other is Jasmine2.
Circus exports describe(), it() and some other keywords, though these are not really useful to us, as these functions only internally create the test suites and test cases but do not expose them to us or give us a way to run them.
Jasmine2 exports an executable function which returns a Promise of a test result. The code for jasmineAsyncInstall creates most of the keywords in the env or globally, and you might be able to use these.
What you'd want to do here is define it() and describe() functiions, either globally, as exports (if you'd like to use them as in the code sample in the question), or hackily by defining them inside the scope of the main module. These functions should register test cases and test suites. You'll want to either keep track of the test cases and run them later, or run them right away and keep track of the test results.
The problem now lies in determining when the test module has finished running. That is, when all of the describe() and it() have executed (whether or not the test cases themselves have executed), as well as any other incident code (code that isn't in any block). There is no good way to handle this, and here's where it may get hacky again. The easiest way is probably to add a listener to process.on('exit'.
Why that doesn't matter
A test harness is generally just a test runner and a reporter. Jest, in particular, is just a collection of components, all of which are configurable.
If we're just pulling a function here and a variable there from Jest, can we really say that we're still using it? Why do we even want to? There's really no reason to use Jest here. If you don't like the way it runs tests, you should use a different test harness instead of trying to modify it. If you like the reporter, Jest exports a package containing only the reporter.
A better way to make test files runnable
Use tap or tape. These are designed to be run the way you want, and are configurable.
Example:
const test = require('tape');
const MyClass = require('../src/my-class');
test('MyClass.doSometing should be true', (t) => {
const result = MyClass.doSomething();
if (result === true) {
t.pass('The test passed! Hooray! Our class MyClass is seemingly error-free!');
} else {
t.fail('Oh noes. Our test has failed. Why am I such a bad programmer....?');
}
t.end();
});
I had to do a lot of research to get to the right library.
The most suitable one that I found so far is Jest Lite. This guy has done a great job.
Another solution I found is a workaround with Chai.js. Install chai and import the expect from the library. Run expect statement. Unfortunately, it does not return a promise or result. You can just get the job done with a try-catch.
Is there any way to auto-generate test-cases in Truffle?
As an example, the AutoFixture Library was helping me to auto-generate test-cases in xUnit. I'm looking for a similar functionality.
Looking at Truffle's tests it seems like it uses mocha for defining tests and chai for checking that things are OK. This is a pretty common setup in JavaScript land. It might not even be mandated, but it's a good start.
This allows you to build tests like:
describe('My tests', () => {
for (const testCase of TEST_CASES) {
it(`also works for ${testCase.name}`, () => {
// check something about testCase
});
}
});
You can have arbitrarily many nestings of describe, building a tree-structure of tests. With auto-generation via the language's own features (for loops, while loops etc) you get a very powerful way of getting a lot of tests at once.
This could be a stupid question. Jasmine, Qunit, Mocha, Unit.js, etc - as far I as I know are Javascript test frameworks. But what is a Javascript test framework? What it is for actually? Is it any different from these Javascript frameworks below?
ember.js
backbone
require.js
Jasmine,
describe('Hello world', function() {
it('says hello', function() {
expect(helloWorld()).toEqual("Hello world!");
});
});
It seems like that is what node.js does, isn't?
What do you need to test?
(Short overview)
A test framework is a set of tools that allows you to test the functionality of your code (in this case your Javascript code).
It allows you to test certain functions and check if the output/result matches your expectations.
They allow you to test certain conditions and how your code react on that, like missing or unset variables, unexpected data in your variables and so on.
And one of the advantages is the test automation. This allows you to run a bunch of test automatically and it will give you result if every single test. This way you can see which test fails after you made some changes in your code.
Also you should consider reading the link mplungjan provided.
(If I missed something mandatory to say, then leave a comment, I will add that)
I'm looking for framework which provides unit tests for JS. Later, I'll have to integrate it with CruiseControl.NET. I have a lot of asp websites using JS and I need to automate testing for them. For example I used local JSUnit (may be other framework), which was quite easy for me, but I have no idea how to integrate it with CC.NET. Maybe you can show me how to start with ?
NOTE: it must be CC.NET, framework for unit test not specified
I realize this is a rather old question, but I have had similar interests so I figure I would post an answer regardless. I am assuming you are writting OO style JavaScript and wanting to test those objects.
A common solution recommended by people is to run JsTestDriver and this is definitely a great tool; that being said, it requires the JVM to be installed on your machine as well as the build server. In addition to requiring the JVM, JsTestDriver is still dependant on running one or more 'Slave' browsers. Personally, I believe that should not be a concern when unit testing JavaScript objects. If the focus is UI testing, then using a tool such as WatiN and is a different type of test in my opinion. The browser should be mock out like you would mock out any other view using JSMock or JsMockito (which I like best).
To that end, I have used Windows Script Host to run my JavaScript unit tests for years. This has several advantages:
No Browser Required (to me that is a good thing).
CScript is already on every Windows machine since XP; so nothing to install.
Runs in background process and does not require an interactive UI when running tests.
Since running inside MSTest, NUnit or xUnit etc, integrating with CC.NET, Hudson, or TeamCity is as easy as running your unit tests as a part of the build.
Full debug support simply by running your unit test inside the debugger.
There are a couple of disadvantages of course:
The machine running the unit tests has to have the ability to spawn off new processes (CSCript) to actually run the tests (has yet to be an issue).
Slower that running regular unit tests (A little over 1,000/minute on an older dual core if run individually - i.e., not bundled together in a single CSCript process).
Using JSTest.NET (approach I have used for years but just posted online), you end up with a test fixture that may look similar to the following:
using JSTest;
using JSTest.ScriptLibraries;
using Xunit;
namespace JSTest.Sample
{
public class EventDispatcherTests
{
private readonly TestScript _testScript = new TestScript();
public EventDispatcherTests()
{
//Append File(s) Under Test.
_testScript.AppendFile(#"..\..\..\Web\Scripts\eventDispatcher.js");
//Append Mock/Assertion libraries.
_testScript.AppendBlock(new JsHamcrestLibrary());
_testScript.AppendBlock(new JsMockitoLibrary());
//Append Common "Setup" Scripts.
_testScript.AppendBlock(#"var dispatcher = new EventDispatcher();
var mockHandler = JsMockito.mockFunction();
var mockPredicate = JsMockito.mockFunction();
var fakeEvent = { eventType: 'CommentAdded', data: 'Sample Comment' };
");
}
[Fact]
public void ProcessEventInvokesAttachedHandler()
{
...
}
[Fact]
public void ProcessEventGracefullyHandlesPredicateException()
{
_testScript.RunTest(#"var mockPredicateAlternate = JsMockito.mockFunction();
JsMockito.when(mockPredicateAlternate)(fakeEvent).thenThrow('MyException');
dispatcher.attachListener(mockHandler, mockPredicateAlternate);
dispatcher.processEvent(fakeEvent);
JsMockito.verify(mockPredicateAlternate)(fakeEvent);
JsMockito.verifyZeroInteractions(mockHandler);
");
}
}
}
This approach works well for me, and is very simple to setup. Basically, it is like writing regular unit tests, but with JavaScript.