Is there any way to auto-generate test-cases in Truffle?
As an example, the AutoFixture Library was helping me to auto-generate test-cases in xUnit. I'm looking for a similar functionality.
Looking at Truffle's tests it seems like it uses mocha for defining tests and chai for checking that things are OK. This is a pretty common setup in JavaScript land. It might not even be mandated, but it's a good start.
This allows you to build tests like:
describe('My tests', () => {
for (const testCase of TEST_CASES) {
it(`also works for ${testCase.name}`, () => {
// check something about testCase
});
}
});
You can have arbitrarily many nestings of describe, building a tree-structure of tests. With auto-generation via the language's own features (for loops, while loops etc) you get a very powerful way of getting a lot of tests at once.
Related
Jest uses describe and it and expect without you having to require them. This is okay because if you have a test file called test.spec.js, you'll never execute it directly by issuing the command node test.spec.js.
I want to execute it using node as a standard js file, without having to use jest's cli or npm test. Is it possible to do that?
For instance, I'd convert the following file:
// taken from documentation
const user = require('./users.js')
it('works with promises', () => {
expect.assertions(1);
return user.getUserName(4).then(data => expect(data).toEqual('Mark'));
});
To something like
var {describe, it, expect} = require('jest-primitives')
const user = require('./users.js')
it('works with promises', () => {
expect.assertions(1);
return user.getUserName(4).then(data => expect(data).toEqual('Mark'));
});
So that it's a self-contained js file, which can be run using just node.
A complete list of globals that jest provides is given here.
Is it technically possible? Yes.
Should you? Probably not. (But there are better ways - tl;dr use tape)
Background
Jest is a test harness.
Jest is not the only test harness to use describe() and it(). These are typical test keywords for behavior-driven development (BDD). You'll also find them used with Mocha and others.
describe() and it() are functions that interface with the test harness, telling it to add a test suite and a test case, respectively. The test harness then runs the test cases, collects the results, formats the results and outputs them.
Why you shouldn't do this with Jest
Generally, you should use technology as idiomatically as possible. This makes it easier for others to read, understand, and work with your technology.
In particular, using Jest in this way will be self-implemented, hacky, buggy and generally incomprehensible to anyone not familiar with your code. That being said, it should be possible.
How you might attempt that with Jest
Jest defines runner packages in their Jest monorepo. One is Circus, the other is Jasmine2.
Circus exports describe(), it() and some other keywords, though these are not really useful to us, as these functions only internally create the test suites and test cases but do not expose them to us or give us a way to run them.
Jasmine2 exports an executable function which returns a Promise of a test result. The code for jasmineAsyncInstall creates most of the keywords in the env or globally, and you might be able to use these.
What you'd want to do here is define it() and describe() functiions, either globally, as exports (if you'd like to use them as in the code sample in the question), or hackily by defining them inside the scope of the main module. These functions should register test cases and test suites. You'll want to either keep track of the test cases and run them later, or run them right away and keep track of the test results.
The problem now lies in determining when the test module has finished running. That is, when all of the describe() and it() have executed (whether or not the test cases themselves have executed), as well as any other incident code (code that isn't in any block). There is no good way to handle this, and here's where it may get hacky again. The easiest way is probably to add a listener to process.on('exit'.
Why that doesn't matter
A test harness is generally just a test runner and a reporter. Jest, in particular, is just a collection of components, all of which are configurable.
If we're just pulling a function here and a variable there from Jest, can we really say that we're still using it? Why do we even want to? There's really no reason to use Jest here. If you don't like the way it runs tests, you should use a different test harness instead of trying to modify it. If you like the reporter, Jest exports a package containing only the reporter.
A better way to make test files runnable
Use tap or tape. These are designed to be run the way you want, and are configurable.
Example:
const test = require('tape');
const MyClass = require('../src/my-class');
test('MyClass.doSometing should be true', (t) => {
const result = MyClass.doSomething();
if (result === true) {
t.pass('The test passed! Hooray! Our class MyClass is seemingly error-free!');
} else {
t.fail('Oh noes. Our test has failed. Why am I such a bad programmer....?');
}
t.end();
});
I had to do a lot of research to get to the right library.
The most suitable one that I found so far is Jest Lite. This guy has done a great job.
Another solution I found is a workaround with Chai.js. Install chai and import the expect from the library. Run expect statement. Unfortunately, it does not return a promise or result. You can just get the job done with a try-catch.
We have a large test-suite based on Mocha (BDD), and run by the Mocha command-line client. Some of our designers / product managers would like to contribute test scenarios written in the Gherkin language (plus World / Step definitions and Hooks in JavaScript).
If we want to integrate those new test scenarios into our existing Mocha test-tree, what would be the recommended way to do this without sacrificing any basic Cucumber functionality? (Ideally, those same tests should also still be runnable with the Cucumber client.)
I don't yet understand Cucumber / Gherkin well enough to answer this myself, though I suspect it shouldn't be too difficult. Both Mocha and Cucumber seem to be quite modular, and I've seen one or two plugins that look promising. But I can't quite see how the pieces fit together.
For example, I imagine something like this should be possible with the right setup:
import gherkinToMocha from 'gherkin-to-mocha'
describe('test-tree', () => {
describe('Mocha test', () => {
it('can run', () => {})
})
describe('Cucumber / Gherkin tests', () => {
return gherkinToMocha('./test-scenario.feature')
})
})
The most direct way I see of quickly allowing product and design people to define feature files and then integrating the scenarios with your current test suite would probably be using the mocha-gherkin tool you mentioned.
You could make a folder available in your repository where product and design people are free to push feature files. Then you can have a script to generate "step-definition" files for all feature files in that folder, based on mocha-gherkin's usage:
cat file.feature | mocha-gherkin > test.js
I'd still make sure before the feature files are merged, they are properly reviewed by engineers so that steps that are similar are made uniform throughout the feature files, and the scenarios are valid.
I am trying to figure out how to restrict my tests, so that the coverage reporter only considers a function covered when a test was written specifically for that function.
The following example from the PHPUnit doc shows pretty good what I try to achieve:
The #covers annotation can be used in the test code to specify which
method(s) a test method wants to test:
/**
* #covers BankAccount::getBalance
*/
public function testBalanceIsInitiallyZero()
{
$this->assertEquals(0, $this->ba->getBalance());
}
If the test above would be executed, only the function getBalance will be marked as covered, and none other.
Now some actual code sample from my JavaScript tests. This test shows the unwanted behaviour that I try to get rid of:
it('Test get date range', function()
{
expect(dateService.getDateRange('2001-01-01', '2001-01-07')).toEqual(7);
});
This test will mark the function getDateRange as covered, but also any other function that is called from inside getDateRange. Because of this quirk the actual code coverage for my project is probably a lot lower than the reported code coverage.
How can I stop this behaviour? Is there a way to make Karma/Jasmine/Istanbul behave the way I want it, or do I need to switch to another framework for JavaScript testing?
I don't see any particular reason for what you're asking. I'd say if your test causes a nested function to be called, then the function is covered too. You are indeed indirectly testing that piece of code, so why shouldn't that be included in the code coverage metrics? If the inner function contains a bug, your test could catch it even if it's not testing that directly.
You can annotate your code with special comments to tell Istanbul to ignore certain paths:
https://github.com/gotwarlost/istanbul/blob/master/ignoring-code-for-coverage.md
but that's more for the opposite I think, not to decrease coverage if you know you don't want a particular execution path to be covered, maybe because it would be too hard to write a test case for it.
Also, if you care about your "low level" functions tested in isolation, then make sure your code is structured in a modular way so that you can test those by themselves first. You can also set up different test run configurations, so you can have a suite that tests only the basic logic and reports the coverage for that.
As suggested in the comments, mocking and dependency injections can help to make your tests more focused, but you basically always want to have some high level tests where you check the integrations of these parts together. If you mock everything then you never test the actual pieces working together.
This could be a stupid question. Jasmine, Qunit, Mocha, Unit.js, etc - as far I as I know are Javascript test frameworks. But what is a Javascript test framework? What it is for actually? Is it any different from these Javascript frameworks below?
ember.js
backbone
require.js
Jasmine,
describe('Hello world', function() {
it('says hello', function() {
expect(helloWorld()).toEqual("Hello world!");
});
});
It seems like that is what node.js does, isn't?
What do you need to test?
(Short overview)
A test framework is a set of tools that allows you to test the functionality of your code (in this case your Javascript code).
It allows you to test certain functions and check if the output/result matches your expectations.
They allow you to test certain conditions and how your code react on that, like missing or unset variables, unexpected data in your variables and so on.
And one of the advantages is the test automation. This allows you to run a bunch of test automatically and it will give you result if every single test. This way you can see which test fails after you made some changes in your code.
Also you should consider reading the link mplungjan provided.
(If I missed something mandatory to say, then leave a comment, I will add that)
I'm looking for framework which provides unit tests for JS. Later, I'll have to integrate it with CruiseControl.NET. I have a lot of asp websites using JS and I need to automate testing for them. For example I used local JSUnit (may be other framework), which was quite easy for me, but I have no idea how to integrate it with CC.NET. Maybe you can show me how to start with ?
NOTE: it must be CC.NET, framework for unit test not specified
I realize this is a rather old question, but I have had similar interests so I figure I would post an answer regardless. I am assuming you are writting OO style JavaScript and wanting to test those objects.
A common solution recommended by people is to run JsTestDriver and this is definitely a great tool; that being said, it requires the JVM to be installed on your machine as well as the build server. In addition to requiring the JVM, JsTestDriver is still dependant on running one or more 'Slave' browsers. Personally, I believe that should not be a concern when unit testing JavaScript objects. If the focus is UI testing, then using a tool such as WatiN and is a different type of test in my opinion. The browser should be mock out like you would mock out any other view using JSMock or JsMockito (which I like best).
To that end, I have used Windows Script Host to run my JavaScript unit tests for years. This has several advantages:
No Browser Required (to me that is a good thing).
CScript is already on every Windows machine since XP; so nothing to install.
Runs in background process and does not require an interactive UI when running tests.
Since running inside MSTest, NUnit or xUnit etc, integrating with CC.NET, Hudson, or TeamCity is as easy as running your unit tests as a part of the build.
Full debug support simply by running your unit test inside the debugger.
There are a couple of disadvantages of course:
The machine running the unit tests has to have the ability to spawn off new processes (CSCript) to actually run the tests (has yet to be an issue).
Slower that running regular unit tests (A little over 1,000/minute on an older dual core if run individually - i.e., not bundled together in a single CSCript process).
Using JSTest.NET (approach I have used for years but just posted online), you end up with a test fixture that may look similar to the following:
using JSTest;
using JSTest.ScriptLibraries;
using Xunit;
namespace JSTest.Sample
{
public class EventDispatcherTests
{
private readonly TestScript _testScript = new TestScript();
public EventDispatcherTests()
{
//Append File(s) Under Test.
_testScript.AppendFile(#"..\..\..\Web\Scripts\eventDispatcher.js");
//Append Mock/Assertion libraries.
_testScript.AppendBlock(new JsHamcrestLibrary());
_testScript.AppendBlock(new JsMockitoLibrary());
//Append Common "Setup" Scripts.
_testScript.AppendBlock(#"var dispatcher = new EventDispatcher();
var mockHandler = JsMockito.mockFunction();
var mockPredicate = JsMockito.mockFunction();
var fakeEvent = { eventType: 'CommentAdded', data: 'Sample Comment' };
");
}
[Fact]
public void ProcessEventInvokesAttachedHandler()
{
...
}
[Fact]
public void ProcessEventGracefullyHandlesPredicateException()
{
_testScript.RunTest(#"var mockPredicateAlternate = JsMockito.mockFunction();
JsMockito.when(mockPredicateAlternate)(fakeEvent).thenThrow('MyException');
dispatcher.attachListener(mockHandler, mockPredicateAlternate);
dispatcher.processEvent(fakeEvent);
JsMockito.verify(mockPredicateAlternate)(fakeEvent);
JsMockito.verifyZeroInteractions(mockHandler);
");
}
}
}
This approach works well for me, and is very simple to setup. Basically, it is like writing regular unit tests, but with JavaScript.