We Keep Coding

How to skip or ignore programmatically a suite in CodeceptJS

As the test suite grows I need to be able to run something in BeforeSuite() which will connect to external suite and skip the suite if an external resource is unavailable.
Feature('External Server');
BeforeSuite((I) => {
// Check if server is available and skip all scenarios if it is not
});
Scenario('Login to the server', (I) => {
// Do not run this if the server is not available
})
I understand I could probably set a variable, but I think it would be nice if there was a way to tell the runner that a suite has been skipped.
The goal is to have a suite marked as skipped in the output eg:
Registration --
✓ Registration - pre-checks in 4479ms
✓ Registration - email validation in 15070ms
✓ Registration - password validation in 8194ms
External Server -- [SKIPPED]
- Login to the server [SKIPPED]

maybe prepend x before every scenario in your feature? example xScenario. I don't think codecept supports something similar to only for features. it currently works with scenarios only as far as I know.

you can use
Scenario.skip
in your step definition to skip a scenario.
Note: if any steps have been executed before skipping then it will still show it in the report
https://codecept.io/basics/#todo-test

My answer is compiled from a number of comments on the CodeceptJS github and stackoverflow. However, I can't recall the exact links or comments which helped me derive this solution, it's been at least a year, maybe two, since I started and have slowly modified this.
Edit: Found the github thread - https://github.com/codeceptjs/CodeceptJS/issues/661
Edit2: I wrote a post about "selective execution" (which avoids tagging unwanted tests with skip status) https://github.com/codeceptjs/CodeceptJS/issues/3544
I'll add a snippet at the bottom.
I'm on CodeceptJS 3.3.6
Define a hook file (eg: skip.js) and link it to your codeceptjs.conf.js file.
exports.config = {
...
plugins: {
skipHook: {
require: "../path/to/skip.js",
enabled: true,
}
}
...
}
The basic code in skip.js is
module.exports = function () {
event.dispatcher.on(event.test.before, function (test) {
const skipThisTest = decideSkip(test.tags);
if (skipThisTest) {
test.run = function skip() {
this.skip();
};
return;
}
});
};
I've defined a decision function decideSkip like so:
function decideSkip(testTags) {
if (!Array.isArray(testTags)) {
output.log(`Tags not an array, don't skip.`);
return false;
}
if (testTags.includes("#skip")) {
output.log(`Tags contain [#skip], should skip.`);
return true;
}
if (
process.env.APP_ENVIRONMENT !== "development" &&
testTags.includes("#stageSkip")
) {
output.log(`Tags contain [#stageSkip], should skip on staging.`);
return true;
}
}
(Mine is a bit more complicated, including evaluating whether a series of test case ids are in a provided list but this shows the essence. Obviously feel free to tweak as desired, the point is a boolean value returned to the defined event listener for event.test.before.)
Then, using BDD:
#skip #otherTags
Scenario: Some scenario
Given I do something first
When I do another thing
Then I see a result
Or standard test code:
const { I } = inject();
Feature("Some Feature");
Scenario("A scenario description #tag1 #skip #tag2", async () => {
console.log("Execute some code here");
});
I don't know if that will necessarily give you the exact terminal output you want External Server -- [SKIPPED]; however, it does notate the test as skipped in the terminal and report it accordingly in allure or xunit results; at least insofar as CodeceptJS skip() reports it.
For "selective execution" (which is related but not the same as "skip"), I've implemented a custom mocha.grep() utilization in my bootstrap. A key snippet is as follows. To be added either to a bootstrap anonymous function on codecept.conf.js or some similar related location.
const selective = ["tag1", "tag2"];
const re = selective.join("|");
const regex = new RegExp(re);
mocha.grep(regex);

NodeJS: Single object with all requires, or "standard" paths in code?

So, I'm a big fan of creating global namespaces in javascript. For example, if my app is named Xyz I normally have an object XYZ which I fill with properties and nested objects, for an example:
XYZ.Resources.ErrorMessage // = "An error while making request, please try again"
XYZ.DAL.City // = { getAll: function() { ... }, getById: function(id) { .. } }
XYZ.ViewModels.City // = { .... }
XYZ.Models.City // = { .... }
I sort of picked this up while working on a project with Knockout, and I really like it because there are no wild references to some objects declare in god-knows-where. Everything is in one place.
Now. This is ok for front-end, however, I'm currently developing a basic skeleton for a project which will start in a month, and it uses Node.
What I wanted was, instead of all the requires in .js files, I'd have a single object ('XYZ') which would hold all requires in one place. For example:
Instead of:
// route.js file
var cityModel = require('./models/city');
var cityService = require('./services/city');
app.get('/city', function() { ...........});
I would make an object:
XYZ.Models.City = require('./models/city');
XYZ.DAL.City = require('./services/city');
And use it like:
// route.js file
var cityModel = XYZ.Models.City;
var cityService = XYZ.DAL.City;
app.get('/city', function() { ...........});
I don't really have in-depth knowledge but all of the requires get cached and are served, if cached, from memory so re-requiring in multiple files isn't a problem.
Is this an ok workflow, or should I just stick to the standard procedure of referencing dependencies?
edit: I forgot to say, would this sort-of-factory pattern block the main thread, or delay the starting of the server? I just need to know what are the downsides... I don't mind the requires in code, but I just renamed a single folder and had to go through five files to change the paths... Which is really inconvenient.

I think that's a bad idea, because you are going to serve a ton of modules every single time, and you may not need them always. Your namespaced object will get quite monstrous. require will check the module cache first, so I'd use standard requires for each request / script that you need on the server.

Organizing code of a midly sized javascript client side app for testing

I'm building a midly size app using backbone and its friends jquery and underscore. My plan is to use QunitJS to create unittests.
I already have a Proof-Of-Concept of the app, so I basicly have a good grasp of how the code should look like without having to test it. It looks like that:
(function() {
// prepare some model
var Query = BackboneModel.extend({});
query = new Query();
// register some events
$('body.something').click(function() {
query.set('key', 'value');
# ...
});
// create some backbone view
var QueryView = Backbone.View.extend({...})
// init backbone view
query.view = new QueryView();
// add some plumbing here ...
// and so on...
})();
Otherwise said:
I wrap the module inside a function to avoid pollution
Class declaration and class use is interleaved
event registration is interleaved with the rest of the code
variables global to the module are reused inside the module
Now I need to test that. The problem is, I think, mainly about event registration and plumbing.
My plan is to wrap the code in functions and export every function and objects I want to test. The code will look like this:
var app = (function() {
var app = {}
// prepare some model
var Query = BackboneModel.extend({});
app.query = new Query();
// register some events
app.registerEvent = function() {
$('body.something').click(function() {
query.set('key', 'value');
# ...
});
};
app.registerEvent(); // XXX: call immediatly
// create some backbone view
app.QueryView = Backbone.View.extend({...})
// init backbone view
app.query.view = new QueryView();
// add some plumbing here ...
// wrapped in function with correct arguments and call it immediatly
// and so on...
// ...
return app;
})();
This is the first time I need to write tests in javascript for this kind of application so I'm wondering whether my approach to make the code testable is correct or can be improved. For instance, It seems silly to me to wrap the registration of events in function without arguments and call them immediatly.
Is there a javascript way to do this?

So I found an excellent way to test private functions while keeping my production code clean. I am not sure if you are using any build system like Grunt or Gulp, but if you are open to it you could do something like this:
//has a dependency of 'test' causing it to run testing suite first
gulp.task('js', ['test'], function() {
return gulp.src(source)
.pipe(plumber())
//create sourcemaps so console errors point to original file
.pipe(sourcemaps.init())
//strip out any code between comments
.pipe(stripCode({
start_comment: 'start-test',
end_comment: 'end-test'
}))
//combine all separatefiles into one
.pipe(concatenate('mimic.min.js'))
//minify and mangle
.pipe(uglify())
.pipe(sourcemaps.write('maps'))
.pipe(gulp.dest('dist/js'));
});
And the file could look like:
var app = (function () {
function somePrivateFunction () {}
function someotherPrivateFunction () {}
var app = {
publicFunction: function(){}
publicVar: publicVar
}
/* start-test */
app.test = {
testableFunction: somePrivateFunction
}
/* end-test */
}());
Everything between the test comments gets stripped out after the tests are run so the production code is clean. Grunt has a version of this and I assume any automated build system can do the same. You can even set a watch task so that it runs the tests on every save. Otherwise you'll have to manually remove the exported test object before deployment.
In the case of Backbone just attach a test object to the module and reference that object in the tests. Or if you really want to separate it, set the object in the global scope. window.testObject = { //list of objects and methods to test... }; and strip that code out before deployment.

Basically what I do is avoid any calls in the library I want to test. So everything is wrapped in a function and exported.
Then I have two other files one is main.js where I do the plumbing and should be tested using integration tests with selenium and tests.js which does unit testing.

Intern functional testing: skip a suite/test

I'm writing some functional tests using Intern/Leadfoot. These tests are end-to-end tests (called also stories) and between them there kind of data dependency. With that I mean that a test (ie. test2) will fail if the previous test (test1) did not complete successfully. So in order to avoid running tests that fail for sure I want to skip them (or part of them). Thus, I wonder if there a way to achieve that.
Consider that all test.js files are like the one below:
define([
"require",
"intern",
"intern!object",
"../../support/executor/executor"],
function(require, intern, registerSuite, Executor) {
var executor;
var steps = [
// set of actions,
// like login, click this button,
// then assert that ....
];
registerSuite(function() {
return {
setup: function() {
executor = new Executor(this.remote, steps.slice(0));
},
"Test 1": function() {
return executor.run();
},
};
});
});
This means that each js file is a suite that contains only one test. In other words, it's like I wanna skip all remaining suites, if a previous one failed.

The Intern docs specify a this.skip method that is available for unit tests (https://theintern.io/docs.html#Intern/4/docs/docs%2Fwriting_tests.md/skipping-tests-at-runtime) which also works for functional/e2e tests.
"skipped test":function(){
this.skip('skipping this');
}
They also specify a grep-regex option for skipping tests via the config (https://theintern.io/docs.html#Intern/4/docs/docs%2Fwriting_tests.md/skipping-tests-at-runtime).

QUnit multiple scripts in one page but no interaction between them

I'm very new to unit testing (this is my first day working with QUnit and I've never worked with anything other testng system before), and I'm a bit unclear on how to test stuff from multiple script files in one QUnit page without letting the scripts interact with each other. What I mean is, say, if I have script1.js, and it calls hello(), and hello() is defined in script2.js, how can I run a unit test on script1.js to make sure it calls hello(), but mock the output of hello() so that it's a true unit test, and then run script2.js's hello().
Basically, how am I supposed to hide one script's global variables and functions from another script in a single QUnit page?

This depends entirely on how the various script files are organized as well as the system as a whole. If you were using Angular, for example, then you are able to inject dependencies when you include a module in another script file. There are tools for mocking out things and "spying" on function calls such as Sinon, but it still depends heavily on how your code is organized.
For the sake of argument, let's say the two files look like so, and we'll ignore the design pattern (although you should seriously be considering that)...
// File A:
window.greeting = function() {
var world = hello();
return 'hello ' + world;
}
// File B:
window.hello = function() {
// possibly lots of code to determine what to return...
var value = 'foobar';
return value;
}
The hello() function could just as easily return any other value based on the state of the system, user input, etc. For our case it doesn't, and what we want to do is mock out File B's code so that we don't have to test what it is doing, just that it gives back a string. We could (and should) do this with a proper mocking/dependency injection library. However, just to give you a sense of the minimal setup you could do, and to see the general approach, here's our QUnit test file:
var _hello;
QUnit.module('File A', {
setup: function() {
_hello = window.hello; // hold onto the old value
// now we mock out hello()
window.hello = function() {
window.hello.called++; // track calls to it
return 'world'; // return our static value
}
window.hello.called = 0;
},
teardown: function() {
// put the old one back
window.hello = _hello || window.hello;
}
});
QUnit.test('Ensure greeting is correct', function(assert) {
var result = greeting();
assert.equal(window.hello.called, 1, 'hello should be called only once');
assert.equal(result, 'hello world', 'The greeting call should be "hello world"');
});
And if you want to see it running, here is a jsfiddle for you. As I said, this is a simple example to show you how you could do this, but you should look into proper code organization (think AMD modules, require, Angular, Ember, things like this) and a proper mocking library.