Is there a way to get JavaScript tests to do continuous integration with MSBuild or in Visual Studio? What I want to do is have it so anytime my JavaScript or my JavaScript tests change they are built again and ran and if they fall below certain values of acceptance criteria (code coverage, assertions passing, test's passing, etc.) it'll fail my build. Anyone know how to do this?
Thanks!
It looks like Chutzpah has a command-line runner, so you can create a PowerShell script which gets called from your build server to run the tests. See the Chutzpah documentation for more information. I'm not sure how you would integrate the test results with TFS.
I've used a Command Line build step to run my javascript tests with chutzpah.
Add this and point to something like
$(Build.SourcesDirectory)\packages\Chutzpah.4.0.3\tools\chutzpah.console.exe
or whereever you have chutzpah setup to be run on build server. You can then add an argument to tell the chutzpah console where the tests are
$(Build.SourcesDirectory)\YourApp.UnitTests\Tests**strong text**
Related
I am QA Engineer with very minimal developer experience. My skills are limited to writing automated tests and in my time I have always used Visual Studio Enterprise (with selenium webdriver) and TestComplete IDE's with their in built debugger. I have recently made a move to something new and want to get to grips with Nightwatch JS for my end to end tests. I have written some tests and they are running fine. However, I am now at a stage where a test is failing and I want to debug just that one test. How I would do this??? Usually in VS enterprise or TestComplete, I would just pop a break point on a line of code, right click on the one test script and it will pause at the break point.
For my Nightwatch JS project I have used Visual Studio Code. I know that I can run a single test by opening the terminal in VS Code and running the command "nightwatch tests/folder/path/myTest.js". There is not much help on nightwatch out there that has helped me and the only post I found on here was this one but it has not solved my problem. Because when I click the debugger "Play" button it starts to run ALL of my test not the one test I am interested in.
I have tried to play the debugger and then in the terminal run my cmd for the one test but the test will run through and does not hit my break points. Any guru's out there that can help?
Please ask away if you need me to post any information.
You can pass additional options for Nightwatch in your terminal, such as -- --tag.
So, you can tag each single test file you have, like this:
module.exports = {
'#tags': ['iamatag'],
'[EXAMPLE TEST]': (client) => {
code
code
code
}
};
Then, in your terminal, you would run:
$ npm test -- --tag iamatag
Only tests with the tag 'iamatag' will run.
Cheers.
I am actually configuring Jenkins to make a continuous integration of a JS project with unit tests. My problem is that after many researchs I didn't found any tutorial or information about how to proceed to write unit tests in Jenkins.
Should I use other tools (grunt, ant...) with Jenkins or could it do the job alone ? Is there any important plugin to install to make it work ?
Thanks for your answers
What you're looking for is a testing framework with a JUnit Reporter. Jenkins can read your test results from the JUnit report and give you metrics from them.
Jenkins is just for handling the automation of building your code and running your tests in whatever language you need. In order to use it for that, you need to use something like Karma, Mocha, HapiJS's Lab, Unit.js, Jasmine, etc. to actually write and test with.
We run a CI environment with Jenkins and the Project using ExtJS 5.1.1.
I try using Chutzpah test runner, using mocha framework JavaScript unit test.
With JavaScript test without ExtJS 5.1.1 testing run successfully.
But JavaScript with reference ext-all.js, there are no success. We have time out error in Blanket.js (error blanket_mocha.js line 5141) testing are not succeed and coverage.xml won't generate.
I'm looking for other way how run JavaScript test (ExtJS) in Jenkins with mocha framework and have code coverage report generate.
We already have Grunt run in Jenkins to do JavaScript combine and minify.
Is this possible to using Grunt to run mocha test and generate coverage.xml in Jenkins?
Please help me out if there is some other way to achieve the same thing. Any suggestion are highly appreciated.
I am just trying to get my head round unit testing in Javascript and RequireJS. I am building a web-app and obviously only want to have tests run in development not production builds.
Questions:
Do you just test when you want to, or do you have JS tests running
on every page load when in development?
If tests are only on demand
then how do you trigger your tests to run? Query strings (eg.
?testing=true) or something like that?
I just need an idea of how people go about testing in development. I am using BackboneJS, RequireJS and jQuery on the front end with a NodeJS/ExpressJS server on the backend.
For a Backbone project at work we have a maven build process that runs our automated javascript tests through jsTestDriver, and we read the results with Sonar. I usually run the tests manually (with 'mvn test'), but I could easily tell maven every time I save a file, for example. I wrote a post that shows how to integrate QUnit, Requirejs, and code coverage with JSTD that is independent of Maven: js-test-driver+qunit+coverage+requirejs. It also contains links to a QUnitAdapter that is way more up-to-date and developed than the one on the jsTestDriver site. I'll update this post when I manage to write about how I got jsTestDriver working with Maven and Sonar. Hope it helps.
Grunt is a popular JS build tool. There's something called grunt-watch that can monitor certain files for change, and execute tasks accordingly. You could easily run unit tests with something like this on every save.
Usually end-to-end tests take longer, and we use the CI for that. I've seen a presentation on Meteor TDD that does end-to-end tests after every save though.
There are many end-to-end test frameworks, and they can run in a headless browser like phantom js using a build tool like grunt. Some frameworks open an actual browser to run the tests, but run via command line and report results using XML.
If you break out your components enough, the tests could have a small enough scope to run on each save.
For some core code I use JsUnit + Rhino on build server. For more complex bits (usually interface) I use selenium (it also runs on build server). I don't test anything on page load, I only use not-compressed versions of scripts.
I don't any solution for integration tests.
We have a rich web client. Our controllers and service facades are written in coffeescript (JavaScript) and jquery. In the past they would have been java.
To run our JavaScript jasmine tests from Jenkins/Hudson, we use java's junit and htmlunit to load a test oriented jsp page which includes the jasmine specs.
When the Htmlunit tries to run, it blows up trying to getPage() probably because of an XML parser class path which is extremely challenging to track down in our world.
We just want to be able to run our JavaScript tests from Jenkins and have it report failure if a JavaScript test does not pass. We are just using jsp and htmlunit in order to run JavaScript tests. Can we load the JavaScript tests and javascript code into a JavaScript engine with Jenkins as the thing that kicks it off? If so, how?
Sounds like you're in a Java environment. My jasmine-maven-plugin might be a good fit.
Jasmine Reporters would also be a solution. It has instructions for running headlessly via PhantomJS for example, and it can generate JUnit XML so Jenkins can understand the test results natively, graphing test count, duration, and failure over time.
Also, the "xvfb-run" wrapper often provided with xvfb is a great help here, so you can do "xvfb-run phantomjs.runner.sh ..." in a truly headless environment.
I've previously solved this problem by running the tests with a node.js plugin called jasmine-node
This solution of course requires node.js and a few node modules to properly run the jasmine tests. There is no real browser running the tests, but an emulated one using a module called jsdom, which basically creates a headless browser, and more specifically, a DOM, which the tests can interact with.
There's node modules for jQuery, underscore and propably other too, so these can be tested too. You can even skip the whole browser emulation if you'd rather run the tests in a browser, though I find it too cumbersome compared to automated Jenkins testing.
jasmine-node generates jUnit test reports, which Jenkins can interpret just fine.
I just realized there is some jenkins-jasmine-node plugin that might ease this process.
Grunt is your friend
use grunt http://gruntjs.com/
with grunt jasimine https://github.com/gruntjs/grunt-contrib-jasmine
with nodejs http://nodejs.org/
on jenkins using https://wiki.jenkins-ci.org/display/JENKINS/NodeJS+Plugin
got this setup and it's really nice, plus this gives you a place to start making your build server do other nice things such as deployment, unit testing, etc you know, other nice things
Can you use selenium? That would actually use a real browser then and get as close to the real environment as possible.