I'm trying to run Karma tests on a custom installation of Firefox using these binaries: https://ftp.mozilla.org/pub/firefox/releases/45.7.0esr/
I want to be able to run the Firefox binaries using Karma. I was wondering if there was a way to set the binary location for Firefox. I was hoping this could apply to pretty much any browser where you can choose to use their binaries at a custom location instead of default installation configurations. Currently here is what I have tried in my karma.conf.js file:
module.exports = function(config) {
config.set({
// For brevity, I only included the configuration for the browsers
// start these browsers
// available browser launchers: https://npmjs.org/browse/keyword/karma-launcher
browsers: ['ChromeHeadless', 'Firefox', 'FirefoxDeveloper', 'FirefoxNightly', 'IE', 'firefox-sdk/bin/firefox.exe'],
customLaunchers: {
FirefoxHeadlessLocal: {
base: 'Firefox',
flags: ['-headless'],
},
FireFoxHeadlessWin: {
base: 'firefox-sdk/bin/firefox.exe',
flags: ['-headless'],
}
}
})
}
I currently have the firefox-sdk directory (which contains the Firefox binary) in the same level as my karma.conf.js file. I then tried to run this command to run my tests:
node node_modules/karma/bin/karma start --single-run --browsers FireFoxHeadlessWin karma.conf.js
Then I get this error:
19 08 2019 11:04:43.551:ERROR [launcher]: Cannot load browser "FireFoxHeadlessWin"!
Error: No provider for "launcher:firefox-sdk/bin/firefox.exe". Can not use provider from the parent!
I am using Windows 7 currently to get this to work. I have considered using environment variables, but I want to to configure the paths locally as I would like to apply this to Linux because I am building this project on Windows and Linux. How do I get Karma to run my binaries at custom locations?
I see that you could run karma start and then karma run -- --grep=testDescriptionFilter to run a test for just one file. However, when I try to do that in a project where I am using Vue and got started with the webpack template, it doesn't work.
I've tried editing the files array in karma.conf.js, in hopes of being able to test one file this way. Normally the array looks like this:
files: [
'../../node_modules/jquery/dist/jquery.min.js',
'../../node_modules/babel-polyfill/dist/polyfill.js',
'./index.js'
],
And index.js looks like this:
import Vue from 'vue'
Vue.config.productionTip = false
// require all test files (files that ends with .spec.js)
const testsContext = require.context('./specs', true, /\.spec$/)
testsContext.keys().forEach(testsContext)
// require all src files except main.js for coverage.
// you can also change this to match only the subset of files that
// you want coverage for.
const srcContext = require.context('../../src', true, /^\.\/(?!main(\.js)?$)/)
srcContext.keys().forEach(srcContext)
First I tried to include the file I want to test instead of ./index.js like so:
files: [
'../../node_modules/jquery/dist/jquery.min.js',
'../../node_modules/babel-polyfill/dist/polyfill.js',
'./specs/players/OutcomeAnalyzerPlayer/setRangesAccordingToFilters/afterPreflop1/afterFlop1.spec.js'
],
When I do that, I get a SyntaxError: Use of reserved word 'import' error:
code/premium-poker-tools [master●] » npm run unit
> premium-poker-tools#1.0.0 unit /Users/adamzerner/code/premium-poker-tools
> BABEL_ENV=test karma start test/unit/karma.conf.js
18 10 2018 12:25:49.014:WARN [karma]: No captured browser, open http://localhost:9876/
18 10 2018 12:25:49.021:INFO [karma]: Karma v1.7.1 server started at http://0.0.0.0:9876/
18 10 2018 12:25:49.022:INFO [launcher]: Launching browser PhantomJS with unlimited concurrency
18 10 2018 12:25:49.026:INFO [launcher]: Starting browser PhantomJS
18 10 2018 12:25:49.801:INFO [PhantomJS 2.1.1 (Mac OS X 0.0.0)]: Connected on socket 1jJ_7wJ0bOiMO299AAAA with id 58854798
PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
SyntaxError: Use of reserved word 'import'
at specs/players/OutcomeAnalyzerPlayer/setRangesAccordingToFilters/afterPreflop1/afterFlop1.spec.js:1
PhantomJS 2.1.1 (Mac OS X 0.0.0): Executed 0 of 0 SUCCESS (0 secs / 0 secs)
TOTAL: 0 SUCCESS
I'm not sure why I'm getting that error, but after looking through that ./index.js file, it looks like it is necessary, and that if I want to specify that I want to only run the specs for a certain file, I'll have to do it there. So I tried creating a set-ranges-according-to-filters.js file that only includes the file i want to test, and is otherwise the same as index.js:
import Vue from 'vue'
Vue.config.productionTip = false
// require all test files (files that ends with .spec.js)
const testsContext = require.context('./specs/players/OutcomeAnalyzerPlayer/setRangesAccordingToFilters/afterPreflop1/afterFlop1.spec.js', true, /\.spec$/)
testsContext.keys().forEach(testsContext)
// require all src files except main.js for coverage.
// you can also change this to match only the subset of files that
// you want coverage for.
const srcContext = require.context('../../src', true, /^\.\/(?!main(\.js)?$)/)
srcContext.keys().forEach(srcContext)
Then I updated the files array in karma.conf.js:
files: [
'../../node_modules/jquery/dist/jquery.min.js',
'../../node_modules/babel-polyfill/dist/polyfill.js',
'./set-ranges-according-to-filters.js'
],
and ran karma. I again get the same error:
code/premium-poker-tools [master●] » npm run unit
> premium-poker-tools#1.0.0 unit /Users/adamzerner/code/premium-poker-tools
> BABEL_ENV=test karma start test/unit/karma.conf.js
18 10 2018 12:30:35.072:WARN [karma]: No captured browser, open http://localhost:9876/
18 10 2018 12:30:35.087:INFO [karma]: Karma v1.7.1 server started at http://0.0.0.0:9876/
18 10 2018 12:30:35.087:INFO [launcher]: Launching browser PhantomJS with unlimited concurrency
18 10 2018 12:30:35.119:INFO [launcher]: Starting browser PhantomJS
18 10 2018 12:30:35.937:INFO [PhantomJS 2.1.1 (Mac OS X 0.0.0)]: Connected on socket 5-U0Mca0_Vgr07-rAAAA with id 87593561
PhantomJS 2.1.1 (Mac OS X 0.0.0) ERROR
SyntaxError: Use of reserved word 'import'
at set-ranges-according-to-filters.js:1
PhantomJS 2.1.1 (Mac OS X 0.0.0): Executed 0 of 0 SUCCESS (0 secs / 0 secs)
TOTAL: 0 SUCCESS
Preferably, I'd like to be able to test one file from the command line, something like karma start --file-path-to-my-file. But if that isn't possible, being able to edit karma.conf.js so that it only runs the tests for my file would be an ok consolation.
By editing test/unit/index.js
One way is by editing test/unit/index.js. The following code for test/unit/index.js tests the file I wanted to test:
import Vue from 'vue'
Vue.config.productionTip = false
// require all test files (files that ends with .spec.js)
const testsContext = require.context('./specs/players/OutcomeAnalyzerPlayer/setRangesAccordingToFilters/afterPreflop1', true, /afterFlop1\.spec\.js/)
testsContext.keys().forEach(testsContext)
// require all src files except main.js for coverage.
// you can also change this to match only the subset of files that
// you want coverage for.
const srcContext = require.context('../../src', true, /^\.\/(?!main(\.js)?$)/)
srcContext.keys().forEach(srcContext)
Note that my initial attempt at using require.context - require.context('./specs/players/OutcomeAnalyzerPlayer/setRangesAccordingToFilters/afterPreflop1/afterFlop1.spec.js', true, /\.spec$/) - wouldn't have worked. The first argument is a folder, not a file. The second argument is a flag indicating whether subdirectories should be searched too, and the third is a regex to match files against.
I initially tried to create a different file in place of test/unit/index.js and include it in files instead of ./index.js. The reason why I was getting the SyntaxError: Use of reserved word 'import' error is because import is an ES6 feature that isn't part of Node. It needs webpack to make it available. To get it working, I had to add it to the preprocessors array in karma.conf.js like so:
files: [
'../../node_modules/jquery/dist/jquery.min.js',
'../../node_modules/babel-polyfill/dist/polyfill.js',
// './index.js'
'./set-ranges-according-to-filters.js'
],
preprocessors: {
'./index.js': ['webpack', 'sourcemap'],
'./set-ranges-according-to-filters.js': ['webpack', 'sourcemap']
},
By directly including the file in the files array
Directly including the file in the files array instead of ./index.js works too:
files: [
'../../node_modules/jquery/dist/jquery.min.js',
'../../node_modules/babel-polyfill/dist/polyfill.js',
// './index.js'
'../../src/services/monkey-patches.js',
'./specs/players/OutcomeAnalyzerPlayer/setRangesAccordingToFilters/afterPreflop1/afterFlop1.spec.js'
],
There are two caveats (that I could think of) though:
1) Using ./index.js includes other specs and src files. When you just add your file directly instead of using ./index.js, those other files don't get loaded, and that could cause stuff to break. It did for me, and I had to add something to the files array in front of my spec to get it to work.
2) I needed to add the following to the preprocessors array:
preprocessors: {
// './index.js': ['webpack', 'sourcemap'],
'../../src/services/monkey-patches.js': ['webpack', 'sourcemap'],
'./specs/players/OutcomeAnalyzerPlayer/setRangesAccordingToFilters/afterPreflop1/afterFlop1.spec.js': ['webpack', 'sourcemap']
},
Using the command line
I wasn't able to get the karma run -- --grep=testDescriptionFilter approach working. And when I run karma run --help and karma start --help, I don't see any options for running the tests of a specific file.
If you really want to, you could create different karma configuration files that run tests for different files, and then create scripts in package.json to run your tests with different configuration files. Eg. karma-1.conf.js would run the tests for file-1.spec.js, and then you could have a npm run unit-1 command that runs karma start test/unit/karma-1.conf.js:
{
"scripts": {
"unit-1": "karma start test/unit/karma-1.conf.js"
}
}
I created a web application using the Yeoman Angular-Fullstack generator. Everything runs correctly from the command line. I can use grunt to start the service and run tests.
I want to run the Karma unit tests inside the Webstorm user interface. I right clicked on karma.conf.js and had Webstorm create a Karma run configuration for me. When I try to run that configuration, it spins for a few seconds then fails.
The Karma Server window in Webstorm contains this:
/usr/local/bin/node /Applications/WebStorm.app/Contents/plugins/js-karma/js_reporter/karma-intellij/lib/intellijServer.js --karmaPackageDir=/Users/williammcneill/Documents/Work/LearnNode/gecko/node_modules/karma --configFile=/Users/williammcneill/Documents/Work/LearnNode/gecko/karma.conf.js
WARN [watcher]: Pattern "/Users/williammcneill/Documents/Work/LearnNode/gecko/client/bower_components/angular-route/angular-route.js" does not match any file.
WARN [watcher]: Pattern "/Users/williammcneill/Documents/Work/LearnNode/gecko/client/app/app.coffee" does not match any file.
WARN [watcher]: Pattern "/Users/williammcneill/Documents/Work/LearnNode/gecko/client/components/**/*.coffee" does not match any file.
WARN [watcher]: Pattern "/Users/williammcneill/Documents/Work/LearnNode/gecko/client/components/**/*.jade" does not match any file.
WARN [watcher]: Pattern "/Users/williammcneill/Documents/Work/LearnNode/gecko/client/app/**/*.coffee" does not match any file.
WARN [watcher]: Pattern "/Users/williammcneill/Documents/Work/LearnNode/gecko/client/app/**/*.jade" does not match any file.
INFO [karma]: Karma v0.12.37 server started at http://localhost:8080/
INFO [launcher]: Starting browser PhantomJS
INFO [PhantomJS 1.9.8 (Mac OS X 0.0.0)]: Connected on socket 29_J57vgWL6mWwyV13tv with id 98142631
Which all looks correct to me. The Test Run window contains this:
/usr/local/bin/node /Applications/WebStorm.app/Contents/plugins/js-karma/js_reporter/karma-intellij/lib/intellijRunner.js --karmaPackageDir=/Users/williammcneill/Documents/Work/LearnNode/gecko/node_modules/karma --serverPort=8080 --urlRoot=/
/Applications/WebStorm.app/Contents/plugins/js-karma/js_reporter/karma-intellij/lib/intellijRunner.js:54
throw e;
^
Error: connect ETIMEDOUT 127.0.0.1:8080
at Object.exports._errnoException (util.js:837:11)
at exports._exceptionWithHostPort (util.js:860:20)
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1060:14)
Process finished with exit code 1
I can't figure out what is timing out. If I open http://localhost:8080 I see a webpage that looks like this.
Which also looks correct to me.
I haven't actually written any code. I'm taking all the defaults that Yeoman and Webstorm give me. I think I have all the relevant Webstorm plugins (AngularJs, Karma).
Does anyone have insight into what might be happening or how I could debug it?
Webstorm 10.0.4, Yeoman 1.4.8, Karma 0.13.9, Node 4.1.0, OS X 10.10.5
The port value in karma.conf.js is set to 8080, even though the server runs on 9000. Change this value to 9000 and everything works.
Not sure exactly how the grunt configuration makes it all work, or why it's set up this way, but that's the workaround.
I'm trying to do some unit testing and I'm at the end of my rope. I've moved files, rewritten things countless times, and I still can't get the damn thing to work. I've followed this tutorial exactly and it still doesn't work.
When I run "gulp tests", and I am in my ~/git/mobile directory, I get the following output
Josh#DAEDALUS ~/git/mobile (unit-testing)
$ gulp test
[10:28:40] Using gulpfile c:\Users\Josh\git\mobile\gulpfile.js
[10:28:40] Starting 'test'...
WARN [watcher]: Pattern "c:/Users/Josh/git/mobile/tests/www/lib/angular/angular.
js" does not match any file.
WARN [watcher]: Pattern "c:/Users/Josh/git/mobile/tests/node_modules/angular-moc
ks/angular-mocks.js" does not match any file.
WARN [watcher]: Pattern "c:/Users/Josh/git/mobile/tests/www/js/app.js" does not
match any file.
WARN [watcher]: Pattern "c:/Users/Josh/git/mobile/tests/www/js/controller.js" do
es not match any file.
WARN [watcher]: Pattern "c:/Users/Josh/git/mobile/tests/www/js/services.js" does
not match any file.
WARN [watcher]: Pattern "c:/Users/Josh/git/mobile/tests/tests/Controllers/contro
llers.tests.js" does not match any file.
WARN [watcher]: Pattern "c:/Users/Josh/git/mobile/tests/tests/Services/services.
tests.js" does not match any file.
INFO [karma]: Karma v0.12.37 server started at http://localhost:9876/
INFO [launcher]: Starting browser PhantomJS
INFO [PhantomJS 1.9.8 (Windows 8 0.0.0)]: Connected on socket --QoOiVt0ZraOpiQ3q
zd with id 22369565
PhantomJS 1.9.8 (Windows 8 0.0.0): Executed 0 of 0 ERROR (0.003 secs / 0 secs)
[10:28:42] Finished 'test' after 2.19 s
See, I don't get why it decides that it wants to add the /tests/ folder out of nowhere. It's not in my directory. For example, a proper path would be c:/users/josh/git/mobile/www/js/controller.js That tests folder isn't there. There is a tests folder in my mobile directory, but it doesn't contain anything but my "my.conf.js" and two subdirectories which contain my test files.
I cannot for the life of me figure out what's wrong. I've tried adding ../, ./, /, or removing that all together from the beginning of my files paths in my.conf.js.
Here is the contents of "my.conf.js"
Here is the contents of "gulpfile.js"
I fixed this by adding ../ to my basePath.
I'm trying to run Karma/Jasmine from Grunt on a project generated with
http://newtriks.com/2013/12/31/automating-react-with-yeoman-and-grunt/
Karma launches PhantomJS (or Chrome) and, depending on singleRun, it either times out or just sits there and does nothing. I've tried changing captureTimeout and browserNoActivityTimeout based on reading solutions from people with similar problems, but it doesn't seem to work.
My relevant pacakge versions etc.:
NodeJS: 0.10.25
Karma: 0.12.16
Webpack: 1.1.11
webpack-dev-server: 1.4.1
karma-jasmine: 0.1.5
Linux: Ubuntu 14.04
I've found someone with the same problem on OS X:
I've tried updating all my dev dependencies to the latest versions but the problem still remains.
My console output is below. The webpack lines referring to bundle is now VALID/INVALID are worrying, but I can't find any info on what they mean. Here's my console output:
Running "karma:unit" (karma) task
DEBUG [config]: autoWatch set to false, because of singleRun
DEBUG [plugin]: Loading karma-* from /home/ed/workspace/wwb-app/node_modules
DEBUG [plugin]: Loading plugin /home/ed/workspace/wwb-app/node_modules/karma-chrome-launcher.
DEBUG [plugin]: Loading plugin /home/ed/workspace/wwb-app/node_modules/karma-coffee-preprocessor.
DEBUG [plugin]: Loading plugin /home/ed/workspace/wwb-app/node_modules/karma-firefox-launcher.
DEBUG [plugin]: Loading plugin /home/ed/workspace/wwb-app/node_modules/karma-html2js-preprocessor.
DEBUG [plugin]: Loading plugin /home/ed/workspace/wwb-app/node_modules/karma-jasmine.
DEBUG [plugin]: Loading plugin /home/ed/workspace/wwb-app/node_modules/karma-phantomjs-launcher.
DEBUG [plugin]: Loading plugin /home/ed/workspace/wwb-app/node_modules/karma-requirejs.
DEBUG [plugin]: Loading plugin /home/ed/workspace/wwb-app/node_modules/karma-script-launcher.
DEBG [plugin]: Loading plugin /home/ed/workspace/wwb-app/node_modules/karma-webpack-plugin.
INFO [karma]: Karma v0.12.16 server started at http://localhost:8080/
INFO [launcher]: Starting browser PhantomJS
DEBUG [temp-dir]: Creating temp dir at /tmp/karma-98204612
DEBUG [launcher]: /home/ed/workspace/wwb-app/node_modules/karma-phantomjs-launcher/node_modules/phantomjs/lib/phantom/bin/phantomjs /tmp/karma-98204612/capture.js
Hash: 89285186567c1bc5bb7f
Version: webpack 1.1.11
Time: 2ms
Asset Size Chunks Chunk Names
webpack: bundle is now VALID.
webpack: bundle is now INVALID.
DEBUG [web-server]: serving: /home/ed/workspace/wwb-app/node_modules/karma/static/client.html
DEBUG [web-server]: serving: /home/ed/workspace/wwb-app/node_modules/karma/static/karma.js
DEBUG [web-server]: upgrade /socket.io/1/websocket/CjC8pnQq5It2z_kWYB98
DEBUG [karma]: A browser has connected on socket CjC8pnQq5It2z_kWYB98
INFO [PhantomJS 1.9.7 (Linux)]: Connected on socket CjC8pnQq5It2z_kWYB98 with id 98204612
DEBUG [launcher]: PhantomJS (id 98204612) captured in 1.704 secs
WARN [PhantomJS 1.9.7 (Linux)]: Disconnected (1 times), because no message in 30000 ms.
DEBUG [karma]: Run complete, exitting.
DEBUG [launcher]: Disconnecting all browsers
DEBUG [launcher]: Process PhantomJS exited with code 0
DEBUG [temp-dir]: Cleaning temp dir /tmp/karma-98204612
Warning: Task "karma:unit" failed. Use --force to continue.
Aborted due to warnings.
Here's my karma.conf.js file:
'use strict';
module.exports = function (config) {
config.set({
basePath: '',
frameworks: ['jasmine'],
files: [
'test/helpers/**/*.js',
'test/spec/components/**/*.js'
],
preprocessors: {
'test/spec/components/**/*.js': ['webpack']
},
webpack: {
cache: true,
module: {
loaders: [{
test: /\.css$/,
loader: 'style!css'
}, {
test: /\.gif/,
loader: 'url-loader?limit=10000&minetype=image/gif'
}, {
test: /\.jpg/,
loader: 'url-loader?limit=10000&minetype=image/jpg'
}, {
test: /\.png/,
loader: 'url-loader?limit=10000&minetype=image/png'
}, {
test: /\.js$/,
loader: 'jsx-loader'
}]
}
},
webpackServer: {
stats: {
colors: true
}
},
exclude: [],
port: 8080,
logLevel: config.LOG_DEBUG,
colors: true,
autoWatch: true,
// Start these browsers, currently available:
// - Chrome
// - ChromeCanary
// - Firefox
// - Opera
// - Safari (only Mac)
// - PhantomJS
// - IE (only Windows)
browsers: ['PhantomJS'],
reporters: ['progress'],
captureTimeout: 60000,
browserNoActivityTimeout: 60000,
singleRun: true
});
};
I had the same problem. From a related GitHub Issue, I learned that you can extend the inactivity timeout.
Set this Karma config option in your gruntfile or karma config file:
browserNoActivityTimeout: 100000
I set it to 100 seconds and my tests ran successfully. I don't know what's causing the delay.
I've changed my Karma config, to
captureTimeout: 60000, // it was already there
browserDisconnectTimeout : 10000,
browserDisconnectTolerance : 1,
browserNoActivityTimeout : 60000,//by default 10000
Also I have 200-300 tests, PhantomJS 1.9.8
and it needs only about 100 mb memory for Phantom
With grunt and karma
They all together used about 300mb of memory.
We encountered a similar problem on our build servers.
Increasing the browserNoActivityTimeout worked to a point. We upped it to 60000ms, but the problem with phantomJS not disconnecting returned as the number of unit tests increasing.
We eventually tracked the problem down to the RAM available to phantomJS. We had 1100 unit tests that would take ~1m30s to run, but phantomJS would fail to disconnect within the 60000ms timeout.
The build node VM RAM was increased from 2GB to 4GB and the 1100 unit tests then took ~45s to run and phantomJS would disconnect with ~5s. A huge improvement.
There are two lessons:
1. PhantomJS is memory hungry, so make sure it has enough RAM to do its thing
2. Profile your code to learn where you can be more efficient with memory usage.
Another likely explanation is RequireJS getting in the way. I'm getting this exact error if I add 'requirejs' to the karma.conf.js in the config.frameworks array.
This seems to override the native require function and cause tests to not be executed. In my case the describe-block was triggered, but none if the it-blocks were.
In my case I hadn't included the following code in my test.js file:
requirejs.config({
callback: window.__karma__.start
});
describe('tests', function() {
...
Once this config was included the tests started running. Hopefully this saves someone else a lot of stress!
Remove the 'requires' from the karma config file, just use frameworks: ['jasmine'].
Check that localhost points correctly to 127.0.0.1 and not an unreachable IP, this can happen in dev environments using virtual machines for example.
This may not be the case for the OP here, but if the code you're testing hits an infinite loop, it will cause a disconnection on timeout just like this.
Here is why I was getting this error, might help someone in similar situation.
My main component had multiple child components which were using different services, one of the services had an HTTP call and the initialisation failed as I was using that service in ngInit() method of the child component.
To fix the issue, I had to import the above service in the main component specs and attach a mock for the service, it started working after that.
I solved this for my own environment. I had a bunch of nodejs packages installed globally. I didn't do a regression to figure out exactly what package caused the problem, but I strongly suspect that having karma installed globally was the cause.
If you have this problem then try
sudo npm -g remove karma
and if that doesn't work then I would remove all global node packages (except truly global packages like yeoman, grunt-cli, for example). And then install locally for your project.
I also noticed that when you run sudo npm -i on OS X, it changes the owner of ~/.npm to root and subsequent npm -i commands will fail with an EACCESS error,.
#Vijender's answer got me on the right track
It was as simple as replacing HttpClientModule in tests with HttpClientTestingModule.
beforeEach(async(() => {
TestBed.configureTestingModule({
imports: [
HttpClientTestingModule
]
}).compileComponents();
}));
I fixed this by removing a call to an async function at global scope, in my Root.tsx.
It was working in a 'real' browser window, but doesn't work in a test run.
It seems to make loading the module itself hang, so it doesn't even get as far as executing the async function (so log statements won't show up)
Edit:
This was caused by another async function, run at app startup, that was trying to do auth and redirect to another URL. That's why it worked in the browser but not in headless mode. Without any async test functions, the auth code never had a chance to jam the system. With an async test, the async auth code started running and caused the lock-up.
Moral of the story: if an async test is hanging, check what other async code is going on in the background.
afterEach(function () {
document.body.innerHTML = '';
});
Adding this fixed issue for me, tests started running much faster. Seems like it decreases load on headless browser.