Since I'm a newbie with automated tests and protractor, I'm having some trouble setting this up in my tests.
According to the guide, every time that I create a new instance of screenshot reporter, I have to pass a directory path. Right, this means that every time I create a new instance in my spec file?
Also, there are functions to take screenshots of my skipped and my failed tests. Where i supposed to use takeScreenShotsForSkippedSpecs and takeScreenShotsOnlyForFailedSpecs? In my config file?
This is my onPrepare:
onPrepare: function () {
browser.driver.manage().window().maximize();
global.dvr = browser.driver;
global.isAngularSite = function (flag) {
browser.ignoreSynchronization = !flag;
}
jasmine.getEnv().addReporter(new ScreenShotReporter({
baseDirectory: '/tmp/screenshots',
takeScreenShotsForSkippedSpecs: true,
takeScreenShotsOnlyForFailedSpecs: true
}));
Note: If you are using jasmine2, use protractor-jasmine2-screenshot-reporter.
For jasmine1:
I've been using successfully using protractor-html-screenshot-reporterpackage. It is based on protractor-screenshot-reporter, but also provides a nice HTML report.
Here is what I have in the protractor config:
var HtmlReporter = require("protractor-html-screenshot-reporter");
exports.config = {
...
onPrepare: function () {
// screenshot reporter
jasmine.getEnv().addReporter(new HtmlReporter({
baseDirectory: "test-results/screenshots"
}));
},
...
}
After running tests, you would get an HTML file containing (example):
You can click "view" to see the test-case specific screenshot in the browser.
The readme in the library is pretty self explanatory. After installing the library, add it onto protractor's onPrepare in your protractor config file.
i.e.
protractorConf.js:
var ScreenShotReporter = require('protractor-screenshot-reporter');
exports.config = {
// your config here ...
onPrepare: function() {
// Add a screenshot reporter and store screenshots to `/tmp/screnshots`:
jasmine.getEnv().addReporter(new ScreenShotReporter({
baseDirectory: '/tmp/screenshots',
takeScreenShotsForSkippedSpecs: true
}));
}
}
then protractor protractorConf.js to run protractor.
Just recently I published a brand new plugin called protractor-screenshoter-plugin that captures for each browser instance a screenshot and console logs. The snapshot is made optionally for each expect or spec. It comes with a beautiful angular and bootstrap based analytics tool to visually check and fix tests results.
Check it out at https://github.com/azachar/protractor-screenshoter-plugin.
Also, I created a list of all available alternatives, so if you find something else, please do not hesitate to add it there.
Related
I'm trying to get webdriver.io and Jasmine working.
Following their example, my script is at test/specs/first/test2.js (in accordance with the configuration) and contains:
var webdriverio = require('webdriverio');
describe('my webdriverio tests', function() {
var client = {};
jasmine.DEFAULT_TIMEOUT_INTERVAL = 9999999;
beforeEach(function() {
client = webdriverio.remote({ desiredCapabilities: {browserName: 'firefox'} });
client.init();
});
it('test it', function(done) {
client
.url("http://localhost:3000/")
.waitForVisible("h2.btn.btn-primary")
.click("h2.btn.btn-primary")
.waitForVisible("h2.btn.btn-primary")
.call(done);
});
afterEach(function(done) {
client.end(done);
});
});
I'm using wdio as the test runner, and set it up using the interactive setup. That config is automatically-generated and all pretty straightforward, so I don't see a need to post it.
In another terminal window, I am running selenium-server-andalone-2.47.1.jar with Java 7. I do have Firefox installed on my computer (it blankly starts when the test is run), and my computer is running OS 10.10.5.
This is what happens when I start the test runner:
$ wdio wdio.conf.js
=======================================================================================
Selenium 2.0/webdriver protocol bindings implementation with helper commands in nodejs.
For a complete list of commands, visit http://webdriver.io/docs.html.
=======================================================================================
[18:17:22]: SET SESSION ID 46731149-79aa-412e-b9b5-3d32e75dbc8d
[18:17:22]: RESULT {"platform":"MAC","javascriptEnabled":true,"acceptSslCerts":true,"browserName":"firefox","rotatable":false,"locationContextEnabled":true,"webdriver.remote.sessionid":"46731149-79aa-412e-b9b5-3d32e75dbc8d","version":"40.0.3","databaseEnabled":true,"cssSelectorsEnabled":true,"handlesAlerts":true,"webStorageEnabled":true,"nativeEvents":false,"applicationCacheEnabled":true,"takesScreenshot":true}
NoSessionIdError: A session id is required for this command but wasn't found in the response payload
at waitForVisible("h2.btn.btn-primary") - test2.js:21:14
/usr/local/lib/node_modules/webdriverio/node_modules/q/q.js:141
throw e;
^
NoSessionIdError: A session id is required for this command but wasn't found in the response payload
0 passing (3.90s)
$
I find this very strange and inexplicable, especially considering that it even prints the session ID.
Any ideas?
Please check out the docs on the wdio test runner. You don't need to create an instance using init on your own. The wdio test runner takes care on creating and ending the session for you.
Your example covers the standalone WebdriverIO usage (without testrunner). You can find examples which use wdio here.
To clarify that: there are two ways of using WebdriverIO. You can embed it in your test system by yourself (using it as standalone / or as a scraper ). Then you need to take care of things like create and end an instance or run those in parallel. The other way to use WebdriverIO is using its test runner called wdio. The testrunner takes a config file with a bunch of information on your test setup and spawns instances updates job information on Sauce Labs and so on.
Every Webdriver command gets executed asynchronously.
You properly called the done callback in afterEach and in your test it test, but forgot to do it in beforeEach:
beforeEach(function(done) {
client = webdriverio.remote({ desiredCapabilities: {browserName: 'firefox'} });
client.init(done);
});
I'm currently running my test suite on AngularJS using Grunt, Karma, Jasmine and Protractor. The database library I'm using is hood.ie, which is a library on top of CouchDB. I start hood.ie using the following code in my Gruntfile:
hoodie: {
start: {
options: {
callback: function(config) {
grunt.config.set('connect.proxies.0.port', config.stack.couch.port);
}
}
}
},
However, I would like to have a separate database for running tests, which automatically resets afterwards. This way, the production data won't conflict with the tests.
How should I approach this? I would assume there's some kind of standard way of doing this, as I can imagine other people have come across the same problem, but I'm unable to find anything on the internet.
Currently, this seems to be impossible as the hoodie server does not support it. The best way to go about this is to modify it yourself at Hood.ie server Github repository by adding a parameter to define the folder in which the data will be stored, which is at the moment hardcoded to 'data' (https://github.com/hoodiehq/hoodie-server/blob/master/lib/core/environment.js#L48)
Something similar to this should work:
app_path: path.resolve(project_dir, argv.folder || 'data')
As the hoodie task is a 'multitask' you could have a test target in your hood.ie grunt task specific to testing, and then reference this in a grunt command used to run tests e.g:
hoodie: {
start: {
options: {
callback: function(config) {
grunt.config.set('connect.proxies.0.port', config.stack.couch.port);
}
}
},
test: {
options: {
callback: function(config) {
// Make test specific changes here.
}
}
}
}
// The task that runs tests first starting test deps. 'runtests' can be anything you want.
grunt.registerTask('test', 'Run unit tests', ['hoodie:test', 'runtests']);
Note: this will mean any other times you're referencing the hoodie task you'll need to be explicit as otherwise all the specified targets will be run. See this documentation on multitasks for more info. In this example you'd change hoodie to hoodie:start to run the 'start' task as previously defined.
I have found Protractor framework which is made for AngularJS web applications.
How can I use Protractor on a website which is not using AngularJS?
I wrote my first test and Protractor triggers this message:
Error: Angular could not be found on the page https://www.stratexapp.com/ : retries looking for angular exceeded
Another approach is to set browser.ignoreSynchronization = true before browser.get(...). Protractor wouldn't wait for Angular loaded and you could use usual element(...) syntax.
browser.ignoreSynchronization = true;
browser.get('http://localhost:8000/login.html');
element(by.id('username')).sendKeys('Jane');
element(by.id('password')).sendKeys('1234');
element(by.id('clickme')).click();
If your test needs to interact with a non-angular page, access the webdriver instance directly with browser.driver.
Example from Protractor docs
browser.driver.get('http://localhost:8000/login.html');
browser.driver.findElement(by.id('username')).sendKeys('Jane');
browser.driver.findElement(by.id('password')).sendKeys('1234');
browser.driver.findElement(by.id('clickme')).click();
waitForAngular should now be used instead of the deprecated ignoreSynchronization property.
The following waitForAngular guidance is taken from the Protractor docs for timeouts:
How to disable waiting for Angular
If you need to navigate to a page which does not use Angular, you can turn off waiting for Angular by setting `browser.waitForAngularEnabled(false). For example:
browser.waitForAngularEnabled(false);
browser.get('/non-angular-login-page.html');
element(by.id('username')).sendKeys('Jane');
element(by.id('password')).sendKeys('1234');
element(by.id('clickme')).click();
browser.waitForAngularEnabled(true);
browser.get('/page-containing-angular.html');
Forget about ignoreSynchronization!!! It no longer exists in protractor starting from protractor 5.3.0
browser.waitForAngularEnabled(false) should be used instead
Step by step instructions how to use it
Protractor has builtin handling of waiting for angular and this is what makes it different. However, if the page is not angular protractor will hang until it hits timeout error, so you need to disable that feature in order to test non-angular page. There is also an edge case, when the page is angular, but this feature will STILL make protractor error out. Perform the steps below, to find out when to disable protractor's waiting for angular
Find out if your page is Angular: open dev console in the browser and inside of 'console' tab run command
getAllAngularTestabilities()
If the output is getAllAngularTestabilities is not defined, then your page is not angular, go to the last step to disable built-in waiting
Run these commands in the console
getAllAngularTestabilities()[0]._ngZone.hasPendingMacrotasks
getAllAngularTestabilities()[0]._ngZone.hasPendingMicrotasks
If any of these return true (if there are micro or macro tasks pending), then go to the last step. If all are false, congrats, you can use the builtin protractor's waiting for angular. But if you don't like it as I don't, then read the last step to find out how to disable it
Run the above mentioned command. BUT! It returns a promise, which needs to be handled, preferably using await keyword
await browser.waitForAngularEnabled(false)
This command can be ran anywhere: in the it block, in beforeAll, or in onPrepare of your configuration. Just make sure, if you use await to make the respective block async
beforeAll(async () => await browser.waitForAngularEnabled(false))
To test on non angular site, you should remove the synchronisation. for that use the following:
browser.ignoreSynchronisation = true;
browser.get('url');
In some occasions, to avoid errors need to add both values .
browser.driver.ignoreSynchronization = true;
browser.waitForAngularEnabled(false);
You can add them in spec.js file .
describe('My first non angular class', function() {
it ('My function', function() {
browser.driver.ignoreSynchronization = true;
browser.waitForAngularEnabled(false);
Or as #Mridul Suggested ,add them in config.js file .
exports.config = {
directConnect: true,
framework: 'jasmine',
onPrepare: function () {
browser.driver.ignoreSynchronization = true;// for non-angular set true. default value is false
browser.waitForAngularEnabled(false); // for non-angular set false. default value is true
},
Personally, I didn't get any success with proposed solutions as the DOM elements weren't properly loaded in time.
I tried many ways of handling that asynchronous behavior, including browser.wait with browser.isElementPresent, but none of them were satisfying.
What did the trick is using Protractor returned Promises from its methods in onPrepare :
onPrepare: () => {
browser.manage().window().maximize();
browser.waitForAngularEnabled(true).then(function () {
return browser.driver.get(baseUrl + '/auth/');
}).then(function () {
return browser.driver.findElement(by.name('login')).sendKeys('login');
}).then(function () {
return browser.driver.findElement(by.name('password')).sendKeys('password');
}).then(function () {
return browser.driver.findElement(by.name('submit')).click();
}).then(function () {
return true;
});
return browser.driver.wait(function () {
return browser.driver.getCurrentUrl().then(function (url) {
return /application/.test(url);
});
}, 10000);
},
I was inspired by https://github.com/angular/protractor/blob/master/spec/withLoginConf.js
add the below snippet in your .js spec file
beforeAll(function() {
browser.waitForAngularEnabled(false);
});
Add the following piece of code in the conf.js file
onPrepare: function () {
browser.ignoreSynchronization = true;
}
Add below snippet for non angular applications:
app- browser.ignoreSynchronization = true;
Use below snippet in your config.js file for non-angular applications-
browser.ignoreSynchronization = true;
and for angular application -
browser.ignoreSynchronization = false;
I am working on aurelia web-app which is a FE framework similar to Angular , React. In this I am using Protractor for automation.
Tech Stack which of my project:-
Protractor
Typescript
Page Object Modal
Cucumber
Chai
node
npm
VS Code (IDE)
The main change happens only in the configuration file, I can add code in github if that would help, here is the config file I am using in my project which works perfect for me.
Posted some blogs as well in my wordpress , hope that can be of help.
const reporter = require('cucumber-html-reporter');
exports.config = {
SELENIUM_PROMISE_MANAGER: false,
directConnect: true,
specs: ["./e2e/features/*/EndToEnd.feature"],
format: 'json:cucumberReport.json',
framework: 'custom',
frameworkPath: require.resolve('protractor-cucumber-framework'),
cucumberOpts: {
strict: true,
format: 'json:cucumberReport.json',
keepAlive: false,
require: [
'./e2e/hooks/*.ts',
'./e2e/stepDefinition/*/*.ts',
],
tags: '#Regression'
},
beforeLaunch: function () {
require('ts-node/register')
},
onPrepare: async () => {
await browser.waitForAngularEnabled(false);
await browser.ignoreSynchronization == true;
await browser.manage().window().maximize();
await browser.manage().timeouts().implicitlyWait(10000);
},
onComplete: async () => {
var options = {
theme: 'bootstrap',
jsonFile: './reports/cucumberReport.json',
output: './reports/cucumberReport.html',
reportSuiteAsScenarios: true,
launchReport: false,
screenshotsDirectory: './reports/screenshots',
storeScreenshots: true,
metadata: {
"Test Environment": "SAND-DEV-1",
"Platform": "Windows 10",
}
};
reporter.generate(options);
},
};
Instead of Protractor, you can use for e2e testing the Testcafe.
Pros:
ES2016 syntax
no need an additional dependencies, configs and browser plugins
flexible selectors
easy setup
As indicated in this stackoverflow answer, it looks like Karma will serve JSON fixtures. However, I've spent too many hours trying to get it to work in my environment. Reason: I'm doing angular testing and need to load mock HTTP results into the test, as Jasmine doesn't support any global setup/teardown with mock servers and stuff.
In my karma config file, I'm defining a fixture as so:
files: [
// angular
'angular/angular.min.js',
'angular/angular-route.js',
'angular/mock/angular-mocks.js',
// jasmine jquery helper
'jquery-1.10.2.min.js',
'angular/jasmine-jquery.js',
// our app
'../public/js/FooApp.js',
// our tests
'angular/*-spec.js',
// fixtures
{ pattern: 'node/mock/factoryResults.json',
watched: 'true',
served: 'true',
included: 'false' }
]
Before I even attempt to use jasmine-jquery.js in my jasmine test to load the JSON, I see karma choking on trying to serve it:
...
DEBUG [web-server]: serving: /Users/XXX/FooApp/spec/node/mock/factoryResults.json
Firefox 25.0.0 (Mac OS X 10.8) ERROR
SyntaxError: missing ; before statement
at /Users/XXX/FooApp/spec/node/mock/factoryResults.json:1
...
Here's what factoryResults.json looks like:
{ "why": "WHY" }
Any idea what's going on here? I see plenty of examples on the web of folks successfully loading JSON into jasmine tests via karma fixtures. Karma can see the file; if I put the wrong path in my fixture block, I see an error stating that it couldn't find any files that match my fixture pattern. I've tried reformatting the .json file in different ways... Any ideas?
Your problem is that 'false' has to be a boolean, not a string.
There is already an issue to validate the config better and fix such a mistakes.
Also, you might write a simple "json" preprocessor (similar to karma-html2js) that would make it valid JS and put the JSON into some global namespace so that you can keep the tests synchronous...
I also needed json fixtures in my karma test suite.
I ended up just using the html2js preprocessor with json files as well as html.
karma.conf.js:
module.exports = function (config) {
config.set({
frameworks: ["jasmine"],
files: [
'**/*.js',
'**/*.html',
'**/*.json',
'**/*.spec.js'
],
plugins: [
'karma-html2js-preprocessor'
]
preprocessors: {
'**/*.html': ['html2js'],
'**/*.json': ['html2js']
}
});
};
Then it is just a matter of getting the json from the __html__ global.
e.g.
var exampleJson = __html__['example.json'];
var jsonObj = JSON.parse(exampleJson);
var exampleHtml = __html__['example.html'];
document.body.innerHTML = exampleHtml;
So, I had a lot of issues with jasmine-jquery and I got a pretty decent workaround.
It's a little hacky, but it works. Basically, I just create a function accessible on the window, then stack the JSON fixtures inside a little switch:
if (typeof(window.fixtures === "undefined")) {
window.fixtures = {};
}
window.setFixture = function(type) {
var json;
if (type == "catalog") {
json = { ... }
}
if (typeof(type) !== "undefined") {
window.fixtures[type] = json;
}
return json;
}
Then, I can just stub it inline in the view:
describe "App.Models.Catalog", ->
it "provides the 'App.Models.Catalog' function", ->
expect(App.Models.Catalog).toEqual(jasmine.any(Function))
it "sets up a fixture", ->
setFixture("catalog")
console.log(fixtures["catalog"])
expect(fixtures["catalog"]).toBeDefined()
Boom, tests pass, and the object comes out in the log:
{
catalog_id: '2212',
merchant_id: '114',
legacy_catalog_id: '2340',
name: 'Sample Catalog',
status: '1',
description: 'Catalog Description ',
}
Now, it's accessible within my test.
It's of course not perfect or ideal, but I kept hitting strange matchErrors and the like with the jasmine-jquery plugin, and it's simple enough (and fast) for me to paste in a couple of JSON blocks and get moving.
You also save yourself the time fiddling around with the configuration and making any changes to the files for Karma.
Anyone have any better suggestions or have any luck getting jasmine-jquery to work?
I am using Karma to test my project and can see tests passing an failing in the console window, however, how do I get these to show in the browser? The browser only has a green bar (even though a test is failing) with
Karma v0.10.2 - connected
Written in it.
I have tried addong singleRun :false to the karma.config.js file.
The config file looks like this:
module.exports = function (config) {
config.set({
basePath: '../',
files: [
'app/lib/angular/angular.js',
'app/lib/angular/angular-*.js',
'test/lib/angular/angular-mocks.js',
'app/js/**/*.js',
'test/unit/**/*.js'
],
autoWatch: true,
singleRun: false,
frameworks: ['jasmine'],
browsers: ['Chrome'],
plugins: [
'karma-junit-reporter',
'karma-chrome-launcher',
'karma-firefox-launcher',
'karma-jasmine'
],
junitReporter: {
outputFile: 'test_out/unit.xml',
suite: 'unit'
}
})
}
matthias-schuetz wrote a plugin that claims to produce html test output.
https://npmjs.org/package/karma-htmlfile-reporter
Along with the instructions on the plugin page I had to include a reference to the plugin in the Karma config -
plugins: [
'karma-htmlfile-reporter'
]
Accroding to Documention, even not very perfect:
If 'singleRun' is false, it will set the ci-mode on, so, make it true, and you will see some failed red status on the top bars of browers.
There is no straight forward way to do this with Karma.
The "best" way to solve this problem would be to hunker down and write a html-reporter for karma (which would make a lot of us other Karma users very happy).
If this is too much work for you, the second best thing is to use the junit reporter which generates an xml file. You can then post-process the xml file in some way that turns it into a HTML-file which you can then view in your browser
I wanted to display HTML5 Web Notifications with Karma so I wrote something quick to get it to work with Karma version 0.11. Might behave slightly different with other versions. I load this script in with the rest of my application scripts, it will store the karma test results and after completion it will determine the success of the test and then reset to the original karma functions so they're not changed when this script gets run again.
// store all my test results
var results = [];
// Wrap the karma result function
var resultFunc = window.__karma__.result;
window.__karma__.result = function(result){
// run the original function
resultFunc(result);
// push each result on my storage array
results.push(result);
}
// wrap the karma complete function
var completeFunc = window.__karma__.complete;
window.__karma__.complete = function(result){
// run the original function
completeFunc(result);
// determine success
var success = results.every(function(r){ return r.success });
if (success) {
// display a success notification
}
else {
// display a test failure notification
}
// reset the result function
window.__karma__.result = resultFunc;
// reset the complete function
window.__karma__.complete = completeFunc;
}