We Keep Coding

Angular/ Protractor E2E Test fails when run in Azure DevOps Pipeline. Passes locally

I am trying to run e2e tests on my angular application. My tests pass locally when I run ng e2e but not in my pipeline. I'm going to share my protractor.conf, the pipeline tasks and the output I get from the task that fails below.
Some more background:
I am trying to run e2e code from an otherwise empty angular app. The
test I am trying to run simply logs into AD by navigating to my
website, entering a username/ password and then checking the that the
user is redirected to my website.
I am running this from a release
pipeline where the repo containing the e2e tests is added as an
artifact.
I am using xpaths to find elements on the page
I've omitted my jasmine code because I do not think it is relevant since the tests pass locally. However, if I'm wrong on this point let me know and I will post it.
protractor.conf.js
const { SpecReporter } = require('jasmine-spec-reporter');
process.env.CHROME_BIN = process.env.CHROME_BIN || require("puppeteer").executablePath();
exports.config = {
allScriptsTimeout: 11000,
specs: [
'./src/**/*.e2e-spec.ts'
],
capabilities: {
chromeOptions: {
args: ["--headless", "--disable-gpu", "--window-size=1200,900"],
binary: process.env.CHROME_BIN
},
'browserName': 'chrome'
},
directConnect: true,
baseUrl: 'http://localhost:4200/',
framework: 'jasmine',
jasmineNodeOpts: {
showColors: true,
defaultTimeoutInterval: 120000,
print: function() {}
},
onPrepare() {
require('ts-node').register({
project: require('path').join(__dirname, './tsconfig.e2e.json')
});
jasmine.getEnv().addReporter(new SpecReporter({ spec: { displayStacktrace: true } }));
}
};
Pipeline Tasks:
Failed Task Output:
Please help me get these tests to pass in azure
Thanks!
EDIT: I have that 'Update Webdriver' task because I read I should do it somewhere, it doesn't actually change the outcome if it is there or not

Have you tried to increase your allScriptsTimeout?
The timeout in milliseconds for each script run on the browser. This should be longer than the maximum time your application needs to stabilize between tasks.

angularjs protractor 3.3.0 not reporting specs

I don't see expected output for a passed run, the assertions are not listed. I expect to see the assertion in this line "1 spec, 0 failures".
The output:
[18:28:06] I/hosted - Using the selenium server at http://localhost:4444/wd/hub
[18:28:06] I/launcher - Running 1 instances of WebDriver
Started
.
1 spec, 0 failures
Finished in 0.854 seconds
[18:28:08] I/launcher - 0 instance(s) of WebDriver still running
[18:28:08] I/launcher - chrome #01 passed
Expected end of run output as seen on protractor's web site, http://www.protractortest.org/#/ "Run the test"):
1 test, 3 assertions, 0 failures
The spec:
describe('Viewing index.html', function() {
'use strict';
beforeEach(function(){
browser.get('/');
});
it('should have pages in left nav', function() {
expect(element.all(by.repeater('page in adminClient.selectedSite.pages')).count()).toBeGreaterThan(0);
});
});
I verified that the by.repeater locator worked:
element.all(by.repeater('page in adminClient.selectedSite.pages')).count()
.then(function(count){
console.log('page count: ' + count);
});
[UPDATE] According to this SO, it is a version issue and there is a recommendation to inject jasmine reporters on the onPrepare hook but that created more runtime errors for me.
stack overflow question
My protractor config:
exports.config = {
allScriptsTimeout: 11000,
chromeOnly: true,
chromeDriver: 'node_modules/protractor/bin/selenium/chromedriver_2.21',
seleniumAddress: 'http://localhost:4444/wd/hub',
specs: ['tests/e2e/*-spec.js'],
capabilities: {
'browserName': 'chrome'
},
baseUrl: 'http://localhost:8889/',
framework: 'jasmine',
jasmineNodeOpts: {
showColors: true,
defaultTimeoutInterval: 30000
}
};

To see the spec names and assertions you must pass the --verbose flag to protractor. If you are using grunt or something to run protractor, you'll need to specify this flag in your config.
EDIT
After reading your edit I believe I have found the solution to your issue. I've tried it with a project of my own and it appears to work.
The underlying problem is that you are likely using protractor 3, which no longer supports many of the previous options particularly within jasmineNodeOpts. To correct this issue, you should downgrade your version of protractor to 2, the latest version is 2.5.1
Here's the related issue on protractor's github repository. It also mentions a custom reporter within the onPrepare hook like you were talking about, but a different one: jasmine-spec-reporter. I got that working as well with a slightly different configuration than what you are using but it does not display the assertions, just has a much better output for the tests, which I quite like:
jasmineNodeOpts: {
print: function () {} // remove dots for each test
},
onPrepare: function () {
var SpecReporter = require('jasmine-spec-reporter');
jasmine.getEnv().addReporter(new SpecReporter({displayStackTrace: true}));
}

How do I Run multiple protractor test suites at once?

First attempt at using Protractor. I would like to be able to run multiple suites in succession.
I have an application that is one big angular form with different scenarios.
I have expected results for each scenario and would like to enter one command and run through each test.
I thought I could just use comma separated like:
protractor config.js --suite=rf1_breast, rf1_ovarian, rf1_pancreatic
But I am getting the error:
Error: more than one config file specified
Which is strange as there is only the one config file which is in the directory where I am running protractor.
Here is my config.js:
exports.config = {
seleniumAddress: 'http://localhost:4444/wd/hub',
capabilities: { 'browserName': 'chrome' },
framework: 'jasmine2',
suites: {
rf1_breast: './rf1-ashkenazi-hboc/Breast/specs/*_spec.js',
rf1_ovarian: './rf1-ashkenazi-hboc/Ovarian/specs/*_spec.js',
rf1_bladder_fail: './rf1-ashkenazi-hboc/Bladder-expected-fail/specs/*_spec.js',
rf1_pancreatic: './rf1-ashkenazi-hboc/Pancreatic/specs/*_spec.js',
rf1_prostate: './rf1-ashkenazi-hboc/Prostate/specs/*_spec.js'
},
onPrepare: function() {
/* global angular: false, browser: false, jasmine: false */
browser.manage().window().setSize(1600, 1600);
// Disable animations so e2e tests run more quickly
var disableNgAnimate = function() {
angular.module('disableNgAnimate', []).run(['$animate', function($animate) {
$animate.enabled(false);
}]);
};
browser.addMockModule('disableNgAnimate', disableNgAnimate);
},
jasmineNodeOpts: { showColors: true }
};
Is there a better way around getting each scenario run?

Don't put spaces after commas:
protractor config.js --suite rf1_breast,rf1_ovarian,rf1_pancreatic

In your config.js
If you want to run the script as a suite comment out the spec line
// specs: ['src/com/sam/scriptjs/alertexamplespec.js']
give the suite name and location
suites: {
//Suite name and location give * to run all the specs or provide the name
smoke: ['./smoke/*.spec.js'],
endtoend: ['src/com/sam/scriptjs/*.spec.js'],
//Futhermore you can select few test specs
testfew: ['./smoke/editorder.spec.js','./smoke/cancelorder.spec.js']
},
then in cmd type protractor filenameconfig.js

Try to make your suite composed by multiple files. I Have a line for the test I currently work on and another with the entire suite of tests:
exports.config = {
allScriptsTimeout: 11000,
specs: [
'./e2e/**/1.Landing.Page.ts',
'./e2e/**/2.Confirmation.Page.ts',
'./e2e/**/3.PersonalData.Page.ts',
'./e2e/**/4.sms.Page.ts',
'./e2e/**/5.idCard.Page.ts'
],
}
This works for me.

Using protractor spyOn() for jQuery ajax() cause error "ajax() method does not exist"

I trying to test the ajax is used to form submit. The protractor code for test is:
describe('login.php', function() {
it("should use ajax on submit", function() {
browser.get('/login.php');
spyOn($, "ajax");
$("form#frmLogin [type='submit']").click();
expect($.ajax).toHaveBeenCalled();
});
});
it fails with the message:
Error: ajax() method does not exist
What is wrong?
Other tests are going well. It's Windows PC. Protractor conf:
exports.config = {
allScriptsTimeout: 11000,
specs: [
'e2e/*.js',
],
capabilities: {
'browserName': 'chrome',
'chromeOptions': {
args: ['--test-type']
}
},
chromeOnly: true,
baseUrl: 'http://mysite.local/',
framework: 'jasmine',
jasmineNodeOpts: {
defaultTimeoutInterval: 30000
}
};
Is something wrong with my environment?

You cannot spy on jQuery because it is defined in your browser, not in your protractor test. Your protractor test is a node process that runs outside of the browser. It uses the webdriver protocol to communicate with the browser.
I would recommend you to write a unit test in karma for this scenario.

Karma E2E Tests found but skipped and marked as fail

Given the e2e config:
module.exports = function(config) {
config.set({
basePath: '../',
files: [
'E2E/**/*.js'
],
autoWatch: false,
browsers: ['Chrome'],
frameworks: ['ng-scenario'],
singleRun: true,
proxies: {
'/': 'http://localhost:8000/'
},
plugins: [
'karma-junit-reporter',
'karma-chrome-launcher',
'karma-jasmine',
'karma-ng-scenario'
],
junitReporter: {
outputFile: 'test_out/e2e.xml',
suite: 'e2e'
},
urlRoot: '/_karma_/'
});
};
and the scenario:
'use strict';
/* http://docs.angularjs.org/guide/dev_guide.e2e-testing */
describe('Mailing App', function () {
it('should filter the phone list as user types into the search box', function () {
expect(true).toBe(true);
});
it('should filter the phone list as user types into the search box', function () {
expect(Element('foo').count()).toEqual(1);
});
describe('Sample Test', function () {
beforeEach(function () {
browser().navigateTo('../../Client/Views/Shared/_Layout.cshtml');
});
it('should filter the phone list as user types into the search box', function () {
pause();
expect(Element('.ng-binding')).toBeDefined();
});
});
});
The programs finds the 3 tests but does not pass or fail them, rather skips them.
Running the script (with windows 8.1, git bash) returns says:
Karma v0.12.1 server started atHTTP://localhost:9876/karma/"
starting Chrome Connected on Socket Chrome 31.0.1650 Executed 0 of 3
(skipped 3) Error
Any idea why tests that don't even need to traverse the site or look at the DOM etc can be found but can not be run?

Ok, not sure if this will help anyone but basically the issue was I didn't realise that angular-scenario.js is not part of the Karma test running suite, it is from before karma when tests were running with testacular.
angular-scenario.js was being included as part of the file includes with the */.js wildcards.
Once I changed it to no longer see that it seems to now be working, guess I would have expect some kind of conflicting functions or stuff not defined to be thrown, if classes were messing each other up.