We Keep Coding

Catching all failures in Jasmine Protractor spec tests

I want to be able to catch any and all failures within my specs and log it into my Test Rail integration. I have this pattern currently where I wrap all the tests in a try/catch and on the CATCH, I mark the test as FAILURE and add it to the results array. Inside the afterAll it will then send all these results to Test Rail's API.
This all works mostly correct. However, occasionally one of my tests will fail but will NOT trigger the catch part of the code. My spec file will proceed all the way to the done() and finish the full test and THEN will show my failure once the code gets to the AfterAll section.
I assume this is because the expect(lookForSomeObject) is still processing/looking in the background and it doesn't finally finish doing its thing until the spec file gets to the AfterAll section.
How can I update this pattern to be better.
afterAll(async () => {
var r = await tr.addResultsForCases(testCases);
});
it('Verifies Search', async (done) => {
var testCaseInfo: AddResultForCase = {
case_id: 75
}
try {
const locatorSearchBarInput = await loc.locatorSearchBarInput();
await waitToBeDisplayed(locatorSearchBarInput);
await locatorSearchBarInput.sendKeys('35011');
const locatorSearchButton = await loc.locatorSearchButton();
await waitToBeDisplayed(locatorSearchButton);
await click(locatorSearchButton);
await expect(await loc.locatorSearchBarText()).toBeDisplayed();
testCaseInfo.status_id = TestRailStatus.Passed;
testCases.results.push(testCaseInfo);
done();
} catch (e) {
testCaseInfo.status_id = TestRailStatus.Failed;
testCases.results.push(testCaseInfo);
done.fail(e);
}
});

I would get rid of try/catch
Add case_id into your it description at the env like so Verifies Search #75
Create your own custom jasmine reporter as described here https://jasmine.github.io/tutorials/custom_reporter
register it within your onPrepare()
specDone function will take spec parameter (or result, whatever you call it). Then add this snippet
specDone: function(spec) {
if (spec.status === 'passed') {
// do anything on success
} else if (spec.status === 'failed') {
let case_id = spec.description.match(/#([^#]*$)/)[1];
// report your case_id as failed
} else {
// block for skipped test cases (marked by `xit` or `xdescribe`)
}
}
That's it. No need to add your handling to each test case anymore.

How to make protractor test fail if function fails?

So basically I have some helpers method to help me debug my protractor test cases, one of my main ones is tho wait for an element to be clickable, I'm creating this loop to give the necessary time for protractor to find and make sure the element is enabled, but when an element is not found, either by the element not being found or a typo on my scrip, I would like the test run to STOP and mark it as a FAILURE..
async WaitToBeClickable(element){
try{
for(var i = 0; i <= 3000; i++){
var wait = await browser.wait(this.EC.elementToBeClickable(element), i);
if(wait == true){
break;
}else{
//this is where I want to fail
}
}
}catch(err){
//this is where I want to fail
await console.log(`WAIT TO BE CLICKABLE FAILED:\n${element.parentElementArrayFinder.locator_.value}\n\nError:\n${err}\n`);
}
};
this would help me a LOT debugging my script since I'm working on VSC, but I can not seem to find a way yet to make the test FAIL and thereby to CLOSE the browser at the first fail, I've seen protractor-fail-fast and protractor-bail-fast but it seems to be for the jasmine test cases not for function, I WOULD REALLY APPRECIATE any help please, protractor its driving me a bit nuts lol...

//method
const checkData = () = {
return new Promise((resolve)=>{
if(true){
// Success case
return resolve(true)
}
else{
// Fail case
return reject(false)
}
})
}
//Specfile
it('sample test',async ()=>{
Let data = await checkData();
expect(data).toEqual(true);
})
Based on resolved value test will pass or fail

If you function throws error you can just use the done function that jasmine provides
Example:
it('should do something', async done => {
try {
await weDoSomething();
} catch (e) {
done.fail(e); // mark test as failed
}
done(); // mark test as done
}, 1 * 60 * 1000);
async function weDoSomething() {
throw Error('failed function');
}

Did you try simply re-throwing the error in the catch? I believe that should cause the test to fail.
Comments:
You do not need to await a console.log as it is an synchronous operation.
broser.waits throw exceptions when the element is not found within the timeout period, it appears your for loop is not required at all.
This function will wait 3000ms for the element to be clickable and if that does not happen it will throw an exception which will be caught in the catch. It will log the message and then rethrow the error which will fail your test (assuming the error is not being caught and handled further up)
async WaitToBeClickable(element){
try {
await browser.wait(this.EC.elementToBeClickable(element), 3000);
} catch (err) {
console.log(`WAIT TO BE CLICKABLE FAILED:\n${element.parentElementArrayFinder.locator_.value}\n\nError:\n${err}\n`);
throw new Error(err);
}
};

Jasmine: Test that returned promise is a concrete exception

I have a method in my node.js server, that returns a Promise - throws a custom Exception (UserNotAuthenticatedError) - and I want to write a test to make sure that this Exception is thrown whenever it has to.
The method is the following:
export function changePassword(userId, oldPass, newPass) {
var query = User.findById(userId);
return query.exec()
.then(user => {
if (user.authenticate(oldPass)) {
user.password = newPass;
return user.save();
} else {
// I want to test that this Exception is thrown
throw new UserNotAuthenticatedError();
}
});
}
I've tried writing the test, and this is what I have so far:
describe('#changePassword', function() {
it('should throw a UserNotAuthenticatedError when passing wrong password', function() {
var userId = user._id;
var wrongPwd = 'wrongpassword';
var newPwd = 'new password';
// so far the following only tells me that the Promise was rejected, but
// I WANT TO TEST THAT THE REJECTION WAS DUE TO A 'UserNotAuthenticatedError'
UserService.changePassword(userId, wrongPwd, newPwd).should.be.rejected;
});
});
I have accomplished checking that the promise returned was rejected, but I also want to be more specific and check that the returned promise is a UserNotAuthenticatedError. How could I do that?

I finally ended up finding a much nicer way to solve it:
it('should throw a UserNotAuthenticatedError when passing wrong password', function() {
var userId = user._id;
var wrongPwd = 'wrongpassword';
var newPwd = 'new password';
var promise = UserService.changePassword(userId, wrongPwd, newPwd);
return promise.should.be.rejectedWith(UserNotAuthenticatedError);
});

Wouldn't you just use a promise.catch(function(error){}) and inspect the error variable to make sure it's the right type of error?

I suppose there are better ways to handle this than the one I propose. This one should work nevertheless.
var promise = UserService.changePassword(userId, wrongPwd, newPwd);
var error = null;
promise.should.be.rejected;
promise.then(function(){}, function(err){
expect(err).toEqual(jasmine.any(UserNotAuthenticatedError));
done();
});
Where done is function which could be passed as parameter to the it method. It specifies that it is asynchronous test.

Working with promises, doing a save

I have the following cloud function returning immediately, with promise full-filled but without doing its job. Can any one see why?
function myFunction(newValArray)
{
console.log("Entered myFunction.");
var query,classPromise;
query = new Parse.Query("myClass");
classPromise = (query.find().then
(function(result) {
result[0].set("myField", result[0].get("myField")+1);
result[0].save(null,{}).then
(function() {
console.log("1)newValArray:"+newValArray.length.toString());
newValArray.push(result[0].get("myField"));
console.log("2)newValArray:"+newValArray.length.toString());
return Parse.Promise.as();
});
}));
return Parse.Promise.when(classPromise);
}
If I then use this code for :
myFunction(newLTR).then
(function() {
console.log("myFunction FULL-FILLED.");
}
I can see the messages "Entered myFunction." and "myFunction FULL-FILLED." in the logs.
But I never see "1)newValArray:..." neither do I see "2)newValArray:..."
I have also checked that the passed parameter has not been processed as expected.
If I replace myFunction with the following version, it doesn't make any difference:
function myFunction(newValArray)
{
console.log("Entered myFunction.");
var query,classPromise;
query = new Parse.Query("Configuration");
classPromise = (query.find().then
(function(result) {
result[0].set("myField", result[0].get("myField")+1);
result[0].save(null,{
success:function(configRcd) {
console.log("1)newValArray:"+newValArray.length.toString());
newValArray.push(configRcd.get("myField"));
console.log("2)newValArray:"+newValArray.length.toString());
return Parse.Promise.as();
},
error:function(error) {
console.log("Something went wrong in incrementLastTouchReference.");
}});
}));
return Parse.Promise.when(classPromise);
}

That's a terrible way to write your promises. The whole reason you want to use promises in the first place, is so you can chain callbacks. In your example it's the worst of both worlds, the complexity of promises but you're still nesting.
The second issue is that you really need to place a final error handler. Any errors emitted right now might just disappear. always end with a catch.
I rewrote your first function to correctly do promises, but I can't guarantee if there's not something else wrong. Hopefully it helps you along your way:
function myFunction(newValArray)
{
console.log("Entered myFunction.");
var query,classPromise;
query = new Parse.Query("myClass");
classPromise = query.find()
.then(function(result) {
result[0].set("myField", result[0].get("myField")+1);
return result[0].save(null,{});
}).then(function() {
console.log("1)newValArray:" + newValArray.length.toString());
newValArray.push(result[0].get("myField"));
console.log("2)newValArray:"+newValArray.length.toString());
return Parse.Promise.as();
}).then(function(result) {
// I added this third then clause. You were returning
// Parse.Promise.as() so presumably you wanted to do something
// with that. Before this then clause it got discarded, with my
// change the result of Parse.Promise.as() is thrown in the
// 'result' argument in this function.
}).catch(function(err) {
// If an error is thrown in any part of the change, it will
// bubble to this final catch statement.
// Do something with err! Log it or whatever ;)
})
return Parse.Promise.when(classPromise);
}

Using Jasmine's done() to do asynchronous testing?

I am attempting to test some asynchronous JavaScript (that was once TypeScript) using Jasmine. I've had a hard time getting this to work correctly, and with this simple example it never makes it to the then(function( code block and I get the following error:
Error: Timeout - Async callback was not invoked within timeout specified by jasmine.DEFAULT_TIMEOUT_INTERVAL.
My test looks something like:
it("Should work", function(done){
dataService.ready = true;
dataService.isReady().then(function(result){
console.log(result);
expect(result).toBe(true);
done();
});
});
And the service I'm testing looks something like (before compiled to JavaScript):
public isReady(): angular.IPromise<any> {
var deferred = this.q.defer();
if (this.ready) {
setTimeout(() => { return deferred.resolve(true); }, 1);
} else {
// a bunch of other stuff that eventually returns a promise
}
return deferred.promise;
}
I am sure I am just misusing done() but I feel like this should work! Any suggestions?
UPDATE:
For further debugging, I added a few console logs within the isReady() function. It now looks like:
public isReady(): angular.IPromise<any> {
var deferred = this.q.defer();
console.log("in isReady()"); // new line to add logging
if (this.ready) {
console.log("this.ready is true"); // new line to add logging
setTimeout(() => {
console.log("returning deferred.resolve"); // new line to add logging
return deferred.resolve(true);
}, 1);
} else {
// a bunch of other stuff that eventually returns a promise
}
return deferred.promise;
}
isReady() works as expected when I manually test in a browser. When running the test, my logs include:
LOG: 'in isReady()'
LOG: 'this.ready is true'
LOG: 'returning deferred.resolve'
Within my test, it appears to never be resolved (code block within then() is never executed) but when running my app this function works just fine. This example is in a controller:
DataService.isReady().then(() => {
console.log("I work!");
});
UPDATE: And more debugging...
In my test:
it("Should work", function(done){
console.log("calling dataService.isReady()");
var prom = dataService.isReady();
console.log("promise before");
console.log(prom);
setTimeout(function(){
console.log("promise after");
console.log(prom);
},1000);
prom.then(function(result){
// never makes it here
done();
}, function(reason) {
// never makes it here either
});
}
Now, in my console, I see:
LOG: 'calling dataService.isReady()'
LOG: 'in isReady()'
LOG: 'this.ready is true'
LOG: 'promise before'
LOG: Object{$$state: Object{status: 0}}
LOG: 'returning deferred.resolve'
LOG: 'promise after'
LOG: Object{$$state: Object{status: 1, pending: [...], value: true, processScheduled: true}}
So, my promise looks like it should. Why isn't then() being invoked?

What was actually happening:
So, it turns out my question should have been something like "Why doesn't my angular promise resolve in my Jasmine test?"
$digest
After a bit of digging and looking at other solutions, I found some good information about when/how promises are resolved. I needed to call $digest() on the $rootScope in order to resolve the promise and execute the then() code block (and therefore call done() to satisfy the spec).
No more request expected error
Adding $rootScope.$digest() got me most of the way there, but then I started discovering a No more request expected error that was causing my tests to fail. This was because the service I am using is sending off various POST and GET requests for another aspect of my application. Stubbing out a whenGET and whenPOST response seemed to solve that issue.
The Final Solution:
Long story short, my spec file now looks like:
describe("Async Tests", function(){
var dataService;
var rootScope;
var httpBackend;
beforeEach(module("myangularapp"));
beforeEach(inject(function(_$httpBackend_, _DataService_, $rootScope){
dataService = _DataService_;
rootScope = $rootScope;
httpBackend = _$httpBackend_;
// solves the 'No more request expected' errors:
httpBackend.whenGET('').respond([]);
httpBackend.whenPOST('').respond([]);
}));
it("Should work", function(done){
dataService.ready = true;
dataService.isReady().then(function(result){
console.log(result);
expect(result).toBe(true);
// still calls done() just as before
done();
});
// digest the scope every so often so we can resolve the promise from the DataService isReady() function
setInterval(rootScope.$digest, 100);
});
});
This solution seems more complex than it needs to be, but I think it'll do the trick for now. I hope this helps anyone else who may run into challenges with testing async code with Angular and Jasmine.

Another way to handle this is to always use .finally(done) when testing promises and then call $timeout.flush() afterwards.
"use strict";
describe('Services', function() {
var $q;
var $timeout;
// Include your module
beforeEach(module('plunker'));
// When Angular is under test it provides altered names
// for services so that they don't interfere with
// outer scope variables like we initialized above.
// This is nice because it allows for you to use $q
// in your test instead of having to use _q , $q_ or
// some other slightly mangled form of the original
// service names
beforeEach(inject(function(_$q_, _$timeout_) {
$q = _$q_;
// *** Use angular's timeout, not window.setTimeout() ***
$timeout = _$timeout_;
}));
it('should run DataService.isReady', function(done) {
// Create a Dataservice
function DataService() {}
// Set up the prototype function isReady
DataService.prototype.isReady = function () {
// Keep a reference to this for child scopes
var _this = this;
// Create a deferred
var deferred = $q.defer();
// If we're ready, start a timeout that will eventually resolve
if (this.ready) {
$timeout(function () {
// *** Note, we're not returning anything so we
// removed 'return' here. Just calling is needed. ***
deferred.resolve(true);
}, 1);
} else {
// a bunch of other stuff that eventually returns a promise
deferred.reject(false);
}
// Return the promise now, it will be resolved or rejected in the future
return deferred.promise;
};
// Create an instance
var ds = new DataService();
ds.ready = true;
console.log('got here');
// Call isReady on this instance
var prom = ds.isReady();
console.log(prom.then);
prom.then(function(result) {
console.log("I work!");
expect(result).toBe(true);
},function(err) {
console.error(err);
}, function() {
console.log('progress?');
})
// *** IMPORTANT: done must be called after promise is resolved
.finally(done);
$timeout.flush(); // Force digest cycle to resolve promises;
});
});
http://plnkr.co/edit/LFp214GQcm97Kyv8xnLp