Lately a colleague and I have had some disagreements on the "right" way to implement Cucumber step definitions using Protractor and Chai as Promised. Our contention comes from a mutual lack of understanding of precisely what is going with promise resolution in the context of Cucumber.
We're testing against an AngularJS application, so resolving promises and asynchronous behavior is a necessary evil. The biggest problem we've had is forcing synchronous test behavior and getting Cucumber to wait on promises between step definitions. In some cases, we've observed situations such that Cucumber seems to plow straight through step definitions before Webdriver even executes them. Our solutions to this problem vary...
Consider the hypothetical scenario:
Scenario: When a user logs in, they should see search form
Given a user exists in the system
When the user logs into the application
Then the search form should be displayed
Most of the confusion originates in the Then step. In this example the definition should assert that all fields for the search form exist on the page, meaning multiple isPresent() checks.
From the documentation and examples I was able to find, I felt the assertion should look something like this:
this.Then(/the search form should be displayed/, function(next) {
expect(element(by.model('searchTerms')).isPresent()).to.eventually.be.true;
expect(element(by.model('optionsBox')).isPresent()).to.eventually.be.true;
expect(element(by.button('Run Search')).isPresent()).to.eventually.be.true.and.notify(next);
});
However, my colleague contends that in order to satisfy promise resolution, you need to chain your expects with then() like this:
this.Then(/the search form should be displayed/, function(next) {
element(by.model('searchTerms')).isPresent().then(function(result) {
expect(result).to.be.true;
}).then(function() {
element(by.model('optionsBox')).isPresent().then(function(result) {
expect(result).to.be.true;
}).then(function() {
element(by.button('Run Search')).isPresent().then(function(result) {
expect(result).to.be.true;
next;
});
});
});
});
The latter feels really wrong to me, but I don't really know if the former is correct either. The way I understand eventually() is that it works similarly to then(), in that it waits for the promise to resolve before moving on. I would expect the former example to wait on each expect() call in order, and then call next() through notify() in the final expect() to signal to cucumber to move on to the next step.
To add even more confusion, I've observed other colleagues write their expects like this:
expect(some_element).to.eventually.be.true.then(function() {
expect(some_other_element).to.eventually.be.true.then(function() {
expect(third_element).to.eventually.be.true.then(function() {
next();
});
});
});
So the questions I think I'm alluding to are:
Is any of the above even kinda right?
What does eventually() really do? Does it force synchronous behavior like then()?
What does and.notify(next) really do? Is it different from calling next() inside of a then()?
Is there a best practices guide out there that we haven't found yet that gives more clarity on any of this?
Many thanks in advance.
Your feeling was correct, your colleague was wrong (though it was a reasonable mistake!). Protractor automatically waits for one WebDriver command to resolve before running a second. So in your second code block, element(by.button('Run Search')).isPresent() will not resolve until both element(by.model('optionsBox')).isPresent() and element(by.model('searchTerms')).isPresent() are done.
eventually resolves promises. An explanation is here: https://stackoverflow.com/a/30790425/1432449
I do not believe it's different from putting next() inside of then()
I do not believe there is a best practices guide. Cucumber is not a core focus of the Protractor team, and support for it is largely provided by the community on github. If you or someone you know would like to write a best practices guide, we (the protractor team) would welcome a PR!
What works for me is this - The function below searches for something that will always equal true - in the case the existence of a html tag. I call this function at the end of each test, passing in the callback
function callbackWhenDone(callback) {
browser.wait(EC.presenceOf(element(by.css('html'))))
.then(function () {callback();})
}
This is the usage in a simple test:
this.Given(/^I go on "([^"]*)"$/, function (arg1, callback) {
browser.get('/' + arg1);
callbackWhenDone(callback);
});
A bit of a hack I know but it gets the job done and looks pretty clean when used everywhere
Related
I know this problem has been dealt with so many times, but none of them seem to solve the issue of reliably detecting dangling promises (even those that resolve properly).
So I want to be able to figure out a way (whether at runtime or better at static time) to root out "dangling promises", especially of this class:
async function a() {
... do some async operation ...
}
async function b() {
a(); // forgot to call await on it
}
where I have accidentally forget to await on one function and some task that executes asynchronously doesn't get awaited on. Often times these types of bugs don't throw exceptions so I can't just use "unhandledRejection" and call it a day.
At this point after lots of desperate attempts I just need a way to detect this kinda faulty pattern at either (optimally) static/compile/lint time or at runtime. Runtime for me I think should work assuming I have good test coverage.
tl;dr basically, I'm searching for some code that would do something like the following:
Error: Potential dangling promise detected!
...
at b (/dangling.js:5:3)
For each dangling promise
My thought process
I first tried finding some static analysis library that would help to detect these things (theoretically it should be possible but I had no luck with finding such a thing). A while back I remember finding something in the depths of stackoverflow that talked about using some typescript checker to check for dangling promises, though now I can't find it anymore :( . Though at the time changing the entire codebase to typescript was a no-go. (Later I learned you can actually use tsc to typecheck javascript (given that you do some type annotation in comments))
Currently I was previously using node version 11, so I thought about using the node.js's async_hooks API and try to listen on events (apparently simply monkey patching the Promise constructor wouldn't work because node.js bypasses the Promise constructor when creating a Promise object to return from an async function). Using node v11, after a bit of code hackery here it seemed to work (though it's not very efficient cause it throws away a lot of promise optimization in the v8 engine, but it does the job). There was a small hiccup in this entire operation in that I had to still monkey patch the then/catch/finally functions of the Promise API to check if we are currently calling that function (somehow that worked with detecting some dangling promises).
Now enter node v12 (apparently I need this for certain other things that are breaking), and now that hackery (unsurprisingly) completely breaks. After scrutinizing the version diffs, seems like they optimized the await/async implementation. After narrowing down the reason, it seems like await no longer calls the then function of the promise and just directly does some native shenanigans (idk what really).
Now I'm actually kinda desperate for some solution (and maybe if Typescript has some way of type-checking these dangling promises, what was the option to enable this check? I know (tested) that tsc doesn't do this by default).
You are most likely looking for the no-floating-promises rule of typescript-eslint
I am new to Jasmine testing and looking for a "best practice" for unit testing a stateful AngularJS service. Most tutorials I could find focus on test cases that run atomic calls to stateless services.
This matches nicely the Jasmine syntax
it("should do something", function(){ expect(target.doSomething()).toBe... })
However, I found no obvious way to extend this pattern to a test case that involves multiple calls to one or several of the service's functions.
Let's imagine a service like this one:
angular.module("someModule").factory("someService", function(){
return {
enqueue: function(item){
// Add item to some queue
}
}
});
For such a service, it makes sense to test that sequential calls to enqueue() process the items in the correct order. This involves writing a test case that calls enqueue() multiple times and checks the end result (which obviously cannot be achieved with a service as simple as the one above, but this is not the point...)
What doesn't work:
describe("Some service", function(){
// Initialization omitted for simplicity
it("should accept the first call", function() {
expect(someService.enqueue(one)).toBe... // whatever is correct
});
it("should accept the second call", function() {
expect(someService.enqueue(two)).toBe... // whatever is correct
});
it("should process items in the correct order", function() {
// whatever can be used to test this
});
});
The code above (which actually defines not one but three test cases) fails randomly as the three test cases are executed... just as randomly.
A poster in this thread suggested that splitting the code into several describe blocks would have these to be executed in the given order, but again this seems to be different from one Jasmine version to another (according to other posters in same thread). Moreover, executing suites and tests in a random order seems to be the intended way; even if it was possible, by setup, to override this behaviour, that would probably NOT be the right way to go.
Thus it seems that the only correct way to test a multi-call scenario is to make it ONE test case, like this:
describe(("Some service", function(){
// Initialization omitted for simplicity
it("should work in my complex scenario", function(){
expect(someService.enqueue(one)).toBe... // whatever is correct
expect(someService.enqueue(two)).toBe... // whatever is correct
expect(/* whatever is necessary to ensure the order is correct */);
});
});
While technical-wise this seems the logical way to go (after all, a complex scenario is one test case not three), the Jasmine "description + code" pattern seems disturbed in this implementation as:
There is no way to associate a message to each "substep" that can fail within the test case;
The description for the single "it" is inevitably bulky, like in example above, to really say something useful about a complex scenario.
This makes me wonder whether this is the only correct solution (or ist it?) to this kind of testing needs. Again I am especially interested in "doing it the right way" rather than using some kind of hack that would make it work... where it should not.
Sorry no code for this... Not sure it needs, I think you just need to adjust your expectations of your tests.
For the general rule of testing you don't really care how external dependencies handle the service, you can't control that. You want to test what you think the expected results are going to be for your service.
For your example you'll just want to invoke the dependencies for your service and call the function and test what the expected results are from calling the enqueue function. If it returns a promise, check success and error. It calls an API check that and so on.
If you want to see how the external dependencies use your service, you'll test it on those dependencies tests.
For example you have a controller that invokes enqueue. In the test you'll have to inject your provider (the service). And handle the expectations.
I know there is similar questions on here about this, but I cannot make sense of them for the life of me.
Here's an example, where I need to click a button and check the url.
My initial thought is I would write it as
element(by.id('button')).click();
expect(browser.getCurrentUrl()).toContain('asyncisconfusing');
I know the expect handles its promise but what about the .click? Shouldn't I have to write it like this?
element(by.id('button')).click().then(() => {
expect(browser.getCurrentUrl()).toContain('asyncisconfusing')
})
Or is protractor/webdriver auto-magically doing this?
In theory, since Protractor maintains a queue of promises via Control Flow and works in sync with an AngularJS application under test, you should not resolve promises explicitly unless you need a real value for further processing. In other words, this should be the prefferred form:
element(by.id('button')).click();
expect(browser.getCurrentUrl()).toContain('asyncisconfusing');
In practice though, explicitly resolving click() promises, or adding explicit waits via browser.wait() helps to deal with occasional and random timing issues.
http://seleniumhq.github.io/selenium/docs/api/javascript/module/selenium-webdriver/lib/promise.html
The first section talks about how the control flow is used to manage promises without having to chain together every single command.
I've got this problem that I couldn't find a solution for by googling.
I've got a library, that I'm using (and do not want to edit, unless it's really necessary) that allows the user to select an item, then calls my custom callback function to modify the item and then continues working with it.
I need to perform some asynchronous tasks on it, which may take some time. This creates a race condition as my async tasks have not yet finished when the callback function is finished and the library continues its work on the item.
library.onItemSelectionCallback = function (item) {
myService.modifyItem(item).then(
function (modifiedItemProperty) {
item.newProperty = modifiedItemProperty;
});
myService.anotherModifyItem(item).then(
function (modifiedItemProperty) {
item.existingProperty = modifiedItemProperty;
});
}
How do I wait for both of my async tasks to finish, before allowing this callback to finish?
Only thing I could think of is looping with while and sleep every hundred or so milliseconds until both of the promises have been resolved, but that doesn't seem to be a very good solution.
I understand that this makes async requests quite synchronous and might possibly be detrimental for UX, but do not really see another way out.
EDIT: I know that i'm risking with removing the generic nature of the question and thus making it too localized, I will say that I'm trying to use angular-file-upload module, specifically, trying to mount a custom imageService, that would resize the picture before it's upload. I'm mounting it on the onBeforeUploadItem callback. The idea is that creating the resized image may take a while and that is why I need to return a promise from my imageService, that needs to be resolved before upload.
If modifyItem and anotherModifyItem work independently (that is, one does not rely on the other), you can just pipe them both into $q.all, eg
library.onItemSelectionCallback = function(item) {
var promises = {
newProperty: myService.modifyItem(item),
existingProperty: myService.anotherModifyItem(item)
};
return $q.all(promises).then(function(values) {
return angular.extend(item, values);
});
}
This will return a promise that resolves with item.
For the first part of my question -- Yes, I guess the only way to really wait for those two promises to be resolved would be something with a while and sleep, making them synchronous, which would probably work and not even be that bad (except for the site pausing until the requests are fulfilled), but would make me feel very, very bad about myself as a person and how my actions affect this world.
It is not possible to correctly mix callbacks and promises without hacks afaik.
For the second part of my question -- as per comments of #georgeawg, figured that an AngularJS module that implements HTML5 API and callbacks instead of $http service and promises is not how a good AngularJS module should be implemented, and so I moved towards a different module ng-file-upload, which, even though one could argue is less stylish, does the job very well and in an Angular way (ng-file-upload provides a simple $upload service, that returns a promise. If you want to modify files before upload, suggested way is to simply $watch and catch the moment user drag-drops or selects a file.).
In writing a fairly large JavaScript module with lots of async operation all dealing with promises, it becomes quite difficult to debug and follow the flow/state of promises.
Are there any tools or frameworks for instrumenting tests or module code to give he a flow of all the Promises in my system? I'm using jQuery promises if it matters at all.
Off the top of my head, and stimulated by #MrLeap's idea to send messages to the console, how about a creating Deferreds via an "adapter" of your own design.
eg (embryonic and untested) :
var debugMode = true;
function DeferredAdapter(name) {
var dfrd = $.Deferred();
if(debugMode) {
dfrd.notify = function() {
console.log(name + ': notify');
if (arguments[0] && typeof arguments[0] == "string")
console.log(arguments[0]);
};
//similar for resolve
//similar for reject
}
return dfrd;
}
This way (when you've got it working), you can do extra stuff when debugging, with a simple mechanism to turn off the extra stuff in production code, without the need to go through all your code purging console.log() statements.
It would add quite a bit of development overhead, but if you're using node for the back end you can implement pretty robust asynchronous unit tests with vows.js. If you want to add even more work for yourself you could use vows to create mock versions of all your responses if you're using some other kind of server tech, and use it for testing and reporting. Using a node.js mock server also lets you do robust, enterprise^^ things like simulate certain requests failing to see how gracefully your front end responds to it.
I wish I had a better answer than this though; console.log and console.dir are your best friends. Cram your callbacks full of them so you can get an idea how long stuff takes to happen, and in what order they happen.