TestCafe 1.8.0,
Firefox 76.0 (any will do),
macOS 10.15.4.
My TestCafe tests (steps after useRole) are trying to execute even before useRole is completely done. Example:
import { Role } from 'testcafe';
const role = Role('http://example.com/login', async t => {
await t
.typeText('#login', 'username')
.typeText('#password', 'password')
.click('#sign-in'); // Redirects to http://example.com/session
});
fixture `My Fixture`;
test('My test', async t => {
await t
.navigateTo('http://example.com/')
.useRole(role);
.click()
.typeText('#search', 'query')
// Further tests.
.......
Role is being used -> user is redirected to: http://example.com/session at the end.
Once role execution is finished -> TestCafe goes back to My test AND it reloads http://example.com/ again.
This is a huge problem because between this one page reload for just a moment 'page is ready' and since TestCafe is working rapidly .click() is executed. Now page reloads so test execution stops. Once page loaded test brakes because it is trying to .typeText(...) without a .click().
Tried this solutions:
Wait until will work only first time. Second time when useRole used (from cache) code will execute even before this second page reload. .expect(getUrl()).eql('desiredurl', { timeout: 10000 })
As we all know hard wait .wait(3000) or slowing down tests .setTestSpeed(0.7) will work but it is not a good solution from code perspective. Tests still might fail from time to time and I need and want stability here.
With { preserveUrl: true } it would just reload http://example.com/session so it doesn't matter if this option is used. Reload is still happening.
Any ideas?
How can I let my test know to wait for exactly the same page reload without using any hardcoded waits, code sleep?
As far as I understand, the main issue is that the click action is executed on the wrong page for some reason. It does not wait until the page is completely reloaded. This behavior is unexpected. We would really appreciate it if you create a reproducible sample.
I agree that the use of wait or setTestSpeed is not a suitable solution.
I see that you tried to use assertions: .expect(getUrl()).eql('desiredurl', { timeout: 10000 }). I think this approach should work, but I cannot be sure because I couldn't reproduce the issue.
You can define your role with preserveUrl: true. Then, extract the useRole method as follows:
async function useRole (t, role) {
await t.useRole(role);
await t.expect(getUrl()).eql('http://example.com/session', {timeout: 10000 });
}
Now, you can use the new useRole(t, role) method, which will wait until the page is completely loaded.
Related
I'm writing E2E tests in Cypress (version 12.3.0). I have a page with a table in a multi-step creation process that requires some data from back-end application. In some cases (rarely, but it occurs) the request gets stuck and the loader never disappears. The solution is simple: go back to the previous step and return to the "stuck" table. The request is sent anew and most likely receives a response - the process and the tests can proceed further. If the loader is not present, then going back and forth should be skipped (most of the times).
I managed to work around that with the code below, but I'm wondering if it could be done with some built-in Cypress functions and without explicit waiting. Unfortunately, I didn't find anything in the Cypress docs and on StackOverflow. I thought that maybe I could use the then function to work on a "conditionally present" element, but it fails on get, that's why I've used find on the jQuery ancestor element.
waitForTableData() {
return cy.get('.data-table')
.should('exist')
.then(table => {
if (this.loaderNotPresent(table)) {
return;
}
cy.wait(200)
.then(() => {
if (this.loaderNotPresent(table)) {
return;
}
cy.get('button')
.contains('Back')
.click()
.get('button')
.contains('Next')
.click()
.then(() => this.waitForTableData());
});
});
}
loaderNotPresent(table: JQuery) {
return !table.find('.loader')?.length;
}
Your code looks to me to be the best you could do at present.
The cy.wait(200) is about the right size, maybe a bit smaller would be better - 50 - 100 ms. The recursive call is going to give you similar behaviour to Cypress retry (which also waits internally, in order not to hammer the test runner).
Another approach would be to cy.intercept() and mock the backend, presuming it's the backend that gets stuck.
Also worth trying a simple test retry, if the loading only fails on a small percentage of times.
Application:
My React application gathers information from the server to populate the page. Whilst it is doing this, there is a spinner to show the page is loading.
const [loading, setLoading] = useState(true);
useEffect(() => getCandidate(), []);
if (loading) {
return <LoadingInformation />;
} else if (candidateInformation) {
And in the getCandidate() function, it'll run setLoading(false) upon success in the try block.
Cypress Testing:
My assumption would be that cypress sees this loading spinner, but still runs the start of the tests. This means, for example, the code below would look at the loading spinner, and throw an error:
cy.get("[datacy='firstName']").find("input").invoke('val')
Whilst I can introduce cy.wait(5000), I feel this is bad practise.
How can I wait for the state to change from true to false before starting the code? Do I need to add a datacy attribute to the loading spinner (<LoadingInformation />), and then run a conditional? How could I do this logic globally? As it'll be on every test.
I guess there is a bigger question here, which is "How do I wait for asynchronous tasks in React to complete before running the next cypress task?"
You're thinking too deep about your app and cypress together. Just think about it at the UI level and what Cypress can see on the page.
If you're just waiting for the candidate information to display, you can use something like .should('be.visible') to validate it's there. Cypress will automatically wait for the condition to pass before continuing or until it times out.
// Assuming [datacy='firstName'] is a candidate information element
// Wait until this element is visible before continuing.
cy.get("[datacy='firstName']")
.should('be.visible')
.find("input")
.invoke('val')
As #jonrsharpe posted, https://docs.cypress.io/guides/core-concepts/retry-ability. Retry-ability is a core concept of cypress, it's what cypress uses to wait on things.
Protractor hangs completely when trying to get any element property after logging in (idk if it's related to logging in or related just to switching pages).
it("Should get location of main container", async function() {
await LoginPage.validLogin();
// Works and logs in the dashboard
await browser.sleep(3000);
// Get the main container by class name
const container = await element(by.css(".main-container"));
// Logs properly the element functions (as expected)
console.log(container);
console.log(await container.getLocation()); // Hangs here
});
In this case, I'm trying to get the location of the main container element on the page. The first console.log fires and shows properly, while the second hangs completely, so I get the script timeout. Increasing the timeout time doesn't help at all...
I found online that misusing $timeout in AngularJS instead of using $interval may lead to this strange behaviour, but I really can't skim through the entire (very big!) project's codebase to change everything hoping that it just works, not to talk about the external libraries using $timeout.
I have SELENIUM_PROMISE_MANAGER = false; in my Protractor config so I disabled the built-in Control Flow in order to manually manage the promises using async/await, but even if I use the built-in Control Flow without using async/await I get the very same behaviour and error.
I'm using Jasmine as testing framework.
Maybe I'm missing something? Any help would be much appreciated, thanks!
This is caused by the fact that angular is not stable. Have a look at the link below. I found my answer there. When the page you are trying to test is open go to the browser dev tools and type in the console getAllAngularTestabilities(). There are a few properties here that indicate whether angular is ready to be tested. hasPendingMicrotasts needs to be false. hasPendingMacroTasks needs to be false. isStable needs to be true. I put a screenshot below. In my screenshot hasPendingMacrotasks is true and it must be false. So the page I looked at was not ready to be tested.
Failed: script timeout: result was not received in 11 seconds From: Task: Protractor.waitForAngular() - Locator: By(css selector, #my-btn)
Try something like this:
it("Should get location of main container", async function() {
await LoginPage.validLogin();
const container = await element(by.css(".main-container"));
await browser.wait(protractor.ExpectedConditions.presenceOf(container), 5000, 'Element taking too long to appear in the DOM');
await console.log(await container.getLocation());
});
I don't think that getLocation() exists in the Javascript bindings for selenium. I couldn't find it in the source code anyway. So that promise will never return which is why it hangs. But I the you can achieve basically the same thing with getRect():
it("Should get location of main container", async function() {
await LoginPage.validLogin();
const container = await element(by.css(".main-container"));
await browser.wait(protractor.ExpectedConditions.presenceOf(container), 5000, 'Element taking too long to appear in the DOM');
await console.log(await container.getRect());
});
tl;dr: When I run my test case, steps executed seem to work, but the test bails out early on a failure to find an element that hasn't even loaded yet. It seems like the waits I have around locating certain elements are loaded/launched as soon as the test is launched, not when the lines should actually be executed in the test case. I think this is happening because the page is barely (correctly) loaded before the "search" for the element to verify the page has loaded bails out. How do I wrangle the event loop?
This is probably a promise question, which is fine, but I don't understand what's going on. How do I implement my below code to work as expected? I'm working on creating automated E2E test cases using Jasmine2 and Protractor 5.3.0 in an Angular2 web app.
describe('hardware sets', () => {
it('TC3780:My_Test', async function() {
const testLogger = new CustomLogger('TC3780');
const PROJECT_ID = '65';
// Test Setup
browser.waitForAngularEnabled(false); // due to nature of angular project, the app never leaves zones, leaving a macrotask constantly running, thus protractor's niceness with angular is not working on our web app
// Navigate via URL to planviewer page for PROJECT_ID
await planListingPage.navigateTo(PROJECT_ID); // go to listing page for particular project
await planListingPage.clickIntoFirstRowPlans(); // go to first plan on listing page
await planViewerPage.clickOnSetItem('100'); // click on item id 100 in the plan
});
});
planViewerPage.po.ts function:
clickOnSetItem(id: string) {
element(by.id(id)).click();
browser.wait(until.visibilityOf(element(by.css('app-side-bar .card .info-content'))), 30000); // verify element I want to verify is present and visible
return expect(element(by.css('app-side-bar .card .info-content')).getText).toEqual(id); //Verify values match, This line specifically is failing.
}
This is the test case so far. I need more verification, but it is mostly done. I switched to using async function and awaits instead of the typical (done) and '.then(()=>{' statement chaining because I prefer not having to do a bunch of nesting to get things to execute in the right order. I come from a java background, so this insanity of having to force things to run in the order you write them is a bit much for me sometimes. I've been pointed to information like Mozilla's on event loop, but this line just confuses me more:
whenever a function runs, it cannot be pre-empted and will run entirely before any other code
runs (and can modify data the function manipulates).
Thus, why does it seem like test case is pre-evaluated and the timer's set off before any of the pages have been clicked on/loaded? I've implemented the solution here: tell Protractor to wait for the page before executing expect pretty much verbatim and it still doesn't wait.
Bonus question: Is there a way to output the event-loop's expected event execution and timestamps? Maybe then I could understand what it's doing.
The behavior
The code in your function is running asynchronously
clickOnSetItem(id: string) {
element(by.id(id)).click().then(function(){
return browser.wait(until.visibilityOf(element(by.css('app-side-bar .card .info-content'))), 30000);
}).then(function(){
expect(element(by.css('app-side-bar .card .info-content')).getText).toEqual(id);
}).catch(function(err){
console.log('Error: ' + err);
})
}
Note: Additional information appended to end of original question as Edit #1, detailing how request-promise in the back-end is causing the UI freeze. Keep in mind that a pure CSS animation is hanging temporarily, and you can probably just skip to the edit (or read all for completeness)
The setup
I'm working on a desktop webapp, using Electron.
At one point, the user is required to enter and submit some data. When they click "submit", I use JS to show this css loading animation (bottom-right loader), and send data asynchronously to the back-end...
- HTML -
<button id="submitBtn" type="submit" disabled="true">Go!</button>
<div class="submit-loader">
<div class="loader _hide"></div>
</div>
- JS -
form.addEventListener('submit', function(e) {
e.preventDefault();
loader.classList.remove('_hide');
setTimeout(function() {
ipcRenderer.send('credentials:submit', credentials);
}, 0)
});
where ._hide is simply
._hide {
visibility: hidden;
}
and where ipcRenderer.send() is an async method, without option to set otherwise.
The problem
Normally, the 0ms delay is sufficient to allow the DOM to be changed before the blocking event takes place. But not here. Whether using the setTimeout() or not, there is still a delay.
So, add a tiny delay...
loader.classList.remove('_hide');
setTimeout(function() {
ipcRenderer.send('credentials:submit', credentials);
}, 100);
Great! The loader displays immediately upon submitting! But... after 100ms, the animation stops dead in its tracks, for about 500ms or so, and then gets back to chooching.
This working -> not working -> working pattern happens regardless of the delay length. As soon as the ipcRenderer starts doing stuff, everything is halted.
So... Why!?
This is the first time I've seen this kind of behavior. I'm pretty well-versed in HTML/CSS/JS, but am admittedly new to NodeJS and Electron. Why is my pure CSS animation being halted by the ipcRenderer, and what can I do to remedy this?
Edit #1 - Additional Info
In the back-end (NodeJS), I am using request-promise to make a call to an external API. This happens when the back-end receives the ipcRenderer message.
var rp = require('request-promise');
ipcMain.on('credentials:submit', function(e, credentials) {
var options = {
headers : {
... api-key...
},
json: true,
url : url,
method : 'GET'
};
return rp(options).then(function(data) {
... send response to callback...
}).catch(function(err) {
... send error to callback...
});
}
The buggy freezing behavior only happens on the first API call. Successive API calls (i.e. refreshing the desktop app without restarting the NodeJS backend), do not cause the hang-up. Even if I call a different API method, there are no issues.
For now, I've implemented the following hacky workaround:
First, initialize the first BrowserWindow with show:false...
window = new BrowserWindow({
show: false
});
When the window is ready, send a ping to the external API, and only display the window after a successful response...
window.on('ready-to-show', function() {
apiWrapper.ping(function(response) {
if(response.error) {
app.quit();
}else {
window.show(true);
}
});
});
This extra step means that there is about 500ms delay before the window appears, but then all successive API calls (whether .ping() or otherwise) no longer block the UI. We're getting to the verge of callback hell, but this isn't too bad.
So... this is a request-promise issue (which is asynchronous, as far as I can tell from the docs). Not sure why this behavior is only showing-up on the first call, so please feel free to let me know if you know! Otherwise, the little hacky bit will have to do for now.
(Note: I'm the only person who will ever use this desktop app, so I'm not too worried about displaying a "ping failed" message. For a commercial release, I would alert the user to a failed API call.)
Worth to check how does request-promise internally setups up module loading. reading it, it seems like there is kind of lazy loading (https://github.com/request/request-promise/blob/master/lib/rp.js#L10-L12) when request is being called. Quick try out
const convertHrtime = require('convert-hrtime');
const a = require('request-promise');
const start = process.hrtime();
a({uri: 'https://requestb.in/17on4me1'});
const end = process.hrtime(start);
console.log(convertHrtime(end));
const start2 = process.hrtime();
a({uri: 'https://requestb.in/17on4me1'});
const end2 = process.hrtime(start2);
console.log(convertHrtime(end2));
returns value like below:
{ seconds: 0.00421092,
milliseconds: 4.21092,
nanoseconds: 4210920 }
{ seconds: 0.000511664,
milliseconds: 0.511664,
nanoseconds: 511664 }
first call is obviously taking longer than subsequent. (number of course may vary, I ran this on bare node.js on relatively fast cpu) If module loading is major cost for first call, then it'll block main process until module is loaded (cause node.js require resolve is synchronous)
I'm not able to say this is concrete reason, but worth to check. As suggested in comment, try other lib or bare internal module (like Electron's net) to rule out.