I'm writing integration tests for my meteor project. I want to test the webhook POST handler in my app. This is how it looks like:
post() {
Meteor.defer(() => {
// some logic here, e.g insert / update database
})
return {
statusCode: 200,
}
}
Note: Meteor.defer is a must because I want to return code 200 (OK) as soon as possible.
To test this webhook, I create a fake POST request to this webhook, then check if the database is updated accordingly. The thing is that in the test I don't know when the code inside Meteor.defer has finished, therefore my assertions are failed because the database hasn't been updated yet.
Any suggestions ?
I came up with a workaround: using Mocha's test timeouts to wait for a specific amount of time before doing assertions. It's not the best solution but it works at the moment.
Related
I'm writing E2E tests in Cypress (version 12.3.0). I have a page with a table in a multi-step creation process that requires some data from back-end application. In some cases (rarely, but it occurs) the request gets stuck and the loader never disappears. The solution is simple: go back to the previous step and return to the "stuck" table. The request is sent anew and most likely receives a response - the process and the tests can proceed further. If the loader is not present, then going back and forth should be skipped (most of the times).
I managed to work around that with the code below, but I'm wondering if it could be done with some built-in Cypress functions and without explicit waiting. Unfortunately, I didn't find anything in the Cypress docs and on StackOverflow. I thought that maybe I could use the then function to work on a "conditionally present" element, but it fails on get, that's why I've used find on the jQuery ancestor element.
waitForTableData() {
return cy.get('.data-table')
.should('exist')
.then(table => {
if (this.loaderNotPresent(table)) {
return;
}
cy.wait(200)
.then(() => {
if (this.loaderNotPresent(table)) {
return;
}
cy.get('button')
.contains('Back')
.click()
.get('button')
.contains('Next')
.click()
.then(() => this.waitForTableData());
});
});
}
loaderNotPresent(table: JQuery) {
return !table.find('.loader')?.length;
}
Your code looks to me to be the best you could do at present.
The cy.wait(200) is about the right size, maybe a bit smaller would be better - 50 - 100 ms. The recursive call is going to give you similar behaviour to Cypress retry (which also waits internally, in order not to hammer the test runner).
Another approach would be to cy.intercept() and mock the backend, presuming it's the backend that gets stuck.
Also worth trying a simple test retry, if the loading only fails on a small percentage of times.
tl;dr: When I run my test case, steps executed seem to work, but the test bails out early on a failure to find an element that hasn't even loaded yet. It seems like the waits I have around locating certain elements are loaded/launched as soon as the test is launched, not when the lines should actually be executed in the test case. I think this is happening because the page is barely (correctly) loaded before the "search" for the element to verify the page has loaded bails out. How do I wrangle the event loop?
This is probably a promise question, which is fine, but I don't understand what's going on. How do I implement my below code to work as expected? I'm working on creating automated E2E test cases using Jasmine2 and Protractor 5.3.0 in an Angular2 web app.
describe('hardware sets', () => {
it('TC3780:My_Test', async function() {
const testLogger = new CustomLogger('TC3780');
const PROJECT_ID = '65';
// Test Setup
browser.waitForAngularEnabled(false); // due to nature of angular project, the app never leaves zones, leaving a macrotask constantly running, thus protractor's niceness with angular is not working on our web app
// Navigate via URL to planviewer page for PROJECT_ID
await planListingPage.navigateTo(PROJECT_ID); // go to listing page for particular project
await planListingPage.clickIntoFirstRowPlans(); // go to first plan on listing page
await planViewerPage.clickOnSetItem('100'); // click on item id 100 in the plan
});
});
planViewerPage.po.ts function:
clickOnSetItem(id: string) {
element(by.id(id)).click();
browser.wait(until.visibilityOf(element(by.css('app-side-bar .card .info-content'))), 30000); // verify element I want to verify is present and visible
return expect(element(by.css('app-side-bar .card .info-content')).getText).toEqual(id); //Verify values match, This line specifically is failing.
}
This is the test case so far. I need more verification, but it is mostly done. I switched to using async function and awaits instead of the typical (done) and '.then(()=>{' statement chaining because I prefer not having to do a bunch of nesting to get things to execute in the right order. I come from a java background, so this insanity of having to force things to run in the order you write them is a bit much for me sometimes. I've been pointed to information like Mozilla's on event loop, but this line just confuses me more:
whenever a function runs, it cannot be pre-empted and will run entirely before any other code
runs (and can modify data the function manipulates).
Thus, why does it seem like test case is pre-evaluated and the timer's set off before any of the pages have been clicked on/loaded? I've implemented the solution here: tell Protractor to wait for the page before executing expect pretty much verbatim and it still doesn't wait.
Bonus question: Is there a way to output the event-loop's expected event execution and timestamps? Maybe then I could understand what it's doing.
The behavior
The code in your function is running asynchronously
clickOnSetItem(id: string) {
element(by.id(id)).click().then(function(){
return browser.wait(until.visibilityOf(element(by.css('app-side-bar .card .info-content'))), 30000);
}).then(function(){
expect(element(by.css('app-side-bar .card .info-content')).getText).toEqual(id);
}).catch(function(err){
console.log('Error: ' + err);
})
}
I am working on an socket.io IRC and I don't want users to have a long username. I wrote the following (mocha) test to verify that the server doesn't send out a response to every connected socket when a longer username is provided:
it("should not accept usernames longer than 15 chars", function (done) {
var username = "a".repeat(server.getMaxUsernameLength() + 1);
client1.emit("username change", username);
client2.on("chat message", function (data) {
throw Error("Fail, server did send a response.");
});
setTimeout(function () {
done();
}, 50);
});
This currently does work, but it's far from optimal. What if my CI platform is slower or the server does respond after more than 50 ms? What's the best way to fail a test when a response is given, or should I structure my tests differently?
Thanks!
P.s. This question is different from Testing asynchronous function with mocha, because while the problem does have to do with asynchronous testing, I am aware of the done() method (and I'm using it obviously).
What you're trying to do is verify that the callback to client2.on("chat message"... is never called. Testing for negative cases can be tough, and your case seems to be exacerbated by the fact that you're trying to do a complete end-to-end (client-to-server-to-client) integration test. Personally, I would try to test this in a unit case suite and avoid introducing the complexity of asynchronicity to the test.
However, if it must be done, here's a tip from Eradicating Non-Determinism in Tests:
This is the trickiest case since you can test for your expected response, but there's nothing to do to detect a failure other than timing-out. If the provider is something you're building you can handle this by ensuring the provider implements some way of indicating that it's done - essentially some form of callback. Even if only the testing code uses it, it's worth it - although often you'll find this kind of functionality is valuable for other purposes too.
Your server should send some sort of notice to client1 that it's going to ignore the name change, even if you aren't testing, but since you are you could use such a notification to verify that it really didn't send a notification to the other client. So something like:
it("should not accept usernames longer than 15 chars", function (done) {
var chatSpy = sinon.spy();
client2.on("chat message", chatSpy);
client1.on('error', function(err) {
assertEquals(err.msg, 'Username too long');
assert(chatSpy.neverCalledWith(...));
done();
});
var username = "a".repeat(server.getMaxUsernameLength() + 1);
client1.emit("username change", username);
});
would be suitable.
Also, if for whatever reason, server.getMaxUsernameLength() ever starts returning something other than 15, the best case scenario is that your test description becomes wrong. It can become worse if getMaxUsernameLength and the server code for handling the name change event don't get their values from the same place. A test probably should not rely on the system under test to provide test values.
I can share frustration with a lot of professionals about default protractor behavior on test failure - it just keeps running the tests and you have to wait until it finishes to fix the error.
I read the related posts and came across jasmine-bail-fast solution. Link to related post is provided here: Exit Protractor e2e test on fail?
However, this solution puts me on the other side of the pickle. I do NOT want to terminate the testing suite when it failed to close the confirmation message or ran into similar minor issues.
I would like to have the ability to control when to exit the script via exitonfailure() function or something similar. For instance, if had this block of code:
> browser.wait(function()
> return browser.isElementPresent(acceptBudgetButton);
> }, 30000, 'Error - Unable to save budget changes because Accept Budget button is not visible. Terminating test run.');
and put exitonfailure() after this block, I want my test run to exit immediately. However, I want to test keep on running if exitonfailure() is not there.
Is there a feasible way to achieve this goal and take control of my own destiny?
Thank you for your help!
You can handle the browser.wait() success and failure cases appropriately:
var EC = protractor.ExpectedConditions;
browser.wait(EC.presenceOf(acceptBudgetButton), 30000).then(
function () {
// success
},
function () {
// failure
console.log('Error - Unable to save budget changes because Accept Budget button is not visible. Terminating test run.');
jasmine.getEnv().bailFast();
}
});
Also, look into using fail() function which fails a test.
I have an app that loads several resources when it's first run, which are stored in localStorage. I have a function that checks whether all the local storage variables are set, so that part is working okay.
My method of working is like this:
Display a loading message.
Initialize the AJAX requests.
Start a timer interval to check if everything has loaded.
When the data has loaded, initialize the application etc.
If the data did not load, display an error message.
The problem is with #5 - how to detect if there was an error? For example if there was a connection problem or the sever sent back invalid data for whatever reason. Here is my current code - downloadData just performs a basic AJAX request:
// check local storage and download if any missing
if ( !checkLocalStorage() )
{
$('#content').before( '<div class="notice" id="downloading">Downloading data, please wait...</div>' );
for ( var i in db_tables )
{
if ( localStorage[db_tables[i]] == null )
downloadData( db_tables[i] );
}
}
// check progress
var timer = setInterval( function() {
if ( checkLocalStorage() )
{
// everything is downloaded
$('#downloading').hide();
clearInterval(timer);
initApp();
}
}, 500 );
Could you turn it around a bit? Something like this (with sensible variable names and a "real" API) would simplify things:
Display a loading message.
Instantiate an application initializer, ai.
Crank up the AJAX requests:
Success handlers call ai.finished(task).
Error handlers call ai.error(task).
Register with the initializer, ai.register(task), in case a "you're taking too long" check is desired.
Once all the AJAX requests have called ai.finished, initialize the application etc.
If any of the AJAX tasks called ai.error, then display an error message and start cleaning things up.
This way you wouldn't need to setInterval() and the individual AJAX tasks will tell you when they have finished or fallen over. You might still want the interval to deal with tasks that are taking too long but most of the logic would be notification based rather than polling based.
Seeing your actual ajax calls in downloadData would help, but I suggest you look over the jquery AJAX API again. Ajax calls have callbacks not just for overall completion but specifically for success and failure including errors. Try to do something like retrying if there is an error and if it continues to fail you can warn the user. You can also use these callbacks to notify your application when the loading is done instead of using an interval timer.