I already have working project with few passing Cypress tests.
Now I'm trying to add contract tests using Cypress + Pact
In developer console I can see that app is calling /api/v1/document-service, but I get:
Pact verification failed - expected interactions did not match actual.
Part of logs:
W, [2021-07-20T12:49:37.157389 #34805] WARN -- : Verifying - actual interactions do not match expected interactions.
Missing requests:
POST /api/v1/document-service
W, [2021-07-20T12:49:37.157489 #34805] WARN -- : Missing requests:
POST /api/v1/document-service
I'm using:
cypress: 7.5.0
#pact-foundation/pact: 9.16.0
Steps I've done:
Added cypress plugin (https://github.com/pactflow/example-consumer-cypress/blob/master/cypress/plugins/cypress-pact.js)
Added commands (https://github.com/pactflow/example-consumer-cypress/blob/master/cypress/support/commands.js)
Added config to cypress.json (https://github.com/pactflow/example-consumer-cypress/blob/master/cypress.json) - not sure what to put to baseUrl, if I don't want interactions with real server.
Added test:
let server;
describe('Simple', () => {
before(() => {
cy.mockServer({
consumer: 'example-cypress-consumer',
provider: 'pactflow-example-provider',
}).then(opts => {
cy.log(opts)
server = opts
})
});
beforeEach(() => {
cy.fakeLogin()
cy.addMockRoute({
server,
as: 'products',
state: 'products exist',
uponReceiving: 'a request to all products',
withRequest: {
method: 'POST',
path: '/api/v1/document-service',
},
willRespondWith: {
status: 200,
body: {
data: {
collections: [
{
id: '954',
name: 'paystubs',
},
{
id: '1607',
name: 'mystubs',
},
],
},
},
},
});
});
it('is ok?', () => {
cy.visit('/new/experiments/FirstProject/collections');
});
})
Tried both using deprecated cy.server()/cy.route() and new cy.intercept(), but still verification failures.
At Pactflow, we've spent about 6 months using Cypress and Pact together, using the Pact mock service to generate pacts from our Cypress tests as suggested in https://pactflow.io/blog/cypress-pact-front-end-testing-with-confidence/
We have this week decided to change our approach, for the following reasons.
Our Cypress tests are much slower than our unit tests (15+ minutes), so generating the pact, and then getting it verified takes a lot longer when we generate the pact from our Cypress tests vs our unit tests.
Our Cypress tests can fail for reasons that are not to do with Pact (eg. layout changes, issues with the memory consumption on the build nodes, UI library upgrades etc) and this stops the pact from being generated.
The Cypress intercept() and mock service don't play very nicely together. Any request that does not have an explicit intercept() for Cypress ends up at the mock service, and every time a new endpoint is used by the page, the request hits the mock service, which then fails the overall test, even if that endpoint was not required for that specific test.
When a Cypress test fails, there is very good debugging provided by the Cypress library. When it fails for reasons to do with Pact, it's currently hard to identify the cause, because that information is not displayed in the browser (this could potentially be improved, however, it won't mitigate the already listed issues).
Our new approach is to generate our Pacts from our unit tests (instead of our Cypress tests) and to share the fixtures between the unit and Cypress tests, using a function that strips of the Pact matchers. eg.
const SOME_INTERACTION = {
state: 'some state',
uponReceiving: "some request",
withRequest: {
method: "GET",
path: "/something"
},
willRespondWith: {
status: 200,
body: {
something: like("hello")
}
}
}
Unit test
describe('Something', () => {
let someProviderMockService
beforeAll(() => {
someProviderMockService = new Pact({
consumer: 'some-consumer',
provider: 'some-provider'
})
return someProviderMockService.setup()
})
afterAll(() => someProviderMockService.finalize())
afterEach(() => someProviderMockService.verify())
describe('loadSomething', () => {
beforeEach(() => {
return someProviderMockService.addInteraction(SOME_INTERACTION)
})
it('returns something', async () => {
const something = await loadSomething()
//expectations here
})
})
})
Function that turns the Pact interaction format into Cypress route format, and strips out the Pact matchers.
const Matchers = require('#pact-foundation/pact-web').Matchers
// TODO map the request body and query parameters too
const toCypressInteraction = (interaction) => {
return {
method: interaction.withRequest.method,
url: Matchers.extractPayload(interaction.withRequest.path),
status: interaction.willRespondWith.status,
headers: Matchers.extractPayload(interaction.willRespondWith.headers),
response: Matchers.extractPayload(interaction.willRespondWith.body)
}
}
In the Cypress test
cy.route(toCypressInteraction(SOME_INTERACTION))
This approach has the following benefits:
The pact is generated quickly.
The pact is generated reliably.
The Cypress tests are more reliable and easier to debug.
The requests used in the Cypress tests are verified to be correct.
There is less chance of "interaction bloat", where interactions are added merely to test UI features, rather than because they provide valuable coverage.
I hope this information is helpful for you. We now recommend this approach over direct use of the mock service in Cypress tests.
Related
I have seen several posts about running a single test with different parameters. It is also documented here.
However, I couldn't find any examples of how to run the entire test suite, i.e. tests across multiple files in the cypress/integration folder multiple times with different data.
My scenario is that I want to stub different responses from an API I'm calling and run all test cases against the different responses. So for the 1st run, I would put in support/index.js:
beforeEach(() => {
cy.intercept("GET", "example/API", { fixture: "fixture1.json" });
});
and for the 2nd run I would put:
beforeEach(() => {
cy.intercept("GET", "example/API", { fixture: "fixture2.json" });
});
and so on. All my test cases are identical for different responses and I expect them to have the same result regardless of the data returned by the API.
Running a suite with different parameters
cypress run --env fixture=fixture1
// or
cypress run --env fixture=fixture2
Ref Environment Variables
In support/index.js
beforeEach(() => {
const fixtureName = `${Cypress.env('fixture')}.json`
cy.intercept("GET", "example/API", { fixture: fixtureName });
})
I expect them to have the same result regardless of the data returned by the API - I'm not sure what that means exactly but to me it suggests you don't have to worry about changing the fixture.
If you want a single call from the command line to run the whole suite, say 3 times with a different fixture each time, consider using the Cypress Module API
In a script file, e.g /scripts/run-fixtures.js
const fixtures = ['fixture1.json','fixture2.json','fixture3.json']
const cypress = require('cypress')
fixtures.forEach((fixtureName) => {
cypress.run({
reporter: 'junit',
browser: 'chrome',
config: {
baseUrl: 'http://localhost:8080',
video: true,
},
env: {
fixture: fixtureName,
},
})
})
Run it with node /scripts/run-fixtures.
support/index.js is the same as above.
I'm running several tests where I'm mocking http calls using sinon fake server:
import sinon from 'sinon';
...
const fakeServer = sinon.fakeServer.create();
fakeServer.respondWith('POST', '/myapp/myendpoint/pathparam', [201, { 'Content-Type': 'application/json' }, myPayload]);
...
However the fake server is returning a not found error: [404, { }, (empty string)].
I cannot figure out what's going wrong.
Is there any way to activate some kind of logs that tells me what is going on?
After going over sinon's documentation, I cannot find anything about logs or debug flags.
After inspecting sinon I found that it does not contain the fake server but it is exposing nise/fake-server/.
Then I went over nise and I ended up finding that method to trace the behaviour:
logger: function () {
// no-op; override via configure()
},
So to enable logs in the fake-server the only thing you need to do is to specify a behaviour when you create the fake server:
fakeServer = sinon.fakeServer.create({logger: str => console.log('Fake server', str)});
I'm wanting to run e2e tests written in javascript with mocha on an Appium server instance running a local android emulator. The app on test is an apk originally written in react-native.
On Windows I have the server up and running with an Android Studio emulator through using the Appium desktop app. The server all looks good and has the apk of the native app I want to test working fine. I also have a basic describe/assert test written in mocha that I want to apply to the app.
My question is what do I need to include (presumably in the test file) to make the tests actually test the emulator application? I'm finding the documentation pretty confusing and the sample code seems pretty specific to a different use case.
Many thanks for your help!
There are at least 2 good js client libraries to use for Appium based project: webdriverio and wd. Personally, I'm using the second one so I can advice you how write tests with it and mocha:
my test file looks like this:
'use strict'
require(path.resolve('hooks', 'hooks'))
describe('Suite name', function () {
before('Start new auction', async function () {
//do before all the tests in this file, e.g. generate test data
})
after('Cancel auction', async function () {
//do after all the tests in this file, e.g. remove test data
})
it('test1', async () => {
// test steps and checks are here
})
it('test2', async () => {
// test steps and checks are here
})
it('test3', async () => {
// test steps and checks are here
})
})
where hooks.js contains global before/after for all the tests:
const hooks = {}
before(async () => {
// before all the tests, e.g. start Appium session
})
after(async () => {
// after all the tests, e.g. close session
})
beforeEach(async () => {
// before each test, e.g. restart app
})
afterEach(async function () {
// e.g. take screenshot if test failed
})
module.exports = hooks
I'm not saying its the best practice of designing tests, but its one of multiple ways.
Cool so I managed to get it working to a degree. I was checking through the Appium console logs as I was trying to run stuff and noticed that the session id was missing from my requests. All that was needed was to attach the driver using the session id. My code looks a bit like this:
"use strict";
var wd = require("wd")
var assert = require("assert")
var serverConfig = {
host: "localhost",
port: 4723,
}
var driver = wd.remote(serverConfig)
driver.attach("0864a299-dd7a-4b2d-b3a0-e66226817761", function() {
it("should be true", function() {
const action = new wd.TouchAction()
action
.press({x: 210, y: 130})
.wait(3000)
.release()
driver.performTouchAction(action)
assert.equal(true, true)
})
})
The equals true assert is just there as a placeholder sanity check. The only problem with this currently is that I'm copy-pasting the alpha-numeric session id inside the attach method each time I restart the Appium server so I need to find a way to automate that.
I'm currently develop an App that is based on NativeScript and Angular2.
My screen freeze for while when my App fetching data through HTTP, and I'd like to put the fetching action into another thread.
I did a lot of search on the web, and all I got is the code in javascript like the official doc - https://docs.nativescript.org/angular/core-concepts/multithreading-model.html
Is there any way to implement the muli-threading with WebWorker in "Typescript"(which contain the support of Angular injected HTTP service) instead of the "Javascript" code(the code from the official doc)
It's appreciated if someone could give me some guide or hint, and it'll be great if I could got some relative example code.
Thanks.
There shouldn't be any big draw back for using WebWorkers in {N} + Angular but be aware that currently the WebWorker is not "exactly" compatible with Angular AoT compilation.
For me when creating an WebwWrker (var myWorker = new Worker('~/web.worker.js');) throws and error after bundling the application with AoT. I have seen soem talk about this in the community and possible the way to fix this is by editing the webpack.common.js and adding an "loaded" like so:
{
test: /\.worker.js$/,
loaders: [
"worker-loader"
]
}
Disclaimer: I have not tried this approach for fixing the error.
If someone have some problems adding workers in Nativescript with Angular and Webpack, you must follow the steps listed here.
Keep special caution in the next steps:
When you import the worker, the route to the worker file comes after nativescript-worker-loader!.
In the webpack.config.js keep caution adding this piece of code:
{
test: /.ts$/, exclude: /.worker.ts$/, use: [
"nativescript-dev-webpack/moduleid-compat-loader",
"#ngtools/webpack",
]
},
because is probable that you already have configured the AoT compilation, like this:
{
test: /(?:\.ngfactory\.js|\.ngstyle\.js|\.ts)$/,
use: [
"nativescript-dev-webpack/moduleid-compat-loader",
"#ngtools/webpack",
]
},
and you only need to add the exclude: /.worker.ts$/,
Finally, there is an example of a worker, in this case it use an Android native library:
example.worker.ts:
import "globals";
const context: Worker = self as any;
declare const HPRTAndroidSDK;
context.onmessage = msg => {
let request = msg.data;
let port = request.port;
let result = HPRTAndroidSDK.HPRTPrinterHelper.PortOpen("Bluetooth," + port.portName);
context.postMessage(result);
};
example.component.ts (../../workers/example.worker is the relative route to my worker):
import * as PrinterBTWorker from "nativescript-worker-loader!../../workers/example.worker";
import ...
connect(printer: HPRTPrinter): Observable<boolean> {
if (this.isConnected()){
this.disconnect(); //Disconnect first if it's already connected
}
return Observable.create((observer) => {
const worker = new PrinterBTWorker();
worker.postMessage({ port: printer });
worker.onmessage = (msg: any) => {
worker.terminate();
if (msg.data == 0) { // 0: Connected, -1: Disconnected
observer.next(true);
}
else {
observer.next(false);
}
};
worker.onerror = (err) => {
worker.terminate();
observer.next(false);
}
}).pipe(timeout(5000), catchError(err => of(false)));
}
Note: I use an Observable to make my call to the worker async and to add a timeout to the call to the native code, because in the case that it is not possible to connect to the printer (ex. it's turned off), it takes almost 10 seconds to notify, and this caused in my case the frezing of the app for all that time.
Important: It seem that it's necessary to run again the code manually every time that a change is made, because the worker isn't compiled using AoT.
I have developed a node application which I would like to have called as an AWS Lambda Application.
The application works as intended as an AWS Lambda, however my CloudWatch logs always finish with the following error: Process exited before completing request.
I wrote some code to ensure that my context.succeed() and context.fail() calls were taking place and they are. However, when ran locally, I also noted a lag between my logging of success in start.js and the command prompt appearing again, making me believe there could be some node processes still taking place once those calls have been made. Could that be causing the error, and if so what is a good way to triage and resolve the issue?
The relevant code is below:
lambda-handle.js
import log from './log';
import database from './database';
import User from './database/models/user';
export function handle(event, context) {
log.info('CS Blogs Feed Aggregator Started');
database.sync()
.then(() =>
User.findAll({
attributes: ['id', 'blogFeedURI']
}))
.then(users => {
users.forEach(user => {
log.info({ user }, 'User loaded from database');
});
})
.then(() => {
// context.done() called so AWS knows function completed successfully
context.succeed();
})
.catch(error => {
context.fail(error);
});
}
start.js (used to test context.succeed/fail being called)
// This function invokes the AWS Lambda Handle as AWS would
// but allows you to do it from your local machine for development
// or from a non-AWS server
import { handle } from './lambda-handle';
import log from './log';
handle({}, {
succeed: result => {
log.info({ result: result || 'No result returned' }, 'Process succeeded');
},
fail: error => {
log.error({ error: error || 'No error returned' }, 'Process failed');
}
});
The code is being transpiled by babel before being deployed. However, I suspect there is more of a logic issue so I have shown you the original source code.
If any more information is required the repository is available here: https://github.com/csblogs/feed-downloader/tree/fix/lambda-implementation-details
Thanks
I am pretty sure this is caused by at least 1 native module dependency in bunyan(dtrace-provider). Native modules need to be built/installed on the system that they will run on. So in the case of Lambda you need to run npm install on a linux ec2 instance or possibly vagrant to get the right version of dtrace-provider built.
See:
Cross-compile node module with native bindings with node-gyp
https://aws.amazon.com/blogs/compute/nodejs-packages-in-lambda/ (scroll to Native Modules)
You could probably just remove Bunyan to verify it works and then go down the ec2/vagrant compile route if that fixes it.