I have a web scraper that uses Puppeteer. I am writing tests for my initial method: loadMainPage
loadMainPage:
const loadMainPage = async () => {
try {
// load puppeteer headless browser
const browser = await puppeteer.launch({
headless: true,
});
const mainPage = await browser.newPage();
await mainPage.goto(URL, { waitUntil: ["domcontentloaded"] });
// make sure page loaded.
console.log(URL + " loaded...");
const links = await getPackLinks(mainPage);
// close mainPage
await mainPage.close();
// loop through all links/pages and run the scraper
if (mainPage.isClosed()) {
await loadSubPage(links[6], browser);
console.log("Closing browser session...");
await browser.close();
}
} catch (e) {
console.error(e);
}
};
My test file:
const puppeteer = require("puppeteer");
const { loadMainPage } = require("./scraper");
jest.mock("puppeteer");
describe("loadMainPage()", () => {
it("should launch a new browser session", () => {
loadMainPage();
expect(puppeteer.launch).toBeCalled();
});
it("should open a new page", () => {
loadMainPage();
expect(???)
});
});
All I want to do is test whether certain methods in the puppeteer module are being called. My first test, checking for puppeteer.launch to be called, works just fine. The launch method returns a new instance of a Puppeteer object (a Browser), on which there is a newPage() method. How can I test to see if this method was called? newPage() itself returns another object (a Page), with its own methods that I will also need to test. I tried mocking my own implementations with the factory function that jest.mock accepts, but it was getting to be too much. I felt like I was missing something. Any help?
Related
I'm playing around with puppeteer to learn a bit about automation in the browser. I wanted to open the chromium browser visable so not in headless. I set the launch option to false, but it's still not opening Chromium.
I tried to use no sandbox args, i did even deflag the --disable-extensions in the args, but nothing helped..
There are no errors in the terminal, it just doesn't launch.
Here is my code:
const puppeteer = require ("puppeteer");
async () => {
const browser = await puppeteer.launch({ headless: false });
const page = browser.newPage();
await page.goto("https://google.de");
await browser.close();
};
Any idea why chromium is not opening? Also there are no logs about errors...
Problem
You are not calling the function, you are just defining it via async () => { ... }. This is why you are not getting any errors, as the function is not executed. In addition, as the other answer already said, you are missing an await.
Solution
Your code should look like this:
(async () => {
const browser = await puppeteer.launch({ headless: false });
const page = await browser.newPage(); // missing await
await page.goto("https://google.de");
await browser.close();
})(); // Here, we actually call the function
newPage() returns a promise so you should await it
const puppeteer = require ("puppeteer");
async () => {
const browser = await puppeteer.launch({ headless: false });
const page = await browser.newPage();
await page.goto("https://google.de");
await browser.close();
};
I'm creating tests using jest puppeteer for my react website. Each test passes when run individually however they do not pass when all run together.
import './testFunctions'
import {
cleanSmokeTest,
testErrorsPage1,
testErrorsPages2,
} from '. /testFunctions'
describe('app', () => {
beforeEach(async () => {
jest.setTimeout(120000)
await page.goto('http://localhost:3000')
})
it('should for through all pages with no issue', async () => {
await cleanSmokeTest()
})
it('test errors on page 1', async () => {
await testErrorsPage1()
})
it('test errors on page 2', async () => {
await testErrorsPage2()
})
My best guess for a solution involves clearing the session storage or opening the browser in a new page (as no errors will occur if the page has already passed once)
The following will not open the webpage url so I'm stuck on how to solve this issue
import './testFunctions'
import {
cleanSmokeTest,
testErrorsPage1,
testErrorsPages2,
} from '. /testFunctions'
describe('app', () => {
beforeEach(async () => {
jest.setTimeout(120000)
const puppeteer = require('puppeteer')
const browser = await puppeteer.launch()
const page = await browser.newPage()
await page.goto('http://localhost:3000')
})
it('should for through all pages with no issue', async () => {
await cleanSmokeTest()
})
it('test errors on page 1', async () => {
await testErrorsPage1()
})
it('test errors on page 2', async () => {
await testErrorsPage2()
})
Using the line:
sessionStorage.clear()
produces the error
ReferenceError: sessionStorage is not defined
and:
window.sessionStorage.clear()
produces the error
ReferenceError: window is not defined
Found my solution
await page.goto('http://localhost:3000')
await page.evaluate(() => {
sessionStorage.clear()
})
await page.goto('http://localhost:3000')
The reason for this was because sessionStorage is not defined unless the page is reached. Once it has cleared the page needs a refresh because redux has kept it in memory.
I think you are running sessionStorage.clear() function in nodejs environment. sessionStorage is defined in client javascript context. The code below is executing the function in the page context.
const html = await page.evaluate(() => {sessionStorage.clear() });
so, I'm using Puppeteer with Jest. After adding
const browser = await puppeteer.launch({ headless: false });
const page = await browser.newPage();
My tests does not perform any actions. It doesn't matter if I'm using headless mode or let's call it "normal" mode. Anybody can help me?
homepage.test.js
const puppeteer = require('puppeteer');
const HomePage = require('./page_objects/HomePage');
const homePage = new HomePage();
describe('Homepage', () => {
beforeAll(async () => {
const browser = await puppeteer.launch({ headless: false });
const page = await browser.newPage();
await page.goto(homePage.path);
await page.waitForSelector(homePage.loginPanel);
});
it('Log into your account', async () => {
await homePage.fillLoginForm();
await expect(page).toMatchElement(homePage.productList);
await page.screenshot({ path: 'example.png' });
});
HomePage.js
module.exports = class HomePage {
constructor() {
this.path = 'https://www.saucedemo.com/index.html';
this.loginPanel = '#login_button_container';
this.productList = 'div[class="inventory_container"]';
this.loginForm = {
fields: {
usernameInput: 'input[id="user-name"]',
passwordInput: 'input[id="password"]',
logInButton: 'input[class="btn_action"]',
},
};
}
async fillLoginForm() {
await page.type(this.loginForm.fields.usernameInput, 'standard_user');
await page.type(this.loginForm.fields.passwordInput, 'secret_sauce');
await page.click(this.loginForm.fields.logInButton);
}
};
The answer has two parts, one with normal jest and another with jest-puppeteer. You can skip to the jest-puppeteer if you want.
Problem (with jest):
The browser and page inside beforeAll block has no relation to the it blocks. It also does not have any relation with the page inside HomePage class as well.
You did not mention if you were using jest-puppeteer or not.
Solution:
Create block scoped variables for the describe block, and pass the page object to the modules.
Refining the HomePage class
Consider the following HomePage class.
// HomePage.js
class HomePage {
constructor(page) {
this.page = page;
}
async getScreenshot() {
await this.page.screenshot({ path: "example.png" });
}
async getTitle(page) {
return page.title();
}
}
As you can see, there are two ways to access to the page inside the class. Either pass inside the constructor, or use with the method directly.
The method getScreenshot has a this.page, while getTitle has access to a page.
Refining the test
You cannot use this inside the jest tests due to this issue, but you can declare a variable on top of a block, then access it later.
describe("Example", () => {
// define them up here inside the parent block
let browser;
let page;
let homepage;
beforeAll(async () => {
// it has access to the browser, page and homepage
browser = await puppeteer.launch({ headless: true });
page = await browser.newPage();
homepage = new HomePage(page); // <-- pass the page to HomePage here
await page.goto("http://example.com");
await page.waitForSelector("h1");
return true;
});
});
Now all other blocks can access to the page. According to our previous example HomePage class, we can do either of following depending on how we defined the methods.
it("Gets the screenshot", async () => {
await homepage.getScreenshot(); // <-- will use this.page
});
it("Gets the title", async () => {
await homepage.getTitle(page); // <-- will use the page we are passing on
});
Finally we cleanup the tests,
afterAll(async () => {
await page.close();
await browser.close();
return true;
});
We probably need to run the jest tests with detectOpenHandles for headfull mode.
jest . --detectOpenHandles
Result:
Problem (with jest-puppeteer):
jest-puppeteer already gives you a global browser and page object. You do not need define anything.
However if you want to use jest-puppeteer and expect-puppeteer, you have to use a custom config file.
Solution:
Create a jest-config.json file and put the contents,
{
"preset": "jest-puppeteer",
"setupFilesAfterEnv": ["expect-puppeteer"]
}
Now, get rid of browser and page creation code, as well as any afterAll hooks for page.close as well.
Here is a working test file,
class HomePage {
async getTitle() {
return page.$("h1");
}
}
describe("Example", () => {
const homepage = new HomePage();
beforeAll(async () => {
// it has access to a global browser, page and scoped homepage
await page.goto("http://example.com");
await page.waitForSelector("h1");
});
it("Gets the screenshot", async () => {
const element = await homepage.getTitle();
await expect(element).toMatch("Example");
});
});
And let's run this,
jest . --detectOpenHandles --config jest-config.json
Result:
Is it possible to retrieve a native interface from the Browser or Page instance in order to check if an object is an instanceof this interface?
For instance, in a jest testing context (where for some reasons CanvasRenderingContext2D isn't available since it is a Node context and not a JSDOM or an other emulation of browser APIs):
it("should create an instance of CanvasRenderingContext2D", async () => {
expect.assertions(1);
const context = await page.evaluate(() => {
return document.createElement("canvas").getContext("2d");
});
// Could a JSHandle be used somehow?
const CanvasRenderingContext2DInterface = await page.evaluateHandle(() => CanvasRenderingContext2D);
expect(context).toBeInstanceOf(CanvasRenderingContext2DInterface);
});
Instance check should be evaluated in the Puppeteer browser/page directly otherwise the execution contexts are different.
it("should create an instance of CanvasRenderingContext2D", async () => {
expect.assertions(1);
const isInstanceOfCanvasRenderingContext2D = await page.evaluate(
() =>
document.createElement("canvas").getContext("2d") instanceof CanvasRenderingContext2D
);
expect(isInstanceOfCanvasRenderingContext2D).toBeTruthy();
});
New to JavaScript and trying to understand how to run the following simple test, which loads the google home page, and gets the title. This title is then tested.
const puppeteer = require("puppeteer");
var page_title = "blank";
assert = require("assert");
async function run() {
const browser = await puppeteer.launch({ headless: true });
const page = await browser.newPage();
await page.goto("http://www.google.co.uk");
page_title = await page.title();
console.log("Page Title: ", page_title)
await browser.close();
}
run();
describe("Google", function() {
it("Title contains Google", async function() {
assert.equal(page_title, "Google");
});
});
The issue is the describe/it block runs before the page_title is obtained. Please could someone advise how I should actually be structuring this?
You just need read the mocha documentation. No need to digging deeper, async code located on the TOC.
mocha offer 3 ways:
callback
Simply invoke the callback when your test is complete. By adding a callback (usually named done) to it().
promise
async and await
So it revised like this with async and await :
const puppeteer = require("puppeteer");
var page_title = "blank";
assert = require("assert");
describe("Google", function() {
// this.timeout(0);
it("Title contains Google", async ()=> {
const browser = await puppeteer.launch(); //headless by default
const page = await browser.newPage();
await page.goto("http://www.google.co.uk");
page_title = await page.title();
console.log("Page Title: ", page_title);
assert.equal(page_title, "Google");
await browser.close()
});
});
My advice is quick reading on every explanation on TOC, and read brief explanation async and await