I'm playing around with puppeteer to learn a bit about automation in the browser. I wanted to open the chromium browser visable so not in headless. I set the launch option to false, but it's still not opening Chromium.
I tried to use no sandbox args, i did even deflag the --disable-extensions in the args, but nothing helped..
There are no errors in the terminal, it just doesn't launch.
Here is my code:
const puppeteer = require ("puppeteer");
async () => {
const browser = await puppeteer.launch({ headless: false });
const page = browser.newPage();
await page.goto("https://google.de");
await browser.close();
};
Any idea why chromium is not opening? Also there are no logs about errors...
Problem
You are not calling the function, you are just defining it via async () => { ... }. This is why you are not getting any errors, as the function is not executed. In addition, as the other answer already said, you are missing an await.
Solution
Your code should look like this:
(async () => {
const browser = await puppeteer.launch({ headless: false });
const page = await browser.newPage(); // missing await
await page.goto("https://google.de");
await browser.close();
})(); // Here, we actually call the function
newPage() returns a promise so you should await it
const puppeteer = require ("puppeteer");
async () => {
const browser = await puppeteer.launch({ headless: false });
const page = await browser.newPage();
await page.goto("https://google.de");
await browser.close();
};
Related
I am trying to type into YouTube's search input using Puppeteer.
Code as follows:
(async () => {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto('https://youtube.com');
await page.type('#search','a');
...
Here is the error I get:
throw new Error('Evaluation failed: ' + (0, util_js_1.getExceptionMessage)(exceptionDetails));
^
Error: Evaluation failed: Error: Cannot focus non-HTMLElement
at pptr://__puppeteer_evaluation_script__:3:23
at ExecutionContext._ExecutionContext_evaluate (/Users/benjaminrubin/node_modules/puppeteer/lib/cjs/puppeteer/common/ExecutionContext.js:286:15)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
at async ExecutionContext.evaluate (/Users/benjaminrubin/node_modules/puppeteer/lib/cjs/puppeteer/common/ExecutionContext.js:117:16)
at async ElementHandle.evaluate (/Users/benjaminrubin/node_modules/puppeteer/lib/cjs/puppeteer/common/JSHandle.js:105:16)
at async ElementHandle.focus (/Users/benjaminrubin/node_modules/puppeteer/lib/cjs/puppeteer/common/ElementHandle.js:486:9)
at async ElementHandle.type (/Users/benjaminrubin/node_modules/puppeteer/lib/cjs/puppeteer/common/ElementHandle.js:516:9)
at async DOMWorld.type (/Users/benjaminrubin/node_modules/puppeteer/lib/cjs/puppeteer/common/DOMWorld.js:449:9)
at async /Users/benjaminrubin/Documents/Software Dev Education/Scraping with Node JS/youtubeScrape.js:60:9
I could not figure out what exactly is wrong. Several examples across the web use the exact same format. What exactly does 'Cannot focus non-HTMLElement' mean?
This is a tricky one. Google sites are notorious for breaching the "one id on a page" rule, so there's actually two elements with the id search:
<ytd-searchbox id="search"> <!-- the one you are actually selecting -->
... bunch of nodes ...
<input id="search"> <!-- the one you think you're selecting -->
await page.type('#search','a'); types into ytd-searchbox, which isn't a standard HTML element, so Puppeteer fails with the Error: Cannot focus non-HTMLElement error.
The fix is to use input#search instead:
const puppeteer = require("puppeteer"); // ^19.1.0
let browser;
(async () => {
browser = await puppeteer.launch();
const [page] = await browser.pages();
await page.goto("https://youtube.com", {waitUntil: "domcontentloaded"});
await page.type("input#search", "hello world");
await page.screenshot({path: "youtube.png"});
})()
.catch(err => console.error(err))
.finally(() => browser?.close());
Although the above solution may work, this is a good example of where simply encoding your search as a URL parameter and navigating directly to the results page is easier and more efficient:
const puppeteer = require("puppeteer");
let browser;
(async () => {
browser = await puppeteer.launch();
const [page] = await browser.pages();
const q = encodeURIComponent("your search here");
const url = `https://www.youtube.com/results?search_query=${q}`;
await page.goto(url, {waitUntil: "networkidle2"});
await page.screenshot({path: "youtube.png"});
})()
.catch(err => console.error(err))
.finally(() => browser?.close());
I have a simple piece of code
describe('My First Puppeeteer Test', () => {
it('Should launch the browser', async function() {
const browser = await puppeteer.launch({ headless: false})
const page = await browser.newPage()
await page.goto('https://github.com/login')
await page.type('#login_field', testLogin)
await page.type('#password', testPassword)
await page.click('[name="commit"]')
await page.waitForNavigation()
let [element] = await page.$x('//h3[#class="text-normal"]')
let helloText = await page.evaluate(element => element.textContent, element);
console.log(helloText);
browser.close();
})
})
Everything worked before but today I get an error + my stacktrace:
Error: Evaluation failed: TypeError: Cannot read properties of undefined (reading 'textContent')
at puppeteer_evaluation_script:1:21
at ExecutionContext._evaluateInternal (node_modules\puppeteer\lib\cjs\puppeteer\common\ExecutionContext.js:221:19)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
at async ExecutionContext.evaluate (node_modules\puppeteer\lib\cjs\puppeteer\common\ExecutionContext.js:110:16)
at async Context. (tests\example.tests.js:16:22)
How I can resolve this?
Kind regards
While I haven't tested the code due to the login and I assume your selectors are correct, the main problem is almost certainly that
await page.click('[name="commit"]')
await page.waitForNavigation()
creates a race condition. The docs clarify:
Bear in mind that if click() triggers a navigation event and there's a separate page.waitForNavigation() promise to be resolved, you may end up with a race condition that yields unexpected results. The correct pattern for click and wait for navigation is the following:
const [response] = await Promise.all([
page.waitForNavigation(waitOptions),
page.click(selector, clickOptions),
]);
As a side point, it's probably better to do waitForXPath rather than $x, although this seems less likely the root problem. Don't forget to await all promises such as browser.close().
const puppeteer = require("puppeteer");
let browser;
(async () => {
browser = await puppeteer.launch({headless: true});
const [page] = await browser.pages();
await page.goto('https://github.com/login');
await page.type('#login_field', testLogin);
await page.type('#password', testPassword);
// vvvvvvvvvvv
await Promise.all([
page.click('[name="commit"]'),
page.waitForNavigation(),
]);
const el = await page.waitForXPath('//h3[#class="text-normal"]');
// ^^^^^^^^^^^^
//const el = await page.waitForSelector("h3.text-normal"); // ..or
const text = await el.evaluate(el => el.textContent);
console.log(text);
//await browser.close();
//^^^^^ missing await, or use finally as below
})()
.catch(err => console.error(err))
.finally(() => browser?.close())
;
Additionally, if you're using Jest, once you get things working, you might want to move the browser and page management to beforeEach/afterEach or beforeAll/afterAll blocks. It's faster to use the same browser instance for all test cases, and pages can be opened and closed before/after each case.
I am using Puppeteer headless browser and doing WebRTC call. At the end of call I want to know statistics like bandwidth, Jitter, ICE details etc.
So far what I have been able to collect from google search is we can get the stats data using getStats api.
But in a puppeteer script how I can call getStats api, I could not find any example.
My code looks as below.
const puppeteer = require('puppeteer');
const sleep = (waitTimeInMs) => new Promise(resolve => setTimeout(resolve, waitTimeInMs));
(async () => {
const browser = await puppeteer.launch({headless: false});
const page = await browser.newPage();
await page.goto('https://janus.conf.meetecho.com/videocalltest.html');
await page.waitForSelector('#start');
await page.click('[id=start]');
await page.waitForSelector('#username', { visible: true });
await page.type('input[id="username"]', 'user1');
await page.click('button[id=register]');
await page.waitFor(5000);
await page.type('input[id=peer]', 'user0');
await page.click('button[id=call]');
await sleep(16000);
await page.click('button[id=start]');
await sleep(3000);
await browser.close();
})();
Just before browser.close(), I want to know stats data. Can you please help me to understand, how can I make use of getStats api in this context to get the stats data.
Is there any better way to get stats data then getsStats api?
You can use evaluate to get the WebRTC stats:
result = await page.evaluate(async () => await videocall.webrtcStuff.pc.getStats());
console.log(result);
Here is my code:
const puppeteer = require('puppeteer');
(async () => {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto('https://google.com/');
await page.screenshot({path: 'example.png'});
await browser.close();
})();
No matter what website I attempt to screenshot, I always get the following error:
(node:9548) UnhandledPromiseRejectionWarning: TimeoutError: Navigation Timeout Exceeded: 30000ms exceeded
I'm running node version 8.16.0. I have no idea why I always get this timeout. Any help is appreciated.
EDIT:
It does seem to work when I run it with headless mode turned off, but I need it to run as a headless browser.
Try to increase the navigation timeout:
await page.goto('https://google.com/', { waitUntil: 'load', timeout: 50000 });
and add try/catch:
try {
await page.goto('https://google.com/', { waitUntil: 'load', timeout: 50000 });
} catch(e) {
console.log('Error--->', e);
}
New to JavaScript and trying to understand how to run the following simple test, which loads the google home page, and gets the title. This title is then tested.
const puppeteer = require("puppeteer");
var page_title = "blank";
assert = require("assert");
async function run() {
const browser = await puppeteer.launch({ headless: true });
const page = await browser.newPage();
await page.goto("http://www.google.co.uk");
page_title = await page.title();
console.log("Page Title: ", page_title)
await browser.close();
}
run();
describe("Google", function() {
it("Title contains Google", async function() {
assert.equal(page_title, "Google");
});
});
The issue is the describe/it block runs before the page_title is obtained. Please could someone advise how I should actually be structuring this?
You just need read the mocha documentation. No need to digging deeper, async code located on the TOC.
mocha offer 3 ways:
callback
Simply invoke the callback when your test is complete. By adding a callback (usually named done) to it().
promise
async and await
So it revised like this with async and await :
const puppeteer = require("puppeteer");
var page_title = "blank";
assert = require("assert");
describe("Google", function() {
// this.timeout(0);
it("Title contains Google", async ()=> {
const browser = await puppeteer.launch(); //headless by default
const page = await browser.newPage();
await page.goto("http://www.google.co.uk");
page_title = await page.title();
console.log("Page Title: ", page_title);
assert.equal(page_title, "Google");
await browser.close()
});
});
My advice is quick reading on every explanation on TOC, and read brief explanation async and await