Hello I try to make a screenshot with Playwright but I have cookie EU law popup on my screenshots. How can I remove them ?
Here is my browser parameters.
const browser = await playwright.firefox.launch({
headless: true,
firefoxUserPrefs: {
"network.cookie.cookieBehavior": 2
}
});
But it don't work.
Thank for your help.
Use the playwright API to click the element. I'm using the text selector in the example below, but you can use any selector.
const { webkit } = require('playwright');
(async() => {
const browser = await webkit.launch({ headless: false });
const page = await browser.newPage();
await page.goto('https://npmjs.com');
await page.click('text=Accept');
await page.screenshot({ path: 'screenshot.png' });
await browser.close();
})();
Related
I'm using Puppeteer.js to crawl some URL. I'm using the default Chromium browser of Puppeteer.All is working well, but the problem is, that when I run the crawling script, and doing other things in the background and the focus is no longer on the Chromium browser of Puppeteer, it's not working: waiting for elements way too long, and abort operations, or in other words: puppeteer is paused (or freeze).
P.S, I'm also using puppeteer-extra and puppeteer-extra-plugin-stealth NPM packages for advance options.
Here is how I create the browser and the page:
async initiateCrawl(isDisableAsserts) {
// Set the browser.
this.isPlannedClose = false;
const browser = await puppeteerExtra.launch({
headless: false,
args: [
'--no-sandbox',
'--disable-setuid-sandbox',
'--disable-dev-shm-usage',
'--start-maximized',
'--disable-background-timer-throttling',
'--disable-backgrounding-occluded-windows',
'--disable-renderer-backgrounding'
]
});
const pid = browser.process().pid;
browser.on('disconnected', () => {
systemUtils.killProcess(pid);
if (!this.isPlannedClose) {
systemUtils.exit(Status.BROWSER_CLOSE, Color.RED, 0);
}
});
process.on('SIGINT', () => {
this.close(browser, true);
});
// Set the page and close the first empty tab.
const page = await browser.newPage();
const pages = await browser.pages();
if (pages.length > 1) {
await pages[0].close();
}
await page.setRequestInterception(true);
await page.setJavaScriptEnabled(false);
await page.setDefaultNavigationTimeout(this.timeout);
page.on('request', (request) => {
if (isDisableAsserts && ['image', 'stylesheet', 'font', 'script'].indexOf(request.resourceType()) !== -1) {
request.abort();
} else {
request.continue();
}
});
return {
browser: browser,
page: page
};
}
I already looked at:
https://github.com/puppeteer/puppeteer/issues/3339
https://github.com/GoogleChrome/chrome-launcher/issues/169
https://www.gitmemory.com/issue/GoogleChrome/puppeteer/3339/530620329
Not working solutions:
const session = await page.target().createCDPSession();
await session.send('Page.enable');
await session.send('Page.setWebLifecycleState', {state: 'active'});
const chromeArgs = [
'--disable-background-timer-throttling',
'--disable-backgrounding-occluded-windows',
'--disable-renderer-backgrounding'
];
var ops = {args:[
'--kiosks',
'--disable-background-timer-throttling',
'--disable-backgrounding-occluded-windows',
'--disable-renderer-backgrounding',
'--disable-canvas-aa',
'--disable-2d-canvas-clip-aa',
'--disable-gl-drawing-for-tests',
'--disable-dev-shm-usage',
'--no-zygote',
'--use-gl=desktop',
'--enable-webgl',
'--hide-scrollbars',
'--mute-audio',
'--start-maximized',
'--no-first-run',
'--disable-infobars',
'--disable-breakpad',
'--user-data-dir='+tempFolder,
'--no-sandbox',
'--disable-setuid-sandbox'
], headless: false, timeout:0 };
puppeteer = require('puppeteer');
browser = await puppeteer.launch(ops);
page = await browser.newPage();
Has anyone faced this issue before and have any idea how to solve this? Thanks.
My issue was solved when I updated to the latest puppeteer version (9.0.0).
I use the following blog to use the playright
login and I need something similar to use for my app, when I use the headless:flase
I see it opens the UI with the user password in however, it doesnt click on the logon button, I adopt the code I try with the following , am I missing something?
(async () => {
const browser = await chromium.launch({ headless: false });
const page = await browser.newPage();
await page.goto('https://app.com');
await page.fill('input[type="text"]', 'user#test.com');
await page.fill('input[type="password"]', 'Abcd1234!');
// page.click('div[data-testid="LoginForm_Login_Button"]');
page.click('div[id="logOnFormSubmit"]');
}
)();
You are currently using
page.click('div[id="logOnFormSubmit"]');
There is no div in your given code example with that ID, but instead there is a button. You'd need to change that line to reflect this. The final code would look like below.
(async () => {
const browser = await chromium.launch({ headless: false });
const page = await browser.newPage();
await page.goto('https://app.com');
await page.fill('input[type="text"]', 'user#test.com');
await page.fill('input[type="password"]', 'Abcd1234!');
// page.click('div[data-testid="LoginForm_Login_Button"]');
page.click('button[id="logOnFormSubmit"]');
}
)();
I'm trying to build an automation for my Linkedin account so that whenever I get a message, I want to do something custom with it.
I'm using Puppeteer and MutationObserver inside page.evaluate call after loading my Linkedin profile with my li_at session cookie.
But it fails to fire an event even when I see the node changing the textContent.
const puppeteer = require('puppeteer');
(async function main() {
try {
const browser = await puppeteer.launch({ devtools: true, headless: false });
const page = await browser.newPage();
await page.setBypassCSP(true);
await page.setDefaultNavigationTimeout(0);
await page.setCookie({
'name': 'li_at',
'value': 'putYourSessionCookieHere',
'domain': '.www.linkedin.com'
})
await page.goto('https://www.linkedin.com', {waitUntil: 'networkidle2'});
await page.exposeFunction('puppeteerMutationListener', puppeteerMutationListener);
await page.evaluate(() => {
const target = document.querySelector('#messaging-nav-item .nav-item__badge-count');
const observer = new MutationObserver((mutationsList) => {
for (const mutation of mutationsList) {
window.puppeteerMutationListener(
mutation.removedNodes[0].textContent,
mutation.addedNodes[0].textContent,
);
}
});
observer.observe(
target,
{ childList: true},
);
});
} catch (err) {
console.error(err);
}
})();
function puppeteerMutationListener(oldValue, newValue) {
console.log(`${oldValue} -> ${newValue}`);
}
To reproduce the issue, you would need:
A Linkedin account
A helpful coworker with a linkedin account up for messaging you OR you can change the textContent of the Node by yourself
Any ideas on why this may be happening?
I'm using Puppeteer for doing some web scraping and I'm having troubles. The website I'm trying to scrape is this one and I'm trying to create a screenshot of a calendar that appears after clicking the button "Reserve now" > "Dates".
const puppeteer = require('puppeteer');
const fs = require('fs');
void (async () => {
try {
const browser = await puppeteer.launch({headless: false});
const page = await browser.newPage();
await page.goto('https://www.marriott.com/hotels/travel/reumd-le-meridien-ra-beach-hotel-and-spa');
await page.setViewport({ width: 1920, height: 938 });
await page.waitForSelector('.m-hotel-info > .l-container > .l-header-section > .l-m-col-2 > .m-button');
await page.click('.m-hotel-info > .l-container > .l-header-section > .l-m-col-2 > .m-button');
await page.waitForSelector('.modal-content');
await page.waitFor(5000);
await page.waitForSelector('.js-recent-search-inputs .js-datepick-container .l-h-field-input')
await page.click('.js-recent-search-inputs .js-datepick-container .l-h-field-input');
await page.waitFor(5000);
await page.screenshot({ path: 'myscreenshot.png'});
await browser.close();
} catch (error) {
console.log(error);
}
})()
This is what myscreenshot.png should contain:
but I'm getting this instead:
As you can see, myscreenshot.png doesn't contain the calendar. I don't understand what I'm doing wrong since I click on the right node and I even give time enough to it for loading everything.
Thank you in advance!
Edit: I forgot to say that I have also tried Puppeteer recorder in order to achieve this and I haven't had luck either.
As you have many .l-h-field-input elements, I would try being more specific there.
This worked for me:
await page.click('.js-recent-search-inputs .js-datepick-container .l-h-field-input');
Node- v8.11.1 Headless Chrome
Im trying to generate PDF but somehow the background image is not captured in the PDF.
Below is the code. Any help is appreciated
const puppeteer = require('puppeteer');
(async () => {
const browser = await puppeteer.launch({headless: true});
const page = await browser.newPage();
await page.goto('http://54.201.139.151/', {waitUntil : 'networkidle0'});
await page.pdf({path: 'hn40.pdf', printBackground: true, width: '1024px' , height: '768px'});
await browser.close();
})();
Update: page.emulateMedia() is dropped in favor of page.emulateMediaType()
As Rippo mentioned, you require page.emulateMedia("screen") for this to work properly. I have updated your script below, but I changed the page to google for testing.
const puppeteer = require('puppeteer');
(async () => {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto('http://google.ca/', {waitUntil : 'networkidle2'});
await page.emulateMedia('screen');
await page.pdf({path: 'hn40.pdf', printBackground: true, width: '1024px' , height: '768px'});
await browser.close();
})();