I installed puppeteer on my debian server, and I'm trying to use it through php :
print shell_exec("node pptscript.js");
pptscript.js:
const puppeteer = require('puppeteer');
(async () => {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto('https://www.google.com');
await page.screenshot({path: '/var/www/web/example.png'});
await browser.close();
})();
But it doesn't work and I have no output.
I read somewhere that chmoding to 777 the chromium executable can solve the problem because user www-data have no right to execute it. But I don't know where is it.
browser has an internal function called process() which returns a ChildProcess. You can read the spawnfile property of that ChildProcess.
console.log(browser.process().spawnfile);
Related
<script setup>
import puppeteer from 'puppeteer';
const onChange = async () => {
// Launch the browser
const browser = await puppeteer.launch();
// Create a page
const page = await browser.newPage();
// Go to your site
await page.goto('https://stackoverflow.com');
// Evaluate JavaScript
const three = await page.evaluate(() => {
return 1 + 2;
});
console.log(three);
// Close browser.
await browser.close();
};
console.log(onChange);
</script>
here I am trying to scrape a website using (puppeteer) but I get an error
in ReferenceError: process is not defined
From the official page
Puppeteer is a Node.js library
You should probably not try to get that one working with VueJS (a client-side framework), gonna have a harder time for not a lot of benefits.
Use regular NodeJS for the scraping.
Otherwise, you could give a try to Nuxt3 with its SSR capabilities if you think that having Puppeteer and Vue alongside is crucial.
When I use a code like this (an example off the website), an error message pops up {shown at the botton} which crashes the code however about a second later a blank google chrome page opens but nothing else happens because the code has crashed
const puppeteer = require('puppeteer');
(async () => {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto('https://bbc.com');
await browser.close();
})();
Error message is
Error: Failed to launch the browser process!
I am trying to run my first code on puppeteer.
Puppeteer v1.20.0
Node v8.11.3
Npm v5.6.0
It is a basic example but it doesn't works :
const puppeteer = require('puppeteer');
puppeteer.launch({headless: false}).then(async browser => {
const page = await browser.newPage();
await page.goto('https://www.linkedin.com/learning/login', { waitUntil: 'networkidle0' });
console.log(0);
await page.waitFor('#username');
console.log(1);
await browser.close();
});
When I run the script, chromium start and I can see the Linkedin login page with the form and the #username form's field, but puppeteer doesn't find the field. It displays 0 but never 1 and then runs a TimeoutError: waiting for selector "#username" failed: timeout 30000ms exceeded.
Increase timeout doesn't change anything and if I check the console in chromium the field is there.
Linkedin login page works as an SPA and I don't know if I'm using the right way here.
Thank you in advance.
username input is inside iframe you cant access it like this , you need to access iframe first
await page.goto('https://www.linkedin.com/learning/login');
await page.waitForSelector('.authentication-iframe');
var frames = await page.frames();
var myframe = frames.find(
f =>
f.url().indexOf("uas/login") > 0);
let username = '123456#gmail.com';
const usernamefild = await myframe.$("#username");
await usernamefild.type(username, {delay: 10});
I want to start a chromium browser instant headless, do some automated operations, and then turn it visible before doing the rest of the stuff.
Is this possible to do using Puppeteer, and if it is, can you tell me how? And if it is not, is there any other framework or library for browser automation that can do this?
So far I've tried the following but it didn't work.
const browser = await puppeteer.launch({'headless': false});
browser.headless = true;
const page = await browser.newPage();
await page.goto('https://news.ycombinator.com', {waitUntil: 'networkidle2'});
await page.pdf({path: 'hn.pdf', format: 'A4'});
Short answer: It's not possible
Chrome only allows to either start the browser in headless or non-headless mode. You have to specify it when you launch the browser and it is not possible to switch during runtime.
What is possible, is to launch a second browser and reuse cookies (and any other data) from the first browser.
Long answer
You would assume that you could just reuse the data directory when calling puppeteer.launch, but this is currently not possible due to multiple bugs (#1268, #1270 in the puppeteer repo).
So the best approach is to save any cookies or local storage data that you need to share between the browser instances and restore the data when you launch the browser. You then visit the website a second time. Be aware that any state the website has in terms of JavaScript variable, will be lost when you recrawl the page.
Process
Summing up, the whole process should look like this (or vice versa for headless to headfull):
Crawl in non-headless mode until you want to switch mode
Serialize cookies
Launch or reuse second browser (in headless mode)
Restore cookies
Revisit page
Continue crawling
As mentioned, this isn't currently possible since the headless switch occurs via Chromium launch flags.
I usually do this with userDataDir, which the Chromium docs describe as follows:
The user data directory contains profile data such as history, bookmarks, and cookies, as well as other per-installation local state.
Here's a simple example. This launches a browser headlessly, sets a local storage value on an arbitrary page, closes the browser, re-opens it headfully, retrieves the local storage value and prints it.
const puppeteer = require("puppeteer"); // ^18.0.4
const url = "https://www.example.com";
const opts = {userDataDir: "./data"};
let browser;
(async () => {
{
browser = await puppeteer.launch({...opts, headless: true});
const [page] = await browser.pages();
await page.goto(url, {waitUntil: "domcontentloaded"});
await page.evaluate(() => localStorage.setItem("hello", "world"));
await browser.close();
}
{
browser = await puppeteer.launch({...opts, headless: false});
const [page] = await browser.pages();
await page.goto(url, {waitUntil: "domcontentloaded"});
const result = await page.evaluate(() => localStorage.getItem("hello"));
console.log(result); // => world
}
})()
.catch(err => console.error(err))
.finally(() => browser?.close())
;
Change const opts = {userDataDir: "./data"}; to const opts = {}; and you'll see null print instead of world; the user data doesn't persist.
The answer from a few years ago mentions issues with userDataDir and suggests a cookies solution. That's fine, but I haven't had any issues with userDataDir so either they've been resolved on the Puppeteer end or my use cases haven't triggered the issues.
There's a useful-looking answer from a reputable source in How to turn headless on after launch? but I haven't had a chance to try it yet.
I wondering how can I get PDF using Chrome Headless (for example puppeteer). It seems like a good PDF maker but only on chrome using #media print. So here is my question:
Can I get PDF by puppeteer on another browser (ie, mozilla) too? I think I can do that if I want print static page with no inputs. But if I have inputs for users and they are saving it on IE. Can I use this somehow?
Ok i downloaded the puppeteer. I've got the code:
$scope.aClick = function(){
const puppeteer = require('puppeteer');
(async () => {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto('/vUrl_form.html', {waitUntil: 'networkidle'});
await page.pdf({path: 'images/asd.pdf', format: 'A4'});
browser.close();
})();
};
and this can't still work (i don't know why, but app can't run).
No - Puppeteer only works with Chromium/Chrome.
Unfortunately Puppeteer only works with Chromium/Chrome.
If you want to use Headless Mozilla Firefox, you might consider checking this out https://developer.mozilla.org/en-US/Firefox/Headless_mode .
If you still want to use Puppeteer, here is a working snippet that creates a .pdf file:
const puppeteer = require('puppeteer');
(async() => {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto('https://news.ycombinator.com', {waitUntil: 'networkidle'});
// page.pdf() is currently supported only in headless mode.
// #see https://bugs.chromium.org/p/chromium/issues/detail?id=753118
await page.pdf({
path: 'hn.pdf',
format: 'letter'
});
browser.close();
})();
Today it's possible to use firefox with puppeter https://firefox-puppeteer.readthedocs.io/en/master/ Maybe when people answered it wasn't. But I can't find url to pdf functionality. Just screenshots.