Puppeteer: Get innerHTML - javascript

Does anybody know how to get the innerHTML or text of an element? Or even better; how to click an element with a specific innerHTML? This is how it would work with normal JavaScript:
var found = false
$(selector).each(function() {
if (found) return;
else if ($(this).text().replace(/[^0-9]/g, '') === '5' {
$(this).trigger('click');
found = true
}
});
Thanks in advance for any help!

This is how i get innerHTML:
page.$eval(selector, (element) => {
return element.innerHTML
})

Returning innerHTML of an Element
You can use the following methods to return the innerHTML of an element:
page.$eval()
const inner_html = await page.$eval('#example', element => element.innerHTML);
page.evaluate()
const inner_html = await page.evaluate(() => document.querySelector('#example').innerHTML);
page.$() / elementHandle.getProperty() / jsHandle.jsonValue()
const element = await page.$('#example');
const element_property = await element.getProperty('innerHTML');
const inner_html = await element_property.jsonValue();
Clicking an Element with Specific innerHTML
You can use the following methods to click on an element based on the innerHTML that is contained within the element:
page.$$eval()
await page.$$eval('.example', elements => {
const element = elements.find(element => element.innerHTML === '<h1>Hello, world!</h1>');
element.click();
});
page.evaluate()
await page.evaluate(() => {
const elements = [...document.querySelectorAll('.example')];
const element = elements.find(element => element.innerHTML === '<h1>Hello, world!</h1>');
element.click();
});
page.evaluateHandle() / elementHandle.click()
const element = await page.evaluateHandle(() => {
const elements = [...document.querySelectorAll('.example')];
const element = elements.find(element => element.innerHTML === '<h1>Hello, world!</h1>');
return element;
});
await element.click();

This should work with puppeteer:)
const page = await browser.newPage();
const title = await page.evaluate(el => el.innerHTML, await page.$('h1'));

You can leverage the page.$$(selector) to get all your target elments and then use page.evaluate() to get the content(innerHTML), then apply your criteria. It should look something like:
const targetEls = await page.$$('yourFancySelector');
for(let target of targetEls){
const iHtml = await page.evaluate(el => el.innerHTML, target);
if (iHtml.replace(/[^0-9]/g, '') === '5') {
await target.click();
break;
}
}

I can never get the .innerHtml to work reliable. I always do the following:
let els = page.$$('selector');
for (let el of els) {
let content = await (await el.getProperty('textContent')).jsonValue();
}
Then you have your text in the 'content' variable.

With regard to this part of your question...
"Or even better; how to click an element with a specific innerHTML."
There are some particulars around innerHTML, innerText, and textContent that might give you grief. Which you can work-around using a sufficiently loose XPath query with Puppeteer v1.1.1.
Something like this:
const el = await page.$x('//*[text()[contains(., "search-text-here")]]');
await el[0].click({
button: 'left',
clickCount: 1,
delay: 50
});
Just keep in mind that you will get an array of ElementHandles back from that query. So... the particular item you are looking for might not be at [0] if your text isn't unique.
Options passed to .click() aren't necessary if all you need is a single left-click.

You can simply write as below. (no need await sentence in the last part)
const center = await page.$eval('h2.font-34.uppercase > strong', e => e.innerHTML);

<div id="innerHTML">Hello</div>
var myInnerHtml = document.getElementById("innerHTML").innerHTML;
console.log(myInnerHtml);

Related

Innertext returning an object in puppeteer

I'm using the following in puppeteer to try return the number in the inner text of the below element. I've tried many different ways but keep getting an empty object returned, what am I doing wrong?!
if (await page.$('.s-pagination-item.s-pagination-next.s-pagination-button.s-pagination-separator') !== null) {
var lastPageNumber = await page.evaluate(() => document.querySelector('s-pagination-item.s-pagination-disabled'), a => a.innerText);
} else {
var lastPageNumber = 1;
}
To get the innerText value of a given object you would need to do it like this:
var lastPageNumber = await page.evaluate(
() => document.querySelector('s-pagination-item.s-pagination-disabled').innerText
);

Puppeteer: Replacing innerHTML inside the same selectors

When taking screenshots using puppeteer, dynamic elements with the .menu__link class are required to change innerHTML to a stub.
I use BackstopJs puppet/onReady.js
When I try this, only the first element on the page is replaced:
module.exports = async (page) => {
const myLocalValue = "Test";
const tweets = await page.$$('.menu__link');
for (const tweet of tweets) {
await page.$eval('.menu__link', (el, value) => el.innerHTML = value, myLocalValue)
}
};
And this code does not work at all:
module.exports = async (page) => {
const myLocalValue = "Test";
const tweets = await page.$$('.menu__link');
for (const tweet of tweets) {
await page.$eval(tweet, (el, value) => el.innerHTML = value, myLocalValue)
}
};
Please tell me how to replace innerHTML on the entire page for all .menu__link using puppeteer?
You can use $$eval
await page.$$eval('. menu__link', (links, value) => links.forEach(el => el.innerHTML = value), 'myLocalValue');

How to delete existing text from input using Puppeteer?

I'm trying to test amending text in an editable input which contains the title of the current record - and I want to able to test editing such text, replacing it with something else.
I know I can use await page.type('#inputID', 'blah'); to insert "blah" into the textbox (which in my case, having existing text, only appends "blah"), however, I cannot find any page methods1 that allow deleting or replacing existing text.
You can use page.evaluate to manipulate DOM as you see fit:
await page.evaluate( () => document.getElementById("inputID").value = "")
However sometimes just manipulating a given field might not be enough (a target page could be an SPA with event listeners), so emulating real keypresses is preferable. The examples below are from the informative issue in puppeteer's Github concerning this task.
Here we press Backspace as many times as there are characters in that field:
const inputValue = await page.$eval('#inputID', el => el.value);
// focus on the input field
await page.click('#inputID');
for (let i = 0; i < inputValue.length; i++) {
await page.keyboard.press('Backspace');
}
Another interesting solution is to click the target field 3 times so that the browser would select all the text in it and then you could just type what you want:
const input = await page.$('#inputID');
await input.click({ clickCount: 3 })
await input.type("Blah");
You can use the page.keyboard methods to change input values, or you can use page.evaluate().
Replace All Text:
// Using page.keyboard:
await page.focus('#example');
await page.keyboard.down('Control');
await page.keyboard.press('A');
await page.keyboard.up('Control');
await page.keyboard.press('Backspace');
await page.keyboard.type('foo');
// Using page.evaluate:
await page.evaluate(() => {
const example = document.getElementById('example');
example.value = 'foo';
});
Append Text:
// Using page.keyboard:
await page.focus('#example');
await page.keyboard.press('End');
await page.keyboard.type(' bar qux');
// Using page.evaluate:
await page.evaluate(() => {
const example = document.getElementById('example');
example.value += ' bar qux';
});
Backspace Last Character:
// Using page.keyboard:
await page.focus('#example');
await page.keyboard.press('End');
await page.keyboard.press('Backspace');
// Using page.evaluate:
await page.evaluate(() => {
const example = document.getElementById('example');
example.value = example.value.slice(0, -1);
});
Delete First Character:
// Using page.keyboard:
await page.focus('#example');
await page.keyboard.press('Home');
await page.keyboard.press('Delete');
// Using page.evaluate:
await page.evaluate(() => {
const example = document.getElementById('example');
example.value = example.value.slice(1);
});
If you are not interested in simulating any key events, you could also use puppeteer's page.$eval method as a concise means to remove the textarea's value...
await page.$eval('#inputID', el => el.value = '');
await page.type('#inputID', 'blah');
...or even completely replace the value in one step, without simulating the subsequent typing:
await page.$eval('#inputID', el => el.value = 'blah');
This works perfect for "clear only" method:
const input = await page.$('#inputID');
await input.click({ clickCount: 3 })
await page.keyboard.press('Backspace')
above answers has an ESLint issues.
the following solution passing ESLint varification:
await page.evaluate(
(selector) => { (document.querySelector(selector).value = ''); },
inputBoxSelector,
);
Use the Keyboard API which simulates keystrokes:
await page.focus(css); // CSS selector of the input element
await page.keyboard.down('Shift');
await page.keyboard.press('Home');
await page.keyboard.up('Shift'); // Release the pressed 'Shift' key
await page.keyboard.press('Backspace');
This keystroke is cross-platform as opposed to using ctrl + A(does not work in Mac to select all characters in a input field)
The most clean way for me is:
Setup
const clearInput = async (page, { selector }) => {
const input = await page.$(selector)
await input.click({ clickCount: 3 })
await page.keyboard.press('Backspace')
}
Usage
const page = await context.newPage()
await clearInput(page, { selector: 'input[name="session[username_or_email]"]' })
await clearInput(page, { selector: 'input[name="session[password]"]' })
Well, the reason you want to delete existing text generally may be want to replace it.
You can use page.evalute
let title = getTitle()
let selector = getSelector()
await page.evaluate(
({selector, title}) => {
let el = document.querySelector(selector)
if ('value' in el) el.value = title
else el.innerText = title
},
{selector, title}
)
someField.type("");
Pass the empty string before typing your content.
This worked for me.

Puppeteer Getting List of Elements with Same Selector

Background:
Using NodeJS/CucumberJS/Puppeteer to build end-to-end regression test for an emberJS solution.
Problem:
Selecting (page.click) and getting textContent of one of the elements when there are several dynamic elements with the same selector? (In my case, I have 4 elements with the same selector = [data-test-foo4="true"])
I know, that with:
const text = await page.evaluate( () => document.querySelector('[data-test-foo4="true"]').textContent );
I can get the text of the first element, but how do I select the other elements with the same selector? I've tried:
var text = await page.evaluate( () => document.querySelectorAll('[data-test-foo4="true"]').textContent )[1];
console.log('text = ' + text);
but it gives me 'text = undefined'
Also, the following:
await page.click('[data-test-foo4="true"]');
selects the first elements with that selector, but how can I select the next one with that selector?
You can use Array.from() to create an array containing all of the textContent values of each element matching your selector:
const text = await page.evaluate(() => Array.from(document.querySelectorAll('[data-test-foo4="true"]'), element => element.textContent));
console.log(text[0]);
console.log(text[1]);
console.log(text[2]);
If you need to click more than one element containing a given selector, you can create an ElementHandle array using page.$$() and click each one using elementHandle.click():
const example = await page.$$('[data-test-foo4="true"]');
await example[0].click();
await example[1].click();
await example[2].click();
https://github.com/puppeteer/puppeteer/blob/v5.5.0/docs/api.md#frameselector-1
const pageFrame = page.mainFrame();
const elems = await pageFrame.$$(selector);
Not mentioned yet is the awesome page.$$eval which is basically a wrapper for this common pattern:
page.evaluate(() => callback([...document.querySelectorAll(selector)]))
For example,
const puppeteer = require("puppeteer"); // ^19.1.0
const html = `<!DOCTYPE html>
<html>
<body>
<ul>
<li data-test-foo4="true">red</li>
<li data-test-foo4="false">blue</li>
<li data-test-foo4="true">purple</li>
</ul>
</body>
</html>`;
let browser;
(async () => {
browser = await puppeteer.launch();
const [page] = await browser.pages();
await page.setContent(html);
const sel = '[data-test-foo4="true"]';
const text = await page.$$eval(sel, els => els.map(e => e.textContent));
console.log(text); // => [ 'red', 'purple' ]
console.log(text[0]); // => 'red'
console.log(text[1]); // => 'purple'
})()
.catch(err => console.error(err))
.finally(() => browser?.close());
If you want to pass additional data from Node for $$eval to use in the browser context, you can add additional arguments:
const text = await page.$$eval(
'[data-test-foo4="true"]',
(els, data) => els.map(e => e.textContent + data),
"X" // 'data' passed to the callback
);
console.log(text); // => [ 'redX', 'purpleX' ]
You can use page.$$eval to issue a native DOM click on each element or on a specific element:
// click all
await page.$$eval(sel, els => els.forEach(el => el.click()));
// click one (hardcoded)
await page.$$eval(sel, els => els[1].click());
// click one (passing `n` from Node)
await page.$$eval(sel, (els, n) => els[n].click(), n);
or use page.$$ to return the elements back to Node to issue trusted Puppeteer clicks:
const els = await page.$$('[data-test-foo4="true"]');
for (const el of els) {
await el.click();
}
// or click the n-th:
await els[n].click();
Pertinent to OP's question, you can always access the n-th item of these arrays with the usual syntax els[n] as shown above, but often, it's best to select based on the :nth-child pseudoselector. This depends on how the elements are arranged in the DOM, though, so it's not as general of a solution as array access.

QuerySelector in Puppeteer doesn't find element (returns null)

I'm trying to access elements on a website with Puppeteer. Following is my minimal working example. The output of the page.evaluate is null when I try to get the span element. When I change it to the p element it correctly outputs an emtpy object. Is this because the span element has display: none ? What do I have to do to get the span element?
(async() => {
const browser = await puppeteer.launch({
headless: true,
ignoreHTTPSErrors: true,
timeout: 1000
});
const page = await browser.newPage();
await page.goto('https://example.com', {waitUntil: 'networkidle2'});
const elem = await page.evaluate(() => {
const elem = document.querySelector('span');
return elem;
})
console.log("ELEM", elem)
browser.close();
})();
An additional span element was being added by an extension to the Page DOM, which resulted in the incorrect selector specified for document.querySelector().
The correct selector, based on the information given, is 'p':
const elem = document.querySelector('p');

Categories