We Keep Coding

remove element by selector in puppeteer [duplicate]

I try to remove elements by classname but it doesn't work.
This is the code i used:
await page.screenshot({path: 'pic.png'}); //for testing purposes
let div_selector_to_remove= ".xj7.Kwh5n";
var thingToRemove = document.querySelectorAll(div_selector_to_remove);
var number_of_elements = thingToRemove.length;
for (var i = 0; i < number_of_elements.length; i++) {
thingToRemove[i].parentNode.removeChild(thingToRemove);
}
The browser loads and i get a screenshot with the elements loaded. The nothing happens. The elements remain there

Run document.querySelector inside page.evaluate. Here's my answer:
await page.goto('<url_here>');
let div_selector_to_remove= ".xj7.Kwh5n";
await page.evaluate((sel) => {
var elements = document.querySelectorAll(sel);
for(var i=0; i< elements.length; i++){
elements[i].parentNode.removeChild(elements[i]);
}
}, div_selector_to_remove)

Probably easier...
Remove the first node matching selector:
await page.$eval(selector, el => el.remove());
Remove all nodes matching selector:
await page.$$eval(selector, els => els.forEach(el => el.remove()));
If for some reason you need to select and remove in console browser-land:
const selector = ".foo";
// single
await page.evaluate(`
document.querySelector("${selector}").remove()
`);
// multiple
await page.evaluate(selector =>
document.querySelectorAll(selector).forEach(el => el.remove()),
selector
);
You can hardcode the selector into the eval string/function as well but I figured it'd be useful to show it coming from a variable in two different ways just in case.
As with anything in Puppeteer, understanding which code runs in Node/Puppeteer-land and which code runs in browser/console-land is extremely important. The rule of thumb is: if it's a callback or stringified function body, it's in the browser and you can only use browser/DOM concepts like HTMLElement.remove(), window and document, otherwise it runs in Node and you can only call Puppeteer API functions. In OP's case, it looks like we're outside of a callback in Node-land, so document isn't a thing, only Puppeteer page. functions.

First, number_of_elements is number.
But you call number_of_elements.length.
Next, thingToRemove[i].parentNode.removeChild(thingToRemove), thingToRemove[i].parentNode is parent of thingToRemove[i], not thingToRemove,
so you can't remove thingToRemove from thingToRemove[i].parentNode.
I think this code may be useful.
await page.screenshot({path: 'pic.png'}); //for testing purposes
let div_selector_to_remove= ".xj7.Kwh5n";
var thingToRemove = document.querySelectorAll(div_selector_to_remove);
var number_of_elements = thingToRemove.length;
for (var i = 0; i < number_of_elements; i++) {
thingToRemove[i].parentNode.removeChild(thingToRemove[i]);
}

//Wait for the element with id="xe7COe" to show up
await page.waitForSelector('#xe7COe');
//delete the element with id="xe7COe"
await page.evaluate(() => document.querySelector('#xe7COe').remove());

Replace innerHTML with another method to avoid destroying event handler

In my app when I search in the search bar, a dropdown menu would appear, and when I click on the dropdown-item, new info is parsed using innerHTML method.
However, the associated event handler is also destroyed. I'm trying to find ways to replace innerHTML but haven't been able to.
The thing is, inside innerHTML it is a whole block of HTML code, including tags and texts. Plus, whenever a new dropdown item is clicked, the previous HTML code should be removed and new ones parsed in.
I've been trying with removeChild() then appendChild() or insertAdjacentHTML(), to no avail.
Weirdly, it seems even with insertAdjacentHTML() the event handler is still destroyed. (I tried insertAdjacentChild() without removing the previous code, and event handler works only on the previous item, not the item added by insertAdjacentChild()).
Original code:
async function init (name) {
await fetchCurrentData(name)
await weatherBackground(name)
const forecastHoursList = await hoursList(name)
document.querySelector('.forecast ul').innerHTML = renderHourlyForecast(forecastHoursList)
const forecastDailyList = await dailyList(name)
document.querySelector('.future ul').innerHTML = renderDailyForecast(forecastDailyList)
}
What I tried but didn't work:
async function init (name) {
await fetchCurrentData(name)
await weatherBackground(name)
const forecast = document.querySelector('.forecast ul')
const future = document.querySelector('.future ul')
const forecastHoursList = await hoursList(name)
const forecastChildren = forecast.childNodes
for(var i = 0; i < forecastChildren.length; i++) {
forecast.removeChild(forecastChildren[i]);
}
forecast.insertAdjacentHTML('afterbegin', renderHourlyForecast(forecastHoursList))
const forecastDailyList = await dailyList(name)
const futureChildren = future.childNodes
for(var i = 0; i < futureChildren.length; i++) {
future.removeChild(futureChildren[i]);
}
future.insertAdjacentHTML('afterbegin', renderDailyForecast(forecastDailyList))
}

Playwright - Find multiple elements or class names

I have read a few different QAs related to this but none seem to be working.
I am trying to target an element (Angular) called mat-radio-button with a class called mat-radio-checked. And then select the inner text.
In Chrome this is easy:
https://i.stack.imgur.com/Ev0iQ.png
https://i.stack.imgur.com/lVoG3.png
To find the first element that matches in Playwright I can do something like:
let test: any = await page.textContent(
"mat-radio-button.mat-radio-checked"
);
console.log(test);
But if I try this:
let test: any = await page.$$(
"mat-radio-button.mat-radio-checked"
);
console.log(test);
console.log(test[0]);
console.log(test[1]);
});
It does not return an array of elements I can select the inner text of.
I need to be able to find all elements with that class so I can use expect to ensure the returned inner text is correct, eg:
expect(test).toBe("Australian Citizen");

I found out the issue was due to the page generating beforehand and the elements were not available. So I added a waitForSelector:
await page.waitForSelector("mat-radio-button");
const elements = await page.$$("mat-radio-button.mat-radio-checked");
console.log(elements.length);
console.log(await elements[0].innerText());
console.log(await elements[1].innerText());
console.log(await elements[2].innerText());

public async validListOfItems(name) {
await this.page.waitForSelector(name, {timeout: 5000});
const itemlists = await this.page.$$(name);
let allitems: string[]=new Array();
for await (const itemlist of itemlists) {
allitems.push(await itemlist.innerText());
}
return allitems;
}
Make a common method and call by sending parameters.

get all spans and click them with puppeteer - fails with "node not visible' and other errors

I have a HTML page that has many div elements, each one with the following structure (the input id and name changes):
<div class="item">
<div class="box">
<div class="img-block">
<label for="check-11">
<input id="check-11" name="result11" type="checkbox">
<span class="fake-input"></span>
</label>
</div>
</div>
</div>
I want to use puppeteer to get all the span with the 'fake-input' class and click on them.
The problem is that it never works, no matter what I try.
In every attempt the start is the same:
(async () => {
const browser = await puppeteer.launch({ headless: false });
const page = await browser.newPage();
await page.goto(baseUrl, { waitUntil: 'networkidle2' });
// FETCHING AND CLICKING
}();
I tried many things:
1:
await page.waitForSelector('span.fake-input');
await page.click('span.fake-input');
2:
await page.waitForSelector('span.fake-input')
.then(()=>{
console.log(`clicked!`);
page.click('span.fake-input')
3:
const spans = await page.evaluate(() => {
return Array.from(document.querySelectorAll('span'), el => el.textContent)
})
console.log('spans', spans)
for (let index = 0; index < 7; index++) {
const element = spans[index];
await page.click('span')
}'=
4:
await page.evaluate(selector=>{
return document.querySelector(selector).click();
},'span.fake-input)
console.log('clicked');
In every solution the page fails to get anything at all (either return null or undefined, so the error is "click" is not a funciton in null) or it fails with the error "Node is either not visible or not an HTMLElement".
No matter the error, in any case I fail to fetch all the spans, and click on them.
Can anyone tell me what I'm doing wrong?

Use page.$$ to return multiple elements (equivalent of document.querySelectorAll). Use page.$ to return a single element (equivalent of document.querySelector).
If you want to extract a certain value from a group of elements, use page.$$eval and page.$eval for a single element.
e.g. return elementHandle to script
const spans = await page.$$('div#item label .fake-input')
spans.forEach(span=>span.click())
If you extracting a value from an element, pass a callback to it that returns what you need to extract
e.g.
const spanTexts = page.$$eval('div#item label .fake-input', spans => {
spans.map(span=>span.innerText)
})
console.log(spanTexts)
I should add that page.$$eval and page.$eval executes your callback in the browser context.

var obj = document.querySelectorAll("span.fake-input");
for(var i=0;i<obj.length;i++){
obj[i].click();
}
Vanilla JavaScript would work much easier

TestCafe using client Functions to get data from a HTML dataTable

I am trying to scrape data from a bricklet in the UI(i.e. HTML dataTable) and using a testCafe client function to do this but I haven't been successful. I have a few questions about my code and would like someone to point me in the right direction.
I first put my client function in the test file(test.js) which houses all my other test cases and called the function from one of my tests. Just like this example here: - https://devexpress.github.io/testcafe/documentation/test-api/obtaining-data-from-the-client/examples-of-using-client-functions.html check section "complex DOM queries" but testCafe gets stuck, the browser closes but the execution is stuck
Here is my client function. It is in my file that houses all my tests - test.js
fixture`Getting Started`
.page`${config.baseUrl}`;
const getTableRowValues = ClientFunction(() => {
console.log("inside client function");
const elements = document.querySelector('#bricklet_summary_studtable > tbody').querySelectorAll('tr td');
const array = [];
console.log(elements.length);
for (let i = 0; i <= elements.length; i++) {
console.log("inside for");
const customerName = elements[i].textContent;
array.push(customerName);
}
return array;
});
Here is my test case:
test('My 4th test - Check the bricklet data matches the expected data', async t => {
await t.navigateTo('https://myurl.com/app/home/students');
await page_studTest.click_studentlink();
await t
.expect(await page_studTest.exists_ListPageHeader()).ok('do something async', { allowUnawaitedPromise: true })//check the compare button does not exists
await t .navigateTo('https://myurl.com/app/home/students/application/stud/id/details/test.html')
await t
.expect(await page_studTest.getText_studHeader(t)).eql('student123',
"the header text does not match");
let arr = await getTableRowValues();
await console.log(arr);
});
I thought this will get the values from the UI in an array and I will compare it to another array of test values that I will hard code later.
At first, I tried client functions in my page class(page object model: https://devexpress.github.io/testcafe/documentation/recipes/use-page-model.html) and I put the client function in the constructor and called it from a async function in same page class and called the async function from my test.js. All my tests are structured this way but this only prints the following in the console
Valuesfunction __$$clientFunction$$() {
const testRun = builder._getTestRun();
const callsite = (0, _getCallsite.getCallsiteForMethod)(builder.callsiteNames.execution);
const args = [];
// OPTIMIZATION: don't leak `arguments` object.
for (let i = 0; i < arguments.length; i++) args.push(arguments[i]);
return builder._executeCommand(args, testRun, callsite);
}
which is not useful to debug the problem.
There are no examples on testCafe site as to how/where to put the client function when you use the page-object model. Could someone, please share some insight? I am interested in knowing the best way to structure my tests.

I didn't find any problems in your code which can make TestCafe hang. I didn't find any syntax errors or incorrect calls to TestCafe methods either. I only wish that you take note that the await keyword should not be called before console.log. Though this should not lead to any issues.
 
Probably the use of a custom promise with the { allowUnawaitedPromise: true } option can lead to problems, however it's difficult to determine it without the full project.
I recommend you prepare a simple project with a sample test file to demonstrate the issue and create a separate bug report in the TestCafe repository using the following form

So, finally I tried to return a promise from my client function and then it worked.
const getTableRowValues = ClientFunction(() => {
const array = [];
return new Promise(resolve => {
var elements = document.querySelectorAll('#bricklet_summary_studtable > tbody > tr > *');
elements.forEach(function (element, i) {
let text = element.textContent;
array[i] = text.trim();
});
resolve(array);
});
});
I resolve a single dimensional array as the assertion wasn't working with a 2D array in the test when I compare the result of the client function to a 2D expected value.However this serves the purpose for now.