In my app when I search in the search bar, a dropdown menu would appear, and when I click on the dropdown-item, new info is parsed using innerHTML method.
However, the associated event handler is also destroyed. I'm trying to find ways to replace innerHTML but haven't been able to.
The thing is, inside innerHTML it is a whole block of HTML code, including tags and texts. Plus, whenever a new dropdown item is clicked, the previous HTML code should be removed and new ones parsed in.
I've been trying with removeChild() then appendChild() or insertAdjacentHTML(), to no avail.
Weirdly, it seems even with insertAdjacentHTML() the event handler is still destroyed. (I tried insertAdjacentChild() without removing the previous code, and event handler works only on the previous item, not the item added by insertAdjacentChild()).
Original code:
async function init (name) {
await fetchCurrentData(name)
await weatherBackground(name)
const forecastHoursList = await hoursList(name)
document.querySelector('.forecast ul').innerHTML = renderHourlyForecast(forecastHoursList)
const forecastDailyList = await dailyList(name)
document.querySelector('.future ul').innerHTML = renderDailyForecast(forecastDailyList)
}
What I tried but didn't work:
async function init (name) {
await fetchCurrentData(name)
await weatherBackground(name)
const forecast = document.querySelector('.forecast ul')
const future = document.querySelector('.future ul')
const forecastHoursList = await hoursList(name)
const forecastChildren = forecast.childNodes
for(var i = 0; i < forecastChildren.length; i++) {
forecast.removeChild(forecastChildren[i]);
}
forecast.insertAdjacentHTML('afterbegin', renderHourlyForecast(forecastHoursList))
const forecastDailyList = await dailyList(name)
const futureChildren = future.childNodes
for(var i = 0; i < futureChildren.length; i++) {
future.removeChild(futureChildren[i]);
}
future.insertAdjacentHTML('afterbegin', renderDailyForecast(forecastDailyList))
}
Related
I try to remove elements by classname but it doesn't work.
This is the code i used:
await page.screenshot({path: 'pic.png'}); //for testing purposes
let div_selector_to_remove= ".xj7.Kwh5n";
var thingToRemove = document.querySelectorAll(div_selector_to_remove);
var number_of_elements = thingToRemove.length;
for (var i = 0; i < number_of_elements.length; i++) {
thingToRemove[i].parentNode.removeChild(thingToRemove);
}
The browser loads and i get a screenshot with the elements loaded. The nothing happens. The elements remain there
Run document.querySelector inside page.evaluate. Here's my answer:
await page.goto('<url_here>');
let div_selector_to_remove= ".xj7.Kwh5n";
await page.evaluate((sel) => {
var elements = document.querySelectorAll(sel);
for(var i=0; i< elements.length; i++){
elements[i].parentNode.removeChild(elements[i]);
}
}, div_selector_to_remove)
Probably easier...
Remove the first node matching selector:
await page.$eval(selector, el => el.remove());
Remove all nodes matching selector:
await page.$$eval(selector, els => els.forEach(el => el.remove()));
If for some reason you need to select and remove in console browser-land:
const selector = ".foo";
// single
await page.evaluate(`
document.querySelector("${selector}").remove()
`);
// multiple
await page.evaluate(selector =>
document.querySelectorAll(selector).forEach(el => el.remove()),
selector
);
You can hardcode the selector into the eval string/function as well but I figured it'd be useful to show it coming from a variable in two different ways just in case.
As with anything in Puppeteer, understanding which code runs in Node/Puppeteer-land and which code runs in browser/console-land is extremely important. The rule of thumb is: if it's a callback or stringified function body, it's in the browser and you can only use browser/DOM concepts like HTMLElement.remove(), window and document, otherwise it runs in Node and you can only call Puppeteer API functions. In OP's case, it looks like we're outside of a callback in Node-land, so document isn't a thing, only Puppeteer page. functions.
First, number_of_elements is number.
But you call number_of_elements.length.
Next, thingToRemove[i].parentNode.removeChild(thingToRemove), thingToRemove[i].parentNode is parent of thingToRemove[i], not thingToRemove,
so you can't remove thingToRemove from thingToRemove[i].parentNode.
I think this code may be useful.
await page.screenshot({path: 'pic.png'}); //for testing purposes
let div_selector_to_remove= ".xj7.Kwh5n";
var thingToRemove = document.querySelectorAll(div_selector_to_remove);
var number_of_elements = thingToRemove.length;
for (var i = 0; i < number_of_elements; i++) {
thingToRemove[i].parentNode.removeChild(thingToRemove[i]);
}
//Wait for the element with id="xe7COe" to show up
await page.waitForSelector('#xe7COe');
//delete the element with id="xe7COe"
await page.evaluate(() => document.querySelector('#xe7COe').remove());
Im making a firefox addon that shows the definitions for words in the subtitles of youtube videos, its almost done, but the buttons to hide the definition (removeFromView()) and add to an exclusion list (addToExclude()) only work in the most recent set of data.
removeFromView() and addToExclude() are 2 functions that are linked to 2 buttons by event listeners.
I thought it might have something to do with iteratively creating event listeners like here: addEventListener using for loop and passing values, but for the latest group of 3-5 (technically 6-10 because there are 2 listeners per iteration) event listeners they all work for their individual sets of data.
It appears as if everytime the callback from browser.storage.local.onChanged.addListener( runs it invalidates the event listeners added within the setTimeout for all the previous times the callback from browser.storage.local.onChanged.addListener( had ran
image of issue: https://a.tommo.team/bMdDlvCY.png
I explain better how the system works in the code comments:
browser.storage.local.onChanged.addListener(async () => { // When new data (text) is seen in storage this runs
let val = await browser.storage.local.get("value"); // Gets the text, for example it could be: "going to skip that so now we're moving"
val = val["value"].split(" "); // Splits it into an array of words
let body = document.getElementById("text"); // References a div element
val.forEach((word) => { // Iterates over each word
if (!(word.includes("-") || word.includes("'"))) { // If the word doesnt contain special characters continue
let valid = true;
for (let i = 0; (i < exclude.length && valid); i++) valid = exclude[i] !== word; // Detect if the word is in the exclusion list
if (valid) { // Continue if the word isnt in the exclusion list
let url = "https://api.dictionaryapi.dev/api/v2/entries/en/" + word; // URL for dictionary lookup
fetch(url)
.then(response => response.json()) // IDK why but this is needed for the following to work
.then(data => {
let def = data[0]["meanings"][1]["definitions"][0]["definition"]; // Extract the definition from the fetch response
// The following line adds the word and definition within a div with some buttons
body.innerHTML += `<div id='${word}'>${word} : ${def} <button id='${word+"a"}'>-</button><button id='${word+"b"}'>X</button></div>`;
setTimeout(() => { // Give some time for the added html to load
// Make the buttons a different color to show that this function ran
document.getElementById((word+"a")).style.backgroundColor = "yellow";
document.getElementById((word+"b")).style.backgroundColor = "red";
// Add an event listener to each button
document.getElementById((word+"a")).addEventListener("click", () => removeFromView(word));
document.getElementById((word+"b")).addEventListener("click", () => addToExclude(word, word));
}, 1000)
})
.catch(err => console.log(err))
}
}
})
})
Heres the code for the 2 other functions aswell:
let removeFromView = (time) => {
document.getElementById(time).remove() // Remove word definition and buttons from view
}
let addToExclude = async (word, time) => {
removeFromView(time)
let data = await browser.storage.sync.get("exclude"); // Get exclusion list
data = data["exclude"]; // Get exclusion list
data.push(word); // Update exclusion list
await browser.storage.sync.set({"exclude": data}); // Save exclusion list
}
So i'm attempting to write a google parser.
The idea of my tool is it takes search queries and searches google for them and returns URLs. It is working good so far but now im trying to set a page configuration and im having troubles, my code is:
const needle = require("needle") //for making get request
const sp = require("serp-parser") //for parsing data from the request
const queryup = "watch movies online free" //my search data
const query = encodeURI(queryup) //my search data so google can read it
var page = 0; //initializing the page counter
let pages = 5; //setting amount of pages to loop through
for (var i = 0; i < pages; i++) { //my loop
needle.get(`https://www.google.com/search?q=${query}&start=${page}`, function(err, response){ //MY MAIN PROBLEM <<<--- The issue is its adding to the page value but its not effecting it here, why?
page += 10 //adding to page value (every 10 page value is 1 extra page)
console.log(`----- Page number: `+ page / 10+" -----") //logging the number of the page to confirm that it is indeed increasing the page value
let results = response.body; //defining the body of my request
parser = new sp.GoogleNojsSERP(results); //initializing the parser
let parsed = parser.serp //parsing the body
let objarray = parsed.organic; //parsed body (returns as an array of json objects)
for (var i = 0; i < objarray.length; i++) { //loop the logging of each url
let url = objarray[i].url //defining url
console.log(url) //logging each url
}
});
}
without a billion comments:
const needle = require("needle")
const sp = require("serp-parser")
const queryup = "watch movies online free"
const query = encodeURI(queryup)
var page = 0;
let pages = 5;
for (var i = 0; i < pages; i++) {
needle.get(`https://www.google.com/search?q=${query}&start=${page}`, function(err, response){
//^^^^^ MY MAIN PROBLEM <<<--- The issue is its adding to the page value but its not effecting it here, why?
page += 10
console.log(`----- Page number: `+ page / 10+" -----")
let results = response.body;
parser = new sp.GoogleNojsSERP(results);
let parsed = parser.serp
let objarray = parsed.organic;
for (var i = 0; i < objarray.length; i++) {
let url = objarray[i].url
console.log(url)
}
});
}
This seems to be an issue with async.
I'm not familiar with needle, but I know that external queries are basically never synchronous.
The problem you're experiencing is basically, the actual web query is happening after the loop first runs and has already incremented page to 50. Then, 5 queries are constructed, each one with page=50, because async is complicated and difficult to manage.
Under the hood, the engine is essentially doing literally everything else it can possibly do first, and THEN doing your web queries.
A trip through the needle npm docs tells me that you can use alternative syntax to get needle to return a promise instead, which can then be wrapped in an asynchronous function and managed through await to force synchronous behavior, which is what you're after:
const needle = require('needle');
const sp = require('serp-parser');
const queryup = 'watch movies online free';
const query = encodeURI(queryup);
let page = 0;
const pages = 5;
const googler = async function () {
for (let i = 0; i < pages; i++) {
try {
const response = await needle('get', `https://www.google.com/search?q=${query}&start=${page}`);// MY MAIN PROBLEM <<<--- The issue is its adding to the page value but its not effecting it here, why?
console.log('----- Page number: ' + page / 10 + ' -----');
const results = await response.body;
const parser = new sp.GoogleNojsSERP(results);
const parsed = parser.serp;
const objarray = parsed.organic;
for (let i = 0; i < objarray.length; i++) {
const url = objarray[i].url;
console.log(url);
}
} catch (err) {
console.error(err);
}
page += 10;
}
};
googler();
The key differences:
Per the needle docs, rather than the request method being a method on the needle object, it's instead the first argument you pass directly to invoking needle itself as a function.
When you manage promises with await, a rejected promise throws an error that should be caught with a traditional try/catch block; I've done that here. Though, if needle is anything like node-fetch it probably basically never throws errors, but it's good practice.
One of my extensions automatically changed your var declarations to let and not-reassigned let declarations to const; you're welcome to change them back.
This is a classic asynchronous problem. Add another console.log() immediately before the needle.get() call (and after the for statement) and you will see what is going wrong: All of the needle.get() calls execute before any of the callbacks where you do the page += 10. Then, after the for loop completes, all of the callbacks are executed. But it is too late for this to have any effect on the start= parameter.
One way to fix this could be to move the body of this for loop (the needle.get() and its callback) into a separate function. Initialize your variables and call this function once. Then at the end of the callback, do your page += 10 and update any other variables you need to, and call this function again from there if there are more pages left that you want to load. If you have completed all of the pages, then don't make that call. The for loop is not needed with this technique.
Or, you could keep your current code but move the page += 10 after the callback but still inside the outer for loop. That way this variable will be incremented as you expect. I don't necessarily recommend this, as Google may get unhappy about receiving the get requests so rapidly and may start blocking your calls or throwing CAPTCHAs at you.
There may be an issue of whether this kind of scraping is allowed by Google's Terms of Service, but I will leave that question to you and your legal advisors.
Also, I would avoid using var anywhere. Use const or let instead, and prefer const over let except when you need to reassign the variable.
One tip: in most cases where you use a numeric for loop to iterate over an array, the code will be cleaner if you use a for..of loop. For example, this bit of code:
let parsed = parser.serp
let objarray = parsed.organic;
for (var i = 0; i < objarray.length; i++) {
let url = objarray[i].url
console.log(url)
}
could be more simply written as:
for (const result of parser.serp.organic) {
console.log(result.url)
}
(I know that is just a bit of debug code, but this is a good habit to get into.)
Finally, watch your indentation and be sure to indent nested blocks or functions. I took the liberty of adding some indentation for you.
I am currently building a to-do list app and I have already saved the data using the localStorage API.
The desired data, however, appears in the console whenever I console.log it but it still doesn't save in the DOM.
I also added the getItem() function and logged it into the console and I can still view it in the console, find here:
getItem content in the console
But it just doesn't store in the browser
Seeing this, it is in your inclination that it should have stored in the DOM and the content still remains after reloading but that just isn't the case here.
This function below adds a new item to the list, it also deletes and crosses out completed items too:
let id = 0;
function addTaskFunc() {
const aTask = `
<div class="task" id="task-${id}">
<button class="done__btn">
<i class="far fa-check-square"></i>
</button>
<p>${box.value}</p>
<button class="priority">Make priority</button>
<button class="cancel__btn">
<i class="far fa-times-circle"></i>
</button>
</div>
`;
const x = box.value;
if (x) {
taskList.insertAdjacentHTML('afterbegin', aTask);
box.value = '';
;
let cid = id;
const cancelBtn = document.querySelector(`#task-${id} .cancel__btn`);
cancelBtn.addEventListener('click', () => {
deleteItem(cid);
});
let cid2 = id;
// code for marking out an item as done
let cid3 = id;
// code for appending an item to the top of DOM
let cid4 = id;
persistData(cid4);
readData(cid4);
id++;
}
}
newTask.addEventListener('click', addTaskFunc); // Button to activate the function
persistData = id => {
const el = document.querySelector(`#task-${id}`);
localStorage.setItem('addedTasks222', el.innerHTML);
};
readData = id => {
const el = document.querySelector(`#task-${id}`);
const saved = localStorage.getItem('addedTasks222');
if (saved) el.innerHTML = saved;
console.log(el.innerHTML); // This line of code appears in the console
}
I also tried doing it this way inside of the addTaskFunc:
const r = document.querySelector(`#task-${id} .task`);
r.addEventListener('load', () => {
persistData(cid4);
readData(cid4);
});
When I try it with the method above I get the error code in the console:
Cannot read property of addEventListener of null.
I feel there is something wrong somewhere but I just cannot seem to find out where I am missing it.
One last thing, the localStorage only seems to store only one item into the key. Do I need a loop to sort that out?
you can store an array in the local storage like this
localStorage.setItem('array', JSON.stringify(YOURARRAY))
and then you can load that with
var restoredArray = JSON.parse(localStorage.getItem('array'));
sorry for the time..
So I was looking in to your code, and appears that you have some problems..
1- The first one I saw, was in the persistData() function. Accidentally you are just selecting one task to store with document.querySelector(.task);
to select all you need to use document.querySelectorAll(.task), this will return you an array. Because of that you only store one task.
2- Secondly is that you are trying to store html. with .innerHtml, and the innerHtml of your (".class") is buttons and etc.. You should store values.
3- At the end, when you are trying to print the data, you do document.querySelector(.task), and as you can see its returning you undefined, that's because you haven't a .task div yet.
So How you can appropriately make your app work.
1- The first thing you need to do is creating an array variable up on your js file.
let tasks = [];
2-That array will be used to store the values of your tasks with something like this
function addItem(){
let value = item.value; //you need to define the item some way
tasks.push(value);
print() //a function to print the values;
persistData() // store the array
}
3-To print the tasks
function print(){
tasks.forEach(task => {
let div = document.createElement("div");
div.innerHTML = `copy your html task to here, use ${task} to print the task value`
})
}
4-Store the tasks
function persistData(){
localStorage.setItem('addedTasks', tasks); //store the array
};
function readData(){
if (typeof(Storage) !== "undefined") { //check if the browser has localStorage
tasks = localStorage.getItem('addedTasks'); //update your array
print();
} else {
//No Web Storage message
}
};
5- Just run readData() when you load your document with
document.addEventListener("DOMContentLoaded", function(event) {
readData();
});
I hope that I can help
I try to remove elements by classname but it doesn't work.
This is the code i used:
await page.screenshot({path: 'pic.png'}); //for testing purposes
let div_selector_to_remove= ".xj7.Kwh5n";
var thingToRemove = document.querySelectorAll(div_selector_to_remove);
var number_of_elements = thingToRemove.length;
for (var i = 0; i < number_of_elements.length; i++) {
thingToRemove[i].parentNode.removeChild(thingToRemove);
}
The browser loads and i get a screenshot with the elements loaded. The nothing happens. The elements remain there
Run document.querySelector inside page.evaluate. Here's my answer:
await page.goto('<url_here>');
let div_selector_to_remove= ".xj7.Kwh5n";
await page.evaluate((sel) => {
var elements = document.querySelectorAll(sel);
for(var i=0; i< elements.length; i++){
elements[i].parentNode.removeChild(elements[i]);
}
}, div_selector_to_remove)
Probably easier...
Remove the first node matching selector:
await page.$eval(selector, el => el.remove());
Remove all nodes matching selector:
await page.$$eval(selector, els => els.forEach(el => el.remove()));
If for some reason you need to select and remove in console browser-land:
const selector = ".foo";
// single
await page.evaluate(`
document.querySelector("${selector}").remove()
`);
// multiple
await page.evaluate(selector =>
document.querySelectorAll(selector).forEach(el => el.remove()),
selector
);
You can hardcode the selector into the eval string/function as well but I figured it'd be useful to show it coming from a variable in two different ways just in case.
As with anything in Puppeteer, understanding which code runs in Node/Puppeteer-land and which code runs in browser/console-land is extremely important. The rule of thumb is: if it's a callback or stringified function body, it's in the browser and you can only use browser/DOM concepts like HTMLElement.remove(), window and document, otherwise it runs in Node and you can only call Puppeteer API functions. In OP's case, it looks like we're outside of a callback in Node-land, so document isn't a thing, only Puppeteer page. functions.
First, number_of_elements is number.
But you call number_of_elements.length.
Next, thingToRemove[i].parentNode.removeChild(thingToRemove), thingToRemove[i].parentNode is parent of thingToRemove[i], not thingToRemove,
so you can't remove thingToRemove from thingToRemove[i].parentNode.
I think this code may be useful.
await page.screenshot({path: 'pic.png'}); //for testing purposes
let div_selector_to_remove= ".xj7.Kwh5n";
var thingToRemove = document.querySelectorAll(div_selector_to_remove);
var number_of_elements = thingToRemove.length;
for (var i = 0; i < number_of_elements; i++) {
thingToRemove[i].parentNode.removeChild(thingToRemove[i]);
}
//Wait for the element with id="xe7COe" to show up
await page.waitForSelector('#xe7COe');
//delete the element with id="xe7COe"
await page.evaluate(() => document.querySelector('#xe7COe').remove());