this functionality is new to me, im building this with the help of a tutorial but since the tutorial fetched a different set of data i got lost.
const puppeteer = require('puppeteer');
const fs = require('fs');
(async () => {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto('https://br.tradingview.com/symbols/TVC-DXY/',{
waitUntill: 'load',
timeout: 0
});
/*const element = await page.$(".tv-symbol-header__first-line");
const text = await page.evaluate(element => element.textContent, element);*/
const textNode = await page.evaluate(()=>{
const nodeText = document.querySelector(".tv-symbol-price-quote__change-value").innerText;
const text = [nodeText];
return text
});
fs.writeFile('arreglo.json', JSON.stringify(textNode), err =>{
if (err) throw new Error ('algo deu errado')
console.log('deu certo')
})
//await browser.close();
})();
<script>
(async() => {
const response = await fetch('./arreglo.json')
const data = await response.json();
document.querySelector(".container").innerHTML = data
})();
</script>
the first part is index.js
the second piece of code is in the html script.
the file arreglo.json is created and the result is like this:
["+0.887"]
i just want the 0.887, but i could format, not a problem, but i cant seem to get it on the html page.
It would be helpful to see the rest of your HTML code, or at least where you have placed the script tag as you have used a querySelector as part of the fetch request to render the data. Javascript code runs in order so if your script tag is before the ".container" element then your script will run before the ".container" element is available to render the data.
If you haven't already then I would place your script tags at the bottom of your HTML page so that all static content is rendered before any scripts start to amend it:
<html lang="en">
<head>
<meta charset="UTF-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Document</title>
</head>
<body>
<div class="container">
Content here...
</div>
<script>
(async() => {
const response = await fetch('./arreglo.json')
const data = await response.json();
document.querySelector(".container").innerHTML = data
})();
</script>
</body>
</html>
Related
I want to click a button in a shadow DOM inside an iframe. Is there a way to do this?
<html>
<head></head>
<body>
<iframe class="iframe_1">
#document
<div class="shadow-root">
#shadow-root (open)
<div>
<button id="btn_1"></button>
<button id="btn_2"></button>
</div>
</iframe>
</body>
</body>
I did this:
const frameHandle = await page.$("iframe.iframe_1");
const frame = await frameHandle.contentFrame();
var button = await frame.querySelector(".shadow-root").shadowRoot.querySelector("button[id='btn_1']");
await button.click();
But got the following error:
Uncaught TypeError TypeError: frame.querySelector is not a function
I know why this error has occurred, but I can't come up with other ideas. Please teach me.
The error is telling you that you're trying to call frame.querySelector, but querySelector is not a function that exists on frame. DOMElement#querySelector is defined in the browser only, which you can access via a frame using an evaluate-family call.
For example, if you want to return an element, you can use evaluateHandle:
const button = await frame.evaluateHandle(() =>
document.querySelector(".shadow-root")
.shadowRoot
.querySelector("#btn_1")
);
Here's a minimal, complete example you can run and see in action on your provided DOM structure:
index.html:
<!DOCTYPE html>
<html><body>
<iframe class="iframe_1"></iframe>
<script>
const html = `<!DOCTYPE html>
<div class="shadow-root"></div>
<script>
const el = document.querySelector(".shadow-root");
const root = el.attachShadow({mode: "open"});
el.shadowRoot.innerHTML = \`
<button id="btn_1">A</button>
<button id="btn_2">B</button>
\`;
el.shadowRoot.querySelector("#btn_1").addEventListener("click", event => {
event.target.textContent = "clicked";
});
<\/script>
`;
const doc = document.querySelector("iframe").contentWindow.document;
doc.open();
doc.write(html);
doc.close();
</script>
</body></html>
index.js:
const fs = require("node:fs/promises");
const puppeteer = require("puppeteer"); // ^19.6.3
let browser;
(async () => {
browser = await puppeteer.launch();
const [page] = await browser.pages();
const html = (await fs.readFile("index.html")).toString();
await page.setContent(html);
const frameHandle = await page.$("iframe.iframe_1");
const frame = await frameHandle.contentFrame();
const button = await frame.evaluateHandle(() =>
document.querySelector(".shadow-root")
.shadowRoot
.querySelector("#btn_1")
);
await button.click();
const result = await button.evaluate(el => el.textContent);
console.log(result); // => clicked
})()
.catch(err => console.error(err))
.finally(() => browser?.close());
I'm trying to get data from https://wttr.in/?format=j1 to show on a webpage. I'm very new to Javascript so I hoped this would be easy but I'm struggling to get it to work, what am I doing wrong?.
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content=
"width=device-width, initial-scale=1.0" />
</head>
<body>
<p id="temp"></p>
</body>
<script>
const api_url = "https://wttr.in/?format=j1";
async function getWeather() {
const response = await fetch(api_url);
const data = await response.json();
const weather = data.results[0].current_condition[0];
let { temp } = weather.temp_C;
document.getElementById("temp").href = "temp:" + temp;
getWeather();
</script>
</html>
As per the comments from CBroe (thanks for the help) the main issues with my code were a few things.
No closing bracket
Not accessing the correct part of the JSON array
Not setting the element correctly
The working code looks like this:
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content=
"width=device-width, initial-scale=1.0" />
</head>
<body>
<p id="temp"></p>
</body>
<script>
const api_url = "https://wttr.in/?format=j1";
async function getWeather() {
const response = await fetch(api_url);
const data = await response.json();
const weather = data.current_condition[0];
let temp = weather.temp_C;
document.getElementById("temp").innerHTML = "temp:" + temp;
}
getWeather();
</script>
</html>
You are not parsing the json response correctly. The data does not have any results array, it is an object.
So to get current_condition you need to do something like this:
const currentCondition = data.current_condition[0];
Also, to get temp_C this is wrong:
let { temp } = weather.temp_C; //WRONG
// reason is that this statement means that `temp_C` is an object, and it contains `temp` property, which is not the case here. Because temp_C is a string.
So to get temp_C you need to simply do this:
let temp_C = currentCondition.temp_C;
Plus, worth mentioning here that if you are using the temp_C only for displaying purposes and not intending to reassign any value to it, then its better to use const instead of let
So it would become:
const temp_C = currentCondition.temp_C;
And if you want to use 'destructuring' you need to write it like this:
const { temp_C } = currentCondition;
Also your function is missing the closing paranthesis }.
I am beginner in web development and I am practicing my fundamental html, css and javascript skills.
I have here a code snippet of a blog-like site that I am doing as practice. I am fetching the data on https://jsonplaceholder.typicode.com/posts and https://jsonplaceholder.typicode.com/users to put in the site. In the code below, you can see how I fetch the data and how do I put them. I also put a photo of the site itself.
I just want to ask a few questions.
You can see in the code that the title and the content is in anchor tag. I want to ask if, just using vanilla javascript, there is a way I can click the title or content of the post and it will open to a new tab and it will only show that post? I think I know how to do this in Vue.js using router-link and props, but I am practicing and I am wondering if this is doable using only plain Javscript.
If you look in the article tag in the HTML, it is quite long. So instead of doing a lot of document.createElement and .appendChild, what I did was I wrote the entire article tag in the HTML. Then in the setPost function, I just selected and cloned it using .cloneNode(true) and from there, I just changed the contents based on the post in the for loop. I want to ask if this is a good way to do this or should I just manually create and append elements? How would you do this?
Thanks a lot in advance!
fetch('https://jsonplaceholder.typicode.com/posts')
.then(res => {
// console.log(res);
if(res.ok) {
return res.json();
}
return Promise.reject(res);
})
.then(data => {
// console.log('data: ', data.slice(0, 3));
const posts = data.slice(0, 3);
setPost(posts);
// getNavLink();
})
.catch(err => console.log('error: ', err))
function setPost(posts) {
const main = document.querySelector('main');
const article = document.querySelector('article');
const loading = document.querySelector('#loading');
for(post of posts) {
const clone = article.cloneNode(true);
const title = clone.querySelector('.article-title');
const author = clone.querySelector('.author');
const body = clone.querySelector('.article-content');
const date = clone.querySelector('.date');
title.href = `#${post.title}`;
// title.target = '_blank';
title.textContent = post.title;
body.textContent = post.body + post.body + post.body + post.body;
date.textContent = new Date().toLocaleString();
(() => fetch(`https://jsonplaceholder.typicode.com/users/${post.userId}`)
.then(res => res.json())
.then(data => {
author.textContent = data.name + ',';
clone.classList.remove('display');
loading.remove();
})
.catch(err => console.log(err)))();
main.appendChild(clone);
}
article.remove();
}
<div class='container container-flex'>
<main class='main-content'>
<div id='loading'></div>
<article class='display'>
<div class='article-head'>
<a href='#walapa' class='article-title'></a>
<div class='article-details art-flex'>
<span class='author subtitle'></span>
<span class='spacer subtitle'></span>
<span class='date subtitle'></span>
</div>
</div>
<div class='article-body'>
<a href='#' class='article-content'></a>
</div>
<div class='article-spacer'></div>
</article>
</main>
<aside class='sidebar'>
Tags
</aside>
</div>
You can make use of the click event on the title then inside your event function you make use of the window.open() method.
https://www.w3schools.com/jsref/met_win_open.asp
let hello=document.getElementById("hello");
hello.addEventListener("click",()=>{
window.open(url of where you are fetching the dat)
})
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Document</title>
</head>
<body>
<h1 id="hello">Hello</h1>
</body>
</html>
I'm trying to write a script to extract email id and name from this website. I tried the following snippet but it doesn't work.
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<title>foo</title>
<meta name="description" content="">
<meta name="viewport" content="width=device-width, initial-scale=1">
<link rel="stylesheet" href="">
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.3.1/jquery.min.js"></script>
</head>
<body>
<div>
<strong style="color: darkgreen;">Can read this tag</strong>
<object id="external_page" type="text/html" data="https://aleenarais.com/buddy/" width="800px" height="600px"
style="overflow:auto;border:5px ridge blue">
<!-- I want to read tag values from this object -->
</object>
</div>
<script>
window.addEventListener('load', function () {
const item = [];
$('strong[style="color: darkgreen;"]').each(function () {
item.push($(this).text())
})
console.log(item)
})
</script>
</body>
</html>
Is there any better way to do this? Or is it possible to convert the whole page into a string and extract the email using RegEx?
The email and name of in the webpage are being rendered in an iframe. The source of iframe is an external source. In order for you to extract the information, you need to use a headless browser to do that.
I would suggest using Node.JS & Puppeteer (https://www.npmjs.com/package/puppeteer)
const puppeteer = require("puppeteer");
(async() => {
const url = "https://aleenarais.com/buddy/";
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto(url, {
waitUntil: "networkidle0"
});
var frames = await page.frames();
var myframe = frames.find(
(f) => f.url().indexOf("https://feedium.app/fetchh.php") > -1
);
const textFeed = await myframe.$$eval("strong", (sElements) =>
sElements.map((el) => el.textContent)
);
console.log(textFeed.splice(1)); //Array contains both name and email
await browser.close();
})();
Puppeteer loads the page similar to how a user loads the page. It waits until all the network calls are done (see network idle0) and then it tries finding the iframe which has the url (fetchh.php). If you observe, name and email are present in strong tags and they are the only strong tags available. Hence, we are extracting the strong tags, removing the count and we are left with just the name and email.
Output:
[ 'JJ', 'j*j#gmail.com' ] //I have just masked the values but the program gives the actual ones
Steps to run the script:
Install Node.Js (https://nodejs.org/en/download/)
Install puppeteer using (npm i puppeteer)
copy the script and place it in file (demo.js)
In the terminal, navigate to the directory in which the demo.js is
present and then run node demo.js
You should see the output.
Try this:
window.addEventListener('load', function () {
let item = [];
$('strong[style*="color: darkgreen;"]').each(function (index, item) {
item.push($(this).text())
})
console.log(item)
}
I would like to retrieve the value of a counter made by a
script I've not access to, to show it on another webpage.
The remote webpage looks like this :
<!doctype html>
<html lang="fr-FR">
<head>
<script>
window.changeTargetingData = {"Count":{"total":123456}, "Week":12345};
</script>
</head>
</html>
Is it possible to get the "total" value since it is inside a script tag ? And to refresh this value, say, every hour ?
Thanks
Every time you want to check the value, you can make a request to the webpage, parse it into a document, then select the <script> tag and examine its textContent:
const textResponse = `<!doctype html>
<html lang="fr-FR">
<head>
<script>
window.changeTargetingData = {"Count":{"total":123456}, "Week":12345};
<\/script>
</head>
</html>`;
/*
fetch(url)
.then(res => res.text())
.then((textResponse) => {
*/
const doc = new DOMParser().parseFromString(textResponse, 'text/html');
const script = doc.querySelector('script');
const objJSON = script.textContent.match(/window.changeTargetingData = (.+);/)[1];
const obj = JSON.parse(objJSON);
console.log(obj.Count);
If you can't make a request directly to the site, and you're on the front-end, you'll have to bounce the request off a server which can relay it without CORS restrictions.