How to print web scrapping results to a file? - javascript

I want to save program results to file. Currently I'm printing them to console. How would I do it?
public static void main(String[] args) throws FileNotFoundException {
System.setProperty("webdriver.chrome.driver", "C:\\Users\\Dejan Kostov\\IdeaProjects\\MavenSigmaSpeakers1\\src\\main\\resources\\chromedriver-74.exe");
WebDriver driver = new ChromeDriver();
driver.get("https://sigma.world/eurasia/speakers/");
String[] expected = {"Gary Vaynerchuk", "John Lee", "Jordan Belfort", "Clinton Sparks", "Johnny Walker", "Dr. Sara Al Madani", "VESA", "Irina S. Litchfield", "Cal Evans", "Dr Jane Thomason", "Gordon Einstein", "Jimmy Nguyen", "Robert Dowling", "Aideen Shortt", "Dustin Plantholt", "Andres Meneses", "Arvin Khamseh", "Agne Linge", "Joshua Ellul", "Mike Prasad", "Paweł Łaskarzewski", "Obediah Ayton", "Jorge Sebastiao", "Wesley Ellul", "Nikita Sachdev", "Joseph F Borg", "Zack Ritchie", "Francisco Fernandez", "Gabriel Laender", "Reem Al Mosabbeh", "Cannelle Maricaux", "Onur Altan Tan", "Jitendra Vaswani", "Marco Funk", "James Bowater", "Susan Oh", "Laura K. Inamedinova", "Isabella Händel", "Tejinder Kumar", "Gustavo Montero", "Adel Bhurtun", "Vas Modinos", "Alexander Fazel", "Alexander Mihailovski", "Marc Taverner", "Pavlina Papalouka", "Emma Todd", "Cordelia Morgan Cooper", "Nick Spanos", "John Murillo", "Alex Tsepaev", "Anna Tutova", "H.E. Mr. Zulfiquar Ghadiyali", "Shafeeq Qureshi", "Bailey Dodds", "Joan de Ramón Brunet", "Amir Student", "Amna Usman Chaudhry", "Ayesha Mubarak Ali", "Nadia Dvoinos", "Mario Nawfal", "Alex J.", "Paula Tavangar", "Cinderella Amar", "Joel Michael", "Kevin Imani", "José. F. Pereira", "Paula Hajduczenia", "Dr Andreas Mathikolonis", "Alena Zharkovskaya", "Edward Condrat", "Wadih Al Sayah", "Aravind Swaminathan","Luca Tagliaferro"};
List<WebElement> elements = driver.findElements(By.tagName("h3"));
System.out.println("Number of elements with tag h3 : " + elements.size());
for (int i = 0; i < elements.size(); i++) {
WebElement element = elements.get(i);
System.out.println(i + " : " + element.getText());
if (element.getText().equals(expected[i])) {
System.out.println("Strings are equal");
} else {
System.out.println("Strings are NOT equal");
driver.quit();
}
}
}
}

Below code would write the console output to a specified file:
PrintStream printStream = new PrintStream(new File("C:\\Users\\username\\consoleOutput.txt"));
System.setOut(printStream);
Add the above 2 lines to your code anywhere before you start printing.

Related

Adding picture to randomized list

I'm adding a button to a website that randomizes NBA teams and provides a picture with the answer. I have the randomizer set up to provide the random team when the button is pushed but I'm unsure how to get a picture to go along with the team selected.
My randomizer in JS:
let teams =
{
atlanta: "Atlanta Hawks",
boston: "Boston Celtics",
brooklyn: "Brooklyn Nets",
charlotte: "Charlotte Hornets",
chicago: "Chicago Bulls",
cleveland: "Cleveland Cavaliers",
dallas: "Dallas Mavericks",
denver: "Denver Nuggets",
detroit: "Detroit Pistons",
goldenState: "Golden State Warriors",
houston: "Houston Rockets",
indiana: "Indiana Pacers",
clippers: "Los Angeles Clippers",
lakers: "Los Angeles Lakers",
memphis: "Memphis Grizzlies",
miami: "Miami Heat",
milwaukee: "Milwaukee Bucks",
minnesota: "Minnesota Timberwolves",
pelicans: "New Orleans Pelicans",
knicks: "New York Knicks",
thunder: "Oklahoma City Thunder",
orlando: "Orlando Magic",
philadelphia: "Philadelphia 76ers",
phoenix: "Phoenix Suns",
portland: "Portland Trail Blazers",
sacramento: "Sacramento Kings",
spurs: "San Antonio Spurs",
toronto: "Toronto Raptors",
utah: "Utah Jazz",
washington: "Washington Wizards"
}
const values = Object.values(teams)
app.get("/api/teams", (req, res)=>{
const randomTeam = values[Math.floor(Math.random()*values.length)]
console.log(randomTeam)
res.status(200).send(randomTeam)
})
To add team name to HTML:
<script>
document.getElementById("button").onclick = function () {
axios.get("http://localhost:5151/api/teams").then(function(response) {
const data = response.data;
document.getElementById("randomTeamCard").innerHTML = [data].map(function(team) {
return `The team you should bandwagon is: ${data}!`;
})
.join("");
}).catch(function (err) {
document.getElementById("randomTeamCard").innerHTML =
'<li class="text-danger">' + err.message + "</li>";
});
};
</script>

NodeJS - how to scrape ld+json data and save it to an object

I've been trying to find a way to get the apllication/ld+json contents and saving it to a local object. What I want to have is save it to an object, and in my program I would be able to console.log(data.offers.availability) which will result in logging: "InStock", and this for each of the data values.
I currently have this:
let content = JSON.stringify($("script[type='application/ld+json']").html())
let filteredJson = content.replace(/\\n/g, '')
let results = JSON.parse(filteredJson)
console.log(results)
Which results in this: - Doesn't let me console.log(results.offers.availability)
{ "#context": "http://schema.org/",
"#type": "Product", "name": "Apex Legends - Bangalore - Mini Epics",
"description": "<div class="textblock"><p><h2>Apex Legends - Bangalore - Mini Epics </h2><p>Helden uit alle uithoeken van de wereld strijden voor eer, roem en fortuin in Apex Legends. Weta Workshop betreedt the Wild Frontier en brengt Bangalore met zich mee - Mini Epics style!</p><p>Verzamel alle Apex Legends Mini Epics en voeg ook Bloodhound en Mirage toe aan je collectie!</p></p></div>",
"brand": {
"#type": "Thing",
"name": "Game Mania"
},
"aggregateRating": {
"#type": "AggregateRating",
"ratingValue": "5",
"ratingCount": "2"
},
"offers": {
"#type": "Offer",
"priceCurrency": "EUR",
"price": "19.98",
"availability" : "InStock"
}
}
Data im trying to scrape and save:
As Bergi pointed out, the problem is that you're using JSON.stringify on the content which is already a string, but out of curiosity I tried this myself. Consider the following test:
index.html (that is served through localhost:4000):
<html>
<script type="application/ld+json">
{
"#context": "http://schema.org",
"#type": "Product",
"name": "Apex Legends - Bangalore - Mini Epics",
"offers": {
"#type": "Offer",
"priceCurrency": "EUR",
"price": "19.98",
"availability": "InStock"
}
}
</script>
<body>
<h2>Index</h2>
</body>
</html>
NodeJS-script:
const superagent = require('superagent');
const cheerio = require('cheerio');
(async () => {
const response = await superagent("http://localhost:4000");
const $ = cheerio.load(response.text);
// note that I'm not using .html(), although it works for me either way
const jsonRaw = $("script[type='application/ld+json']")[0].children[0].data;
// do not use JSON.stringify on the jsonRaw content, as it's already a string
const result = JSON.parse(jsonRaw);
console.log(result.offers.availability);
})()
result now is an object that holds the data from the script tag and logging result.offers.availability, will print InStock as expected.

Display the length of a string on the page

I have a simple program that stores male, female and unisex names, you click a button and one is chosen at random and displayed on the page.
I am constantly adding new names into the database, so I want a simple javascript that records how many names are stored in each separate database (male, female and unisex) and displays in in a text that reads "There are currently ### males name, ### female names, and ### unisex names."
The site is live at: http://surrealmayhem.com/stuff/name_generator.html
The only things I can find online is a console.log using
var = randomOtherNames.length;
but I need it to display in the html via a span or bold tag and an ID name. I know it would be something along the lines of
document.getElementById("male_num");
but passed that, Im not sure how to put it all together and make it display on the page. Im still very new to javascript, and I dont entirely understand how the text that does display when you click the buttons even works. I feel like I might be able to reuse the script I already have, but I could use some help.
<button Onclick="click" id="other_button" style="font-size: 25px;"> Click Me </button>
<br /><br />
<b id="other_name" style="font-size: 25px;"></b>
<script type="text/javascript" charset="utf-8">
var other_string = randomOtherNames.length;
var randomOtherNames = [
"Jamie",
"Shannon",
"Kelly",
"Aeron",
"Shain",
"Kasey",
"Jordan",
"Jesse / Jessie",
"Sage",
"Evan",
"Alex",
"Stephen",
"Renee",
"Tracy",
"Avery",
"Dallas",
"Denver",
"Taylor",
"Elliot",
"Terry",
"Jeri",
"Percy",
"Raven",
"Jean",
"Jan",
"Christian",
"Adrian",
"Jude",
"Quinn",
"Piper",
"Harper",
"Payton",
"Walker",
"Cameron",
"Terran",
"Riley",
"River",
"Andren",
"Camden",
"Jiles",
"Pax",
"Rayne",
"Skylar",
"Kalin",
"Justice",
"July",
"Kensington",
"Kendall",
"Grey",
"Genesis",
"Hollis",
"Keagan",
"Kai",
"Bristol",
"Angel",
"Azure",
"Bailey",
"Carmen",
"Echo",
"Embry",
"Ember",
"Gale",
"Freedom",
"Haven",
"Kyler",
"Kylan",
"Miracle",
"Myka",
"Peace",
"Zene",
"Winter",
"Xannon",
"Valor",
"Urban",
"Valentine",
"Storm",
"Tai",
"Shae",
"Sailor",
"Red",
"Reign",
"Ramsey",
"Reed",
"Reggie",
"Ocean",
"Nalo",
"Neo",
"London",
"Ash",
"Brook",
"Crane",
"Crimson",
"Corin",
"Cresant",
"Indigo",
"Jazz",
"Cassidy",
"Kin",
"Monroe",
"Moral",
"Poet",
"Ryder",
"Saxton",
"Sidney",
"Tanner",
"Torrance",
"Torrin",
"Lex / Lexi",
"Zane",
"Zanas",
"Ari",
"Brynn",
"Greer",
"Kennedy",
"Lyle",
"Lonnie",
"Rune",
"Rylan",
"Sparrow",
"Sunny",
"Emerald",
"Cloud",
"Star",
"Kensey",
];
var other_div = document.getElementById("other_name");
document.getElementById("other_button").addEventListener("click", function() {
randomIndex = Math.ceil((Math.random()*randomOtherNames.length-1));
newText = randomOtherNames[randomIndex];
other_div.innerHTML = newText;
});
</script>
At the top, add an element that you will use to display the count. Then in your script, set the innerHTML for the new element:
var randomOtherNames = ["Jamie", "Shannon", "Kelly", "Aeron", "Shain", "Kasey", "Jordan", "Jesse / Jessie", "Sage", "Evan", "Alex", "Stephen", "Renee", "Tracy", "Avery", "Dallas", "Denver", "Taylor", "Elliot", "Terry", "Jeri", "Percy", "Raven", "Jean", "Jan", "Christian", "Adrian", "Jude", "Quinn", "Piper", "Harper", "Payton", "Walker", "Cameron", "Terran", "Riley", "River", "Andren", "Camden", "Jiles", "Pax", "Rayne", "Skylar", "Kalin", "Justice", "July", "Kensington", "Kendall", "Grey", "Genesis", "Hollis", "Keagan", "Kai", "Bristol", "Angel", "Azure", "Bailey", "Carmen", "Echo", "Embry", "Ember", "Gale", "Freedom", "Haven", "Kyler", "Kylan", "Miracle", "Myka", "Peace", "Zene", "Winter", "Xannon", "Valor", "Urban", "Valentine", "Storm", "Tai", "Shae", "Sailor", "Red", "Reign", "Ramsey", "Reed", "Reggie", "Ocean", "Nalo", "Neo", "London", "Ash", "Brook", "Crane", "Crimson", "Corin", "Cresant", "Indigo", "Jazz", "Cassidy", "Kin", "Monroe", "Moral", "Poet", "Ryder", "Saxton", "Sidney", "Tanner", "Torrance", "Torrin", "Lex / Lexi", "Zane", "Zanas", "Ari", "Brynn", "Greer", "Kennedy", "Lyle", "Lonnie", "Rune", "Rylan", "Sparrow", "Sunny", "Emerald", "Cloud", "Star", "Kensey"];
var other_string = randomOtherNames.length;
var other_div = document.getElementById("other_name");
var other_count_div = document.getElementById("other_name_count");
//populate the innerHTML of "other_count_div" on page load:
other_name_count.innerHTML = randomOtherNames.length;
var findName = function() {
randomIndex = Math.ceil((Math.random() * randomOtherNames.length - 1));
newText = randomOtherNames[randomIndex];
other_div.innerHTML = newText;
other_name_count.innerHTML = randomOtherNames.length;
};
document.getElementById("other_button").addEventListener("click", findName);
<button Onclick="click" id="other_button" style="font-size: 25px;"> Click Me </button>
<br /><br />
<b id="other_name" style="font-size: 25px;"></b>
<span id="other_name_count"></span>

how to get complex json object and render it in views in node js?

I have following json with arrays in it
In server I'm sending json like this
getTrips: function getTrips(req, res, next){
var url = '/CTB-WS/rest/trips?from='+ req.tripinfo.fromCityId + '&to=' + req.tripinfo.toCityId + '&depart-date=' + req.tripinfo.departDate+ '&pax=1';
console.log(url);
rest.get(url).on('complete', function(trips) {
if (trips instanceof Error) {
console.log('Error:', trips.message);
} else {
console.log('trips'+ JSON.stringify(trips));
console.log('onward trips'+ JSON.stringify(trips['onwardTrips']));
trips = trips || [];
req.trips = trips['onwardTrips'];
next();
}
});
},
sendTrips: function sendTrips(req, res, next){
res.render('trips', { trips: req.trips});
}
In view , I'm able to catch trip id, but not pickupPointDetails and dropoffPointDetails which are inside array, how can I render it in view?
extends layout
block content
h3 Trip Selection
form.form-horizontal(id="Findtrips", accept-charset="UTF-8", action="", method="post" enctype="multipart/form-data")
each trip, key in trips
p
a(href="") #{trip.tripId} #{key} #{trips.length}
p= trip.pickupPointDetails[0].pickupPointId //[0] [1] works but when i give key as value Unexpected token =
JSON object
{
"onwardTrips": [
{
"tripId": "1285758",
"fromCity": "Singapore",
"toCity": "Shah Alam",
"operatorCode": "SA",
"operatorName": "Starmart Express",
"departTime": "2014-01-24 11:30:00.0",
"busType": "Executive",
"pickupPointDetails": [
{
"pickupPointId": "78",
"departureTime": "2014-01-24 11:30:00.0",
"pickupPointName": "Golden Mile Tower, Beach Road"
}
],
"dropoffPointDetails": [
{
"dropOffPointName": "Shah Alam Bus Terminal",
"dropOffPointId": "1285758"
}
],
"fareDetails": {
"adultFare": "91.0"
}
},
{
"tripId": "1285856",
"fromCity": "Singapore",
"toCity": "Shah Alam",
"operatorCode": "SA",
"operatorName": "Starmart Express",
"departTime": "2014-01-24 21:00:00.0",
"busType": "Executive",
"pickupPointDetails": [
{
"pickupPointId": "78",
"departureTime": "2014-01-24 21:00:00.0",
"pickupPointName": "Golden Mile Tower, Beach Road"
}
],
"dropoffPointDetails": [
{
"dropOffPointName": "Shah Alam Bus Terminal",
"dropOffPointId": "1285856"
}
],
"fareDetails": {
"adultFare": "91.0"
}
}
],
"errorCode": 0
}
You need to nest another loop if you wish to access each object in the sub array:
Example JSON object:
{
"array": [
{
"property": "Hello",
"nestedArray": [
{
"nestedArrayProp": "World"
}
]
}
]
}
Example Jade template:
each object in array
each nestedObject in object.nestedArray
p= object.property + ' ' + nestedObject.nestedArrayProp
Which will output:
<p>Hello World</p>

array and parsing through sentence construction

I'm really getting a hard time in running the parse button to show the result from the input box ...
these are my codes:
<html>
<head>
<title> sentence detector</title> </head>
<body background="english.jpg">
<font color="black">
<h1 align = center> TABLE OF WORDS </h1>
<br>
<h3>
<script>
var noun = new Array ("time", "year", "people", "way", "man", "day", "thing", "child", "Mr", "government", "work", "life", "woman", "system", "case", "part", "group", "number", "world", "house", "area", "company", "problem", "service", "place", "hand", "party", "school", "country", "point", "week", "member", "end", "state", "word", "family", "fact", "head", "month", "side", "business", "night", "eye", "home", "question", "information" , "power", "change", "interest", "development ");
document.write("<b>");
document.write("NOUN: ");
for (i=0; i<noun.length; i++)
{
document.write(" " + noun[i] + "," + " ");
}
document.write("<br>");
document.write("<br>");
</script>
<script>
var verb = new Array ("allow","answer","arrive","ask" , "be", "become" , "begin" , "believe", "bring", "burn", "buy" , "call" , "can" , "decide", "describe", "destroy" , "die" , "do", "drink" , "drive", "eat", "end", "explain", "fall" , "feel" , "hope" , "hurt", "improve", "jump", "know", "laugh", "learn", "leave", "lie", "listen", "live", "look" , "press" , "promise" , "pull" , "push" , "put" , "read" , "receive", "recognize", "remember", "repeat" , "rest", "return");
document.write("<b>");
document.write("VERB: ");
for (i=0;i<verb.length;i++)
{
document.write(" " + verb[i] + "," + " ");
}
document.write("<br>");
document.write("<br>");
<script>
var adj = new Array ("abandoned", "able", "absolute", "adorable", "adventurous", "academic", "acceptable", "adored", "advanced" , "afraid", "belated", "beloved", "beneficial", "better", "best", "careful", "careles" , "caring", "circular", "classic", "clean", "clear" , "damaged" , "damp", "dangerous", "dapper", "daring");
document.write("<b>");
document.write("ADJECTIVE: ");
for (i=0;i<adj.length;i++)
{
document.write(" " + adj[i] + "," + " ");
}
document.write("<br>");
document.write("<br>");
</script>
<script>
var adv = new Array ("financially","willfully", "abruptly", "endlessly", "firmly", "delightfully", "quickly", "lightly", "eternally", "delicately","wearily", "sorrowfully","beautifully","truthfully" ,"now", "first", "last", "early", "yesterday", "tomorrow", "today", "later", "regularly", "often", "never", "monthly", "always", "usually", "very", "too", "almost", "also", "only", "enough", "so", "quite", "almost", "rather", "back", "gamely", "helplessly" , "entirely", "absently", "bodily", "bloodily", "boldly" , "crazily" , "heartbrokenly", "healthily", "hurtfully");
document.write("<b>");
document.write("ADVERB: ");
for (i=0;i<adv.length;i++)
{
document.write(" " + adv[i] + "," + " ");
}
document.write("<br>");
document.write("<br>");
<script>
var prep = new Array ("about", "above", "according to", "along with", "among", "apart from", "around as", "behind ","below", "beneath", "beside", "between", "beyond", "but", "by", "by means of", "concerning", "despite", "down", "except", "for", "excepting", "for", "in case of", "instead of", "into", "like", "near", "next of", "off", "on", "onto", "on top of", "out", "out of", "outside", "round since", "through", "throughout", "till", "under", "until", "up upon", "up", "to", "with", "within", "without");
document.write("<b>");
document.write("PREPOSITION: ");
for (i=0;i<prep.length;i++)
{
document.write(" " + prep[i] + "," + " ");
}
document.write("<br>");
document.write("<br>");
</script>
<script>
var pronoun = new Array ("all", "another", "any", "anybody", "anyone", "anything", "both","each", "each", "other", "either", "everybody", "everyone", "everything", "few", "he", "her", "hers", "herself", "him", "himself", "his", "I", "it", "its", "itself", "little", "many", "me", "mine");
document.write("<b>");
document.write("PRONOUN: ");
for (i=0;i<pronoun.length;i++)
{
document.write(" " + pronoun[i] + "," + " ");
}
document.write("<br>");
document.write("<br>");
</script>
<script>
var conj = new Array("for", "and", "nor", "but", "or", "yet", "so", "either", "or", "neither", "nor", "both", "and", "not only", "but", "also", "after all", "in addition", "next", "also", "incidentally", "nonetheless", "as a result", "indeed", "on the contrary", "besides", "in fact", "on the other hand", "consequently", "in other words", "otherwise", "finally", "instead", "still", "for example", "likewise", "then","furthermore", "meanwhile", "hence", "moreover", "thus", "however", "nevertheless");
document.write("<b>");
document.write("CONJUNCTION: " );
for (i=0;i<conj.length;i++)
{
document.write(" " + conj[i] + "," + " ");
}
document.write("<br>");
document.write("<br>");
</script>
<script>
var interjection = new Array ( "Absolutely", "Achoo", "Ack", "Adios", "Aha", "Ahoy", "Agreed", "Alack", "Alright", "Alrighty", "Alrighty-roo", "Alack", "Alleluia", "All hail", "Aloha", "Amen", "Anytime", "Argh", "Anyhoo", "Anyhow", "As if", "Attaboy", "Attagirl", "Awww", "Awful", "Ay", "bam", "Bah", "hambug", "Begorra", "Behold", "Bingo", "Blah", "Bravo", "Brrr", "Bye", "Cheers", "Ciao", "Cripes", "Crud", "Darn", "Dear", "Doh", "Drat", "Eek", "Encore", "Eureka", "FiddlesticksFie", "Gadzooks", "Gee");
document.write("<b>");
document.write("INTERJECTION: ");
for (i=0;i<interjection.length;i++)
{
document.write(" " + interjection[i] + "! " + "," + " ");
}
document.write("<br>");
document.write("<br>");
</script>
<script>
var deter = new Array ("the", "a", "an", "another", "no","some", "any", "my", "our", "their", "her", "his", "its", "each", "every" , "certain" , "its", "this", "that", "your", "whose", "little", "|ess", "least", "few", "fewer", "fewest");
document.write("<b>");
document.write("DETERMINER: ");
for (i=0;i<deter.length;i++)
{
document.write(" " + deter[i] + "," + " ");
}
document.write("<br>");
document.write("<br>");
</script>
<br>
<form>
<center>
<font size="+1">
Input Words: <input type = "text" name = "enter" size = "70">
</form>
<input name="parse" type="button" value="parse?">
</center>
</font>
</h4>
</font>
</body>
</body>
</html>
the output must be like this
choose from the words above
type it in the input box
click the button parse?
show results:
ex: she is mine
she-- pronoun
is-- preposition
mine-- pronoun
So there are a couple of points to this:
You have your code directly in a couple of script tags, this means it gets executed immediately when the page gets loaded - and obviously this is before anyone can enter the text.
You need to wrap your code into some sort of a function that then gets called on input:
<script>
function parseWords() {
//All of your code goes here
}
</script>
Then from your button you probably want to call this function I would assume when the
button is pressed.
<input name="parse" type="button" value="parse?" onclick="parseWords()">
Your text input needs an ID so we can get the data from it easily:
<input id="textInput" type = "text" name = "enter" size = "70">
Next you will have to get the data from the input and then split it up.
function parseWords() {
var myText = document.getElementById("textInput").value;
var myWords = myText.split(" ");
}
Now you have to loop over each of the words and find what type they are:
function parseWords() {
var myText = document.getElementById("textInput").value;
var myWords = myText.split(" ");
for (int i = 0; i < myWords.length; i++) {
// Check for your words here and generate the output
}
}
This is not the full solution, but these are the major points I see wrong with your code. I hope you can figure out the rest from here.

Categories