We Keep Coding

Extract a portion code from an external page

I am creating a personal website for vacation rentals (a joomla website).
The owner has created an ad here: https://www.armor-vacances.com/locat...tml#calendrier
Do you know if there is a way to extract the entire "calendar" portion to display on my website site?
I try some scripts find with "file_get_html" for example but I don't arrive to my goals.
Thanks for your help.

php can do it, as long as you're not treading on site copyright or robot control issues, and that you can rely on the site you're reading will always have the container you're after, but it's not going to be easy.
It would be good if the site you're scraping has the information in machine ready format using meta tags that it knows programs will be looking for.
Here's a starting point for some scraping code for you (I've cached the page content to a local file so that you don't hit the website too many times each day):
<?php
// php7.0
$src ="https://stackoverflow.com/questions/52678213/extract-a-portion-code-from-an-external-page";
$tmpfn="C:/temp/temp.$srcX.$now.html";
$findDivId="Place your ID here";
$now = date('Y-m-d', time());
$srcX = preg_replace("/[^a-zA-Z0-9]+/", "", $src);
$srcX = substr($srcX, 0, 155);
if ( file_exists($tmpfn) ) {
$html=file_get_contents($tmpfn);
}
else {
$ch = curl_init($src);
curl_setopt($ch, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
curl_setopt($ch, CURLOPT_AUTOREFERER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_ANY);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'GET');
curl_setopt($ch, CURLOPT_HEADER, true);
$html = curl_exec($ch);
if ( !$html ) {
echo curl_error($ch);
exit;
}
curl_close($ch);
file_put_contents($tmpfn, $html);
}
echo "<LI>html size = ".strlen($html)." bytes";
if ( strcmp($html, "") != 0 ) {
$dom = new DOMDocument;
#$dom->loadHTML($html);
$divs = $dom->getElementsByTagName("div"); // or ->getElementsById($id);
if ( $divs ) {
echo "<UL>";
foreach ($divs as $div) {
echo "<LI>Tag::".$div->nodeName;
if ( $div->hasAttributes() ) {
foreach ($div->attributes as $attr) {
echo "<BR>Attribute::".$attr->nodeName . "=" . $attr->nodeValue . " ";
if ( strcmp($attr->nodeName,'id')==0
and strcmp($attr->nodeValue,$findDivId)==0 ) {
echo "<LI>Found $findDivId!!";
}
}
}
echo "<BR>Value::".$div->nodeValue."<BR><BR>";
}
echo "</UL>";
}
}
?>

Open links from an array without triggering the popup filter

I'm trying to develop a tool which gets the links from a page, stores them in an array and opens with one click in a new tab. Is there a way to accomplish that without triggering the browser's popup spam filter?
Here's my code:
<?php
$base = "web.archive.org";
$time = "/web/20160101000000*/";
$domain = #$_POST["domain"];
$ch = curl_init();
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_URL, "$base$time$domain");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_TIMEOUT_MS, 20000); //in miliseconds
$output = curl_exec($ch);
curl_close($ch);
# REPLACES ALL URLs OF WAYBACK MACHINE
$replacedHrefHtml = preg_replace('/(href=")/', "href=\"https://web.archive.org", $output);
# REPLACES ALL SRCs OF WAYBACK MACHINE
$replacedsrc = preg_replace('/(src=")/', "src=\"https://web.archive.org", $replacedHrefHtml);
#get wbCalendar
$html = str_get_html($replacedsrc);
$elem = $html->find('div[id=wbCalendar]', 0);
#extract the links and store them into an array
$data = array();
$open = '';
foreach($elem->find('a') as $element) {
$extracted = $element->href;
$open .= "window.open('{$extracted}'); ";
$data[] = $extracted;
}
echo "Open multiple links";
?>
You can test it here: seotify.com/archive-opener
Thanks in advance.

As far as I know there is no way you can ensure the popup filter won't catch you. I think the only thing you can do is to make the windows open in an "on-click" event someElement.onclick = function(){ /*POPUP CODE*/ } (so that the code is triggered in response to a user action)

Replace whateverorigin.org call with local proxy in php

I'm currently using whateverorigin.org in some javascript to retrieve a URL as a JSON object because a 3rd party site hasn't made one of their functions available via their JSON API.
I'd like to remove this dependancy from my website as whateverorigin.org breaks the HTTPS/SSL browser checks for secure content because it's a clear http call.
Has anyone done this? I haven't found an example of it anywhere.
Thanks in advance for a response!

Ok, so since I first typed up this question, I've now already found some examples and cobbled together a working proxy function in php... Feel free to use it for your own purposes!
<?php
// Sourced from: http://stackoverflow.com/questions/2511410/curl-follow-location-error
function curl_exec_follow(/*resource*/ &$ch, /*int*/ $redirects = 20, /*bool*/ $curlopt_header = false) {
if ((!ini_get('open_basedir') && !ini_get('safe_mode')) || $redirects < 1) {
curl_setopt($ch, CURLOPT_HEADER, $curlopt_header);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, $redirects > 0);
curl_setopt($ch, CURLOPT_MAXREDIRS, $redirects);
return curl_exec($ch);
} else {
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, false);
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FORBID_REUSE, false);
do {
$data = curl_exec($ch);
if (curl_errno($ch))
break;
$code = curl_getinfo($ch, CURLINFO_HTTP_CODE);
if ($code != 301 && $code != 302)
break;
$header_start = strpos($data, "\r\n")+2;
$headers = substr($data, $header_start, strpos($data,"\r\n\r\n", $header_start)+2-$header_start);
if (!preg_match("!\r\n(?:Location|URI): *(.*?) *\r\n!",$headers, $matches))
break;
curl_setopt($ch, CURLOPT_URL, $matches[1]);
} while (--$redirects);
if (!$redirects)
trigger_error('Too many redirects. When following redirects, libcurl hit the maximum amount.', E_USER_WARNING);
if (!$curlopt_header)
$data = substr($data, strpos($data, "\r\n\r\n")+4);
return $data;
}
}
header('Content-Type: application/json');
$retrieveurl = curl_init(urldecode($_GET['url']));
$callbackname = $_GET['callback'];
$htmldata = curl_exec_follow($retrieveurl);
if (curl_error($retrieveurl))
die(curl_error($retrieveurl));
$status = curl_getinfo($retrieveurl, CURLINFO_HTTP_CODE);
curl_close($retrieveurl);
$data = array('contents' => $htmldata, 'status' => $status);
$jsonresult = json_encode($data);
echo $callbackname . '(' . $jsonresult . ')';
?>
Hope this helps someone!

Extract data from a web page

I have a link to an app on Apple App Store. I need to use its data on my own webpage. To be more precise, I want to extract the app icon, its category and whether or not it is free and to add this data to my webpage. How can I do this using JavaScript?

To my knowledge, you can't do this using Javascript due to Cross Platform security issues.
You would nee a Web Server and possibly write a scraper in PHP to read that page. then call your PHP script from your JavaScript.

Here is the PHP Script:
<?php
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $Url);
curl_setopt($ch, CURLOPT_REFERER, "https://itunes.apple.com/us/app/google+/id447119634?mt=8");
curl_setopt($ch, CURLOPT_USERAGENT, "Mozzila/1.0");
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_TIMEOUT, 10);
$output = curl_exec($ch);
curl_close($ch);
$logo = explode('<div class="artwork">', $output);
$logo = explode('src="', $logo[1]);
$logo = explode('"', $logo[1]);
$logo = $logo[0];
$category = explode('<span class="label">Category:', $output);
$category = explode('">', $category[1]);
$category = explode('<', $category[1]);
$category = $category[0];
echo $logo;
echo $category;
?>

Accessing and printing HTML source code using PHP or JavaScript

I am trying to access and then print (or just be able to use) the source code of any website using PHP. I am not very experienced and am now thinking I might need to use JS to accomplish this. So far, the code below accesses the source code of a web page and displays the web page... What I want it to do instead is display the source code. Essentially, and most importantly, I want to be able to store the source code in some sort of variable so I can use it later. And eventually read it line-by-line - but this can be tackled later.
$url = 'http://www.google.com';
function get_data($url)
{
$ch = curl_init();
$timeout = 5;
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
echo get_data($url); //print and echo do the same thing in this scenario.

Consider using file_get_contents() instead of curl. You can then display the code on your page by replacing every opening bracket (<) with < and then outputting it to the page.
<?php
$code = file_get_contents('http://www.google.com');
$code = str_replace('<', '<', $code);
echo $code;
?>
Edit:
Looks like curl is actually faster than FGC, so ignore that suggestion. The rest of my post still stands. :)

You should try to print the result between <pre></pre> tags;
echo '<pre>' . get_data($url) . '</pre>';

I rewrote your function. The function can return the source with lines or without lines.
<?php
function get_data($url, $Addlines = false){
$ch = curl_init($url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_BINARYTRANSFER, true);
$content = curl_exec($ch);
$content = htmlspecialchars($content); // Prevents the browser to parse the html
curl_close($ch);
if ($Addlines == true){
$content = explode("\n", $content);
$Count = 0;
foreach ($content as $Line){
$lines = $lines .= 'Line '.$Count.': '.$Line.'<br />';
$Count++;
}
return $lines;
} else {
$content = nl2br($content);
return $content;
}
}
echo get_data('https://www.google.com/', true); // Source code with lines
echo get_data('https://www.google.com/'); // Source code without lines
?>
Hope it gets you on your way.

Add a header Content-Type: text/plain
header("Content-Type: plain/text");

Use htmlspecialchars() in php to print the source code.
In your code, use
return htmlspecialchars($data);
instead of
return $data;