We Keep Coding

PHP - Create a directory listing of specified folder

I've got small piece of code which I'm using to scan a certain directory on my server for all the files and folders inside. But there's a catch... It can only scan the specified folder, and no sub-directories. Let's say share/ is the root directory for this scan, if there are files in any sub-directory like share/folder/file.png they won't be listed, where the entire purpose of my project is to create an HTML file explorer.
This is the code I'm using to receive the list of files from the server to later display it in the browser using JavaScript:
<?php
$listFiles = array();
if ($handle = opendir('./share')) {
while (false !== ($entry = readdir($handle))) {
if ($entry != "." && $entry != "..") {
$listFiles[] = $entry;
}
}
closedir($handle);
}
$FS_files = json_encode($listFiles);
?>
And then I "push" it rude to my actual JavaScript code like this:
let FS_files = <?php echo $FS_files ?>
The code I above basically generates an array of file names from scanned directory. What I wanna know is how can I get an entire JSON tree of files in this directory and all folders inside it, or possibly change my code so it works that way?

<?php
function scanFolder($dir) {
$result = array();
$cdir = scandir($dir);
foreach ($cdir as $key => $value) {
if (!in_array($value,array(".",".."))) {
if (is_dir($dir . DIRECTORY_SEPARATOR . $value)) {
$result[$value] = scanFolder($dir . DIRECTORY_SEPARATOR . $value);
} else {
$result[] = $value;
}
}
}
return $result;
}
$FS_files = json_encode(scanFolder("./share"));
?>
The function scans the folder and checks each entry with is_dir. If its true, this folder is scanned with the same function.

PHP 5.4.16 DOMDocument removes parts of Javascript

I try to load an HTML page from a remote server into a PHP script, which should manipulate the HTML with the DOMDocument class. But I have seen, that the DOMDocument class removes some parts of the Javascript, which comes with the HTML page. There are some things like:
<script type="text/javascript">
//...
function printJSPage() {
var printwin=window.open('','haha','top=100,left=100,width=800,height=600');
printwin.document.writeln(' <table border="0" cellspacing="5" cellpadding="0" width="100%">');
printwin.document.writeln(' <tr>');
printwin.document.writeln(' <td align="left" valign="bottom">');
//...
printwin.document.writeln('</td>');
//...
}
</script>
But the DOMDocument changes i.e. the line
printwin.document.writeln('</td>');
to
printwin.document.writeln(' ');
and also a lot of others things (i.e. the last script tag is no longer there. As the result I get a complete destroyed page, which I cannot send further.
So I think, DOMDocument has problems with the HTML tags within the Javascript code and tries to correct the code, to produce a well-formed document. Can I prevent the Javascript parsing within DOMDocument?
The PHP code fragment is:
$stdin = file_get_contents('php://stdin');
$dom = new \DOMDocument();
#$dom->loadHTML($stdin);
return $dom->saveHTML(); // will produce wrong HTML
//return $stdin; // will produce correct HTML
I have stored both HTML versions and have compared both with Meld.
I also have tested
#$dom->loadXML($stdin);
return $dom->saveHTML();
but I don't get any things back from the object.

Here's a hack that might be helpful. The idea is to replace the script contents with a string that's guaranteed to be valid HTML and unique then replace it back.
It replaces all contents inside script tags with the MD5 of those contents and then replaces them back.
$scriptContainer = [];
$str = preg_replace_callback ("#<script([^>]*)>(.*?)</script>#s", function ($matches) use (&$scriptContainer) {
$scriptContainer[md5($matches[2])] = $matches[2];
return "<script".$matches[1].">".md5($matches[2])."</script>";
}, $str);
$dom = new \DOMDocument();
#$dom->loadHTML($str);
$final = strtr($dom->saveHTML(), $scriptContainer);
Here strtr is just convenient due to the way the array is formatted, using str_replace(array_keys($scriptContainer), $scriptContainer, $dom->saveHTML()) would also work.
I find it very suprising that PHP does not properly parse HTML content. It seems to instead be parsing XML content (wrongly so as well because CDATA content is parsed instead of being treated literally). However it is what it is and if you want a real document parser then you should probably look into a Node.js solution with jsdom

If you have a <script> within a <script>, the following (not so smart) solution will handle that. There is still a problem: if the <script> tags are not balanced, the solution will not work. This could occur, if your Javascript uses String.fromCharCode to print the String </script>.
$scriptContainer = array();
function getPosition($tag) {
return $tag[0][1];
}
function getContent($tag) {
return $tag[0][0];
}
function isStart($tag) {
$x = getContent($tag);
return ($x[0].$x[1] === "<s");
}
function isEnd($tag) {
$x = getContent($tag);
return ($x[0].$x[1] === "</");
}
function mask($str, $scripts) {
global $scriptContainer;
$res = "";
$start = null;
$stop = null;
$idx = 0;
$count = 0;
foreach ($scripts as $tag) {
if (isStart($tag)) {
$count++;
$start = ($start === null) ? $tag : $start;
}
if (isEnd($tag)) {
$count--;
$stop = ($count == 0) ? $tag : $stop;
}
if ($start !== null && $stop !== null) {
$res .= substr($str, $idx, getPosition($start) - $idx);
$res .= getContent($start);
$code = substr($str, getPosition($start) + strlen(getContent($start)), getPosition($stop) - getPosition($start) - strlen(getContent($start)));
$hash = md5($code);
$res .= $hash;
$res .= getContent($stop);
$scriptContainer[$hash] = $code;
$idx = getPosition($stop) + strlen(getContent($stop));
$start = null;
$stop = null;
}
}
$res .= substr($str, $idx);
return $res;
}
preg_match_all("#\<script[^\>]*\>|\<\/script\>#s", $html, $scripts, PREG_OFFSET_CAPTURE|PREG_SET_ORDER);
$html = mask($html, $scripts);
libxml_use_internal_errors(true);
$dom = new DOMDocument();
$dom->loadHTML($html);
libxml_use_internal_errors(false);
// handle some things within DOM
echo strtr($dom->saveHTML(), $scriptContainer);
If you replace the "script" String within the preg_match_all with "style" you can also mask the CSS styles, which can contain tag names too (i.e. within comments).

Saving PHP array as javascript array variable

I am using the following pair of javascript and php files to get the names of all the image files in a directory.
Javascript
$(document).on("click", "ul#matchListUL li a", function(event){
event.preventDefault();
var matchLst = $(this).attr('href');
var filenames = new Array();
if (matchLst !== null && matchLst !=="") {
$.ajax({
url : "filenames.php",
dataType : "JSON",
type : "POST",
success:function(data) {
console.log(data);
}
});
}
return false;
})
filename.php
$tardir = "mysite.com/projects/" . $seldir . "/" . $match . "/*.jpg" ;
$files = glob($tardir);
$fileName = array();
for ($i=0; $i<count($files); $i++)
{
$num = $files[$i];
$fileName = basename($num, ".jpg");
}
echo json_encode($fileName);
However, this just prints only one file name - essentially the last file in the folder.
How can i get all the filenames and can is save the filenames locally using local storage?

You are doing it wrong, you should do this:
for ($i=0; $i<count($files); $i++)
{
$num = $files[$i];
// [] will add the value to the array
$fileName[] = basename($num, ".jpg");
}
And instead of using a for loop to go through your array, use a foreach loop instead, like this:
foreach($files as $file){
$fileName[] = basename($file, ".jpg");
}

get all images in an array from directory and sub-directories

Currently I've similar to the following tree structure:
+images
+sub-directory
-image1.jpg
-image2.jpg
+sub-directory-2
-image3.jpg
-image4.jpg
-some-image.jpg
-another.jpg
<script>
<?php
//path to directory to scan. i have included a wildcard for a subdirectory
$directory = "images/*/";
//get all image files with a .jpg extension.
$images = glob("" . $directory . "*.jpg");
$imgs = '';
// create array
foreach($images as $image){ $imgs[] = "$image"; }
echo "var allImages = ".$imgs.";\n";
?>
console.log(allImages);
</script>
As I'm extremely new to php, I'm blindly getting logged as Array() in the console.
Also, I've set $directory = "images/*/"; which will get all images inside the subfolders only but not getting images inside parent directory that likely to images/some-image.jpg and I wanted to get this too.
I want all the images in an array like this (when I use console.log(allImages);):
['some-image.jpg','another.jpg','image1.jpg','image2.jpg','image3.jpg','image4.jpg']

I love JSON, keeps things nice and simple:
<?php
$images = glob("images/*/*.jpg");
$imgs = array();
foreach($images as $image){ $imgs[] = $image; }
?>
<script>
var allImages = JSON.parse('<?php echo json_encode($imgs);?>');
console.log( allImages);
</script>
Now you have the php array available in javascript, with the same structure. You can loop them with a for, of if you have jQuery, $.each().
I changed a code more to your style (mixing php and html), but you should try to split those in htmltemplates.
Im not 100% sure about this, but you can regex in your glob, if this works you don't need the foreach, this will return only the filenames:
$images = glob("~(?:images/*/)*\.jpg~");

What about this:
<script>
<?php
$directory = "images/*/";
$images = glob("" . $directory . "*.jpg");
echo "var allImages = ['".implode("', '", $images)."'];\n";
?>
console.log(allImages);
</script>

How about a recursive function?
function loadImages($folder)
{
$files = glob($folder);
foreach( $files as $file )
{
if(is_dir($file))
{
loadImages($file);
} else {
$images[] = $file; // add some file type validation here
}
}
return $images;
}
$images = json_encode(loadImages($startFolderPath));
I'm on an iPad so I can't test it.

Screen-scraping JavaScript in PHP

I can successfully scrape all the items on this page using this script:
$html = file_get_contents($list_url);
$doc = new DOMDocument();
libxml_use_internal_errors(TRUE);
if(!empty($html))
{
$doc->loadHTML($html);
libxml_clear_errors(); // remove errors for yucky html
$xpath = new DOMXPath($doc);
/* FIND LINK TO PRODUCT PAGE */
$products = array();
$row = $xpath->query($product_location);
/* Create an array containing products */
if ($row->length > 0)
{
foreach ($row as $location)
{
$product_urls[] = $product_url_root . $location->getAttribute('href');
}
}
else { echo "product location is wrong<br>";}
$imgs = $xpath->query($photo_location);
/* Create an array containing the image links */
if ($imgs->length > 0)
{
foreach ($imgs as $img)
{
$photo_url[] = $photo_url_root . $img->getAttribute('src');
}
}
else { echo "photo location is wrong<br>";}
$was = $xpath->query($was_price_location);
/* Create an array containing the was price */
if ($was->length > 0)
{
foreach ($was as $price)
{
$stripped = preg_replace("/[^0-9,.]/", "", $price->nodeValue);
$was_price[] = "£".$stripped;
}
}
else { echo "was price location is wrong<br>";}
$now = $xpath->query($now_price_location);
/* Create an array containing the sale price */
if ($now->length > 0)
{
foreach ($now as $price)
{
$stripped = preg_replace("/[^0-9,.]/", "", $price->nodeValue);
$stripped = number_format((float)$stripped, 2, '.', '');
$now_price[] = "£".$stripped;
}
}
else { echo "now price location is wrong<br>";}
$result = array();
/* Create an associative array containing all the above values */
foreach ($product_urls as $i => $product_url)
{
$result[] = array(
'product_url' => $product_url,
'shop_name' => $shop_name,
'photo_url' => $photo_url[$i],
'was_price' => $was_price[$i],
'now_price' => $now_price[$i]
);
}
}
However, a problem arises if I want to get page two, or if I view 100 per page file_get_contents($list_url) will always return page one with its 24 values.
I presume that the page changes are being handled via AJAX request (though I can't find any evidence of this in the source). Is there a way to scrape exactly what I see on the screen?
I've seen talk of PhantomJS in previous answers but I'm not sure it'd be appropriate here given that I'm working in PHP.

It's because of a hashtag in the link which is generated by some js script. Turn off javascript for that site and check the output links it generates.
For example for page two it is http://www.hm.com/gb/subdepartment/sale?page=1

// Create DOM from URL or file
$file= file_get_html('http://stackoverflow.com/');
// Find your links
foreach($file->find('a') as $youreEement) {
echo $yourElement->href . '<br>';
}