We Keep Coding

Raspberry Pi Server php does not work from outside local network

I have been making a small page on my server to display some photo albums. This is on an nginx server on a raspberry pi 4. When I try to access the page from my laptop or phone when they are on wifi, everything loads correctly. However, when I use my phone with LTE, for example, the php requests do not work. I checked with my laptop using my wireless hotspot and I had the same issue. I never get the "200" code that my browser is waiting for. I do not get an error either. It is as though the http request disappears. I have other pages on this same server where the php is working correctly, one of which is almost the exact same as this code. The php I am trying to use is one which returns file and album names as a long string. Expected output would be:
Siblings|Wyoming|Utah|Skiing|
This all works when accessed from my local network. In the past I have used much more elaborate php files to access security camera footage and also send shell scripts to turn a relay on and off. These worked from anywhere in the world, not just my network. I know very little about this stuff, so I'm hoping I am just missing something obvious that an expert can recognize immediately. :) I will include my code below.
getfiles.php
<?php
function clean_scandir($dir){
return array_values(array_diff(scandir($dir),array('.','..')));
}
$directory = $_GET['dir'];
$files = clean_scandir("/var/www/html/" . $directory);
foreach($files as $file){
$fileStr .= $file . "|";
}
echo $fileStr;
?>
And the javascript that is trying to use this php:
function getAlbums(){
var xmlhttp = new XMLHttpRequest();
xmlhttp.onreadystatechange = function(){
if(this.readyState == 4 && this.status == 200){
console.log("return packet = " + this.response);
var albumArray = this.response.split("|");
albumArray.pop(); //remove empty last element
for (var i = 0; i < albumArray.length; i++){
console.log("album " + i + " = " + albumArray[i]);
}
getThumbnail(albumArray);
setTimeout(() => {
populateAlbums(albumArray); //waits 1 sec for thumbnail requests to finish before loading thumbnails
}, 1000);
}
}
var urlString = "getfiles.php?dir=" + "photos/";
xmlhttp.open("GET", urlString, true);
xmlhttp.send();
}
The code never arrives at the console.logs that are within the big 'if' statement, I am assuming because the status never reaches 200. I do not get any other number like a 500 code. It just remains blank.
I have tried slowly modifying the php code. If the file is simply:
echo "php successful";
then it works. But for some reason the full code is causing issues. Sometimes the issues do not act the same each time either.

I'm trying to make a website load forever. How can I do this?

I'm trying to make a website load forever. My current idea is to request a PHP file:
<?php
sleep(30);
This will delay the load by 30 seconds, which a quick Google search tells me should be within most browsers' timeouts. I was thinking of writing some JavaScript to append a new link tag after a bit less than 30 seconds to keep the page loading, but I found that this didn't keep the loading icon spinning (with Chrome at least):
window.addEventListener( 'load', () => {
var i = 0;
setInterval( () => {
i++;
var newScript = document.createElement('script');
newScript.src = 'infinite-loading.php?i=' + i;
document.querySelector('#infinite-loading').after(newScript);
console.log('The deed is done');
}, 25000)
} )
<script id="infinite-loading" src="infinite-loading.php"></script>
The code above appends a script tag every 25 seconds, and the browser loads the PHP file each time, but it doesn't show the loading icon. I added the URL parameter because I wasn't sure if browsers would cache the page.
I also want to make sure that the server with the PHP file won't be overloaded. I'm not sure if many sleep() functions running constantly at the same time will cause any issues.
Is there a better way to do this client-side? Should I use something other than PHP? Something multi-threaded?
(Edit: Sorry for the awkward title, Stack Overflow didn't like my first one.)

You need that browser will continue reading your page forever (I'm talking about HTML, not other linked objects). So you need not to break timeout and feed some data from backend to frontend.
Example of sending portion of data to client:
ob_end_flush();
# CODE THAT NEEDS IMMEDIATE FLUSHING
ob_start();
Now we need to understand the minimum data packet size that is expected by the browser. Minimal googling tells us a limit of 8-10 bytes.
So combining this together we can try to check (I did not checked, it is just my version):
<?php
while (true) {
sleep(25);
ob_end_flush();
echo " "; // 10 spaces...
ob_start();
}

Not sure why you would want to do anything like this but the simplest solution I think is an endless loop.
<?php
while(true)
{
}

javascript function to round current time (from epoch) to nearest minute

I've been banging my head against the wall trying to get a JavaScript equivalent to this php snippet:
<?php
$id = 'uniqueID'
$now = round(time()/60);
$lock = md5($now . $id);
?>
I've been trying variations on this:
var timeInMin = new Date().getTime() / 60000;
var timestamp = Math.round(timeInMin);
var key = md5(timestamp + 'uniqueID');
utilizing an md5 script from here
I merely need lock and key to match. Seems simple to me. What am I doing wrong?

As said before me, if time not matching it will not create the same hash. What I do in situations like that is to find way to pass the time from php to the client side so they can use the same exact time.
PHP side:
<?php
$id = 'uniqueID';
$now = round(time()/60);
$lock = md5($now . $id);
print $lock;
setcookie("time",$now);
?>
Client Side:
<script type="text/javascript">
var timestamp = getCookie("time");
var key = md5(timestamp + 'uniqueID');
console.log(key);
</script>
Note that getCookie is a shortcut function
The following example is here to present the idea in a simple form. I would not use time as the name of the cookie nor give access to the vars (wrap in function). Uglify scripts goes a long way in cases like this.

To put it in concrete terms (my comment was slightly facetious):
PHP is a server-side language. When your browser fires a request for a page over the internet (or even to a listening port on your local machine), the instance of apache running on the server (or your local machine) executes the PHP on the page, then spits it back out to the browser.
The browser then executes any JavaScript on the page, we refer to this as client-side.
Because you are using Math.round, if it takes more than 30 seconds between the time your server executes the PHP (server-side) and the time your browser starts executing the relevant Javascript (client-side) then the time in minutes will be different. Using Math.floor instead would give you 59 seconds of wiggle room but that's still dicey, especially on mobile.
Even assuming the page executes the JavaScript immediately after loading and loads pretty quickly 30 seconds of latency is not totally off the table, and on mobile neither is 59.

Optimizing for mobile/ low speed connection - full background video

I am developing a website with a full background video.
To optimize for low speed connections / mobile, I am using a media query to detect screen sizes smaller then 768 px, then doing a display:none on the video container and displaying a background image instead.
My question here is:
Is this the correct way to optimize for low speed connections / mobile?
Will it have any impact on my optimization when not displaying containers with css or should I be doing it in JavaScript instead, when loading the page?

Media queries will allow you to load different images if they are set as backgrounds, so that's a start for small screens, but not for low speed on a computer, and it won't work in the case of a video, or additionnal files being loaded or not.
In JS
This is what I can think of at the moment, probably not very reliable, because it depends on how much content you have on your website.
It would consist in only having the most important stuff loaded (low speed connexion), and getting an approximate loading time for the content (DOM, images, css, js...). Then you can choose to either load the rest or not.
// get the current time as soon as you can (directly in the head tag)
var start = new Date().getTime();
// do the same after the page has loaded and find out the difference
window.onload = function(){
var end = new Date().getTime();
var timeTaken = end - start;
alert('It took ' + timeTaken + ' ms to load');
if(timeTaken < 2000){
// load more stuff if it took less than 2 seconds, for example
}
}
Again: not very reliable (a page with lots of images is going to take longer, and finding the perfect "timeout" (2 seconds here) won't be easy. Also, this won't work is your users have JS disabled, but that's not a concern I'm worried about these days :) You should probably wait for other answers.
In PHP
Another method I can think of is doing it in PHP if that's an option for you. You could have your php page get the time of its request by the client. Then for example if you have an external JS, you can do this:
index.php
<script src="myScript.php?time=<?=microtime()?>"></script>
myScript.php would be a php page that will get the time of this request, compare it with the first one , and then you can choose to serve different JS files based on that (That is called a proxy page).
From the JS file you choose, you can load different stuff based on what you want to do.
myScript.php
<?php
header("Content-Type: text/javascript");
$start = intval( $_GET['time'] );
$end = microtime();
$timeTaken = $end - $start;
if( $timeTaken < 2000 ){
echo file_get_contents('JSForHighSpeed.js');
} else {
echo file_get_contents('JSForLowSpeed.js');
}
?>

What are you using as a player for your videos?
For what you're doing, the answers will be in jQuery, not CSS. With videos, it's important to know what the user's bandwidth is so that you can supply the correct video resolution. Most phones can support 1080p resolutions (often times double, especially with Apple's Retina Display, or Samsung's 5K screens). In other words, it shouldn't matter if they are using a phone or a cinema display; what matters is their connection speed.
I've had good luck with JWPlayer and using Amazon S3 for storage. It's also been my experience that H.264 MP4's are the way to go.
Whatever you're using, you should be able to supply multiple versions of your video(s). For example, you might create different resolutions - 360, 720 and 1080.
Here's a jQuery utility you can use to determine the user's bandwidth. Make sure to create a file named "10.kb.file.zip" (and make sure it's exactly 10 kb).
/*
* measureBandwidth.js
* Directory: ~/lib/js/
* jQuery utility for measuring a user's bandwidth
*/
var url = 'js/10.kb.file.zip?{0}';
var start = '';
function getBandwidth(callback) {
start = new Date();
getFile(1, callback);
}
function getFile(i, callback) {
$.get(url.f(Math.random()), function () {
i++;
if (i < 6) {
getFile(i, callback);
} else {
var end = new Date();
var speed1 = Math.round(((50 / ((end - start) * .001) * 8) / 1000) * 10) / 10;
var speed2 = Math.round(50 / ((end - start) * .001) * 10) / 10;
callback(speed1, speed2);
}
});
}
String.prototype.f = function () { var args = arguments; return this.replace(/\{(\d+)\}/g, function (m, n) { return args[n]; }); };
Then, you can use it like this:
getBandwidth(function (Mbits, kBs) {
$('#speed1').html(Mbits + ' Mbit/s');
$('#speed2').html(kBs + ' kB/s');
});
Based on those results, you can then set the appropriate video for the user.
For example, I route kBs < 128 to default to 360p video, and kBs > 128 to the 720p video.
In JWPlayer, you would add all of your videos to the "playlist" and give them labels like "360p", "720p" etc.

caching JavaScript files

Which is the best method to make the browser use cached versions of js files (from the serverside)?

Have a look at Yahoo! tips: https://developer.yahoo.com/performance/rules.html#expires.
There are also tips by Google: https://developers.google.com/speed/docs/insights/LeverageBrowserCaching

or in the .htaccess file
AddOutputFilter DEFLATE css js
ExpiresActive On
ExpiresByType application/x-javascript A2592000

I just finished my weekend project cached-webpgr.js
which uses the localStorage / web storage to cache JavaScript files. This approach is very fast. My small test showed
Loading jQuery from CDN: Chrome 268ms, FireFox: 200ms
Loading jQuery from localStorage: Chrome 47ms, FireFox 14ms
The code to achieve that is tiny, you can check it out at my Github project https://github.com/webpgr/cached-webpgr.js
Here is a full example how to use it.
The complete library:
function _cacheScript(c,d,e){var a=new XMLHttpRequest;a.onreadystatechange=function(){4==a.readyState&&(200==a.status?localStorage.setItem(c,JSON.stringify({content:a.responseText,version:d})):console.warn("error loading "+e))};a.open("GET",e,!0);a.send()}function _loadScript(c,d,e,a){var b=document.createElement("script");b.readyState?b.onreadystatechange=function(){if("loaded"==b.readyState||"complete"==b.readyState)b.onreadystatechange=null,_cacheScript(d,e,c),a&&a()}:b.onload=function(){_cacheScript(d,e,c);a&&a()};b.setAttribute("src",c);document.getElementsByTagName("head")[0].appendChild(b)}function _injectScript(c,d,e,a){var b=document.createElement("script");b.type="text/javascript";c=JSON.parse(c);var f=document.createTextNode(c.content);b.appendChild(f);document.getElementsByTagName("head")[0].appendChild(b);c.version!=e&&localStorage.removeItem(d);a&&a()}function requireScript(c,d,e,a){var b=localStorage.getItem(c);null==b?_loadScript(e,c,d,a):_injectScript(b,c,d,a)};
Calling the library
requireScript('jquery', '1.11.2', 'http://ajax.googleapis.com/ajax/libs/jquery/1.11.2/jquery.min.js', function(){
requireScript('examplejs', '0.0.3', 'example.js');
});

From PHP:
function OutputJs($Content)
{
ob_start();
echo $Content;
$expires = DAY_IN_S; // 60 * 60 * 24 ... defined elsewhere
header("Content-type: x-javascript");
header('Content-Length: ' . ob_get_length());
header('Cache-Control: max-age='.$expires.', must-revalidate');
header('Pragma: public');
header('Expires: '. gmdate('D, d M Y H:i:s', time()+$expires).'GMT');
ob_end_flush();
return;
}
works for me.
As a developer you'll probably quickly run into the situation that you don't want files cached, in which case see Help with aggressive JavaScript caching

In your Apache .htaccess file:
#Create filter to match files you want to cache
<Files *.js>
Header add "Cache-Control" "max-age=604800"
</Files>
I wrote about it here also:
http://betterexplained.com/articles/how-to-optimize-your-site-with-http-caching/

I am heavily tempted to close this as a duplicate; this question appears to be answered in many different ways all over the site:
will a script in html's script tag with extension php be cached?
When does browser automatically clear cache of external JavaScript file?
Help with aggressive JavaScript caching
How to control web page caching, across all browsers?
How can I make the browser see CSS and Javascript changes?

The best (and only) method is to set correct HTTP headers, specifically these ones: "Expires", "Last-Modified", and "Cache-Control". How to do it depends on the server software you use.
In Improving performance… look for "Optimization on server side" for general considerations and relevant links and for "Client-side cache" for the Apache-specific advice.
If you are a fan of nginx (or nginx in plain English) like I am, you can easily configure it too:
location /images {
...
expires 4h;
}
In the example above any file from /images/ will be cached on the client for 4 hours.
Now when you know right words to look for (HTTP headers "Expires", "Last-Modified", and "Cache-Control"), just peruse the documentation of the web server you use.

I have a simple system that is pure JavaScript. It checks for changes in a simple text file that is never cached. When you upload a new version this file is changed. Just put the following JS at the top of the page.
(function(url, storageName) {
var fromStorage = localStorage.getItem(storageName);
var fullUrl = url + "?rand=" + (Math.floor(Math.random() * 100000000));
getUrl(function(fromUrl) {
// first load
if (!fromStorage) {
localStorage.setItem(storageName, fromUrl);
return;
}
// old file
if (fromStorage === fromUrl) {
return;
}
// files updated
localStorage.setItem(storageName, fromUrl);
location.reload(true);
});
function getUrl(fn) {
var xmlhttp = new XMLHttpRequest();
xmlhttp.open("GET", fullUrl, true);
xmlhttp.send();
xmlhttp.onreadystatechange = function() {
if (xmlhttp.readyState === XMLHttpRequest.DONE) {
if (xmlhttp.status === 200 || xmlhttp.status === 2) {
fn(xmlhttp.responseText);
}
else if (xmlhttp.status === 400) {
throw 'unable to load file for cache check ' + url;
}
else {
throw 'unable to load file for cache check ' + url;
}
}
};
}
;
})("version.txt", "version");
just replace the "version.txt" with your file that is always run and "version" with the name you want to use for your local storage.