I've Looked at ajax progress bar solutions on this site,
However i can't find anything to help with the below,
basically,
[browser] ----> makes ajax call to [server running php]
my php code
<?php
//input and process ajax request
//[MySQL] processing 5000 rows of a table
//after 5000/5000 row, end of ajax call
?>
Now, I need to show the user the status of the rows being processed
eg Processing::::::::: 2322/5000. row
The code for processing Mysql looks like this
foreach($result_set as $table_row)
{
// process each table row
}
How am i going to let the client know of this progress via xmlhttprequest?
cannot use JQUERY, only Javascript
Tried implementing the below,
the server continously echos the progress in a loop,
<?php
//progress
while($i < $total_progress)
{
//php processing
if(#ob_get_contents()){ ob_end_flush(); flush(); }
echo $i; // progress
$i++;
}
?>
The Data (variable $i) is continously being printed and pushed into the response,
Is there a way i can read the printed variable from the server,constantly before the ajax call ends?
This can be done with a combination of AJAX, sessions, and javascript. I've never made any graphical versions of a bar, but I have made text-based ones in the past.
In your client side page, have AJAX make a call to your page that will do the table processing. Have this server side page start a session and create a session variable to track what your progress is at (e.g. $_SESSION['processedRows']). Now inside your processing loop after you processed a row, increase the value of your session var and then make a call to session_write_close(). This will write your session info to disk and then close the session. This is important, as I will explain in a minute. You must also make another call to session_start() again prior to updating your session variable in the next iteration of your foreach loop.
Back on the client side, you will have another AJAX request being sent on an interval (I use setInterval() in javascript for this) which will call up another server side page that will open a session and return the value of $_SESSION['processedRows']. You can take this value and use it update your counter, progress bar, etc. You could return several values as JSON or an HTML snippet, doesn't matter. Just make sure that you have a callback method of some kind in place that will kill the interval that has been setup on the client side. It is important that you call session_write_close() in your long running script. If you do not then your concurrent script to check the progress will not be able to access your session variables and will lag while it waits for the processing script to end. The reason is that the session file has not been released until you call session_write_close().
EDIT:
Here is a very basic, no frills example of what you are looking for. This doesn't have much in regards to error checking, timeouts, etc. But it should illustrate what you want done. No jQuery either, as per you question. For this you will need three files. First you need your long running script, this one basically puts PHP to sleep very briefly 5000 times. It is to illustrate your table operations.
startCounter.php
<?php
//Start a long-running process
$goal = 5000;
session_start();
for($i = 0; $i < $goal; $i++) {
session_start(); //Reopen session to continue updating the progress
if(!isset($_SESSION['progress'])) {
$_SESSION['progress'] = $i;
$_SESSION['goal'] = $goal;
}
//wait a wink
usleep(rand(500, 2000));
$_SESSION['progress'] = $i;
session_write_close(); //Close session to gain access to progress from update script
header_remove();
}
?>
Next you need an update script, this one is very simple, it will only output a string with the progress update.
updateCounter.php
<?php
//Get the progress
session_start();
print("Processed " . $_SESSION['progress'] . " of " . $_SESSION['goal'] . " rows.");
?>
Next you need your client side ready to go with AJAX. You need to have a way to start the long running script, and then while that request is incomplete, you set and interval to hit the server up for the current progress. Just make sure you kill the interval when operation is done. Here is the test page:
progress.html
<!DOCTYPE html>
<html>
<head>
<title>Progress Test</title>
<script language="javascript">
function go()
{
var xhr = new XMLHttpRequest();
xhr.open("GET", "startCounter.php", true);
interval = window.setInterval( function () {
update();
}, 200);
xhr.onreadystatechange = function () {
if(xhr.readyState == 4) {
window.clearInterval(interval);
//Extremely likely that the AJAX update function won't make it in time to catch
//the script at 5000 before it wins. So let's display a message instead showing
//that everything is done.
document.getElementById("updateme").innerHTML = "Operation has completed.";
}
}
xhr.send();
}
function update()
{
var xhr = new XMLHttpRequest();
xhr.onreadystatechange = function () {
if(xhr.readyState == 4 && xhr.status == 200) {
document.getElementById("updateme").innerHTML = xhr.responseText;
}
}
xhr.open("GET", "updateCounter.php", true);
xhr.send();
}
</script>
</head>
<body>
<form>
<input type="button" onclick="go()" value="Go!" />
</form>
<h3>AJAX update test</h3>
<h4 id="updateme"></h4>
</body>
</html>
Hope that gets you going in the right direction.
Related
I want to make a progress bar on my website, which tracks execution of a PHP script.
The PHP script makes a bunch of connections with Google API and stores the data it receives in the database. Sometimes the process can take a minute.
The PHP script is located in ajax/integrations-ajax.php file and launched by GET AJAX request sent, if on the website to click #link button. Below is jQuery code for the request:
$('#link').on('click', function () {
var interval = setInterval(trackStatus, 1000);
$.getJSON('ajax/integrations-ajax.php', {action: 'link'}).done(function (json) {
if (json.result == true) {
showMessage('The account is successfully linked.', 'success');
} else {
showMessage('There is an error happened.', 'danger');
}
})
});
This #link button, also sets interval which fires trackStatus function each second:
function trackStatus() {
$.getJSON('ajax/status-ajax.php', {
action: 'status'
}).done(function (json) {
console.log(json.status);
});
}
As you can see, trackStatus function sends GET AJAX requests to ajax/status-ajax.php file and should show status in browser console every second.
To implement tracking ability on the server I made the PHP script in ajax/integrations-ajax.php file to store status in the database. Its code you can see below:
<?php
if(!is_ajax_request()) { exit; }
$action = isset($_GET['action']) ? (string) $_GET['action'] : '';
if ($action == 'link') {
set_status_in_database(0);
// some execution code;
set_status_in_database(1);
// some execution code;
set_status_in_database(2);
// some execution code;
set_status_in_database(3);
// some execution code;
echo json_encode(['result' => true ]);
}
And created another PHP file axax/status-ajax.php which can recover the status from the database:
<?php
if(!is_ajax_request()) { exit; }
$action = isset($_GET['action']) ? (string) $_GET['action'] : '';
if ($action == 'status') {
$return['result'] = get_status_from_database();
echo json_encode($return);
}
But the requests appear not to be working simultaneously. I can't receive responses for trackStatus function until the response on completion ajax/integrations-ajax.php script isn't received.
I made a profiling record in browser, which show that:
So, is there a possibility to execute requests simultaneously? Or to implement the tracking ability I need to rethink the whole approach?
Thanks in advance for help!
Update
Thank you all for your advice! And especially to #Keith, because his solution is the easiest and works. I have put session_write_close() function in the beginning for the script and everything works:
<?php
if(!is_ajax_request()) { exit; }
$action = isset($_GET['action']) ? (string) $_GET['action'] : '';
if ($action == 'link') {
session_write_close();
set_status_in_database(0);
// some execution code;
set_status_in_database(1);
// some execution code;
set_status_in_database(2);
// some execution code;
set_status_in_database(3);
// some execution code;
echo json_encode(['result' => true ]);
}
Here you can see profiling record from a browser:
While PHP can handle concurrent requests without issue, one area that does get serialized is the Session, basically PHP during a request will place an exclusive lock on the SESSION, for that user. IOW: While this lock is on, other requests from the same user will have to wait. This is normally not an issue, but if you have long running requests it will block other requests, like AJax requests etc.
As a default PHP will write session data at then end off the request,. But if you are sure you no longer need to write any session data, calling session_write_close will release the lock much sooner.
More info here -> http://php.net/manual/en/function.session-write-close.php
Would advise trying EventSource. Here is an example.
PHP
<?php
header('Content-Type: text/event-stream');
// recommended to prevent caching of event data.
header('Cache-Control: no-cache');
function send_message($id, $message, $progress) {
$d = array('message' => $message , 'progress' => $progress);
echo "id: $id" . PHP_EOL;
echo "data: " . json_encode($d) . PHP_EOL;
echo PHP_EOL;
ob_flush();
flush();
}
for($i=0; $i<4; $i++){
set_status_in_database($i);
// some execution code;
send_message($i, "set status in database " . $i + 1 . " of 3' , $i*4);
sleep(1);
}
send_message('CLOSE', 'Process complete');
?>
JavaScript
var es;
function startTask() {
es = new eventSource('ajax/status-ajax.php');
es.addEventListener('message', function(e) {
var result = JSON.parse(e.data);
console.log(result.message);
if(e.lastEventId == 'CLOSE') {
console.log('Received CLOSE closing');
es.close();
showMessage('The account is successfully linked.', 'success');
} else {
$('.progress').css("width", result.progress + '%');
}
});
es.addEventListener('error', function(e) {
console.log('Error occurred', e);
es.close();
});
}
function stopTask() {
es.close();
console.log('Interrupted');
}
$('#link').on('click', function(e) {
e.preventDefault();
startTask($(this));
});
Reference:
Show Progress Report for Long-running PHP Scripts
Hope that is useful for you.
Both php logic and JavaScript syntax seem to be fine; however, with the minimal amount of php code example it is assumed that it’s resource heavy. MySQL might be busy, which is why get status may wait for MySQL.
I have gone around such a problem by making the update status written to a file instead of competing for database resources.
Since you consider using a different approach, let me recommend GraphQL as a thin layer / api above your database.
There are quite a few Php-solutions out there, for example Siler. Look for one that has subscriptions (not all do), as this would be the feature you are looking for. Subscriptions are used to create a websocket (stream between your Php and Javascript), reducing all status-related communication to one call.
Yes, this may be "shooting cannons at birds", but maybe you have other things flying around, then it might be worth considering. There is a fantastic document to familiarize with the intriguing concept. You'd be able to reuse most of your database-related Php within the resolver functions.
I am facing a strange issue here.
I am using javascript ajax(I used jquery). Now the scenario is;
One ajax call is invoking a php script which is basically a long running process and it sets some session variables.
Later in some intervals(lets say in each 2 sec) I am running another ajax calls to check the session variables to know when the process(first php script execution) is completed.
First php script is fetching data from database and wring it into a file. In each fetching I am counting the loop number and storing it into a session variable to keep some kind of tracking record. Like;
$i=0;
$_SESSION['time']=date('m-d-Y H:i:s');
while(...)
{
ini_set('session.use_only_cookies', false);
ini_set('session.use_cookies', false);
ini_set('session.use_trans_sid', false);
ini_set('session.cache_limiter', null);
session_start();
$_SESSION['tracksatus']="loop number : ".$i." time is :"$_SESSION['time'];
session_write_close();
$i++;
......
......
}
Another php script which I am invoking via setInterval ajax is just doing like;
echo $_SESSION['trackstatus']
The set interval ajax is returning me like;
loop number 1 time is m-d-Y H:i:s
loop number 5 time is m-d-Y H:i:s
loop number 8 time is m-d-Y H:i:s
......
Then after few call again;
loop number 1 time is m-d-Y H1:i1:s1
.....
Notice the change of H:i:s to H1:i1:s1
So as per my understanding the php script is invoking twice. And for your information same code was working just before 12 hrs may be. And I faced this issue before and somehow solved it(trial and error so I don't know how or may be automatically....ok actually I have no clue).
Can you please give me an insight what I am doing wrong?
Please mention if you need more information.
And the funny thing is that it is working as expected just after asking this question without even changing a single line of code. But I want to know the reason.
I think that I know what the reason, PHP writes session variables to file, but it do it only on end of script execution, so you can`t see the changes of session in another script before end of long one.
You can fix it by adding session_write_close(); session_start(); after each change of session data.
session_write_close will write changes to HD, so another script can read it.
session_start will load session from HD, but make sure that your another script make no changes for a session, these changes will be overwritten by your long script.
And one more thing if you are using separate domains:
Before actual AJAX call happen your browser sends OPTIONS request to the same URL for checking CORS headers. So on start of your script check the HTTP METHOD and if it HEAD or OPTIONS make die();
Instead of using sessions, try using a temp file to keep count with a dynamic ID
Javascript
var time = Date.now();
$.get('/firstURL?time='+time);
setInterval(function(){
$.get('/secondURL?time='+time, function(response){
console.log(response);
}
}, 1000);
PHP 1st URL
<?php
$id = $_GET['time'];
$count = 0;
while(...) {
// Do your stuff
$count++;
file_put_contents("/tmp/{$id}", $count);
}
?>
PHP 2nd URL
<?php
$id = $_GET['time'];
$count = 0;
try {
$count = file_get_contents("/tmp/{$id}");
} catch(Exception $e) {}
echo $count;
?>
As other have said PHP does not write the session until execution has finished. you better off creating a php function that you call that writes a file with the progress and then your second ajax call just reads the file.
function updateCreateProgress($jobStartTime, $progress){
file_put_contents('/tmp/'.$jobStartTime.'.txt', $progress);
}
function completeProgress($jobStartTime){
unlink('/tmp/'.$jobStartTime.'.txt')
}
now your second script can check for '/tmp/'.$jobStartTime.'.txt' if it's there read it using file_get_contents if its not there report back it has finished.
Try adjusting to something like this:
$i=0;
ini_set('session.use_only_cookies', false);
ini_set('session.use_cookies', false);
ini_set('session.use_trans_sid', false);
ini_set('session.cache_limiter', null);
session_start();
$_SESSION['time']=date('m-d-Y H:i:s');
while(...)
{
$_SESSION['tracksatus']="loop number : ".$i." time is :"$_SESSION['time'];
session_write_close();
session_start();
$i++;
......
......
}
You started talking about $_SESSION before calling session_start();
If you call ajax with GET method - you must set "cache:false" option.
Yes, you must protect your php script from other requests. With unique key (GET parameter) or session.
php lock session data for single call and release it only when this call end. Using session_write_close() when script still working - bad practice. Maybe you want save into session something more after loop but before using this data from other requests.
Flexible and clear solution:
1) script1.php - invoke from ajax for start long job.
2) script2.php (long job here) - run directly from script1.php as background without wait, or add new cron job (insert into table) and run script2.php from cron (check jobs every second or other time).
3) script3.php - check job status (ajax).
For "communication" between script2.php and script3.php can be use database or special file with flock(), clearstatcache() and flush().
I looked over a few of the questions, namely
Show progress for long running PHP script
How do you run a long PHP script and keep sending updates to the browser via HTTP?
and neither one seems to answer my question, part of which seems to be "how do I do this?" and the other half is "Hey, the way I'm doing this right now - is it the best way? Could I code this better?"
I have a simple ajax script that sends some data over to a PHP script:
$.ajax({
type: 'POST',
url: 'analysis.php',
data: { reportID:reportID, type:type, value:value, filter_type:filter_type, filter_value:filter_value, year:year },
success:function(dataReturn){
analysis_data = JSON.parse(dataReturn);
/* do stuff with analysis_data... */
});
This PHP script takes about 3 minutes to run, as it loops through a database and runs some pretty complex queries:
<?php
session_start();
ob_start();
ini_set('max_execution_time', 180);
$breaks = [ 1000, 2000, 4000, 6000, 8000, 10000, 20000, 50000, 99999999 ];
$breaks_length = count($breaks);
$p = 0;
foreach ( $breaks as $b ) {
$p++;
$percentage_complete = number_format($p / $breaks_length,2) . "%";
$sql = "query that takes about 20 seconds to run each loop of $b....";
$query = odbc_exec($conn, $sql);
while(odbc_fetch_row($query)){
$count = odbc_result($query, 'count');
}
$w[] = $count;
/* tried this... doesn't work as it screws up the AJAX handler success which expects JSON
echo $percentage_complete;
ob_end_flush();
*/
}
echo json_encode($w);
?>
All of this works - but what I'd really like to do is find a way after each foreach loop, to output $percentage_complete back to the user so they can see it working, instead of just sitting there for 2 minutes with a FontAwesome icon spinning in front of them. I tried using ob_start();, but not only does it not output anything until the page is done running, it echoes the value, which is then part of what is sent back to my AJAX success handler, causing it to screw up. (I need the output in a JSON_encoded format as I use it for something else later.)
So far in threads I've read, my only thought is to start the $breaks array loop on the previous page, so instead of looping 6 times on the same page, I loop once, return an answer, then call analysis.php again using the second element of the $breaks array, but I'm not sure this is the best way to go about things.
Also - during the 3 minutes that the user is waiting for this script to execute, they cannot do anything else on the page, so they just have to sit and wait. I'm sure there's a way to get this script to execute in such a way it doesn't "lock down" the rest of the server for the user, but everything I've searched for in Google doesn't give me a good answer for this as I'm not sure exactly what to search for...
You are encountering what is know as Session Locking. So basically PHP will not accept another request with session_start() until the first request has finished.
The immediate fix to your issue is to remove session_start(); from line #1 completely because I can see that you do not need it.
Now, for your question about showing a percentage on-screen:
analysis.php (modified)
<?php
ob_start();
ini_set('max_execution_time', 180);
$breaks = [ 1000, 2000, 4000, 6000, 8000, 10000, 20000, 50000, 99999999 ];
$breaks_length = count($breaks);
$p = 0;
foreach ( $breaks as $b ) {
$p++;
session_start();
$_SESSION['percentage_complete'] = number_format($p / $breaks_length,2) . "%";
session_write_close();
$sql = "query that takes about 20 seconds to run each loop of $b....";
$query = odbc_exec($conn, $sql);
while(odbc_fetch_row($query)){
$count = odbc_result($query, 'count');
}
$w[] = $count;
/* tried this... doesn't work as it screws up the AJAX handler success which expects JSON
echo $percentage_complete;
ob_end_flush();
*/
}
echo json_encode($w);
check_analysis_status.php get your percentage with this file
<?php
session_start();
echo (isset($_SESSION['percentage_complete']) ? $_SESSION['percentage_complete'] : '0%');
session_write_close();
Once your AJAX makes a call to analysis.php then just call this piece of JS:
// every half second call check_analysis_status.php and get the percentage
var percentage_checker = setInterval(function(){
$.ajax({
url: 'check_analysis_status.php',
success:function(percentage){
$('#percentage_div').html(percentage);
// Once we've hit 100% then we don't need this no more
if(percentage === '100%'){
clearInterval(percentage_checker);
}
}
});
}, 500);
I have done this a couple different ways, but the pattern I like the best is to have three scripts (or one controller to handle all of this), analysis_create.php, analysis.php, and analysis_status.php. The key is to create a DB object that you reference in your status checks (analysis_status.php). analysis_create.php will store all the data in the post into a DB table that will also have a column for percent_complete. The analysis_create.php function should return an ID/Token for the analysis. Once the front-end has the ID, it would post to analysis.php and then after a delay (250ms) kill the request, because you don't want to wait for it to finish. analysis.php should read the data out of the DB and start doing the work. You will need to make sure ignore_user_abort is set properly in your analysis.php script. Once the request to analysis.php is killed, you will start long polling to analysis_status.php with that ID. As analysis.php is working through the query, it should be updating the corresponding DB record with the percentage complete. analysis_status.php should look up this record and return the percentage complete to the front end.
I ran into the same issue. What caused it is different to what people are suggesting here.
Reason was gzip was enabled. Leading to a type of session locking even without an actual session.
Several ways to disable for one specific file:
How to disable mod_deflate in apache2?
Put this in httpd.conf
SetEnvIfNoCase Request_URI getMyFile\.php$ no-gzip dont-vary
I have page with customers and with ajax im loading info on whether they send us email or not.
Code looks like this:
$hostname = '{imap.gmail.com:993/imap/ssl}INBOX';
$username = 'email';
$password = 'password';
$this->session->data['imap_inbox'] = $inbox = imap_open($hostname,$username,$password) or die('Cannot connect to Gmail: ' . imap_last_error());
foreach($customers as $customer){
$emails = imap_search($inbox, 'FROM ' . $email);
// Processing info
}
But there are roughly 20-30 customers on one page, so the proccess takes sometimes about 10-20 seconds to show and I was unable to optimize the process.
But when client tries to reload a page, it is still waiting before imap_search finishes, so when reloading it could take 20 seconds before the page is actually reloaded.
I have tried to abort the ajax with beforeunload function and close the imap but this is not working.
My code:
Ajax:
$(window).bind('beforeunload',function(){
imap_email.abort(); // the ajax is succesfully aborted(as showed in console), yet the page still takes considerable time to reload
$.ajax({
type: 'GET',
url: 'getimapmails&kill=1',
async:false
}); // ajax call to the same function to call imap_close
});
PHP:
if($this->request->get['kill'] == '1'){
imap_close($this->session->data['imap_inbox']);
unset($this->session->data['imap_inbox']);
$kill == 1;
exit;
}
But even though the ajax is aborted and imap_close is called on variable holding imap_open, it still takes 10-20 seconds for page to reload, so I'm assuming the imap was not closed.
How do I close the imap so the page can reload immediately?
I would recommend killing it by creating a file that causes a break:
$hostname = '{imap.gmail.com:993/imap/ssl}INBOX';
$username = 'email';
$password = 'password';
$this->session->data['imap_inbox'] = $inbox = imap_open($hostname,$username,$password) or die('Cannot connect to Gmail: ' . imap_last_error());
foreach($customers as $customer){
clearstatcache(); //Can't use the cached result.
if(file_exists('/tmp/kill_imap.'.$this->session->id)) break; //making the assumption that /tmp and session->id are set, but the idea is a temporary folder and a unique identifier to that session.
$emails = imap_search($inbox, 'FROM ' . $email);
// Processing info
}
if(file_exists('/tmp/kill_imap.'.$this->session->id)) unlink('/tmp/kill_imap.'.$this->session->id);
Then on your exit ajax, just call to a php script that simply creates that file. and it will break your loop and remove the file.
If I understood correctly, the time-consuming code lies within the foreach() loop.
Now, even if you make a second request to kill the IMAP session, that foreach() loop will continue until either it finishes or PHP kills it if (and when) execution time exceeds your max_execution_time setting.
In any case, you need something within your foreach() loop that will check on each round if a condition to abort has been met, so as to switfly terminate the current request and allow the client to make new one.
I suggest you look at the PHP function connection_aborted(), that you could use to detect once the client aborts the current request, and more generally you could read on the topic of connection handling to get a better sense of how connections and requests are handled in PHP.
I wrote some PHP code to help me connect to a REST API for a telephone system (ie. ICWS.php.)
Then to make my life easier, I wrote a small script (ie. interations.php) that accepts two parameters: a method and an ID. This script will basically call a public method in my PHP connector.
In addition, I have another script (ie. poll.php). This script will ping the API once every half a second to see if there is a new message available. I am using server-side polling to handle this. The code below will show how poll.php
while(1){
//process Messages
$icws->processMessages();
//show the Calls Queue
$result = $icws->getCallsQueue();
//$current = $icws->getCurrentUserStatusQueue();
echo 'event: getMessagingQueue' . "\n";
echo 'data: ' . json_encode( array('calls' => $result));
echo "\n\n"; //required
ob_flush();
flush();
putToSleep($sleepTime);
}
function putToSleep($val = 1){
if( is_int($val) ){
sleep($val);
} else {
usleep($val * 1000000);
}
}
From my site (ie. phonesystem.html) I start server-side polling "which pings the API once every 1/2 seconds." From the same page, I can also make other direct calls (ie. Dial 7204536695); all requests are done via Ajax.
Here is my code that generates the server-side polling
//Server Side Message Polling
function startPolling(){
var evtSource = new EventSource("poll.php");
evtSource.addEventListener("getMessagingQueue", function(e) {
var obj = JSON.parse(e.data);
if(!obj.calls || obj.calls.length === 0){
console.log('no messages');
phoneKeyPad(false);
return;
}
processMessages(obj.calls);
}, false);
}
$(function(){
startPolling();
});
The problem that I am facing is that when making an ajax call, the response takes way too long (+1 minute).
It seems that the Apache server slows down as using other application becomes a little slower.
What can I check and how can I trouble shoot this problem?