3

I've coded a php script which uploads CSV files, parse them and update data into a SQL database. Now that this part is done, I've found out this treatment takes a lot of time and I'd like to add a progress bar to monitor the process.

Sadly, I've found out that I should have done some kind of jQuery/JSON call from the main page from the beginning, so I would have the value returned once updated.

In order to not change the whole code, I was trying to retrieve the progression by just calling a javascript function which will printout the php value each 5 seconds. As you can imagine, it didn't work at all. If I'm not wrong, since circa 2012, in php code, no echo is sent while task is running, even using flush(); ob_flush(); etc. Is it any possible workaround? Thanks poy

main page code:

<!DOCTYPE html>
<html lang="en">
<head>
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<link rel="stylesheet" href="../../css/bootstrap.min.css">
<script src="../../src/jquery-1.9.1.js"></script>
<script src="../../src/jquery-ui.js"></script>
<script>

$(document).ready(function () {
    $('.box').css({ 'display': 'none'});
    function updb(){
            var progress = '<?= $progress;?>';
            $('.progress-bar').css('width',progress+'%');
            $('.progress-bar').text(progress+'%');
    }
    $("#planningupload").submit(function(){ 
            $('.box').removeAttr('style');
            $('.progress-bar').text('0%');
            setInterval(function(){
                updb() // this will run after every 5 seconds
            }, 5000);
    });
});
</script>
</head>
<body>
<div id="container">
<h1>CSVParser</h1> 
<form action="#" id="csvparser" method="post" enctype="multipart/form-data">
<p>Select csv filetype:</p>
  <input type="radio" name="plantype" value="fa" required> FA<br>
  <input type="radio" name="plantype" value="pi"> Pi<br><br>
<p>Select a planning to upload:</p>
<input type="file" name="csvfile" size="40" />
<input type="hidden" name="upload">
<br />
<input type="submit" class="button" id="upload" name="upload" value="upload" />
</form>
</div>
<div class="box">
 <div class="progress">
  <div class="progress-bar progress-bar-striped active" role="progressbar" style="width:0%"> 0% </div>
 </div>
</div>
<?php
if (isset($_POST["upload"])) { 
 $uploaddir = 'csv/';
 if ($_POST["plantype"]==="fa") $filename = $uploaddir . 'latestfa.csv';
 if ($_POST["plantype"]==="pi") $filename = $uploaddir . 'latestpi.csv';

...
 $progress = number_format((($k/$amount)*100), 2, '.', '');
}
?>

</html>

Update 1: Based on your recommendations, I've done the following modifications to the code & created a status.php page which shall return in real time the progress. Sadly, I still get the values updated at the end of the task. Is it any way to get the values as they come, in real time? FYI session_start(); has been added on the very top of each php file.

main file changes :

function updb(){

 $.ajax({
    url : 'status.php',
    dataType : 'json',
    success : function (data) {
       alert(data.progress); 
    },
    error : function () {
       alert("error");
    }
 })
}

...
 $progress = number_format((($k/$amount)*100), 2, '.', '');
 $_SESSION['progress'] = $progress;

status.php file :

<?php
session_start();
if (isset($_SESSION['progress'])){
     echo json_encode($_SESSION);
 } else {
    echo "none%";
 }
?>
poypoy
  • 107
  • 1
  • 13
  • See http://stackoverflow.com/questions/28856729/upload-multiple-image-using-ajax-php-and-jquery/ , http://stackoverflow.com/questions/27690326/upload-progress-with-post-processing/ – guest271314 Apr 19 '16 at 14:11
  • The short answer is: no you cannot do this. The long answer is that you could do something like set a database or session value for 'progress' in your php script and then write a short Javascript method that calls another PHP script that checks the 'progress' variable and returns it to the page. – Pete Thorne Apr 19 '16 at 14:12
  • You can still use AJAX calls to call a PHP function which will check the current progress and return it. Of course, this will only work if the second PHP functino can somehow see the progress of the other, currently running function. – JimmyB Apr 19 '16 at 14:13
  • Another option is to use a "progress bar" that doesn't move - like a spinner or similar. Or you could fake it, some software guesses how long a process with take and moves the progress bar along based on time. Of course this is not optimal – Pete Thorne Apr 19 '16 at 14:17
  • To get any sort of real-time data, you really need to use web sockets – Halfpint Apr 20 '16 at 09:00
  • @Alex Not necessarily. If you do it correctly, you can ping a PHP server, say every 5 seconds, and check for updates. I agree that web sockets would make it easier, but in this case I don't think OP will be switching to web sockets any time soon – Huw Jones Apr 20 '16 at 10:23
  • 1
    @HuwJones I misunderstood the context of the problem, the OP actually wants sudo-realtime, this would most definitely be achieved by pinging the server periodically...realtime would be achieved through the use of web-sockets. All I'm saying is the current architecture for the current system isn't easy to scale. Your solution with the database works, but if that system was rolled out to a large scale, having multiple SELECT requests on the database to retrieve a percentage could get extremely resource intensive. – Halfpint Apr 20 '16 at 10:50
  • @Alex, you're fully right as it's the case. I'm currently taking a deep look into the web sockets. Thanks for suggestion! – poypoy Apr 20 '16 at 12:58
  • @poypoy have you considered using a full Javascript development stack? You could use Node and Express on the back-end and then something like Angular or React on the front end, using socket-io and socket-io-client modules. If you read through the documentation you could have it up and running in a couple of days (although im not sure if you need to support a legacy codebase, or if this is a new application) – Halfpint Apr 20 '16 at 13:10
  • @Alex I just bought the Angular book, so yes. I got so much positive comments about running fully Javascript that I wanted to try. I'm still more oriented php as I know it better. Thanks for suggestion! – poypoy Apr 20 '16 at 16:07
  • @poypoy I migrated from php to a full JavaScript stack 12 months ago and it's the best move I've made for my career. I'm so much more productive. Feel free to get in touch if you need any help/advice, my email is alexander.Sims92@gmail.com – Halfpint Apr 20 '16 at 18:00
  • @Alex very cool! Thanks for sharing your experience. I appreciate. Will do, cheers :) – poypoy Apr 21 '16 at 08:20

1 Answers1

1

The problem is, that PHP variable will be fixed everytime you request the page. To get PHP variables into Javascript on the client side, you have to use an XMLHttpRequest (Ajax).

To do it, you could create a small script that works out the progress (and is separate from you

An example workflow for this would be as follows:

  1. Client submits CSV files.
  2. Server send client to waiting page.
  3. Client pings the server (using Ajax) every 5 seconds to get progress.
  4. Server responds with progress amount.

Since you are putting data into a SQL database, and you know how many files you are processing, you can use this to produce a progress percentage.

In a new script, check the database to see how many of the files have been processed. Turn that into a percentage, send the result to the client, and voila, progress bar data source.

Huw Jones
  • 150
  • 1
  • 9
  • For step three take a look at this answer: http://stackoverflow.com/questions/10341434/get-variable-from-php-file-using-jquery-ajax – Pete Thorne Apr 19 '16 at 14:22
  • The concept looks great ! Sadly, as mentioned in my updated topic, I still don't get the data in "real time" but only at the very end of the process. Is it somehow a way to force this data to be released? – poypoy Apr 20 '16 at 08:58
  • @poypoy You could easily do it… I'm going to update my answer with it – Huw Jones Apr 20 '16 at 09:34
  • @HuwJones thanks a lot for your suggestion. It would have work perfectly but sadly, in my case, I can't use it. The SQL is updated only if the data has changed. To avoid resources consuming, I do a query at the first beginning and I keep all the data in an array. Then I run a function which compares the CSV data and the array which contains the SQL data. If there is a change, then there is an update on the SQL server. The 90% of the php task is taken by the function doing the compare. I guess the web socket could be the "only" alternative? – poypoy Apr 20 '16 at 09:43
  • If you posted your checking script, I'm sure this could be optimised (in SQL) rather than making PHP do all the work. – Huw Jones Apr 20 '16 at 09:44