PHP cURL download progress using jquery

Question

I can't figure out how to push the progress out every time it updates using php. For the sake of clarity, I will write an example.

jQuery:

function uploadMovieDownload(link){
    $.post("php/downloadmovie.php", { source:link }, function(json){ console.log(json); });
}

uploadMovieDownload(url);

PHP (php/downloadmovie.php):

session_start();
ob_start();
date_default_timezone_set("Europe/Bucharest");
ini_set('display_errors',true);

require_once(dirname(__FILE__)."/functions.php");


$url = $_POST['source'];
$headers = getHeaders($url);
$url = $headers['url'];
$path = dirname(__FILE__)."/temp/test.mp4";

$fp = fopen ($path, 'w+');
$ch = curl_init();
curl_setopt( $ch, CURLOPT_URL, $url );
curl_setopt( $ch, CURLOPT_RETURNTRANSFER, false );
curl_setopt( $ch, CURLOPT_PROGRESSFUNCTION, 'progress' );
curl_setopt( $ch, CURLOPT_NOPROGRESS, false );
curl_setopt( $ch, CURLOPT_BINARYTRANSFER, true );
curl_setopt( $ch, CURLOPT_SSL_VERIFYPEER, false );
curl_setopt( $ch, CURLOPT_CONNECTTIMEOUT, 100 );
curl_setopt( $ch, CURLOPT_FILE, $fp );
curl_exec( $ch );
curl_close( $ch );
fclose( $fp );


function progress($resource,$download_size, $downloaded, $upload_size, $uploaded){
    if($download_size > 0) echo $downloaded / $download_size  * 100;
    ob_flush();
    flush();
}

echo "Done";
ob_flush();
flush();

The problem I have is that it returns the progress after it completes, it isn't pushing it while downloading. Thanks in advance if you have any sugestions.

a few ideas: 1. load balancers can inadvertently save up gradual responses. 2. you need to ship a certain # of bytes right away to get some browsers to behave right. 3. you need to return HTML with the data methodically wrapped in ` — dandavis, Dec 30 '16 at 21:42

hanshenrik · Answer 1 · 2019-03-12T12:58:27.553

with that method, you'll run in to all kind of caching problems, is php output buffering? you'll have a problem. are you behind a web server, ala nginx/apache/lighthttp/anything? you'll have a problem. is the browser cacheing output? (all mainstream browsers do), you'll have a problem.

i suggest an alternative which will have none of those problems: using $_SESSION to store download percentage, and querying the percentage with XMLHttpRequests (actually querying the percentage over WebSockets would be optimal, lagless, use less bw, etc, but much harder to implement)

downloadmovie.php

<?php 
require_once(dirname(__FILE__)."/functions.php");


$url = $_POST['source'];
$headers = getHeaders($url);
$url = $headers['url'];
//after validation of input
session_start();
$_SESSION['download_percentage']=0.0;//initialize it
session_write_close();
fastcgi_finish_request();//or if you're not using fcgi, some equivalent..

$path = dirname(__FILE__)."/temp/test.mp4";

$fp = fopen ($path, 'w+');
$ch = curl_init();
curl_setopt( $ch, CURLOPT_URL, $url );
curl_setopt( $ch, CURLOPT_RETURNTRANSFER, false );
curl_setopt( $ch, CURLOPT_PROGRESSFUNCTION, 'progress' );
curl_setopt( $ch, CURLOPT_NOPROGRESS, false );
curl_setopt( $ch, CURLOPT_BINARYTRANSFER, true );
curl_setopt( $ch, CURLOPT_SSL_VERIFYPEER, false );
curl_setopt( $ch, CURLOPT_CONNECTTIMEOUT, 100 );
curl_setopt( $ch, CURLOPT_FILE, $fp );
curl_exec( $ch );
curl_close( $ch );
fclose( $fp );


function progress($resource,$download_size, $downloaded, $upload_size, $uploaded){
    $percentage=$download_size==0? 0.0 : (($downloaded/$download_size)*100);
    session_start();
    $_SESSION['download_percentage']=$percentage;
    session_write_close();
}

getProgress.xhr.php

<?php 
if(""===session_id()){
session_start();
}
echo $_SESSION['download_percentage']??'?';

then monitoring the progress in the browser:

(function checkProgress() {
    "use strict";
    var xhr = new XMLHttpRequest();
    xhr.open("GET", "getProgress.xhr.php");
    xhr.addEventListener("readystatechange", function(ev) {
        var xhr = ev.target;
        if (xhr.readyState !== 4) {
            return;
        }
        console.log(xhr.responseText + " percent downloaded!");
        if (xhr.responseText === "100") {
            return; /*stop checking for progress when its finished*/
        }
        setTimeout(checkProgress, 1000); //<<check for progress every 1 second

    });
    xhr.send();
})();

important edit: as @drew010 pointed out, it won't work without session_write_close();session_start(); each time curl updates the value, fixed that.

I think this is the best solution, but don't forget you'll need to use `session_write_close` after each update and then `session_start` again before, otherwise the Ajax request will get stuck until the session closes since it'll be locked and block subsequent requests using the same session. — drew010, Jan 03 '17 at 03:20
Why are you wrapping session_start in an if just call it the first thing it does it check for a session_id from the cookie if it's there knows to load it. — Barkermn01, Jan 03 '17 at 03:51
@drew010 wow. i didn't know about that, thanks! ill update the code in a sec — hanshenrik, Jan 03 '17 at 07:06
@MartinBarker i don't know what runs before the actual php file is called, but some sites begins sessions in a `auto_prepend_file` php.ini directive or the like x.x in that case, calling session_begin() without checking if its already begun, would cause an error (E_NOTICE i believe?) - just checked, yup: `Notice: A session had already been started - ignoring session_start() in C:\WTServer\WWW\sesstest\2.php on line 4` — hanshenrik, Jan 03 '17 at 07:10
Strange if i try to use $_SESSION without a session start even if session already in it does not work for me... — Barkermn01, Jan 03 '17 at 11:21

score 1 · Answer 2 · edited May 23 '17 at 12:24

Quote from cURL download progress in PHP

echo "<pre>";
echo "Loading ...";

ob_flush();
flush();

$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "http://stackoverflow.com");
//curl_setopt($ch, CURLOPT_BUFFERSIZE,128);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_PROGRESSFUNCTION, 'progress');
curl_setopt($ch, CURLOPT_NOPROGRESS, false); // needed to make progress function work
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
$html = curl_exec($ch);
curl_close($ch);


function progress($resource,$download_size, $downloaded, $upload_size, $uploaded)
{
    if($download_size > 0)
         echo $downloaded / $download_size  * 100;
    ob_flush();
    flush();
    sleep(1); // just to see effect
}

echo "Done";
ob_flush();
flush();

?>

score 0 · Answer 3 · edited Oct 23 '15 at 21:02

0

Don't use ob_flush(), just flush(). (Remove all the ob_ functions). Also, consider modifying the buffersize:

curl_setopt($ch, CURLOPT_BUFFERSIZE, 16000);

Experiment with the size.

edited Oct 23 '15 at 21:02

Walery Strauch

6,792
8
50
57

answered Oct 23 '15 at 20:37

Lee Nicholls

1

LF00 · Answer 4 · 2016-12-31T21:41:25.247

0

PHP will show the results when all the scripts are done. This is the reason that you just the finish progress.

You can execute the curl downloading in the background, and write the progress to session, database, session or memory such as redis, memchche. Then in the client side read the process every very short time.

You also can use Iframe to do this.

edited Dec 31 '16 at 21:41

answered Dec 31 '16 at 21:34

LF00

27,015
29
156
295

PHP cURL download progress using jquery

4 Answers4

Linked