HTTP File Download: Monitoring Download Progress

Question

I am in a situation, when I have to implement downloading of large files(up to 4GB) from a Web server: Apache 2.4.4 via HTTP protocol. I have tried several approaches, but the best solution looks to be the usage of X-SendFile module.

As I offer progress bar for file uploads, I would need to have the same feature for file downloads. So here are my questions:

• Is there any way, including workaround, to achieve file downloads progress monitoring?
• Is there any way, including workaround, to calculate file download transfer speed?
• Is there better way to provide efficient file downloads from a web server than usage of X-Sendfile module?

Is there better file download option in general, that would allow me to monitor file download progress? It can be a client (JavaScript) or server solution(PHP). Is there any particular web server that allows this?

Currently I use:

• Apache 2.4.4
• Ubuntu

Many times thanks.

Answer 1

2 ideas (not verified):

First:

Instead of placing regular links to files (that you want to download) on your page place links like .../dowanload.php which may look sth like this:

<?php

    // download.php file
    session_start(); // if needed

    $filename = $_GET['filename']);

    header( 'Content-type: text/plain' ); // use any MIME you want here
    header( 'Content-Disposition: attachment; filename="' . htmlspecialchars($filename) . '"' );
    header( 'Pragma: no-cache' );

    // of course add some error handling

    $filename = 'c:/php/php.ini';

    $handle = fopen($filename, 'rb');

    // do not use file_get_contents as you've said files are up to 4GB - so read in chunks
    while($chunk = fread($handle, 1000)) // chunk size may depend on your filesize
    {
        echo $chunk;
        flush();
        // write progress info to the DB, or session variable in order to update progress bar
    }

    fclose($handle);
?>

This way you may keep eye on your download process. In the meantime you may write progress info to the DB/session var and update progress bar reading status from DB/session var using AJAX of course polling a script that reads progress info.

That is very simplified but I think it might work as you want.

Second:

Apache 2.4 has Lua language built in:

• mod_lua
• Creating hooks and scripts with mod_lua

I bet you can try to write LUA Apache handler that will monitor your download - send progress to the DB and update progress bar using PHP/AJAX taking progress info from the DB.

Similarly - there are modules for perl and even python (but not for win)

Answer 2

I see main problem in that: In a php+apache solution output buffering may be placed in several places:

Browser <= 1 => Apache <= 2 => PHP handler <= 3 => PHP Interpreter process

You need to control first buffer. But directly from PHP it is impossible.

Possible solutions:

1) You can write own mini daemon which primary function will be only send files and run it on another than 80 port 8880 for example. And process downloading files and monitor output buffer from there. Your output buffer will be only one and you can control it:

Browser <= 1 => PHP Interpreter process

2) Also you can take mod_lua and control output buffers directly from apache.

3) Also you can take nginx and control nginx output buffers using built-in perl (it is stable)

4) Try to use PHP Built-in web server and control php output buffer directly. I can't say anything about how it is stable, sorry. But you can try. ;)

I think that nginx+php+built-in perl is more stable and powerful solution. But you can choose and maybe use other solution non in that list. I will follow this topic and waiting your final solution with interest.

Answer 3

Read and write to the database at short intervals is killing performance.

I would suggest to use sessions (incrementing the value of sent data in the loop) with which you can safely off by quite another php file, you can return data as JSON which can be used by the javascript function/plugin.