2

When running PHP, and you want it to immediately return HTML to the browser, close the connection (ish), and then continue processing...

The following works when the connection is HTTP/1.1, but does not when using Apache 2.4.25, with mod_http2 enabled, and you have a browser that supports HTTP/2 (e.g. Firefox 52 or Chrome 57).

What happens is the Connection: close header is not sent.

<?php

    function http_connection_close($output_html = '') {

        apache_setenv('no-gzip', 1); // Disable mod_gzip or mod_deflate

        ignore_user_abort(true);

        // Close session (if open)

        while (ob_get_level() > 0) {
            $output_html = ob_get_clean() . $output_html;
        }

        $output_html = str_pad($output_html, 1023); // Prompt server to send packet.
        $output_html .= "\n"; // For when the client is using fgets()

        header('Connection: close');
        header('Content-Length: ' . strlen($output_html));

        echo $output_html;

        flush();

    }

    http_connection_close('<html>...</html>');

    // Do stuff...

?>

For similar approaches to this problem, see:

  1. close a connection early
  2. Continue processing after closing connection
  3. Continue php script after connection close

And as to why the connection header is removed, the documentation for the nghttp2 library (as used by Apache) states:

https://github.com/nghttp2/nghttp2/blob/master/doc/programmers-guide.rst

HTTP/2 prohibits connection-specific header fields. The 
following header fields must not appear: "Connection"...

So if we cannot tell the browser to close the connection via this header, how do we get this to work?

Or is there another way of telling the browser that it has everything for the HTML response, and that it shouldn't keep waiting for more data to arrive.

Community
  • 1
  • 1
Craig Francis
  • 1,855
  • 3
  • 22
  • 35
  • Actually I doubt that your initial, general statement you make in your first sentence is valid... – arkascha Mar 17 '17 at 15:29
  • How so? the header tells the browser to close the connection. – Craig Francis Mar 17 '17 at 15:30
  • I did not doubt that... But one does _not_ typically want some server side script to continue after having sent some payload to a client. That smells if poor mans background task handling... – arkascha Mar 17 '17 at 15:32
  • I don't know how/if that's possible yet, but what you actually need is just to tell the client not to wait for more output in that particular response. In HTTP/1.x that just happens to be easily achievable by closing the connection, but the entire point of HTTP/2 is to reuse connections, so there's a subtle difference now. – Narf Mar 17 '17 at 15:32
  • Re my previous comment: Should be achievable by explicitly stating the `Content-Length`, as then browsers should consider a request complete as soon as they read that much output. – Narf Mar 17 '17 at 15:36
  • It's not typical, I use this very rarely... e.g. a user asks for something that will take a few minutes to generate, so I show a "loading" page, one that does a refresh every couple of seconds to show its progress :-) – Craig Francis Mar 17 '17 at 15:36
  • Unfortunately the browsers don't do this... there are too many websites out there which set the `Content-Length` header incorrectly. – Craig Francis Mar 17 '17 at 15:38
  • First time I hear about browsers doing this ... I'm not saying it's false (I don't know), but how sure are you about it? Have you tested? :) – Narf Mar 17 '17 at 15:40
  • Yep, the code above already sets the `Content-Length` header for this purpose... it needs the `Connection: close` as well to work on HTTP/1. As to how this should work on HTTP/2 I'm not to sure, because I don't really want the connection to be closed (e.g. it might want to request other resources). – Craig Francis Mar 17 '17 at 15:43
  • If you're using `php-fpm` and if you stop embedding PHP Into the web server (I'm surprised it's still being done), then all you have to do to achieve what you want is `echo 'your output here'; fastcgi_finish_request(); // code to be executed once output is sent to browser`. – Mjh Mar 17 '17 at 15:52
  • Must admit it's more that I've been using Apache and PHP like this for years, and haven't gotten around to setting up `php-fpm` (pro: better separation, con: more moving parts, pro: fastcgi_finish_request might fix this problem). – Craig Francis Mar 17 '17 at 15:57
  • @Mjh If you would like to create an answer saying that `fastcgi_finish_request()` would work in a `php-fpm` setup, then I can mark that as correct (I've not had any luck in getting it to work using `mod_php`). – Craig Francis Mar 24 '17 at 15:39

2 Answers2

2

How to return HTTP response to the user and resume PHP processing

This answer works only when web server communicates to PHP over FastCGI protocol.

To send the reply to user (web server) and resume processing in the background, without involving OS calls, invoke the fastcgi_finish_request() function.

Example:

<?php

echo '<h1>This is a heading</h1>'; // Output sent 

fastcgi_finish_request(); // "Hang up" with web-server, the user receives what was echoed

while(true)
{
    // Do a long task here
    // while(true) is used to indicate this might be a long-running piece of code
}

What to look out for

  • Even if user does receive the output, php-fpm child process will be busy and unable to accept new requests until they're done with processing this long running task.

If all available php-fpm child processes are busy, then your users will experience hanging page. Use with caution.

nginx and apache servers both know how to deal with FastCGI protocol so there should be no requirement to swap out apache server for nginx.

Mjh
  • 2,904
  • 1
  • 17
  • 16
  • Thanks @Mjh - and just to confirm, I was using `mod_php` before, but there does not seem to be a way to get it to close the connection. The common approach I described in the original question relies on the `Connection` header being sent to the browser, which no longer happens in HTTP/2 when Apache is using the `nghttp2 ` library, so the browser just waits for the server to close the connection (as it does not trust the `Content-Length` header by itself). – Craig Francis Mar 24 '17 at 16:14
1

You can serve your slow PHP scripts via HTTP/1.1 using a special subdomain.

All you need to do is to set a second VirtualHost that responds with HTTP/1.1 using Apache's Protocols directive : https://httpd.apache.org/docs/2.4/en/mod/core.html#protocols

The big advantage of this technic is that your slow scripts can send some datas to the browser long after everything else has been sent thru the HTTP/2 stream.

yanhl
  • 178
  • 1
  • 7
  • That's an interesting idea; it's a bit of a hacky solution, but it should work for those that need to use mod_php... so I'll give you a +1, but I still think FastCGI is the better answer for now... As to the big advantage you mention, I'm not sure that's the case, as a single HTTP/2 connection can be used for all of the resources (the connection for the HTML being "complete", and signalled as such with `fastcgi_finish_request()` well before script execution has actually finished). – Craig Francis May 26 '17 at 22:42
  • I'am also using fastcgi :-) – yanhl May 27 '17 at 11:54
  • In which case this shouldn't be a problem, or it wasn't for me when I switched over... just calling `fastcgi_finish_request()` worked fine over HTTP/2 :-) – Craig Francis May 27 '17 at 11:56
  • It works, but if you need a result after a long operation, this is a simple way to get it without breaking the link to the server, and set up a system to check when the result is available. You can think of it as the async attribute on script tags: you set up a domain with the same DocumentRoot and Protocols http/1.1, then everything you call from, for instance, http11.mydomain.com will be "async". Of course, you must restrict the access to that subdomain to prevent bots and users to access every URL with that subdomain. If you don't need a result, fastcgi_finish_request() is the way to go. – yanhl May 27 '17 at 12:12
  • Maybe I'm missing something, but `fastcgi_finish_request()` doesn't close the connection, it just tells the web server (Apache) that it's done responding to the original request (but the PHP code can continue executing), this means Apache and the browser will be able to keep their HTTP/2 connection open for everything else that's going on... whereas using a separate HTTP/1 sub domain, in the original case where we used a `Connection: close` header, with mod_php, meant that the browser would close the connection, and need to establish a new one if it needed extra resources... which isn't ideal. – Craig Francis May 27 '17 at 12:21
  • I wrote a long reply with examples explaining why I think it's sometimes useful to use HTTP/1.1 and HTTP/2: https://pastebin.com/MFYm2cv5 In a nutshell: not blocking the HTTP/2 stream when you send parallel requests expecting results that take time to generate. – yanhl May 27 '17 at 13:42
  • One of the big features of HTTP/2 is that it allows multiple things to be uploaded/downloaded though the single connection (called multiplexing), so where you say "subsequent request[s] from the browser will be queued", that's not the case... you can have many requests being processed at the same time, it was just that mod_php has the issue that you can't tell Apache you're done with the response (while still continuing processing data). :-) – Craig Francis May 27 '17 at 15:05
  • `` During the fake long task, every subsequent request is queued, right? Isn't it why you are using `fastcgi_finish_request()`? In your case it doesn't matter, but what if you need to send something to the browser after the long task, and perform other requests to the server while it is executing? – yanhl May 27 '17 at 16:51
  • Nope, subsequent requests aren't queued, this is one of the cool things about HTTP/2 :-) ... if the browser needs to wait for the result (unlike the original question), then you (as the programmer) don't do anything special, you let it take a long time, and don't use `fastcgi_finish_request()`; in the mean time, while waiting for that thing, Apache will be able to respond to other requests from the browser (as a side note, if you're using PHP sessions, you will need to close the session early, because that prevents PHP handling 2 things at once for that user - ref session locks). – Craig Francis May 27 '17 at 17:34
  • As to my original question though, I'm sending back a "loading" page, which allows the browser to display something (no waiting), and the script continues executing by itself; but you're right, you can't send anything else back via that request (the browser believes it has everything now)... but you can write the results to a temporary file (or database, or PHP session), and get the browser to keep refreshing to see when the result does appear (each time, if the result hasn't appeared yet, keep sending back the "loading" page, so it tries again a few seconds later). – Craig Francis May 27 '17 at 17:44
  • What, really ?! Because that's not how it works on my Apache installation. If I call a `sleep(120)` from a H2 connection, I can't access any other file on the same FQDN over H2 for two minutes (unless the sleep is placed after a `fastcgi_finish_request()`). Even from another tab. Chrome, Safari, Firefox... it doesn't matter, that's how my server behaves :-/ That's why I assumed you wanted to close the connection: to let some other files go thru the H2 stream. – yanhl May 27 '17 at 17:57
  • Ok, that's not right; mine handles it fine, but I'm not at my computer at the moment to create a demo. In the mean time, can you do a `` ... as it's usually the PHP session that blocks subsequent requests... and if using php-fpm, can you check the `max_children` and `max_requests` config, as it might be setup to only handle one request at a time :-) – Craig Francis May 27 '17 at 18:49
  • Just in case somebody has the same problem: I solved it by moving from Apache's Prefork MPM to Event MPM. – yanhl Jun 19 '17 at 17:31