PHP cURL Realtime proxy (stream file)

Question

Currently I have a script like the following:

<?php
$filename = "http://someurl.com/file.ext";
header('Content-Type: application/octet-stream');
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,$filename);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 500);
$data=curl_exec($ch);
curl_close($ch);
echo $data;
?>

The problem is that the server just send the response after download the whole file. I want to make it work like a "stream", sending chunks of data as response while the file is downloaded.

Is that possible to achieve with PHP and cURL?

[This](http://mtdowling.com/blog/2012/01/27/chunked-encoding-in-php-with-guzzle/) might be what you're looking for. — Alexis Côté, Jul 17 '16 at 02:34
I never used composer before and I am in shared hosting, but I will search more about it. — kekit, Jul 17 '16 at 03:01

drew010 · Accepted Answer · 2016-07-18T15:29:35.067

12

It's possible. You can use the curl option CURLOPT_WRITEFUNCTION to specify a callback where you'll receive chunks of data so you can send them directly to the client as curl downloads the file.

<?php

$filename = "http://someurl.com/file.ext";
header('Content-Type: application/octet-stream');
header('Content-Disposition: attachment; filename="' . basename($filename) . '"');
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,$filename);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 500);
curl_setopt($ch, CURLOPT_WRITEFUNCTION, function($curl, $data) {
    echo $data;
    return strlen($data);
});
curl_exec($ch);
curl_close($ch);

edited Jul 18 '16 at 15:29

answered Jul 17 '16 at 05:01

drew010

68,777
11
134
162

Tested it with a 5 mb file. Instead of just gradually send the chuncks I believe that it send the data already transfered from begin many times or it is in some way corrupted. This requires a specific php version? – kekit Jul 18 '16 at 13:02
1

The writefunction should never receive the same data twice. The minimum PHP version is 5.3 to allow the anonymous function to be supplied to CURLOPT_WRITEFUNCTION. Otherwise no requirement. I tried it again with various files from 5 MB to 480 MB and it started downloading on my computer nearly immediately and the files checksums matched up and opened fine. – drew010 Jul 18 '16 at 15:26
I finally found the problem, was caused by cloudflare... The code works as expected. Thank you. – kekit Jul 24 '16 at 14:52
1

Working fine..! Thanks @drew010 – Giri Annamalai M Feb 15 '18 at 13:41
Is it possible to redirect the original content-length to the php proxy and pass it to header-function? – SuperNova Jan 15 '19 at 11:05
1

@SuperNova There is no guarantee a length is sent. If they send a content-length header, it's easy to grab that using `CURLOPT_HEADERFUNCTION` and send it before any data is sent. You may also be able to borrow some logic from [this](https://stackoverflow.com/a/33636206/892493) other answer of mine which shows how to get the content length of a remote resource. – drew010 Jan 15 '19 at 16:05
And when? After curl_exec is to late. Before there is no header data. – SuperNova Jan 17 '19 at 08:49
actually there's no need for a CURLOPT_WRITEFUNCTION here - the default action for curl if CURLOPT_FILE is null and CURLOPT_WRITEFUNCTION is null, is to just write it to stdout anyway, which is exactly what your custom CURLOPT_WRITEFUNCTION here does :) – hanshenrik Sep 24 '19 at 12:09
@SuperNova CURLOPT_HEADERFUNCTION makes it fairly easy to proxy all headers, but you must be careful to filter out the "Content-Encoding" header if you're using CURLOPT_ENCODING... something like (untested) ```curl_setopt($ch,CURLOPT_HEADERFUNCTION,function($ch,string $header){$ret=strlen($header);if(0===stripos($header,"Content-Encoding")){return $ret;}header(substr($header,0,-2));return $ret;});``` (the substr -2 is because the headers from curl includes the \r\n separators, but php's header() function does not want the \r\n separators.) – hanshenrik Sep 24 '19 at 12:25
Code can be made a little more concise by using `curl_setopt_array($ch, [CURLOPT_URL,...])` – Simon Jun 29 '21 at 15:23

score 1 · Answer 2 · answered Nov 29 '18 at 21:37

1

Curl will by default output the response directly, unless you specify CURLOPT_RETURNTRANSFER.

Your code will work just by removing CURLOPT_RETURNTRANSFER and the last echo:

<?php
$filename = "http://someurl.com/file.ext";
header('Content-Type: application/octet-stream');
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,$filename);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 500);
$data=curl_exec($ch);
curl_close($ch);
?>

answered Nov 29 '18 at 21:37

pdey

39
3

`CURLOPT_RETURNTRANSFER` is 1 of several options that stops that default behavior. others include: `CURLOPT_NOBODY` and `CURLOPT_FILE`, and `CURLOPT_WRITEFUNCTION` – hanshenrik Sep 24 '19 at 12:11

PHP cURL Realtime proxy (stream file)

2 Answers2

Linked