0

There is a web server (a) on a hosting with limited disk space. And there is a remote server (b) with video files and unlimited disk space. The client requests a php-cURL proxy from the web server, which in turn uses cURL to request a video file on the remote server (c) and using callback returns the pieces of the video file to the client (a). Everything works fine, except for one. There is no need to save the contents of the video file on the hosting (b), while the video file from the remote server (c) is accumulated in the buffer and overloads the hosting disk memory of the server (b). Where it accumulates and how to clean it is difficult for me to understand. Any tricks with CURLOPT_BUFFERSIZE do not bring results. How to prevent buffering of the entire video file on the hosting (server b). Thanks!

<?php
$filename = "http://some.remote.video.server/mp4/V36.mp4";
$ch = curl_init($filename);
curl_setopt($ch, CURLOPT_HEADERFUNCTION, function($curl, $header)
{
    return strlen($header);
}
);

bcurl_setopt($ch, CURLOPT_WRITEFUNCTION, function($curl, $body)
{
    echo $body;
    $len = strlen($body);
    return $len;
}
);
curl_setopt($ch, CURLINFO_HEADER_OUT, true);
curl_exec($ch);
curl_close($ch);
?>
nikolaos
  • 11
  • 2
  • Possible duplicate of [PHP cURL Realtime proxy (stream file)](https://stackoverflow.com/questions/38417350/php-curl-realtime-proxy-stream-file) – Koala Yeung Sep 24 '19 at 10:37
  • Yes. The callback functions work with chunks that are returned to the client as it is received. There is no need to store the entire file in memory - RAM or HDD. – nikolaos Sep 24 '19 at 17:15
  • @hanshenrik: From what I read from this question: (a) a client, or "requester", in your words, would request a file from the web server; and (b) the web server need to request for the file from a remote server, then stream the response directly back to the client without storing the file locally. This seems to be exactly what [that other question](https://stackoverflow.com/questions/38417350/php-curl-realtime-proxy-stream-file) is about. As a friendly discussion between fellow users, can you please elaborate more on the difference from what you read from it? – Koala Yeung Sep 25 '19 at 01:44
  • @KoalaYeung dang, yeah you're right, somehow i got the question very wrong yesterday, not sure how that happened. you're also right, that is a duplicate. – hanshenrik Sep 25 '19 at 17:17
  • @hanshenrik "Without storing the file locally". Not really. Indeed, chunks are sent to the client (a) in callback function. But the functions curl_exec ($ch) is executed until it uploads the entire file to server (b). If it is for example 1 GB, then it takes just so much 1 GB server (b) disk space. Resources (b) are freed only when the curl_close($ch) is called. And my question is how to avoid uploading the entire file to the server (b). – nikolaos Sep 26 '19 at 20:39
  • @nikolaos no that's only if you were using CURLOPT_FILE or CURLOPT_RETURNTRANSFER. with your CURLOPT_WRITEFUNCTION + echo approach, it's just (likely 16k bytes) chunks that are stored in memory until being echoed, then the memory is freed/re-used when it reaches the end of the WRITEFUNCTION callback - only exception to this would be if you're using ob_start(); & co ... are you using ob_start() ? or perhaps, are you using a web framework that is using ob_start() behind the scene for you? – hanshenrik Sep 27 '19 at 07:38
  • @hanshenrik Yes, you are probably right. But the disk space on the server (b) and during file transfer is constantly out of limit. Perhaps this is a some temporary file/stream in the bowels of cURL? I don’t even know how to solve this problem. Thank You! – nikolaos Sep 28 '19 at 09:39
  • no, there isn't. put this on a page on the proxy server and load it with curl/wget: ``` – hanshenrik Sep 28 '19 at 09:42
  • @hanshenrik (1) wget -O /dev/null http://speedtest.tele2.net/100GB.zip -> No problem. No disk space used. (2) Your first example with curl -> After a few seconds of the request in the hosting control panel: 2257 Mb from 1800 used! Limit exceeded! – nikolaos Sep 28 '19 at 10:36
  • @nikolaos 2257Mb of what? is it talking about RAM? or disk space? or bandwidth? – hanshenrik Sep 28 '19 at 10:45
  • @hanshenrik Disk space of the virtual hosting. – nikolaos Sep 28 '19 at 10:55
  • Also. Disk space limit. – nikolaos Sep 28 '19 at 10:57
  • if you use CURLOPT_RETURNTRANSFER then it will store everything in ram, and unless your proxy server has over 100GB of ram, it will ofc get an error with CURLOPT_RETURNTRANSFER – hanshenrik Sep 28 '19 at 11:00
  • The same result. – nikolaos Sep 28 '19 at 11:02
  • This is the solution. Thanks to @hanshenrik: – nikolaos Oct 09 '19 at 22:05

0 Answers0