Transalte curl command to PHP

Question

Hello I have this Linux command that downloads a compressed file

curl -L -O http://www.url.com

The problem is that when I do curl inside PHP I get the HTML code instead of the compressed file.

The PHP code is this:

$url = https://www.example.com
$filePath = '/app/storage/temp/' . $fileName;
$fp = fopen($filePath . 'me', "w");
$ch = curl_init($url);

curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_FTPAPPEND, 0);
curl_setopt($ch, CURLOPT_FILE, $fp);

$data = curl_exec($ch);
fwrite($fp, $data);
curl_close($ch);
fclose($fp);

I can't share the real URL since it contains secret keys.

EDIT

The curl command downloaded the same HTML file as the curl command when I added the -L -O options to the curl command it started working, so the thing here is, how can I add those lines with PHP

Hard to say without the actual URL to try it on, but I'd bet they've blocked access from bots or cURL clients specifically. — ceejayoz, Jul 14 '17 at 20:29
Your example doesn't show $url being set. What HTML is in $data specifically? Is it an error page? — Matt, Jul 14 '17 at 20:29
@ceejayoz I don't think they blocked curl client since if I run the command inside a Linux terminal it downloads the file — Jesus Walker, Jul 14 '17 at 21:42
Huh. Checking your original code using google.com as the URL and it works fine. This may be specific to the URL you are hitting. Maybe check the return result, dump curl_getinfo, etc.? — Jeremy, Jul 14 '17 at 22:09
https://stackoverflow.com/questions/6409462/downloading-a-large-file-using-curl — Muhammad Usman, Jul 14 '17 at 22:35
@MuhammadUsman the size of the file is not the problem is a compressed file of 123 KB. — Jesus Walker, Jul 14 '17 at 22:52

score 1 · Answer 1 · answered Jul 14 '17 at 20:34

1

Using CURL_FILE means that the output is written to the file handle. You don't need to also use fwrite (especially since without setting CURLOPT_RETURNTRANSFER, the return from curl_exec is just true or false).

If it is indeed possible to load from this URL, either remove the fwrite, or remove the CURLOPT_FILE and use:

curl_setopt($ch, CUROPT_RETURNTRANFER, TRUE)

That way, the return from curl_exec will be the loaded data.

answered Jul 14 '17 at 20:34

Jeremy

521
2
9

I've tried both ways, taking the fwrite and tge CURLOPT_FILE and it just downloads the HTML :c I also added the line you told me, but it's all the same – Jesus Walker Jul 14 '17 at 21:44
On further testing, this solution isn't even necessary. See comment on question! – Jeremy Jul 14 '17 at 22:09

score 0 · Answer 2 · answered Jul 14 '17 at 22:59

Create an empty zip file where you want to download your file.

function downloadFile($url) {
  $ch = curl_init();
  curl_setopt($ch, CURLOPT_URL, $url);
  curl_setopt($ch, CURLOPT_HEADER, 0);
  curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
  curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
  curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 2);
  curl_setopt($ch, CURLOPT_TIMEOUT, 60);
  set_time_limit(65);

  $rawFileData = curl_exec($ch);
  $info = curl_getinfo($ch);

  if (curl_errno($ch)) {
    return  curl_error($ch);
  }
  curl_close($ch);
  $filepath = dirname(__FILE__) . '/testfile.zip';
  file_put_contents($filepath, $rawFileData); // Put the downloaded content in the file
  return $info;
}

Hope this helps!

Jesus Walker · Accepted Answer · 2021-02-23T00:48:25.167

Guys sorry for making this hard for you since I couldn't give much information about my doubt.

The problem is solved, I realized that the URL did work with

curl -O -L htttp://www.example.com

and also by the web browser.

This last thing was actually the one that gave me the path:

Open the web browser
Click F12
Paste the URL and hit enter

I came to realize I needed to add some headers the headers were these:

accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8

accept-encoding:gzip, deflate, br

accept-language:en-US,en;q=0.8,es-MX;q=0.6,es;q=0.4

user-agent:Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36

After I added these headers to the curl inside PHP the result was the compressed zip file I was searching for.

Transalte curl command to PHP

3 Answers3