2

I'm trying to download a csv that's been gz'd directly to a file:

$fp = fopen ($file.'.csv', 'w+');
$ch = curl_init($url.'/'.$file.'.csv.gz');
curl_setopt($ch, CURLOPT_TIMEOUT, 50);
curl_setopt($ch, CURLOPT_FILE, $fp);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_ENCODING, 'gzip');
curl_exec($ch);
curl_close($ch);
fclose($fp);

What ends up happening is {$file}.csv is written to disk, however it's still encoded. If I rename the stored file {$file}.csv.gz and gunzip it the data is decoded properly.

lewicki
  • 480
  • 8
  • 21
  • Are you sure the content wasn't gzipped twice? http://stackoverflow.com/questions/27024900/how-to-decode-content-encoding-gzip-gzip-using-curl – Motomotes Mar 31 '15 at 04:48
  • Also be aware that version of curl are in circulation that weren't built against zlib, you can check yours using `phpinfo()` looking under ZLib version. – Motomotes Mar 31 '15 at 04:54
  • @Motes No, removing the CURLOPT_ENCODING flag and using ht ecode block below decodes it in one pass. – lewicki Mar 31 '15 at 15:07
  • Well, if your `phpinfo()` says you have Zlib then it should be working, you could try `curl_setopt($ch, CURLOPT_ENCODING, '');`, that should make curl decompress whatever encoding it finds, if it can. – Motomotes Mar 31 '15 at 15:15

1 Answers1

2

I ended up downloading the file and creating another block of code to decode it afterwards.

$buffer_size = 262144;
$in_file_handle = gzopen($symbol.'.csv.gz', 'rb');
$out_file_handle = fopen($symbol.'.csv', 'wb');
// Keep repeating until the end of the input file
while(!gzeof($in_file_handle)) {        
    fwrite($out_file_handle, gzread($in_file_handle, $buffer_size));
}
// Files are done, close files
fclose($out_file_handle);
gzclose($in_file_handle);

I would however, still be very much interested in getting curl to do it all in one step.

lewicki
  • 480
  • 8
  • 21