4

I have been trying to download a file from an API in 2 different way without any success: https://example.com/export/banana/by_date/v4?api_token=666&from=$today&to=$today

*Notice the file doesn't end with .csv, it just pop out the download file for the file.

The file that gets downloaded is a .CSV file.

I tried using CURL: 

// Date looks like this: 2016-01-31     
$today = date("Y-m-d");

    $output_filename = "test.csv";

    $host = "https://example.com/export/banana/by_date/v4?api_token=666&from=$today&to=$today";
    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL, $host);
    curl_setopt($ch, CURLOPT_VERBOSE, 1);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
    curl_setopt($ch, CURLOPT_AUTOREFERER, false);
    curl_setopt($ch, CURLOPT_REFERER, "https://www.example.com");
    curl_setopt($ch, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_1);
    curl_setopt($ch, CURLOPT_HEADER, 0);
    $result = curl_exec($ch);
    curl_close($ch);

    print_r($result); // prints the contents of the collected file before writing..


    // the following lines write the contents to a file in the same directory (provided permissions etc)
    $fp = fopen($output_filename, 'w');
    fwrite($fp, $result);
    fclose($fp);
  • With this code I got a black test.csv file.
  • I don't get anything printed on the screen after running the function (print_r($result))

and I tried using file_put_contents function: 

$today = date("Y-m-d");
echo $today;
file_put_contents("", fopen("https://example.com/export/banana/by_date/v4?api_token=666&from=$today&to=$today", 'r'));
// I TRIED THIS ONE TOO: 
// file_put_contents("temp.csv", "https://example.com/export/banana/by_date/v4?api_token=666&from=$today&to=$today");

*I get a CSV file with the URL inside it on the first row (https://example.com/export/banana/by_date/v4?api_token=666&from=2016-01-31&to=2016-01-31).

Can someone help me by telling me if I am doing this right? (since this isn't a direct link to a file maybe I'm working the wrong way). And what is the right way doing this.

Imnotapotato
  • 5,308
  • 13
  • 80
  • 147

2 Answers2

1

The target url is https so perhaps you need to add certain ssl specific options

curl_setopt( $ch, CURLOPT_SSL_VERIFYPEER, FALSE );
curl_setopt( $ch, CURLOPT_SSL_VERIFYHOST, 2 );
curl_setopt( $ch, CURLOPT_CAINFO, realpath( '/path/to/cacert.pem' ) );

another common cause of curl requests failing is the lack of a useragent string.

curl_setopt( $ch, CURLOPT_USERAGENT, 'my useragent string' );

You can set similar options when using file_get_contents by setting options for the $context

Based on your last comment, add:

 curl_setopt( $ch, CURLOPT_FOLLOWLOCATION, TRUE );
Professor Abronsius
  • 33,063
  • 5
  • 32
  • 46
  • I'm not sure what is: `curl_setopt( $ch, CURLOPT_CAINFO, realpath( '/path/to/cacert.pem' ) );` – Imnotapotato Jan 31 '16 at 16:14
  • I added: `curl_setopt( $ch, CURLOPT_SSL_VERIFYPEER, FALSE );` and `curl_setopt( $ch, CURLOPT_SSL_VERIFYHOST, 2 );` and I get: and I get `The resource was found at https://example.com/export/banana/by_date/v4?api_token=666&from=$today&to=$today_KJAGS; you should be redirected automatically.` the `_KJAGS` in the end of the URL is added, and changes the string every time I refresh the page. – Imnotapotato Jan 31 '16 at 16:17
  • you can download `cacert.pem` from the interwebs - easy to find. It is a collection of certificates for validation tasks. The `realpath()` - replace the path with the actual path the the file on your system if you intend to use it – Professor Abronsius Jan 31 '16 at 16:18
  • I added this after `print_r($result);`: `$match = ''; preg_match_all('#\bhttps?://[^\s()<>]+(?:\([\w\d]+\)|([^[:punct:]\s]|/))#', $result, $match); print_r($match[0][0]); file_put_contents($match[0][0]);` – Imnotapotato Jan 31 '16 at 16:36
  • and I got the right output. I noticed that the api URL generates a new URL so I just download it using `file_put_contents`. – Imnotapotato Jan 31 '16 at 16:38
  • Tnx mate! You helped me figure this out :) ! – Imnotapotato Feb 01 '16 at 07:47
0

This is well known (for me) problem with accesing https resources with cURL from php - it can't verify certificate in default configuration. So easiest thing to get this script works, you should add two additional lines for curl_config:

$today = date("Y-m-d");

$output_filename = "test.csv";

$host = "https://example.com/export/banana/by_date/v4?api_token=666&from=$today&to=$today";
$ch = curl_init();
$curl_config = [
    CURLOPT_URL => $host,
    CURLOPT_VERBOSE => 1,
    CURLOPT_RETURNTRANSFER => 1,
    CURLOPT_AUTOREFERER => false,
    CURLOPT_REFERER => "https://www.example.com",
    CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
    CURLOPT_HEADER => 0,
    CURLOPT_SSL_VERIFYHOST => 0, //do not verify that host matches one in certifica
    CURLOPT_SSL_VERIFYPEER => 0, //do not verify certificate's meta
];

curl_setopt_array($ch, $curl_config); //apply config

$result = curl_exec($ch);

if (empty($result)){
    echo  curl_error($ch); //show possible error if answer if empty and exit script
    exit;
}

curl_close($ch);

print_r($result); // prints the contents of the collected file before writing..

// the following lines write the contents to a file in the same directory (provided permissions etc)
file_put_contents($output_filename, $result);
Aleksey Ratnikov
  • 569
  • 3
  • 11