I am working with api ( using curl and php ) that is supposed to return a batch of csv records based on a date. First, you make a request that returns the number of records that will be in the batch. The second request is to actually retrieve the records. My problem is when I make the second request I do NOT get the number of records that the first request indicates I am to get. Regardless of the date selected I ALWAYS get 7167 records ( the number of records for any particular day will be over 8000 ). The content length I received is what the header states I am to get. I do not get any errors. Here is the rub, if I put the url request in a browser's address bar I get ALL the records I am supposed to get.
The script runs on a linux platform. In fact, I have tried it on another linux server with the same results. I have tried changing execution time and timeout setting. I am really baffled. Thank you in advance for your suggestions.
<?php
error_reporting(E_ALL);
ini_set('display_errors', 1);
set_time_limit(0); // no limit
$username = "*********";
$pwd = "*********";
$urlCount = "https://*********&fromDate=". $today . "&toDate=" . $tomorrow";
$TotalPages = 0;
$CurrentPage = 0;
$PageSize = 1;
$RecordCount = 0;
$TotalRecordsProcessed = 0;
$today = date( "Y-m-d" );
$tomorrow = date( 'Y-m-d', strtotime( "+1 days" ) );
echo "today " . $today . "<br>";
echo "tomorrow: " . $tomorrow . "<br>";
$urlCount = "https://*********&fromDate=". $today . "&toDate=" . $tomorrow;
$headers = array("Content-Type:text/plain", 'Connection: Keep-Alive');
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $urlCount );// Get the count of the records we asked for
curl_setopt($ch, CURLOPT_HEADER, true );
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers );
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true );
curl_setopt($ch, CURLOPT_BINARYTRANSFER, true );
curl_setopt($ch, CURLOPT_TIMEOUT, 6000);
curl_setopt($ch, CURLOPT_VERBOSE, true );
curl_setopt($ch, CURLOPT_USERPWD, $username . ":" . $pwd );
$result = curl_exec( $ch ); // execute the curl function
$lines = explode( "\r\n", $result );
$idx = 0;
while( $lines[ $idx ] != null ) $idx++; // find the empty line
$RecordCount = $lines[ $idx + 1 ]; // This is the index to the number of records in the set
$PageSize = $RecordCount;
echo "Records in set: " . $RecordCount . "<br>";
$url = "https://***********&fromDate=2015-10-19&toDate=2015-10-20&pageSize=" . $PageSize . "&pageNumber=" . $CurrentPage . "&timezone=Eastern&outputType=csv";
curl_setopt($ch, CURLOPT_URL, $url ); // set the url for getting the records
$result = curl_exec( $ch ); // execute the function
$info = curl_getinfo($ch);
curl_close( $ch );
$size_download = $info["size_download"];
echo "Http Code: " . $info['http_code'] . "<br>";
echo "Size download: " . $info["size_download"] . "<br>";
// Make sure the length of the content received is the same as the "size_download" from the http header
$pos = strpos( $result, "UUID" ) - 2; // find the begining of the content
$result = substr( $result , $pos, -1 ); // remove the header data...keep just the content
$len = strlen( $result ); // get the length of the received content
if( $len != $size_download )
{
echo "length:" . strlen( $result ) . "<br>";
echo "Content recieved not = size_download<br>";
}
// Check to make sure records downloaded match the number of records that the first curl request says there are
// Records are in csv format
$lines = explode( "\r\n", $result );
$RecordsReceived = count( $lines );
if( $RecordsReceived != $RecordCount )
{
echo "Record Count = " . $RecordCount . " RecordsReceived = " . $RecordsReceived . "<br>";
}
?>