Php file_get_contents not capturing certain group of cookies

Question

I have created a script to automate downloading a file created by an app that sends an email notice to the user when the file is generated. This script logs in, then executes a second time with some session items set on the host site and is able to download the file.

Recently the host site created an update with some addition verification and since my script has not been working. I have implemented everything related to the updated verification process to ensure basically that XSS attacks are mitigated but I am running into a road block where now a set of GUID values are not in "Set-Cookie:" any longer.

I am first making a GET to the login page to get the unique values for that attempt.

$url = "https://loginurl.com/";

$options = array(
    'https' => array(
        'header' => "",
        'method' => 'GET',
        'content' => "",
    ),
);
$post_context = stream_context_create($options);

$post_raw_result = file_get_contents($url, false, $post_context);

$unique_headers = $http_response_header;

I assign the http_response_header and then pick some values I need but it doesnt contain all the cookies I see in the browser.

I have worked with curl, executing curl from shell_exec() and over the past few weeks and file_get seems to be the best option for me but I have no ideal why I cant see these GUID items any more.. I can see them in chrome or Mozilla when inspecting the communication. I thought it may be something with multiple cookies and tried several items like:

$cookies_array = array();
foreach($http_response_header as $s)
{
  if (preg_match('|^Set-Cookie:\s*([^=]+)=([^;]+);(.+)$|', $s, $parts))
  {
    $cookies_array[] = $parts[1] . '=' . $parts[2];
  }
}

found here and on a few other threads for similar questions but this doesnt seem to capture any additional info.

I had some luck finding more data by checking $_COOKIE but still didnt contain 2 of the GUID values I needed.

I even moved to postman and could see the same cookies defined there that I cannot seem to capture with php. Any ideas why they wouldnt be in the http_header_response or where else to look?

score 0 · Answer 1 · answered Apr 26 '18 at 14:02

Might be http://php.net/manual/en/reserved.variables.httpresponseheader.php - first comment:

Note that the HTTP wrapper has a hard limit of 1024 characters for the header lines. Any HTTP header received that is longer than this will be ignored and won't appear in $http_response_header.

If not I would recommend using CURL

$curl = curl_init();
curl_setopt($curl, CURLOPT_HEADER, 1); // This is what you looking for
curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($curl, CURLOPT_VERBOSE, 1);

// Your other options - see http://php.net/manual/en/function.curl-setopt.php

$response = curl_exec($curl);

// Check for $response success
$header = substr($response, 0, curl_getinfo($curl, CURLINFO_HEADER_SIZE)); // There you will get headers

Or you can get it of this headache and you something like Gout - http://goutte.readthedocs.io/en/latest/ which handles cookies for you.

Thank you for the response, working on a curl version again to see if i can make it work and checking out goutte — Allen Craig, Apr 26 '18 at 16:40

score 0 · Accepted Answer · answered Apr 28 '18 at 23:00

Several items to incorporate to solve this. 1. moved to curl 2. remove content length (this caused redirect following to timeout everytime when set) 3. Make a second request and store the cookie and present all stored cookies.

Had to use all the options below

    CURLOPT_RETURNTRANSFER => true,
    CURLOPT_ENCODING => "",
    CURLOPT_MAXREDIRS => 10,
    CURLOPT_TIMEOUT => 30,
    CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
    CURLOPT_CUSTOMREQUEST => "POST",
    CURLOPT_POSTFIELDS => "",
    CURLOPT_FOLLOWLOCATION => true,
    CURLOPT_COOKIEJAR => 'cookie.txt',
    CURLOPT_COOKIEFILE => 'cookie.txt',
    CURLOPT_SSL_VERIFYPEER => false,

Php file_get_contents not capturing certain group of cookies

2 Answers2