PHP Curl And Cookies

Question

I have some problem with PHP Curl and cookies authentication.

I have a file Connector.php which authenticates users on another server and returns the cookie of the current user.

The Problem is that I want to authenticate thousands of users with curl but it authenticates and saves COOKIES only for one user at a time.

The code for connector.php is this:

    <?php
    if(!count($_REQUEST)) {
        die("No Access!");
    }


    //Core Url For Services
    define ('ServiceCore', 'http://example.com/core/');


    //Which Internal Service Should Be Called
    $path = $_GET['service'];


    //Service To Be Queried
    $url = ServiceCore.$path;

    //Open the Curl session
    $session = curl_init($url);

    // If it's a GET, put the GET data in the body
    if ($_GET['service']) {
        //Iterate Over GET Vars
        $postvars = '';
        foreach($_GET as $key=>$val) {
            if($key!='service') {
                $postvars.="$key=$val&";
            }
        }
        curl_setopt ($session, CURLOPT_POST, true);
        curl_setopt ($session, CURLOPT_POSTFIELDS, $postvars);
    }


    //Create And Save Cookies
    $tmpfname = dirname(__FILE__).'/cookie.txt';
    curl_setopt($session, CURLOPT_COOKIEJAR, $tmpfname);
    curl_setopt($session, CURLOPT_COOKIEFILE, $tmpfname);

    curl_setopt($session, CURLOPT_HEADER, false);
    curl_setopt($session, CURLOPT_RETURNTRANSFER, true);
    curl_setopt($session, CURLOPT_FOLLOWLOCATION, true);

    // EXECUTE
    $json = curl_exec($session);
        echo $json;
    curl_close($session);
?>

Here is the process of authentication:

User enters username and password: Connector.php?service=logon&user_name=user32&user_pass=123
Connector.php?service=logosessionInfo returns info about the user based on the cookies saved earlier with logon service.

The problem is that this code saves the cookie in one file for each user and can't handle multiple user authentications.

I solved my problem with creating different cookie files for different users with unique PHPSESSID. `$tmpfname = dirname(__FILE__).'/'.$_COOKIE['PHPSESSID'].'.txt';` — Shalva Kakauridze, Oct 14 '12 at 19:36
No If you have many users this is bad. It causes apache server crash. I fix this issue with apache proxy. And removed all my CURL code. — Shalva Kakauridze, Mar 08 '15 at 18:38

Adam · Answer 1 · 2012-10-14T19:19:46.050

You can specify the cookie file with a curl opt. You could use a unique file for each user.

curl_setopt( $curl_handle, CURLOPT_COOKIESESSION, true );
curl_setopt( $curl_handle, CURLOPT_COOKIEJAR, uniquefilename );
curl_setopt( $curl_handle, CURLOPT_COOKIEFILE, uniquefilename );

The best way to handle it would be to stick your request logic into a curl function and just pass the unique file name in as a parameter.

    function fetch( $url, $z=null ) {
            $ch =  curl_init();

            $useragent = isset($z['useragent']) ? $z['useragent'] : 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:10.0.2) Gecko/20100101 Firefox/10.0.2';

            curl_setopt( $ch, CURLOPT_URL, $url );
            curl_setopt( $ch, CURLOPT_RETURNTRANSFER, true );
            curl_setopt( $ch, CURLOPT_AUTOREFERER, true );
            curl_setopt( $ch, CURLOPT_FOLLOWLOCATION, true );
            curl_setopt( $ch, CURLOPT_POST, isset($z['post']) );

            if( isset($z['post']) )         curl_setopt( $ch, CURLOPT_POSTFIELDS, $z['post'] );
            if( isset($z['refer']) )        curl_setopt( $ch, CURLOPT_REFERER, $z['refer'] );

            curl_setopt( $ch, CURLOPT_USERAGENT, $useragent );
            curl_setopt( $ch, CURLOPT_CONNECTTIMEOUT, ( isset($z['timeout']) ? $z['timeout'] : 5 ) );
            curl_setopt( $ch, CURLOPT_COOKIEJAR,  $z['cookiefile'] );
            curl_setopt( $ch, CURLOPT_COOKIEFILE, $z['cookiefile'] );

            $result = curl_exec( $ch );
            curl_close( $ch );
            return $result;
    }

I use this for quick grabs. It takes the url and an array of options.

Doug · Answer 2 · 2017-12-01T14:21:43.280

In working with a similar problem I created the following function after combining a lot of resources I ran into on the web, and adding my own cookie handling. Hopefully this is useful to someone else.

      function get_web_page( $url, $cookiesIn = '' ){
        $options = array(
            CURLOPT_RETURNTRANSFER => true,     // return web page
            CURLOPT_HEADER         => true,     //return headers in addition to content
            CURLOPT_FOLLOWLOCATION => true,     // follow redirects
            CURLOPT_ENCODING       => "",       // handle all encodings
            CURLOPT_AUTOREFERER    => true,     // set referer on redirect
            CURLOPT_CONNECTTIMEOUT => 120,      // timeout on connect
            CURLOPT_TIMEOUT        => 120,      // timeout on response
            CURLOPT_MAXREDIRS      => 10,       // stop after 10 redirects
            CURLINFO_HEADER_OUT    => true,
            CURLOPT_SSL_VERIFYPEER => true,     // Validate SSL Certificates
            CURLOPT_HTTP_VERSION   => CURL_HTTP_VERSION_1_1,
            CURLOPT_COOKIE         => $cookiesIn
        );

        $ch      = curl_init( $url );
        curl_setopt_array( $ch, $options );
        $rough_content = curl_exec( $ch );
        $err     = curl_errno( $ch );
        $errmsg  = curl_error( $ch );
        $header  = curl_getinfo( $ch );
        curl_close( $ch );

        $header_content = substr($rough_content, 0, $header['header_size']);
        $body_content = trim(str_replace($header_content, '', $rough_content));
        $pattern = "#Set-Cookie:\\s+(?<cookie>[^=]+=[^;]+)#m"; 
        preg_match_all($pattern, $header_content, $matches); 
        $cookiesOut = implode("; ", $matches['cookie']);

        $header['errno']   = $err;
        $header['errmsg']  = $errmsg;
        $header['headers']  = $header_content;
        $header['content'] = $body_content;
        $header['cookies'] = $cookiesOut;
    return $header;
}

Very usefool code. It works. Thank you. If people want the HTTP status code use this syntax : $header['http_code']; — Samuel Dauzon, Jul 25 '14 at 13:24
@Doug, i've checked the regex `#Set-Cookie:\\s+(?[^=]+=[^;]+)#m` of your code in [here](https://regex101.com/r/dU3nD8/1) and the regex engine explains this part `\\s+` as: `\\` matches the character \ literally `s+` matches the character s literally (case sensitive) -- isn't it a mistake in this part of regex, shouldn't it be `\s+` (only one backslash) ? — Igor Savinkin, Jan 02 '15 at 02:52
@Igor, the \\ is because when dealing with strings in PHP [the \ is viewed as an escape character](http://php.net/manual/en/language.types.string.php). As a result you need two slashes to equal one slash in the evaluated string (Which will then be used for the RegEx match). You are correct from a RegEx standpoint I only want one slash. — Doug, Jan 03 '15 at 04:11
This helped me a lot. I changed it a bit and use: `curl_setopt($ch, CURLOPT_COOKIEFILE, ''); $func = function($cookie_entry) { $entry = explode("\t", $cookie_entry); return implode('=', array_slice($entry, -2)); }; $cookie_list = curl_getinfo($ch, CURLINFO_COOKIELIST); $cookies_str = implode('; ', array_map($func, $cookie_list)); ` — Kjeld, Oct 25 '22 at 02:48

score 18 · Answer 3 · edited May 19 '20 at 15:17

18

First create temporary cookie using tempnam() function:

$ckfile = tempnam ("/tmp", "CURLCOOKIE");

Then execute curl init witch saves the cookie as a temporary file:

$ch = curl_init ("http://uri.com/");
curl_setopt ($ch, CURLOPT_COOKIEJAR, $ckfile);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, true);
$output = curl_exec ($ch);

Or visit a page using the cookie stored in the temporary file:

$ch = curl_init ("http://somedomain.com/cookiepage.php");
curl_setopt ($ch, CURLOPT_COOKIEFILE, $ckfile);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, true);
$output = curl_exec ($ch);

This will initialize the cookie for the page:

curl_setopt ($ch, CURLOPT_COOKIEFILE, $ckfile);

edited May 19 '20 at 15:17

Lorenz Meyer

19,166
22
75
121

answered Oct 14 '12 at 19:16

Marin Sagovac

3,932
5
23
53

What about if huge number of users try to access the page at once and you already have some files in /tmp for instance? PHP docs say: Note: If PHP cannot create a file in the specified dir parameter, it falls back on the system default. On NTFS this also happens if the specified dir contains more than 65534 files. – trainoasis Feb 21 '14 at 09:19
Remove cookie files if stored for longer than 24h. Store a cookie data for $ckfile separate for checking all files inside /tmp. – Marin Sagovac Feb 24 '14 at 01:31
2

After 3 years, You are still a life saver Marin for non-PHP persons like me. After going through a load of stackoverflow answers, here is a one nicely explained in orderly manner. – Wahaj Ahmed Ansari Aug 11 '17 at 14:53

score 4 · Answer 4 · answered May 17 '15 at 17:44

Here you can find some useful info about cURL & cookies http://docstore.mik.ua/orelly/webprog/pcook/ch11_04.htm .

You can also use this well done method https://github.com/alixaxel/phunction/blob/master/phunction/Net.php#L89 like a function:

function CURL($url, $data = null, $method = 'GET', $cookie = null, $options = null, $retries = 3)
{
    $result = false;

    if ((extension_loaded('curl') === true) && (is_resource($curl = curl_init()) === true))
    {
        curl_setopt($curl, CURLOPT_URL, $url);
        curl_setopt($curl, CURLOPT_FAILONERROR, true);
        curl_setopt($curl, CURLOPT_AUTOREFERER, true);
        curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
        curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, false);
        curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);

        if (preg_match('~^(?:DELETE|GET|HEAD|OPTIONS|POST|PUT)$~i', $method) > 0)
        {
            if (preg_match('~^(?:HEAD|OPTIONS)$~i', $method) > 0)
            {
                curl_setopt_array($curl, array(CURLOPT_HEADER => true, CURLOPT_NOBODY => true));
            }

            else if (preg_match('~^(?:POST|PUT)$~i', $method) > 0)
            {
                if (is_array($data) === true)
                {
                    foreach (preg_grep('~^@~', $data) as $key => $value)
                    {
                        $data[$key] = sprintf('@%s', rtrim(str_replace('\\', '/', realpath(ltrim($value, '@'))), '/') . (is_dir(ltrim($value, '@')) ? '/' : ''));
                    }

                    if (count($data) != count($data, COUNT_RECURSIVE))
                    {
                        $data = http_build_query($data, '', '&');
                    }
                }

                curl_setopt($curl, CURLOPT_POSTFIELDS, $data);
            }

            curl_setopt($curl, CURLOPT_CUSTOMREQUEST, strtoupper($method));

            if (isset($cookie) === true)
            {
                curl_setopt_array($curl, array_fill_keys(array(CURLOPT_COOKIEJAR, CURLOPT_COOKIEFILE), strval($cookie)));
            }

            if ((intval(ini_get('safe_mode')) == 0) && (ini_set('open_basedir', null) !== false))
            {
                curl_setopt_array($curl, array(CURLOPT_MAXREDIRS => 5, CURLOPT_FOLLOWLOCATION => true));
            }

            if (is_array($options) === true)
            {
                curl_setopt_array($curl, $options);
            }

            for ($i = 1; $i <= $retries; ++$i)
            {
                $result = curl_exec($curl);

                if (($i == $retries) || ($result !== false))
                {
                    break;
                }

                usleep(pow(2, $i - 2) * 1000000);
            }
        }

        curl_close($curl);
    }

    return $result;
}

And pass this as $cookie parameter:

$cookie_jar = tempnam('/tmp','cookie');

score 1 · Answer 5 · answered Oct 14 '12 at 19:12

1

You can define different cookies for every user with CURLOPT_COOKIEFILE and CURLOPT_COOKIEJAR. Make different file for every user so each one would have it's own cookie-based session on remote server.

answered Oct 14 '12 at 19:12

s.webbandit

16,332
16
58
82

What about when this page is used by lots of users (10 thousand plus)? Isn't that a huge load on server? – trainoasis Feb 21 '14 at 09:15

Shalva Kakauridze · Accepted Answer · 2017-06-26T06:52:39.543

Solutions which are described above, even with unique CookieFile names, can cause a lot of problems on scale.

We had to serve a lot of authentications with this solution and our server went down because of high file read write actions.

The solution for this was to use Apache Reverse Proxy and omit CURL requests at all.

Details how to use Proxy on Apache can be found here: https://httpd.apache.org/docs/2.4/howto/reverse_proxy.html

PHP Curl And Cookies

6 Answers6

Linked

Related