PHP Curl Reuse Optimization

Question

So currently I am working with a REST API provided by a third party tool that requires me to make thousands of requests per run of my script.

For the most part, everything works well the only issue is that it takes some time. Now that the logic is finished and complete, I am looking to improve the performance of the script by playing with the CURL requests.

Some notes:

• Using a third party app (like POSTMAN) I get a faster response on average per request ~ 600ms(POSTMAN) vs 1300ms(PHP Curl). I was kind of able to achieve this rate which means I think I have the best optimization I can get

• I am currently using curl_multi in other parts of my script, but the part I am currently targeting has one CURL request depend on the return value of another.

• These are all GET requests, I have POST,PUT,DELETE, and PATCH, but those are used rather sparingly so the area I am targeting are the linear GET requests.

I have done some research and for the most part everyone recommends using curl_mutli as the default, but I can't really do that as the requests are chained. I was looking at the PHP documentation and I thought about going past my basic GET request and adding some more options.

---

Original Code

So below is my first simplistic take, creates a new curl object, sets the request type, transfer, header, credentials, and SSL verification (Needed this to bypass the third part tool moving to a cloud instance). For the specific query I was testing this ran at about ~1460ms

(From test code below = 73sec/50runs = 1.46seconds)

 function executeGET($getUrl)
{
    /* Curl Options */
    $ch = curl_init($this->URL . $getUrl);
    curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "GET");
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type: application/json'));
    curl_setopt($ch, CURLOPT_USERPWD, /*INSERT CREDENTIALS*/);
    curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);

    /* Result handling and processing */
    $result = curl_exec($ch);

    return $result;
}

---

First Attempt at optimization

Next I started going through already posted stack overflow questions and the PHP documentation to look at different CURL options to see what would affect the request. The first thing I found was the IP resolve, I forced that to default to IPV4 and that sped it up by ~ 100ms. The big one however was the encoding which sped it up by about ~300ms.

(From test code below = 57sec/50runs = 1.14seconds)

    function executeGET($getUrl)
{
    /* Curl Options */
    $ch = curl_init($this->URL . $getUrl);
    curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "GET");
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type: application/json'));
    curl_setopt($ch, CURLOPT_USERPWD, /*INSERT CREDENTIALS*/);
    curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
    curl_setopt($ch, CURLOPT_IPRESOLVE, CURL_IPRESOLVE_V4 ); //NEW
    curl_setopt($ch, CURLOPT_ENCODING, ''); //NEW

    /* Result handling and processing */
    $result = curl_exec($ch);

    return $result;
}

---

Re-using same curl object

The last part I decided to try was using a single curl object but just changing the URL before each request. My logic was that all the options were the same I was just querying a different endpoint. So I merged the newly discovered options for IPRESOLVE and the ENCODING along with the re-use and this is what I got in its most striped down version.

(From test code below = 32sec/50runs = 0.64seconds)

    private $ch = null;
function executeREUSEGET($getUrl)
{
    /* Curl Options */
    if ($this->ch == null) {
        $this->ch = curl_init();
        curl_setopt($this->ch, CURLOPT_CUSTOMREQUEST, "GET");
        curl_setopt($this->ch, CURLOPT_RETURNTRANSFER, true);
        curl_setopt($this->ch, CURLOPT_HTTPHEADER, array('Content-Type: application/json'));
        curl_setopt($this->ch, CURLOPT_USERPWD, /*INSERT CREDENTIALS*/);
        curl_setopt($this->ch, CURLOPT_SSL_VERIFYPEER, false); //Needed to bypass SSL for multi calls
        curl_setopt($this->ch, CURLOPT_IPRESOLVE, CURL_IPRESOLVE_V4 ); //NEW
        curl_setopt($this->ch, CURLOPT_ENCODING, '');
    }
    curl_setopt($this->ch, CURLOPT_URL, $this->URL . $getUrl);

    /* Result handling and processing */
    $result = curl_exec($this->ch);
    return $result;
}

---

Testing Code

Reuse GET taken in seconds is: 32 
Normal GET(with extra options) taken in seconds is: 58
Normal GET taken in seconds is: 73

$common = new Common();
$startTime = time();
for ($x = 0; $x < 50; $x++){
    $r = $common->executeREUSEGET('MY_COMPLEX_QUERY');
}
echo 'Time taken in seconds is: '.(time()-$startTime);
$startTime = time();
for ($x = 0; $x < 50; $x++){
    $r = $common->executeGET('MY_COMPLEX_QUERY');
}
echo 'Reuse taken in seconds is: '.(time()-$startTime);

---

Conclusion

This was my thought process when going my code and trying to speed up my request to more closely match what I was receiving using POSTMAN. I was hoping to get some feedback on what I can improve on to speed up these specific GET requests or if there is even anything else I can do?

EDIT: I didn't really do much testing with the curl_multi in terms of optimization, but now that I have my default GET request down under a second, would it be better to convert those curl_multi into my executeReuseGet? The difference is when I use curl_multi I just need the data, none of it depends on previous input like my executeGets.