CURL with PHP - Very slow

Question

All,

I have to request a URL which returns a JSON request. I am using PHP and CURL to do this. Currently it takes around 3-4 seconds for the request and response.

Following is the curl code

    $ch = curl_init();
    $devnull = fopen('/tmp/curlcookie.txt', 'w');

    curl_setopt($ch, CURLOPT_STDERR, $devnull);
    curl_setopt($ch, CURLOPT_POST, 1);
    curl_setopt($ch, CURLOPT_URL, $desturl);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
    curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
    curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
    curl_setopt($ch, CURLOPT_HEADER, false);
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 0);
    curl_setopt($ch, CURLOPT_CONNECTTIMEOUT,1);
    curl_setopt($ch, CURLOPT_POSTFIELDS, $postdata);

    $ret = curl_exec($ch);

    curl_close($ch);

    if ($devnull)
    {
        fclose($devnull);
    }

Following is the CURL_GETINFO array

Array
(
    [url] => https://xx.xx.xxx.xx/portalsite/tester
    [content_type] => application/json
    [http_code] => 200
    [header_size] => 198
    [request_size] => 835
    [filetime] => -1
    [ssl_verify_result] => 20
    [redirect_count] => 0
    [total_time] => 2.054561
    [namelookup_time] => 6.5E-5
    [connect_time] => 0.016048
    [pretransfer_time] => 0.123947
    [size_upload] => 699
    [size_download] => 46735
    [speed_download] => 22746
    [speed_upload] => 340
    [download_content_length] => 0
    [upload_content_length] => 0
    [starttransfer_time] => 1.743973
    [redirect_time] => 0
)

How can I speed up the CURL processing time?

Thanks

How long does it take if you visit `$desturl` with your browser? If that also takes 3-4 seconds, I don't think cURL is to blame. — Tatu Ulmanen, Dec 29 '09 at 20:41
It's actually pretty fast if I visit the URL explicitly. Less than 2 seconds — Jake, Dec 29 '09 at 20:47
I've also found out that setting CURLOPT_TIMEOUT forces cURL to use that said time, its better to not set this constant and let cURL curl — Andre Dublin, Sep 25 '12 at 18:09

score 6 · Answer 1 · edited May 23 '17 at 11:52

6

According to this answer (similar problem) cURL can be slow if you are on Mac OS X and you access to your project with xxxx.local (with 127.0.0.1 myproject.localin your /etc/hosts/

As @lepix said:

It is because the .local tld is reserved for Bonjour service, and this since Mac OS X Lion (10.7).

Hope it will help, thanks to lepix.

edited May 23 '17 at 11:52

Community

1
1

answered Sep 18 '14 at 12:15

rap-2-h

30,204
37
167
263

score 4 · Accepted Answer · answered Dec 29 '09 at 20:47

4

it looks like most of the time is waiting for the server to respond (starttransfer_time - pretransfer_time = 1.620026)... maybe the server is doing some database or other operation that takes time?

answered Dec 29 '09 at 20:47

jspcal

50,847
7
72
76

1

any way to reduce the namelookup_time? – Jake Dec 29 '09 at 20:48
use the ip instead of the name. – Femaref Dec 29 '09 at 20:52
you could add it to /etc/hosts or specify the ip address (with a Host: header) – jspcal Dec 29 '09 at 20:54
I am using the IP. Still the same. :( Wonder if it's need to be in /etc/resolv.conf? – Jake Dec 29 '09 at 20:55
2

i would guess the hostname was cached (the lookup tooks 65 microsecs) – jspcal Dec 29 '09 at 20:57
How do you specify IP address with the HOST header? I mean, what's the syntax for it? – Jake Dec 29 '09 at 21:11
something like: curl_setopt($ch, CURLOPT_HTTPHEADER, array('Host: www.host.com')); – jspcal Dec 29 '09 at 21:20

score 4 · Answer 3 · answered Jan 15 '10 at 15:17

I had some problem like that, using wget it was fast (1 second max), using cURL it took about 5 seconds to get the page, when tcpdump-ing I have found that cURL try to do a revers DNS lookup, and if the server doesn't have the revers DNS registred it will slow you down, I set up an reverse DNS on my local DNS server so every request to that site using cURL now goes very fast. I didn't find a way to disable revers DNS lookup from cURL settings.

My sugestion is to scan your trafic to see where it is waiting so long.

CURL with PHP - Very slow

3 Answers3

Linked