Setting Curl's Timeout in PHP

Question

I'm running a curl request on an eXist database through php. The dataset is very large, and as a result, the database consistently takes a long amount of time to return an XML response. To fix that, we set up a curl request, with what is supposed to be a long timeout.

$ch = curl_init();
$headers["Content-Length"] = strlen($postString);
$headers["User-Agent"] = "Curl/1.0";

curl_setopt($ch, CURLOPT_URL, $requestUrl);
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_USERPWD, 'admin:');
curl_setopt($ch,CURLOPT_TIMEOUT,1000);
$response = curl_exec($ch);
curl_close($ch);

However, the curl request consistently ends before the request is completed (<1000 when requested via a browser). Does anyone know if this is the proper way to set timeouts in curl?

score 422 · Answer 1 · edited Dec 21 '17 at 18:34

422

See documentation: http://www.php.net/manual/en/function.curl-setopt.php

CURLOPT_CONNECTTIMEOUT - The number of seconds to wait while trying to connect. Use 0 to wait indefinitely.
CURLOPT_TIMEOUT - The maximum number of seconds to allow cURL functions to execute.

curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 0); 
curl_setopt($ch, CURLOPT_TIMEOUT, 400); //timeout in seconds

also don't forget to enlarge time execution of php script self:

set_time_limit(0);// to infinity for example

edited Dec 21 '17 at 18:34

Frankie

24,627
10
79
121

answered Jun 16 '12 at 19:25

msangel

9,895
3
50
69

19

You don't need `set_time_limit(0);` if the script is running on the console. – Pedro Lobito Nov 19 '16 at 18:41
7

@PedroLobito what you mention is the default config of the php on cli, but it is possible that this may have been modified. – cherouvim Jul 19 '17 at 14:13
4

@cherouvim is obviously correct here (just run `php -d max_execution_time=1 -r 'while(true){$r=1*1;}'` or something to observe in action that the cli does not have a magic 'always unlimited' flag. – Wrikken Jul 16 '18 at 22:12
@Pedro Lobito you don't need `set_time_limit(0)` if you are not using it inside a loop. – Viktor Joras Sep 14 '18 at 16:18
5

CURLOPT_CONNECTTIMEOUT = 0 doesn't mean "indefinitely" https://curl.se/libcurl/c/CURLOPT_CONNECTTIMEOUT.html "Set to zero to switch to the default built-in connection timeout - 300 seconds." – C.L. Apr 16 '21 at 07:09

score 61 · Answer 2 · answered Apr 06 '10 at 01:10

61

Hmm, it looks to me like CURLOPT_TIMEOUT defines the amount of time that any cURL function is allowed to take to execute. I think you should actually be looking at CURLOPT_CONNECTTIMEOUT instead, since that tells cURL the maximum amount of time to wait for the connection to complete.

answered Apr 06 '10 at 01:10

Chad Birch

73,098
23
151
149

While the [docs in PHP](http://php.net/manual/en/function.curl-setopt.php) say `CURLOPT_TIMEOUT` is about how long the function takes, the [underlying curl library docs](http://curl.haxx.se/libcurl/c/CURLOPT_TIMEOUT.html) seem to say it's about how long the request takes, which is an interesting distinction - not sure which way to read that! – fideloper Dec 21 '15 at 19:16
I think here is the best interpretation: http://stackoverflow.com/questions/27776129/php-curl-curlopt-connecttimeout-vs-curlopt-timeout – fideloper Dec 21 '15 at 19:17

score 49 · Answer 3 · edited Jun 20 '20 at 09:12

There is a quirk with this that might be relevant for some people... From the PHP docs comments.

If you want cURL to timeout in less than one second, you can use CURLOPT_TIMEOUT_MS, although there is a bug/"feature" on "Unix-like systems" that causes libcurl to timeout immediately if the value is < 1000 ms with the error "cURL Error (28): Timeout was reached". The explanation for this behavior is:

"If libcurl is built to use the standard system name resolver, that portion of the transfer will still use full-second resolution for timeouts with a minimum timeout allowed of one second."

What this means to PHP developers is "You can't use this function without testing it first, because you can't tell if libcurl is using the standard system name resolver (but you can be pretty sure it is)"

The problem is that on (Li|U)nix, when libcurl uses the standard name resolver, a SIGALRM is raised during name resolution which libcurl thinks is the timeout alarm.

The solution is to disable signals using CURLOPT_NOSIGNAL. Here's an example script that requests itself causing a 10-second delay so you can test timeouts:

if (!isset($_GET['foo'])) {
    // Client
    $ch = curl_init('http://localhost/test/test_timeout.php?foo=bar');
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    curl_setopt($ch, CURLOPT_NOSIGNAL, 1);
    curl_setopt($ch, CURLOPT_TIMEOUT_MS, 200);
    $data = curl_exec($ch);
    $curl_errno = curl_errno($ch);
    $curl_error = curl_error($ch);
    curl_close($ch);

    if ($curl_errno > 0) {
        echo "cURL Error ($curl_errno): $curl_error\n";
    } else {
        echo "Data received: $data\n";
    }
} else {
    // Server
    sleep(10);
    echo "Done.";
}

From http://www.php.net/manual/en/function.curl-setopt.php#104597

Hello, this code works but source file is 7MB and this download me only 52KB, what could be wrong ? URL is something like http://webserver.tld/folder/download/file.do?name=file.zip&_lg=en_US — Muflix, Aug 24 '14 at 10:33
@Simon East can you please help me http://stackoverflow.com/questions/30861112/scrap-amazon-all-deals-php-curl — Padmanathan J, Jun 16 '15 at 07:32
It should be noted that you are *expecting* a timeout error with this script — kmoney12, Aug 08 '17 at 05:35

score 32 · Answer 4 · edited Apr 23 '12 at 14:49

32

Your code sets the timeout to 1000 seconds. For milliseconds, use CURLOPT_TIMEOUT_MS.

edited Apr 23 '12 at 14:49

Josh Darnell

11,304
9
38
66

answered Apr 23 '12 at 10:41

Matt Humphreys

329
3
2

MarcoP · Answer 5 · 2017-05-14T14:06:53.270

You will need to make sure about timeouts between you and the file. In this case PHP and Curl.

To tell Curl to never timeout when a transfer is still active, you need to set CURLOPT_TIMEOUT to 0, instead of 1000.

curl_setopt($ch, CURLOPT_TIMEOUT, 0);

In PHP, again, you must remove time limits or PHP it self (after 30 seconds by default) will kill the script along Curl's request. This alone should fix your issue.
In addition, if you require data integrity, you could add a layer of security by using ignore_user_abort:

# The maximum execution time, in seconds. If set to zero, no time limit is imposed.
set_time_limit(0);

# Make sure to keep alive the script when a client disconnect.
ignore_user_abort(true);

A client disconnection will interrupt the execution of the script and possibly damaging data,
eg. non-transitional database query, building a config file, ecc., while in your case it would download a partial file... and you might, or not, care about this.

Answering this old question because this thread is at the top on engine searches for CURL_TIMEOUT.

score 8 · Answer 6 · answered Apr 06 '10 at 01:33

8

You can't run the request from a browser, it will timeout waiting for the server running the CURL request to respond. The browser is probably timing out in 1-2 minutes, the default network timeout.

You need to run it from the command line/terminal.

answered Apr 06 '10 at 01:33

Brent Baisley

12,641
2
26
39

2

+1 -- the timeout is probably external to curl. You can actually work around the browser timeout by making sure to periodically output something; browsers generally reset their timeout every time they receive more data. But that's a hack; running via CLI is (almost?) always preferable. – Frank Farmer Apr 06 '10 at 02:28

score 3 · Answer 7 · edited May 23 '17 at 12:34

3

If you are using PHP as a fastCGI application then make sure you check the fastCGI timeout settings. See: PHP curl put 500 error

edited May 23 '17 at 12:34

Community

1
1

answered Jan 05 '14 at 07:58

wbinky

160
1
11

Setting Curl's Timeout in PHP

7 Answers7

Linked

Related