1

I'm trying to get curl to retry as follows:

curl -vvv --retry 100 -H "Authorization: SNIP" "localhost:8086/foo"

localhost:8086/foo is configured to just always send back a "bad" HTTP status code. I've tried sending 503, 429 and 408 but in all cases, curl only tries once. Here are some example logs:

*   Trying 127.0.0.1:8086...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8086 (#0)
> GET /foo HTTP/1.1
> Host: localhost:8086
> User-Agent: curl/7.68.0
> Accept: */*
> Authorization: SNIP
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 408 Request Time-out
< Expires: 0
< Cache-Control: no-cache, no-store, max-age=0, must-revalidate
< X-XSS-Protection: 1; mode=block
< Pragma: no-cache
< X-Frame-Options: DENY
< Date: Sun, 01 Nov 2020 21:27:00 GMT
< Connection: keep-alive
< X-Content-Type-Options: nosniff
< Content-Length: 0
< 
* Connection #0 to host localhost left intact

As you can see, the logs confirm curl received a 408 but curl returns immediately. It doesn't retry another 100 times.

Do I need to specify something beyond --retry to get this to work?

Here is my curl version:

curl -V
curl 7.68.0 (x86_64-pc-linux-gnu) libcurl/7.68.0 OpenSSL/1.1.1f zlib/1.2.11 brotli/1.0.7 libidn2/2.2.0 libpsl/0.21.0 (+libidn2/2.2.0) libssh/0.9.3/openssl/zlib nghttp2/1.40.0 librtmp/2.3
Release-Date: 2020-01-08
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp 
Features: AsynchDNS brotli GSS-API HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL SPNEGO SSL TLS-SRP UnixSockets
Rupert Madden-Abbott
  • 12,899
  • 14
  • 59
  • 74
  • Does this answer your question? [Curl retry mechanism](https://stackoverflow.com/questions/42873285/curl-retry-mechanism) – Mike Doe Nov 01 '20 at 21:52
  • 1
    Gah, I actually accidentally broke retry for 408 in 7.66.0 and later! :-( I'll fix: https://github.com/curl/curl/pull/6155 – Daniel Stenberg Nov 01 '20 at 23:08
  • @emix unfortunately that does not answer my question as I am doing exactly what that answer suggests but it doesn't work. – Rupert Madden-Abbott Nov 02 '20 at 16:00
  • @DanielStenberg wow thank you! My issue is not just with 408 but also 503 and 429 but I'm glad you managed to find a bug and fix it. I've tested with https://httpstat.us/ and it does seem to work for 503 and 408 but not for 429. However, with my local Spring Boot server, none of these work. I'll try and see if I can work out the difference in the response. – Rupert Madden-Abbott Nov 02 '20 at 16:03

1 Answers1

1

I was running curl 7.68.0 on my host. Daniel Stenberg notes in a comment on my question, that 7.66.0+ has a bug that breaks the retry function for at least status 408. It seems to work fine for 503 so I can switch to that.

I was also running a docker container (debian/10.5) where the latest stable package of curl is 7.64.0. With this version, 408, 429 and 503 all seemed to be broken. I resolved this by upgrading libcurl and curl to the next available debian package (bullseye) which at the time of writing is 7.72.0 by following these steps:

  1. Download curl from appropriate mirror.
  2. Download libcurl from appropriate mirror.
  3. Install libcurl: dpkg -i libcurl4_7.72.0-1_amd64.deb
  4. Install curl: dpkg -i curl_7.72.0-1_amd64.deb

Now `curl --retry 100 localhost/foo" works when encountering a 503 status.

Rupert Madden-Abbott
  • 12,899
  • 14
  • 59
  • 74