47

I have a server with Rackspace that I use for a few Ruby sites. When I try bundle install on a new site I get

Retrying download gem from http://rubygems.org/ due to error (2/4): Gem::RemoteFetcher::UnknownHostError timed out (http://rubygems.org/gems/rake-12.3.1.gem)

Or trying gem update --system I get

ERROR:  While executing gem ... (Gem::RemoteFetcher::UnknownHostError)
timed out (http://api.rubygems.org/specs.4.8.gz)

What I have tried:

  • Using curl to download the gem URL in the first error (successful)
  • Change gem source from https://rubygems.org to http://rubygems.org
  • Change the name servers on the slice to use Google's
  • Downloading and updating gem from source (on 2.7.6)
  • Run apt-get update
  • Adding AddTrustExternalCARoot-2048.pem

What I haven't tried:

  • Animal sacrifice
  • Lucky rabbit foot
  • Deleting all files and reinstalling the image

I was finally (just before submitting this) able to get bundle install to work by removing rubygems.org as a source and adding https://gems.ruby-china.org/. Why would gem be unable to access rubygems.org?

mrturtle
  • 609
  • 1
  • 5
  • 14

9 Answers9

93

api.rubygems.org is currently experiencing issues with IPv6 setup: this hostname has 4 IPv6 addresses, but responds on neither of them. Neither to ping, nor to TCP connection attempts. When you are running gem, your gem tries IPv6 addresses first and times out on them, not having time to even try IPv4 addresses.

The solution is to lower priority of IPv6 addresses for api.rubygems.org, so that gem will try IPv4 addresses first. In order to do it, put these lines into /etc/gai.conf:

# Debian defaults.
precedence  ::1/128         50
precedence  ::/0            40
precedence  2002::/16       30
precedence  ::/96           20
precedence  ::ffff:0:0/96   10

# Low precedence for api.rubygems.org IPv6 addresses.
precedence  2a04:4e42::0/32  5
Alexei Khlebnikov
  • 2,126
  • 1
  • 21
  • 21
  • Alexei, this fixed it. Thank you very much for your helpful reply! – mrturtle May 15 '18 at 17:03
  • Found this answer through this discussion: http://help.rubygems.org/discussions/problems/31074-timeout-error , which may be useful to others for context. Any case, thank you Alex for the solution, saved me lots of time. – Carlo Mazzaferro Jun 10 '18 at 00:07
  • Hasn't fixed it yet on my server, do I need to restart any services for this change to take effect? – wmakley Jul 26 '18 at 15:01
  • In most cases you don't need to restart anything. The next DNS resolution via `getaddrinfo()` will use the new settings. But if your services cache the resolved IP addresses - they may need a restart. – Alexei Khlebnikov Jul 28 '18 at 13:58
  • 12
    November 2018 and this still worked. Rubygems.org should really just pull the ipv6 addresses out of DNS until they straighten things out. – Brad Hein Nov 22 '18 at 04:00
  • 3
    Is there an equivalent fix for OSX? I don't see an `/etc/gai.conf` file. – B Seven Jan 23 '19 at 21:13
  • 2
    May 2019 and this still worked. Please fix your ipv6, rubygems! – Maximillian Laumeister May 24 '19 at 00:34
  • @AlexeiKhlebnikov Your answer saved the life of my neighbors dog (I was about to offer it as a sacrifice). Thank you! – Tino Jun 16 '19 at 09:46
  • Fedora 30, the file /etc/gai.conf does not exist and I cant find any information about this, so I used the /etc/hosts method with success – Weston Ganger Jul 09 '19 at 23:26
  • I just started having this problem today. Fortunately, I was able to find this thread quickly and can confirm this is still a working solution as of Aug10 2019. – Gino Aug 10 '19 at 10:23
  • This is _still_ a problem on Sep 27, 2019. This answer still works. – Aearnus Sep 27 '19 at 21:46
  • 11
    Febuary, 2020 and this solution still works! Please rubygems, fix it! – Luccas Paroni Feb 28 '20 at 03:44
  • 1
    After days of searching and trying solutions, this fixed my issue. Can't believe this still hasn't been fixed. Thanks for the workaround! – djilo Apr 02 '20 at 06:32
  • Saved me a lot of time fiddling with my system settings, or maybe re-installing Ruby directly without apt. Thanks!!! – Bruno P. Kinoshita May 16 '20 at 21:48
  • @AlexeiKhlebnikov Thank you savior!!! Worked like a charm! <3 – Dhrumil Panchal May 29 '20 at 01:31
  • 1
    July 2020 and this solution helped me. @AlexeiKhlebnikov how did you come up with this? – Kartikey Singh Jul 04 '20 at 06:55
  • 1
    I diagnosed the issue with ping, traceroute, telnet and nmap. Before that, I read articles on IPv6, Happy Eyeballs, RFC 3484 and 6724, documentation on getaddrinfo(), gai.conf and so on. Once I implemented Happy Eyeballs in a browser. So, I just have some technical knowledge about networking and used it to help myself and other people. :) – Alexei Khlebnikov Jul 07 '20 at 10:42
  • 5
    February 5 2021, and this solved the issue for me :) – gasc Feb 05 '21 at 17:12
  • 1
    As of 02/18/2021 on a fresh Ubuntu 20.04 Server install this fixed the issue for me as well. – Jason Shultz Feb 18 '21 at 20:24
  • 2
    Apr 21, 2021 on Ubuntu 20.03 this worked for me. – Joseph Apr 21 '21 at 17:54
  • 2
    21.04.2021, still working on `Pop!_OS 20.04 LTS (x86-64)` with kernel `Linux 5.11.0-7612-generic`. Just to mention that on my install, the `/etc/gai.conf` file is entirely commented. Simply adding the last line `precedence 2a04:4e42::0/32 5` worked like a charm. – pierre_loic Apr 25 '21 at 17:40
  • 2
    29.06.2021 Wow, still fixing the issue on Ubuntu 20.04. Also, the file was entirely commented – SMSk Jun 28 '21 at 23:10
  • 2
    This is still an issue in Aug 20th 2021 and the answer fixed it on Ubuntu 20.04.2 – Alan Evangelista Aug 20 '21 at 21:05
  • 1
    Sept 6th 2021, still an issue from Manjaro – SeedyROM Sep 06 '21 at 11:01
  • 1
    I have the same problem in 2022. Corporate VPN was my problem. Installing after disconnecting from the VPN fixed the problem. – ThE uSeFuL Mar 16 '22 at 17:31
  • 1
    October 8, 2022 still an issue and this solution still fixes it. – rojs Oct 06 '22 at 15:34
51

I didn't find /etc/gai.conf on MacOS, so as a workaround I just disabled IPV6 to download the gems. That worked for me.

'System Preferences' -> 'Network' -> Select WiFi -> click 'Advanced' button -> select 'TCP/IP' tab -> set the Configure IPV6 select to 'link-local only'.

Mauricio Mora
  • 749
  • 6
  • 13
  • 4
    I found this comment after about an hour searching for `Fetching source index from https://rubygems.org/` and `Network error while fetching https://rubygems.org/quick/Marshal.4.8/libv8-3.16.14.19-x86_64-darwin-16.gemspec.rz (execution expired)` with no luck. So simple, but this was really painful. Leaving those keywords to assist others in their searches. – Max P Magee Nov 08 '18 at 22:29
  • 2
    This worked for me. Does it mean that IPV6 is disabled? Does that affect other sites or services? – B Seven Jan 23 '19 at 21:12
  • Fedora 30, the file `/etc/gai.conf` does not exist and I cant find any information about this, so I used the /etc/hosts method with success – Weston Ganger Jul 09 '19 at 23:26
  • On Fedora 33, the file doesn't exist by default but can be created. See `man gai.conf` for defaults and details. – Roger Lehmann Feb 22 '21 at 13:12
  • This helped. However, I would only do it temporarily – eljefedelrodeodeljefe Jun 25 '21 at 06:06
21

For everybody experiencing this issue, as of rubygems 3.2.11, we're now shipping an opt-in fix for this issue inside rubygems.

If you're having this issue, you first need to upgrade rubygems to at least 3.2.11. Normally you do that by running gem update --system, but it's likely that you run into this same issue when running that command. If that's the case, you can upgrade rubygems manually following these instructions: https://bundler.io/v2.2/guides/rubygems_tls_ssl_troubleshooting_guide.html#updating-rubygems.

Once you have upgraded, put

:ipv4_fallback_enabled: true

inside your ~/.gemrc file and rubygems should no longer timeout.

The reason we haven't yet enabled this by default is that this required monkeypatching core ruby classes, so we wanted to get the fix battle tested a bit first. And also, because the underlying issue is a problem in ruby which currently lacks a happy eyeballs implementation, but it is currently being worked on, so we might not need to enable our monkeypatches by default at all if this fix in ruby lands soon.

For reference, the fix landed to our repo here: https://github.com/rubygems/rubygems/pull/2662, and the ruby issue is being worked on here: https://github.com/ruby/ruby/pull/4038.

Community
  • 1
  • 1
deivid
  • 4,808
  • 2
  • 33
  • 38
10

I added the line below to my /etc/hosts and it works.

151.101.192.70 rubygems.org

Taka Suzuki
  • 117
  • 1
  • 3
  • 7
    That line didn't work for me, but 151.101.192.70 api.rubygems.org did it – Angel Velásquez Mar 25 '19 at 15:28
  • 1
    In Aug 2020 this fix worked for me. Adding 151.101.192.70 api.rubygems.org to my /etc/hosts file using sudo. But it didn't work until I restarted my iTerm2 app on Mac. – teknix Aug 08 '20 at 05:31
  • 1
    I needed to define both `rubygems.org` and `api.rubygems.org` entries. I suppose it depends on your gemset. – raddrick Jan 03 '21 at 16:40
  • It wasn't an issue until I installed ruby 3 – raddrick Jan 03 '21 at 16:46
  • Thanks Taka Suzuki. Also RGB, i got this same error after installing ruby 3 from updating rvm. Had you experience this only scenario or stucking when `gem install or bundle install ` too? – cdrrazan Jan 13 '21 at 07:10
  • I think IP address is changed. I tried (`151.101.1.227 rubygems.org`) and it worked. – Cem Karakurt Dec 03 '21 at 13:02
  • 2023 and this is still an issue... `151.101.1.227 api.rubygems.org` works for me – Mirror318 Jul 31 '23 at 15:05
4

I still had the same issue as well. A combination of the above worked for me so thank you all. Here is what worked for me (I'm on Linux):

In the terminal enter:

dig AAAA api.rubygems.org

Find the following:

;; ANSWER SECTION:
api.rubygems.org.   60  IN  CNAME   rubygems.org.
rubygems.org.        0  IN  A   151.101.192.70

copy the IP (in my case 151.101.192.70 but according to above answers this might vary for you so make sure you do this step and check for yourself) and go to:

etc/

and open the host file like so:

sudo vim hosts

You have to open the file as a superuser otherwise you will not be able to save it since it is a read-only file. I used vim but you can use another editor. At the bottom, ad the IP you copied followed by 'rubygems.org' like so:

151.101.192.70 rubygems.org

The save the file and exit. You can try the connection in your terminal by typing:

ping rubygems.org

You should now see a continuous packet stream. Running any gem install command should now also work providing the rest of your environment is set up correctly.

Again I'm just posting this here for visibility and to help other but he heavy lifting was done by the answers above so thank you for that :)

Benedikt Schnatterbeck
  • 63
  • 1
  • 5
  • I get no IP, my output is quite different...$ dig AAAA api.rubygems.org ; <<>> DiG 9.11.3-1ubuntu1.13-Ubuntu <<>> AAAA api.rubygems.org ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 54499 ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 65494 ;; QUESTION SECTION: ;api.rubygems.org. IN AAAA ;; Query time: 0 msec ;; SERVER: 127.0.0.53#53(127.0.0.53) ;; WHEN: Fri Nov 06 16:28:20 -03 2020 ;; MSG SIZE rcvd: 45 – Wheepy Nov 06 '20 at 19:34
  • 1
    You should use one A instead of 4. This will give you IPv4 address. – Lachezar Todorov Dec 16 '20 at 22:41
2

As of this date, the IPv6 issues with rubygems.org still exist, although it seems not all the time or everywhere. I ran into it with VPSs in one data center, but not those in another. Alexei Khlebnikov's answer above is by far the best and easiest way to work around the problem (at least on Linux). However, make sure to do your own lookup for the api.rubygems.org IPv6 addresses; I found that the specific addresses above are no longer correct.

$ dig AAAA api.rubygems.org +short
rubygems.org.
2a04:4e42::70
2a04:4e42:400::70
2a04:4e42:600::70
2a04:4e42:200::70
bpope
  • 29
  • 1
  • 1
    Thanks for the kind words and for the finding that api.rubygems.org changed its IP addresses. I've updated my answer for deprioritizing the whole 2a04:4e42::0/32 subnet. – Alexei Khlebnikov Sep 14 '18 at 13:16
2

I had the same issue on Ubuntu 20.04 as of Nov 11 2020. I disabled ipv6 and it started working for me. Here are the commands to disable ipv6:

    sudo sysctl -w net.ipv6.conf.all.disable_ipv6=1
    sudo sysctl -w net.ipv6.conf.default.disable_ipv6=1
    sudo sysctl -w net.ipv6.conf.lo.disable_ipv6=1
Dipen Chauhan
  • 99
  • 1
  • 4
  • These issues are going on for so long now... Just letting everybody know that this worked for me on Ubuntu 18 @ 7 December 2020 – Marcin Dec 07 '20 at 13:50
0

For Windows users, the problem can be resolved by prioritizing IPv4 addresses over IPv6 (see https://superuser.com/a/436944). Check your prefix policies by using PowerShell as administrator:

netsh interface ipv6 show prefixpolicies

You should see that IPv6 addresses (::/0) have higher precedence over IPv4 (::/96 and ::ffff:0:0/96). To fix this, delete the IPv6 entry and re-add it with a lower precedence, for example:

netsh interface ipv6 del prefixpolicy ::/0
netsh interface ipv6 add prefixpolicy ::/0 3 6

where the precedence 3 on my machine was lower than all others, and the label 6 was unused.

Boon
  • 1,073
  • 1
  • 16
  • 42
0

This is still a problem for me at least in Jan 2020. I used Takayuki Suzuki's solution above and it worked for me.

Gordon
  • 9
  • 2