Git RPC failed; curl 18 transfer closed with outstanding read data remaining

Question

Any git clone of a repo above a certain size fails halfway through, I can use ssh method however for this case http is required as it is a secondary action as part of brew install.

$ brew cask install homebrew/cask-versions/adoptopenjdk8

==> Tapping homebrew/cask
Cloning into '/usr/local/Homebrew/Library/Taps/homebrew/homebrew-cask'...
remote: Enumerating objects: 422119, done.
error: RPC failed; curl 18 transfer closed with outstanding read data remaining
fatal: the remote end hung up unexpectedly
fatal: early EOF
fatal: index-pack failed
Error: Failure while executing; `git clone https://github.com/Homebrew/homebrew-cask /usr/local/Homebrew/Library/Taps/homebrew/homebrew-cask` exited with 128.
Follow the instructions here:
  https://github.com/Homebrew/homebrew-cask#reporting-bugs
/usr/local/Homebrew/Library/Homebrew/utils.rb:266:in `safe_system'
/usr/local/Homebrew/Library/Homebrew/tap.rb:273:in `install'
/usr/local/Homebrew/Library/Homebrew/cask/cmd.rb:157:in `run'
/usr/local/Homebrew/Library/Homebrew/cask/cmd.rb:123:in `run'
/usr/local/Homebrew/Library/Homebrew/cmd/cask.rb:9:in `cask'
/usr/local/Homebrew/Library/Homebrew/brew.rb:103:in `<main>'

This is on a brand new machine, with a very fast internet connection (verified the git clone works on other machines). Additionally I have increased buffer size

git config --global http.postBuffer 524288000

I have no problem cloning small repos in both http and ssh method. I have reviewed similar stack questions such as the one below but they don't cover this scenario.

error: RPC failed; curl transfer closed with outstanding read data remaining

Do you have any proxies, firewalls, antivirus software, SSL/TLS MITM devices, or other network interceptors in place? Also, `http.postBuffer` has no effect on clones and increasing it will only serve to waste lots of memory when you push. — bk2204, Mar 27 '20 at 01:57
No proxies or antivirus, its a new mac out the box. No devices either, but thanks for the heads up on the http.postbuffer ! — Murchie85, Mar 27 '20 at 02:00
@Murchie85 Which of the two options I proposed below worked in your case? — VonC, Mar 27 '20 at 11:13

score 25 · Answer 1 · answered Mar 27 '20 at 11:19

25

It turns out in my case it was indeed an anti-virus that I wasn't aware of, Virgin Broadband ISP use a built in Virus Safe that I did not know was activated without my consent. For those with this isp, it can be deactivated via https://my.virginmedia.com/my-apps/onlinesecurity/websafe/settings

answered Mar 27 '20 at 11:19

Murchie85

815
2
12
27

1

WOW I would never have worked this out -- thank you! – Neil Vass Dec 11 '20 at 22:30
I had to turn off all the Home Safe settings from my TalkTalk account – s_diaconu Jan 09 '21 at 17:04
Tried increasing the buffer, try to shallow unshallow the git clone, and here it is Virgin (Wales, UK) was filtering out the git repo... – Cyril Feb 19 '21 at 11:32
1

I found it was "Web Safe" rather than "Virus Safe" on virginmedia. Simply adding github.com to the "Allowed Websites" whitelist was sufficient to solve this – Rob Cowell Nov 09 '21 at 11:46

score 10 · Accepted Answer · answered Mar 27 '20 at 05:38

10

If the issue persists, meaning if you see the same error message when trying yourself:

git clone https://github.com/Homebrew/homebrew-cask /usr/local/Homebrew/Library/Taps/homebrew/homebrew-cask

Then try:

either an SSH clone:

git clone git@github.com:Homebrew/homebrew-cask /usr/local/Homebrew/Library/Taps/homebrew/homebrew-cask

or a shallow clone as here

git clone --depth 1 https://github.com/Homebrew/homebrew-cask /usr/local/Homebrew/Library/Taps/homebrew/homebrew-cask
cd /usr/local/Homebrew/Library/Taps/homebrew/homebrew-cask
git fetch --unshallow

answered Mar 27 '20 at 05:38

VonC

1,262,500
529
4,410
5,250

2

Marked as the right answer, although in my case it did indeed turn out to be anti-virus software that was installed on my ISP end without my knowledge. I realised when switching to ssh and noticed the port was blocked. The methods you have listed work well when there is a poor connection speed. – Murchie85 Mar 27 '20 at 11:14
I'm getting the following when trying to unshallow - `sudo git fetch --unshallow remote: Enumerating objects: 482427, done. error: RPC failed; curl 18 transfer closed with outstanding read data remaining fatal: the remote end hung up unexpectedly fatal: protocol error: bad pack header` – Toby Beresford Oct 01 '20 at 08:33
1

@TobyBeresford `sudo`? Nobody makes Git commands as root, usually. What OS and OS version are you using, what Git version are you using? – VonC Oct 01 '20 at 08:36
Mac OS X 10.15.6 Catalina - Git 2.23.0. I was trying to overwrite existing partial clones hence the sudo, agree not great practice. :) – Toby Beresford Oct 01 '20 at 08:50
@TobyBeresford OK, if the issue persists after upgrading Git to 2.28, you might want to consider posting a separate question, in order to make this more visible than buried in comments. – VonC Oct 01 '20 at 08:54
@VonC Thanks, though I've found the answer for my own setup which was the ISP settings needed github to be an Allowed website. – Toby Beresford Oct 01 '20 at 08:55

score 7 · Answer 3 · answered Oct 01 '20 at 07:08

&tldr; brew update-reset

I just had the same issue. First I tried @VonC 's answer but it didn't help. I got message

➜  ~ git clone git@github.com:Homebrew/homebrew-cask /usr/local/Homebrew/Library/Taps/homebrew/homebrew-cask
fatal: destination path '/usr/local/Homebrew/Library/Taps/homebrew/homebrew-cask' already exists and is not an empty directory.

Googling next for that message gave me this link with solution (in case of mine) of running command

brew update-reset

After that problem was solved

➜  ~ brew update-reset
==> Fetching /usr/local/Homebrew...
remote: Enumerating objects: 61, done.
remote: Counting objects: 100% (34/34), done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 7 (delta 4), reused 3 (delta 0), pack-reused 0
Unpacking objects: 100% (7/7), 3.55 KiB | 454.00 KiB/s, done.
From https://github.com/Homebrew/brew
 * [new branch]          imgbot     -> origin/imgbot

==> Resetting /usr/local/Homebrew...
Branch 'master' set up to track remote branch 'master' from 'origin'.
Reset branch 'master'

==> Fetching /usr/local/Homebrew/Library/Taps/dart-lang/homebrew-dart...

==> Resetting /usr/local/Homebrew/Library/Taps/dart-lang/homebrew-dart...
Branch 'master' set up to track remote branch 'master' from 'origin'.
Reset branch 'master'
Your branch is up to date with 'origin/master'.

==> Fetching /usr/local/Homebrew/Library/Taps/elastic/homebrew-tap...

==> Resetting /usr/local/Homebrew/Library/Taps/elastic/homebrew-tap...
Branch 'master' set up to track remote branch 'master' from 'origin'.
Reset branch 'master'
Your branch is up to date with 'origin/master'.

==> Fetching /usr/local/Homebrew/Library/Taps/heroku/homebrew-brew...

==> Resetting /usr/local/Homebrew/Library/Taps/heroku/homebrew-brew...
Branch 'master' set up to track remote branch 'master' from 'origin'.
Reset branch 'master'
Your branch is up to date with 'origin/master'.

==> Fetching /usr/local/Homebrew/Library/Taps/homebrew/homebrew-cask...

==> Resetting /usr/local/Homebrew/Library/Taps/homebrew/homebrew-cask...
Branch 'master' set up to track remote branch 'master' from 'origin'.
Reset branch 'master'
Your branch is up to date with 'origin/master'.

==> Fetching /usr/local/Homebrew/Library/Taps/homebrew/homebrew-cask-versions...

==> Resetting /usr/local/Homebrew/Library/Taps/homebrew/homebrew-cask-versions...
Branch 'master' set up to track remote branch 'master' from 'origin'.
Reset branch 'master'
Your branch is up to date with 'origin/master'.

==> Fetching /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core...
fatal: unable to access 'https://github.com/Homebrew/homebrew-core/': transfer closed with outstanding read data remaining

==> Resetting /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core...
Branch 'master' set up to track remote branch 'master' from 'origin'.
Reset branch 'master'
Your branch is up to date with 'origin/master'.

==> Fetching /usr/local/Homebrew/Library/Taps/homebrew/homebrew-services...

==> Resetting /usr/local/Homebrew/Library/Taps/homebrew/homebrew-services...
Branch 'master' set up to track remote branch 'master' from 'origin'.
Reset branch 'master'
Your branch is up to date with 'origin/master'.

==> Fetching /usr/local/Homebrew/Library/Taps/sass/homebrew-sass...

==> Resetting /usr/local/Homebrew/Library/Taps/sass/homebrew-sass...
Branch 'master' set up to track remote branch 'master' from 'origin'.
Reset branch 'master'
Your branch is up to date with 'origin/master'

Although I see there were one issue with fetching homebrew-core all operation finished successfully and I was able to update packages

➜  ~ brew update && brew upgrade && brew cleanup
Updated 1 tap (homebrew/core).
==> New Formulae
cloudformation-guard
==> Updated Formulae
aws-okta              bitrise               cbmc                  crowdin               frpc                  monero                openrct2              ssh-audit             traefik
==> Upgrading 1 outdated package:
heroku/brew/heroku 7.43.2 -> 7.44.0
==> Upgrading heroku/brew/heroku 7.43.2 -> 7.44.0
==> Downloading https://cli-assets.heroku.com/heroku-v7.44.0/heroku-v7.44.0.tar.xz
######################################################################## 100.0%
==> Caveats
To use the Heroku CLI's autocomplete --
  Via homebrew's shell completion:
    1) Follow homebrew's install instructions https://docs.brew.sh/Shell-Completion
        NOTE: For zsh, as the instructions mention, be sure compinit is autoloaded
          and called, either explicitly or via a framework like oh-my-zsh.
    2) Then run
      $ heroku autocomplete --refresh-cache

  OR

  Use our standalone setup:
    1) Run and follow the install steps:
      $ heroku autocomplete

Bash completion has been installed to:
  /usr/local/etc/bash_completion.d

zsh completions have been installed to:
  /usr/local/share/zsh/site-functions
==> Summary
  /usr/local/Cellar/heroku/7.44.0: 27,477 files, 59.4MB, built in 40 seconds
Removing: /usr/local/Cellar/heroku/7.43.2... (27,475 files, 59.4MB)
Removing: /Users/tpojka/Library/Caches/Homebrew/heroku--7.43.2.tar.xz... (6.9MB)
Removing: /Users/tpojka/Library/Caches/Homebrew/libyaml--0.2.5.catalina.bottle.tar.gz... (107.4KB)
Removing: /Users/tpojka/Library/Caches/Homebrew/nghttp2--1.41.0.catalina.bottle.tar.gz... (968.6KB)
Removing: /Users/tpojka/Library/Logs/Homebrew/htop... (64B)
Removing: /Users/tpojka/Library/Logs/Homebrew/guile... (64B)
Removing: /Users/tpojka/Library/Logs/Homebrew/gnupg... (64B)

I went one more time with brew update-reset command because of error previously seen and then everything went good with no errors:

➜  ~
➜  ~ brew update-reset
==> Fetching /usr/local/Homebrew...

==> Resetting /usr/local/Homebrew...
Branch 'master' set up to track remote branch 'master' from 'origin'.
Reset branch 'master'
Your branch is up to date with 'origin/master'.

==> Fetching /usr/local/Homebrew/Library/Taps/dart-lang/homebrew-dart...

==> Resetting /usr/local/Homebrew/Library/Taps/dart-lang/homebrew-dart...
Branch 'master' set up to track remote branch 'master' from 'origin'.
Reset branch 'master'
Your branch is up to date with 'origin/master'.

==> Fetching /usr/local/Homebrew/Library/Taps/elastic/homebrew-tap...

==> Resetting /usr/local/Homebrew/Library/Taps/elastic/homebrew-tap...
Branch 'master' set up to track remote branch 'master' from 'origin'.
Reset branch 'master'
Your branch is up to date with 'origin/master'.

==> Fetching /usr/local/Homebrew/Library/Taps/heroku/homebrew-brew...

==> Resetting /usr/local/Homebrew/Library/Taps/heroku/homebrew-brew...
Branch 'master' set up to track remote branch 'master' from 'origin'.
Reset branch 'master'
Your branch is up to date with 'origin/master'.

==> Fetching /usr/local/Homebrew/Library/Taps/homebrew/homebrew-cask...

==> Resetting /usr/local/Homebrew/Library/Taps/homebrew/homebrew-cask...
Branch 'master' set up to track remote branch 'master' from 'origin'.
Reset branch 'master'
Your branch is up to date with 'origin/master'.

==> Fetching /usr/local/Homebrew/Library/Taps/homebrew/homebrew-cask-versions...

==> Resetting /usr/local/Homebrew/Library/Taps/homebrew/homebrew-cask-versions...
Branch 'master' set up to track remote branch 'master' from 'origin'.
Reset branch 'master'
Your branch is up to date with 'origin/master'.

==> Fetching /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core...

==> Resetting /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core...
Branch 'master' set up to track remote branch 'master' from 'origin'.
Reset branch 'master'
Your branch is up to date with 'origin/master'.

==> Fetching /usr/local/Homebrew/Library/Taps/homebrew/homebrew-services...

==> Resetting /usr/local/Homebrew/Library/Taps/homebrew/homebrew-services...
Branch 'master' set up to track remote branch 'master' from 'origin'.
Reset branch 'master'
Your branch is up to date with 'origin/master'.

==> Fetching /usr/local/Homebrew/Library/Taps/sass/homebrew-sass...

==> Resetting /usr/local/Homebrew/Library/Taps/sass/homebrew-sass...
Branch 'master' set up to track remote branch 'master' from 'origin'.
Reset branch 'master'
Your branch is up to date with 'origin/master'.

➜  ~

I am not quite sure why was this but maybe has something with OS update (happened around that point here). I had notification to upgrade OS to 10.15.7 for a day maybe and initial brew error stood even after updating OS.

While the update-reset worked, I am still having the `RPC failed; curl 18 transfer closed with outstanding read data remaining` error when trying to run `brew install wget` (as an example). :( — Toby Beresford, Oct 01 '20 at 08:28
@TobyBeresford Can you tell was it failing on some specific line described in log described in my answer? Do you see any similarities comparing your log with mine? — Tpojka, Oct 01 '20 at 08:36
Thanks for asking but I've now found the issue - it was the ISP stopping large downloads from github - adding github.com to my Allowed websites list solved it. :) — Toby Beresford, Oct 01 '20 at 08:46

score 6 · Answer 4 · answered Oct 01 '20 at 08:44

6

It was the Virgin ISP settings. I added https://github.com as an Allowed website: https://my.virginmedia.com/my-apps/onlinesecurity/websafe/settings

Note that I removed the /usr/local/HomeBrew directory from previous attempts (sudo rm -rf /usr/local/HomeBrew) and then ran /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)" as per the standard installation instructions.

And hooray...!

==> Installation successful!

answered Oct 01 '20 at 08:44

Toby Beresford

1,028
9
18

I initially upvoted but there is already an answer for the Virgin websafe. Upvoting Murchie85's answer instead. – Reg Edit Mar 11 '21 at 09:40

score 2 · Answer 5 · answered Jul 15 '20 at 09:58

2

I manually download the install.sh file, and modify BREW_REPO as BREW_REPO="git@github.com:Homebrew/brew" , then run sh install.sh

answered Jul 15 '20 at 09:58

zhaozhi

1,491
1
16
19

score 1 · Answer 6 · answered Mar 08 '21 at 16:47

1

Open your git folder and delete FETCH_HEAD file and then again push hop

answered Mar 08 '21 at 16:47

Deepanshu tyagi

61
1

Sam Bacha · Answer 7 · 2022-08-14T19:19:30.227

These answers do not actually fix the issue, some of them may but your problem will come back.

Disable `tcp-udp` offloading

Darwin/MacOS

sudo sysctl -w net.link.generic.system.hwcksum_tx=0
sudo sysctl -w net.link.generic.system.hwcksum_rx=0

GNU/Linux;

sudo ethtool -K eth0 tx off rx off`

Your computer is trying to, in 'real time', tune your network card to accommodate the load, and fails.

Other related issues you may encounter:

NPM Timeout on install while overseas

Yarn - There appears to be trouble with your network connection. Retrying

Note: I use 'real time' as the relative passage of time, not as a measurement for CPU Cycles/performance modulation for your OS

Alt: Restart your Internet Connection

Simply disconnect your wifi and reconnect, this will fix it as well.

Git RPC failed; curl 18 transfer closed with outstanding read data remaining

7 Answers7

Disable `tcp-udp` offloading

Alt: Restart your Internet Connection

Linked

Git RPC failed; curl 18 transfer closed with outstanding read data remaining

7 Answers7

Disable tcp-udp offloading

Alt: Restart your Internet Connection

Linked

Disable `tcp-udp` offloading