How to fetch all branches - including the remotes remote branches?

Question

There are several upvoted questions here "how to fetch all the remote branches", and the short (upvoted) answer seems to be that git clone and git fetch should do this by default, and that one can see the remote branches by running git branch -r. However, I'm still a bit at short.

I've cloned an "upstream" central repository, and then I've cloned this clone again. Say, we have repository A on github, B is a clone of A and C is a clone of B. The problem is that C contains only branches local at B. I want to clone/pull A -> B -> C, with C having all the branches of A.

I'm partly expecting a "why would you want to do that?" in the comments. I think that given the distributed nature of git, it just ought to be possible. However, consider bandwidth to be expensive, the repository to be huge, B and C to live on the same filesystem or LAN, and most of the work to be done in branches - in this circumstance it's not desirable to clone A->B and A->C, neither to pull from A->B and A->C, as it means twice as much network traffic. There can also be other reasons, i.e. firewalls making direct pull from A->C impossible.

Answer 1

It appears that you can't (yet) configure multiple default refspecs for a remote, but you can specify them on the command line:

git fetch origin '+refs/heads/*:refs/remotes/origin/*' \
                 '+refs/remotes/*:refs/remotes/upstream/*'

This will map origin's remote-tracking branches for its origin to your remote-tracking branches to "upstream" as well as doing the ordinary fetch. Just having those branches is enough, you don't have to define the remote unless you have some reason to use it directly.

To avoid the unfamiliar command line you can define the upstream remote with your origin's url and the indirect-origin-tracking refspec:

git remote add upstream `git config --get remote.origin.url`
git config remote.upstream.fetch '+refs/remotes/*:refs/remotes/upstream/*'

If you additionally want to do both fetches with a single command, add

git config remotes.origin-all 'origin upstream'

and then

git fetch origin-all

Testing on v2.2.2:

~/sandbox/38$ ls -a
.  ..
~/sandbox/38$ git clone ~/src/git .
Cloning into '.'...
done.
~/sandbox/38$ git --version
git version 2.2.2
~/sandbox/38$ git remote add upstream `git config --get remote.origin.url`
~/sandbox/38$ git config remote.upstream.fetch '+refs/remotes/*:refs/remotes/upstream/*'
~/sandbox/38$ git config remotes.origin-all 'origin upstream'
~/sandbox/38$ git fetch origin-all
Fetching origin
Fetching upstream
From /home/jthill/src/git
 * [new ref]         origin/HEAD -> upstream/origin/HEAD
 * [new ref]         origin/maint -> upstream/origin/maint
 * [new ref]         origin/master -> upstream/origin/master
 * [new ref]         origin/next -> upstream/origin/next
 * [new ref]         origin/pu  -> upstream/origin/pu
 * [new ref]         origin/todo -> upstream/origin/todo
~/sandbox/38$ git checkout -b lala -t upstream/origin/master
Previous HEAD position was fdf96a2... Git 2.2.2
Branch lala set up to track remote ref refs/remotes/origin/master.
Switched to a new branch 'lala'
~/sandbox/38$ 

Answer 2

Been wanting similar - to keep a work or internal drive repo B and a USB backup drive repo C of the same remote repo A in sync, without re-fetching across the net from A for each separately. Only after beginning to write up my own question did the link to this question come up as well as this one: How can I convert all the remote branches in a local git repo into local tracking branches which may be useful for someone who finds your second workaround to be preferable; like you, I don't like that either, its messy.

Reading your first workaround got me thinking and reading the git man pages - git clone --mirror is very close to what we/ I want, since as the man page says:
Set up a mirror of the source repository. This implies --bare. Compared to --bare, --mirror not only maps local branches of the source to local branches of the target, it maps all refs (including remote-tracking branches, notes etc.) and sets up a refspec configuration such that all these refs are overwritten by a git remote update in the target repository.

So git clone --mirror does set up all the branches etc that we need on a local clone (somewhere) in order to effectively make other local clones/ repos (e.g. B and C). The problem with "mirror" in a not bare repo is that it is messy; so it seems we really do want at least one bare repo "D" for our "primary mirror", but using git clone --mirror (as opposed to --bare).

The last piece of the puzzle is to have our "working clone" B share as much of the objects/ store as possible of this "primary" repo D, since they're in the same filesystem "mount space" (if they is) meaning you can't cross mount points for this to work.

Create D from scratch (see below for links to convert existing repos):
git clone --mirror git://blah.com/A.git D.git

Create B from D:
git clone ../D.git/ B.git
and git will do the right thing automatically - implying git clone --local if D and B do not cross a mount barrier.

Unfortunately this still requires two steps to update the "local working clone" B as far as I can tell - git fetch --all in D followed by git fetch --all or git pull in B. You could of course write a git-fetch wrapper script to do both steps in one if that suits your chosen workflow.

In my case, C is also a --mirror repo (at least it will be in about 5 minutes from now - it's just a backup of the remote repo, in other words, it's a second "primary mirror").

Repo C can be configured to also copy objects from B (not only D). First create C, then add the appropriate remote(s), e.g. something like:
Create C from D, as a mirror:
git clone --mirror blah://my.work.pc/my/mirrors/D.git C.git
or normal clone:
git clone blah://my.work.pc/my/mirrors/D.git C.git
followed by:
git remote add B blah://my.work.pc/my/work/B.git
or in your specific case where you already have C just add D as a new remote.

Then git fetch --all on C should grab all updates from both B and D.

You may need some of the magic at How do I make existing non-bare repository bare? and also...

Another way to create D from B, use something like:
git clone --bare B.git D.git
combined with some of the magic at How to change a git repository cloned with --bare to match one cloned with --mirror? - also appropriately updating the remotes for D and of course B.

Note that the hardlinks created by git clone --local are broken by git gc which happens automatically from time to time. You may prefer git clone -s ... which uses a symbolic git link and therefore knows how to cross local filesystem "mount barriers", or the git-new-workdir command (in the git repo's contrib/ dir - this only works on non-bare repos), which are both mentioned here: git as an alternative to unison. See also Single working branch with Git and the new Git 2.5 (Q2 2015) git checkout --to=path option for some good discussion on git-new-workdir and its ultimate git official replacement.

To avoid surprises when using git clone -s be sure to read how it works e.g. in man git-clone, and you may want to configure D as follows (and read man git-prune):
git config gc.pruneExpire never

and for a comparison between these two ways, see Brandon Casey's comments in this discussion on git-new-workdir namely:

"If you want to have _multiple_different_ branches checked out from the same repository, and do development in all of them, then git-new-workdir is the right choice."

"If you want to have the _same_branch_ checked out in multiple work directories, then cloning with -s is what you want. In this case I assume development will be performed in the original repo, and the clones will do a pull to update."

Which suggests clone -s is possibly only useful for "testing" work dirs and not for development work dirs.

With two "local master" repos, say D and C configured with each as a remote of the other and of repo A, for bonus points edit the git config file and cut and paste the local repo entry (e.g. D or C) to move it above the remote (repo A) entry - this way git fetch --all does the right thing by getting new changes and branches locally first (where possible), without having to manually fetch first the local repo, then A. This is now my set up for all my cloned public repos, and it works a treat!

Good luck.