60

I'm following this documentation: https://help.github.com/articles/duplicating-a-repository/

git clone --mirror https://github.com/exampleuser/repository-to-mirror.git

cd repository-to-mirror.git

git push --mirror https://github.com/exampleuser/mirrored

The output shows that the repository is pushed as a mirror, but for some reason I'm getting these errors as well:

 ! [remote rejected] refs/pull/1/head -> refs/pull/1/head (deny updating a hidden ref)
 ! [remote rejected] refs/pull/1/merge -> refs/pull/1/merge (deny updating a hidden ref)

What are these errors? Can I assume the repository was mirrored ?

deez
  • 1,472
  • 3
  • 18
  • 28

7 Answers7

56

As mentioned in this issue, that happens when you mirror a GitHub repo which has pull requests made to it.

The refs beginning 'refs/pull' are synthetic read-only refs created by GitHub - you can't update (and therefore 'clean') them, because they reflect branches that may well actually come from other repositories - ones that submitted pull-requests to you.

So, while you've pushed all your real refs, the pull requests don't get updated

You would need to mirror a GitHub repo without their pull requests.

Simply replace the catch-all refspec above with two more specific specs to just include all heads and tags, but not the pulls, and all the remote pull refs will no longer make it into your bare mirror:

fetch = +refs/heads/*:refs/heads/*
fetch = +refs/tags/*:refs/tags/*
fetch = +refs/change/*:refs/change/*

If push still fails, as commented by Ofek Shilon, add the push entries:

push = +refs/heads/*:refs/heads/*
push = +refs/tags/*:refs/tags/*
push = +refs/change/*:refs/change/*

As mention in Git Refspec:

The + tells Git to update the reference even if it isn’t a fast-forward.

This change is to be done on the bare repository folder repository-to-mirror.git, using git config --local --edit.

The named remote to modify is the default "origin" one.

The goal is to not fetch the unwanted references, and push only the ones we need.

Chris Stryczynski
  • 30,145
  • 48
  • 175
  • 286
VonC
  • 1,262,500
  • 529
  • 4,410
  • 5,250
  • 1
    Thanks, I went through the procedure but for some reason i'm still getting the same error. I clone the repo with --mirror, I edit the git configuration with `git config -e` and run - `git remote update` and `git push mirror` and still get the same error. – deez Dec 14 '15 at 13:19
  • @deezx What does return `git config --get-regex remote.origin.fetch`? – VonC Dec 14 '15 at 13:29
  • This is the output- $ git config --get-regex remote.origin.fetch remote.origin.fetch +refs/heads/*:refs/heads/* remote.origin.fetch +refs/tags/*:refs/tags/* Actually, when I cloned the repository without --mirror, edited the git config as you wrote, it worked. it fetched only branches and tags, so it worked! My only question now is that I see that it didn't mirror the `refs/changes`. Is there a way to tell git to fetch ALL except for pull requests (`refs/pull`)? – deez Dec 14 '15 at 13:41
  • @deezx no, you need to add `+refs/changes/*:refs/changes/*` for it to fetch `heads`, `tags` and `changes`, but not `pull`. Does the `git push --mirror` works after that? Make sure to try again in a new local clone (not `clone --mirror`, just simple clone, in which you add the fetch refspecs, do a `git fetch`, then `push --mirror` to another repo) – VonC Dec 14 '15 at 13:45
  • @VonC I'm having the same trouble as deez had. On a fresh repo, added the config into your suggested fetch refspec, and push still fails with the same error. Do you know if this recipe still works in 2021? – Ofek Shilon Dec 12 '21 at 18:11
  • @OfekShilon it should still work. Try and describe your use case in a separate question, with OS and Git version. – VonC Dec 12 '21 at 20:11
  • @VonC adding ‘push’ keys to the config, with identical refspecs, solved my problem (fetch keys weren’t enough). – Ofek Shilon Dec 12 '21 at 21:13
  • @OfekShilon OK. I have edited the answer to include your comment. Thank you for this feedback. – VonC Dec 12 '21 at 21:51
  • A small aside: The '+' at the refspecs start is said to "tell Git to update the reference even if it isn’t a fast-forward" (from the Pro-git book). Not sure if it is relevant to the `push` lines. – Ofek Shilon Dec 13 '21 at 10:47
  • 1
    @OfekShilon It does. I have updated the answer accordingly. – VonC Dec 13 '21 at 14:09
  • BTW, can I just safely ignore the errors when I don't care about pull requests anyway? Do I still get all original branches, tags, notes, etc.? (Basically everything "git", that is not specific to the github platform.) – Qwerty Aug 10 '22 at 00:26
  • 1
    @Qwerty To be tested but yes, I assume all the rest should have been pushed. – VonC Aug 10 '22 at 04:56
  • It's not clear if this change should be done on the github remote or the remote you're pushing to. – Chris Stryczynski Jun 03 '23 at 12:55
  • @ChrisStryczynski Good point. I have edited the answer to make that clearer. Let me know if that edit is enough. – VonC Jun 03 '23 at 14:36
  • Hmm sorry I meant it's not clear under which `[remote ?]` (within the .git/config) this should be done under (the both have a `fetch` line, as opposed to which 'git config' this should be done to). – Chris Stryczynski Jun 03 '23 at 16:08
  • 1
    @ChrisStryczynski It should be the origin one. I have edited the answer. – VonC Jun 03 '23 at 19:36
34

Full steps:

git clone --bare https://github.com/exampleuser/old-repository.git
cd old-repository
git push --mirror https://github.com/exampleuser/new-repository.git
Vovan
  • 478
  • 4
  • 5
  • 2
    wow, this should be the selected answer ! – HoseinGhanbari Oct 23 '21 at 12:01
  • 1
    Easy and 100% working – Vinkal Vishnoi Dec 14 '21 at 09:36
  • 1
    Working 100%. Some explanation: https://stackoverflow.com/questions/3959924/whats-the-difference-between-git-clone-mirror-and-git-clone-bare – user45097 May 24 '22 at 06:21
  • +1. This is a perfect solution if you need to mirror a repository without using the massive overhead of the --mirror option. It handles all feature branches and tags. I hat to mirror a GitLab repository to another server. With the --mirror I get separate branches for each Commit that has a pipeline run associated with it. – Nazar Aug 30 '23 at 13:35
15

instead of

git clone --mirror

use

git clone --bare

instructions

Smart Networks
  • 199
  • 2
  • 6
5

Found working and simple solutions there https://www.metaltoad.com/blog/git-push-all-branches-new-remote

git push newremote refs/remotes/oldremote/*:refs/heads/*

or

git push newremote refs/remotes/oldremote/features/*:refs/heads/features/*
Panoptik
  • 1,094
  • 1
  • 16
  • 23
2

(I wanted this to be a comment, but not enough reputation)

Based on @VonC's answer, this sounds like a non-problem.

So, while you've pushed all your real refs, the pull requests don't get updated

I see two scenarios in which you want to duplicate your repository.

  1. You want a backup/copy of a repo that you have full control over.
  2. You're modifying the history of a repo and you need a backup locally in case you need to undo your changes.

In either case, it seems like git clone --mirror is your safest option because even if you see errors in your push, all of the non-pull request related content was successfully pushed, which takes care of scenario 1. For scenario 2, you'd want those pull request references as part of your backup.

zhanga
  • 89
  • 2
  • 10
  • I am more in senario 2, where I would like to fully migrate my repository (and so interested in keeping the history of any pull requests). How would I go about a solution that keeps them intact (ie github repo --> mirror clone --> gitlab (or other vcs)) – Cole Apr 25 '21 at 03:21
1

Adding these three lines to the git config file after the fetch = +refs/:refs/ line fixed it for me:

push = +refs/heads/*:refs/heads/*
push = +refs/tags/*:refs/tags/*
push = +refs/change/*:refs/change/*
bcoder
  • 39
  • 4
0

I faced a similar issuer when trying to delete some big files from gitlab by rewriting the history, following the procedure here: https://docs.gitlab.com/ee/user/project/repository/reducing_the_repo_size_using_git.html

IMPORTANT: Before starting use gitlab export function to export your project, these steps went wrong for me many time before I figured it out.

The issue I faced as mentioned by others is that you can not rewrite read-only refs specifically does for PRs (merge requests as they are called in gitlab)

The obvious idea that come then is to create a new repo, but this would loose all the issues, releases...

Fortunately, gitlab allows to export you repo with all this metadata unto a zip file, that contains all the git bundled, (more info here about git bundle).

A bundle is basically a full archive of the repo, and ou can clone form it:

git clone --mirror ./project.bundle

Not you have a mirror of your project (the same way if you have mirror cloned form the server), and you can use git filter-repo or (BFG)[https://rtyley.github.io/bfg-repo-cleaner/] as explaind in the docs, to find the biggest files and delete them from history.

use du -h before and after to make sure that the size of your repo is really smaller, now you can create a new git bundle with:

git bundle create --progress project.bundle --tags --branches master

although I recommend handling all branches before this procedure to avoid messing up branches.

You can replace project.build in the export downloaded from gitlab (ie create a new tar.gz with exact same structure, be careful of prefix...) this can be done by:

tar -xvzf project_name...export.tar.gz

replace the project.bundle

cd project_name...export
tar -cvzf updated.tar.gz . # important to avoid messing up tar structure

And the re-importing the project.
You can rename you project to PROJECT_NAME-bkp in gitlab, and reimport with same name as before.

Hope it save someone else the time.

user1928596
  • 1,503
  • 16
  • 21