0

I would like to fork a huge repository (10GB), filter it to a specific dir and push the filtered version to a new Github repository.

I've used these instructions to pull and filter to my directory, which works. But when I try to push the filtered repo back to Github I'm getting this error:

$ git push origin master --force
Enumerating objects: 2292154, done.
Counting objects: 100% (2292154/2292154), done.
Delta compression using up to 8 threads
Compressing objects: 100% (562030/562030), done.
error: RPC failed; curl 55 SSL_write() returned SYSCALL, errno = 32
fatal: the remote end hung up unexpectedly
Writing objects: 100% (2292154/2292154), 7.03 GiB | 40.92 MiB/s, done.
Total 2292154 (delta 1726549), reused 2292154 (delta 1726549)
fatal: the remote end hung up unexpectedly
Everything up-to-date

I've tried the solution proposed here, but the error persists.

Is there a way to fork a specific dir and reduce the total repo size to make it more easily managable?

mbrg
  • 498
  • 10
  • 24

1 Answers1

1

Given the links you gave, the problem may be linked to the size of the commit. Maybe filtering your repo to just this dir, but keeping commits related to it, can help.

From git filter-branch manual :

To rewrite the repository to look as if foodir/ had been its project root, and discard all other history:

git filter-branch --subdirectory-filter foodir -- --all

Hope it can help.

To keep only this directory (and keeping it in the structure), it's slightly more complicated.

Asjudting the example from git-filter-branch manual, that should do the trick :

git filter-branch --index-filter \
   'git ls-files -s foodir/ |
       GIT_INDEX_FILE=$GIT_INDEX_FILE.new \
       git update-index --index-info &&
       mv "$GIT_INDEX_FILE.new" "$GIT_INDEX_FILE"
   ' -- --all

In short : the output of git ls-files -s foodir/ show the content of the index (blobtree and filename) only for files in foodir/; this file list is put back into the index.

Anyway, as I noted in comment, you won't be able to merge it simply - as commits are now different.

Pierre-Olivier Vares
  • 1,687
  • 15
  • 20
  • Thank you! this works. Is there a way to do it while keeping my dir at its current location (instead of root)? This will make it easier to merge back to upstream later. – mbrg Apr 10 '19 at 13:17
  • You can try with `git filter-branch --index-filter`. Sorry to be short, I'll detail that when I'll have some more time. Note that git filter-branch rewrites the history, you won't be able to merge it simply in your original repository. – Pierre-Olivier Vares Apr 10 '19 at 14:23