3

I am trying to remove (huge) files from the history of a repository.

I used git-rebase to drop the commits with the files that I want to remove, and it didn't complain, but running git push origin master fails (see below) with errors precisely on the files that I am trying to remove. These are the largest files in the repo (using this script): 

bash-3.2$ ./gitlistobjectbysize.sh | head -n 5
100644 blob 4f77d4c09bc74ade25fbc9e02c499776db50c923 8268002855 appointments-six-weeks.csv
100644 blob de9e2ab4a1e38bb229a82537cf7d5bc1dd1c740c 4047367885 appointments-three-weeks.csv
100644 blob c68619098142ce84c9140ccaef70a68b7636c935 1043590349 appointments-one-week.csv
100644 blob 0cd9c4d3e574583bd3ccc3948d221a16b97030a7 291783067  appointments-24-hours.csv
100644 blob 1703ddcee60fb5f14c15bfe8113ef0ecc5750424 187968216  appointments-one-week-two-centers.csv

To find which commits modify the first file (for example), I cooked up the following (amateurish) script: 

for commitSHA1 in $(git rev-list --all); do
    echo "commit: $commitSHA1"
    git ls-tree -r --long "$commitSHA1" | grep 4f77d4c09bc74ade25fbc9e02c499776db50c923
done

Here is the relevant portion: 

bash-3.2$ ./dirtyblobfollower.sh 
commit: 3a5c9334615da329700d43f9db7f0eb13f7e2594
commit: f65363e5eea21f9a9c3f877edc66c1a350e2d0dd
commit: b5107588f32469f9fd4be8da52ea4c71370f4341
commit: 657527d79daada62b853ce6bf894da3a1ae3de2f
commit: 2100bbe9cb5a981936b4b5edeba9bffe97eb30be
commit: 11282459cf7e8d778d41b7b8324f571d266323e0
commit: 1c6be79eb630fb4c676621d8f6afc03f63a4e8cd
commit: e716da1a40bc72affd069d74efffa18904750ccf
commit: fd08313462478e56a60ebf475ed8230d9f898f8f
commit: 6a9e4f68c7b23a5513b912d5452c02c57c0a683d
commit: edbd096c5329c63a32c902f2553e3cffe3ad3f29
commit: a02f52dad35b63e5c0ab9b6a0eee6025b745b9c8
commit: d03842d37701a559e842effdade2f80367440b75
commit: 5c3ba451a0fa46bceb62036bbe0e5b43bfb26648
commit: 525f31365acdeacf2f8cacf75144df605e7763ae
100644 blob 4f77d4c09bc74ade25fbc9e02c499776db50c923 8268002855 appointments-six-weeks.csv
commit: 07117aae31062097941f32e7e8f4ef609a9c9108
100644 blob 4f77d4c09bc74ade25fbc9e02c499776db50c923 8268002855 appointments-six-weeks.csv
commit: da184af64d103277621193bf1c09a6f10708ce16
...

But currently the last two commits of the repo are: 

bash-3.2$ git log --oneline -2
3a5c933 (HEAD) [Commit message]
da184af (origin/master) [Commit message]

which are the first and last commits in the previous list.

In other words, the huge files are still there (somewhere) because the deleted commits are still there (somewhere), and they cause trouble when pushing to the remote.

Why are deleted commits still showing up in git-rev-list? I tried this tip : 

git reflog expire --expire=now --all
git gc --aggressive --prune=now

to force a bit of garbage collection, but nothing changed.

What would happen if I manually deleted the git-objects (i.e. the files in .git/objects)? It doesn't sound like a solution, but I am curious.

Here is the failure of the push: 

bash-3.2$ git push origin master
Enumerating objects: 49, done.
Counting objects: 100% (49/49), done.
Delta compression using up to 12 threads
Compressing objects: 100% (23/23), done.
Writing objects: 100% (47/47), 700.38 MiB | 691.00 KiB/s, done.
Total 47 (delta 21), reused 46 (delta 20)
remote: Resolving deltas: 100% (21/21), completed with 1 local object.        
remote: error: GH001: Large files detected. You may want to try Git Large File Storage - https://git-lfs.github.com.        
remote: error: Trace: e5232721406ab3670bfc771048b4dcd6        
remote: error: See http://git.io/iEPt8g for more information.        
remote: error: File appointments-24-hours.csv is 278.27 MB; this exceeds GitHub's file size limit of 100.00 MB        
remote: error: File appointments-one-week.csv is 995.25 MB; this exceeds GitHub's file size limit of 100.00 MB        
remote: error: File appointments-six-weeks.csv is 7884.98 MB; this exceeds GitHub's file size limit of 100.00 MB        
remote: error: File appointments-three-weeks.csv is 3859.87 MB; this exceeds GitHub's file size limit of 100.00 MB        
remote: error: File appointments-one-week-two-centers.csv is 179.26 MB; this exceeds GitHub's file size limit of 100.00 MB        
To github.com:myaccountname/mygitrepo.git
 ! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to 'git@github.com:myaccountname/mygitrepo.git'
Antoine
  • 600
  • 7
  • 19
  • 1
    Run `git fetch` to update `origin/master` if needed, then run `git log --graph origin/master..master` to see which commits those are. One of those commits snapshots some large file(s). – torek Mar 08 '19 at 22:53
  • Ok, so I am discovering that I have a detached head (not entirely sure I am understanding what this means, but there is plenty for me to read on that). `git branch` shows that I am currently on "no branch", and the other branch is "master". What happens if I simply delete master? – Antoine Mar 08 '19 at 23:39
  • 2
    "detached HEAD" just means "not on any branch". That's a normal state if you're in the middle of—i.e., have not yet finished, nor terminated—a rebase. It's also normal if you have used `git checkout` to move off a branch. To get back on a branch, use `git checkout` with a branch name. – torek Mar 08 '19 at 23:45
  • 2
    In modern Git, `git status` will tell you if you're in the middle of an ongoing rebase (or an unfinished merge or any number of other things that can stop in the middle like that). (In the bad old days of Git 1.5 or so it was less helpful. Somewhere during the 1.7 and 1.8 time period is when it got much better.) – torek Mar 08 '19 at 23:46
  • 1
    Just an update: I created a branch for my detached HEAD and renamed branches so that now `master` refers to the branch that I care about (I could delete the other one, actually). Now `git push origin master` is pushing the "correct" branch. What I hadn't realised until Torek's comment, was that I was trying to push the wrong branch... Thanks! – Antoine Mar 09 '19 at 10:28

0 Answers0