6

my bitbucket repo got very big (1.6 GB) after I did some awful commits some months ago. I did not realized how serious was the situation (noob..), since a collegue tried to clone it and failed (too big).

I read carefully this post Why is my git repository so big? and did the following (as @Vi suggested):

  • Detect fat files in my repo history

    git rev-list --all --objects |     sed -n $(git rev-list --objects --all | \
cut -f1 -d' ' | \
git cat-file --batch-check | \
grep blob | \
sort -n -k 3 | \
tail -n40 | \
while read hash type size; do 
 echo -n "-e s/$hash/$size/p ";
done) |
sort -n -k1

    Let's say one of the fat files is mybigfile.gz

  • Delete mybigfile.gz from repo

    git filter-branch -f  --index-filter \
'git rm --force --cached --ignore-unmatch mybigfile.gz' \
-- --all
rm -Rf .git/refs/original && \
git reflog expire --expire=now --all && \
git gc --aggressive && \
git prune

Actually, it worked since now my local repo directory is 850MB. The problem is that the remote repository did not change size. Then I tried to

git push origin --force --all

but the situation got worse, now my remote repo is 2GB! How can I solve this awful situation? Do you suggest to create a new repo or is there something else I can do to sort it out?

Thank you.

EDIT: I try to formulate better the problem. Some months ago, I committed to my repo some big files, several times. When I realised it, I added these files to .gitignore. Then I kept committing to the repo without these file. I was not taking care to the bitbucket warning (your repo is too big). Now, I need to get rid of these files stored in old commits, both locally and remotely. I successfully cleaned up my local git directory with the procedure described above. My problem is that when I push to the remote master branch, the remote repo is not affected by the local clean up.

EDIT 2: I tried BFG repo cleaner on my local .git directory

java -jar bfg-1.12.3.jar --strip-blobs-bigger-than 100M

here the output.

According to this tutorial, this should be enough to remove blobs on remote repo, but actually this did not happen. Locally my repo is slim, but remotely is still huge. I think I'm missing just a step, but do not know how to do it. Do you think it is easier to just create a new repo?

Community
  • 1
  • 1
user123892
  • 1,243
  • 3
  • 21
  • 38
  • Have a look at [BFG Repo-Cleaner](https://rtyley.github.io/bfg-repo-cleaner/). – Enrico Campidoglio Oct 08 '15 at 10:46
  • I answered a similar question some days ago: http://stackoverflow.com/a/32953706/1420197 – Ionică Bizău Oct 08 '15 at 10:47
  • @IonicăBizău: Actually, I don't think it is a duplicate question. I do not need to know how to stop uploading files which are now in .gitignore. I need to delete big files I pushed months ago, and that are stored in my remote repo in old commits. – user123892 Oct 08 '15 at 12:55
  • @user123892 Did you try my answer? It should work for you too, using the BFG tool. Btw, I retracted the close vote. :) – Ionică Bizău Oct 08 '15 at 12:56
  • @IonicăBizău and Enrico Campidoglio I did as you suggested, locally the repo now is slim, but on the bitbucket remote repo nothing changes.. – user123892 Oct 08 '15 at 14:31
  • @user123892 Hmm, try my answer. Not sure if it will work. – Ionică Bizău Oct 08 '15 at 14:39
  • 1
    Possible duplicate of [How to reduce git repo size on Bitbucket?](http://stackoverflow.com/questions/24493071/how-to-reduce-git-repo-size-on-bitbucket) – AD7six Oct 08 '15 at 14:50
  • @AD7six thank you for redirecting me to this question. Actually I found the solution there, I contacted Bitbucket support and they did "git gc" on their size. Now my remote repo is back to the size of my local one. Anyway, the procedure I followed to clean up local repo is a bit different. I can't say my question is a true duplicate but as I noob I leave the decision to you. Tell what you do think and I'll close the question or not. Thank you again. – user123892 Oct 09 '15 at 08:52
  • If the other question/answer helped - upvote it/them. If you have a more up-to-date answer for that question answer it. For this question you don't need to take any action about that, if it's closed this question acts as a signpost to the other. – AD7six Oct 09 '15 at 08:56
  • ok, thank you. I'll upvote the other question. No, the answer to my question does not "update" the other question. Ok, so you think I shall close this post. – user123892 Oct 09 '15 at 09:02
  • 1
    That's not what I said - it takes several people to close a question as a duplicate. In your position I would accept the below answer, or write your own answer and accept that. I would not delete the question myself. – AD7six Oct 09 '15 at 09:05

2 Answers2

1

From the comments I understood that the problem is fixed locally, but not on remote. Let's do some mad science to force all objects to be dereferenced and garbage collected with the following commands (create backup first):

git reflog expire --expire=now --all
git gc --prune=now --aggressive
git push -f

Maybe this will clean up the remote repository.

Ionică Bizău
  • 109,027
  • 88
  • 289
  • 474
  • maybe your solution works, the problem is that my repo now is in read-only mode since bigger then 2.0 GB, so I get "conq: repository is in read only mode (over 2 GB size limit). Transferred: sent 3120, received 2856 bytes, in 0.4 seconds Bytes per second: sent 7432.0, received 6803.1 fatal: Could not read from remote repository. Please make sure you have the correct access rights and the repository exists. " Am I doomed? Thank you for the help. – user123892 Oct 08 '15 at 14:46
  • 1
    @user123892 I took the commands from the BFG output you posted. Probably it helps. If you don't have any issues important data there, just create a new repository... :) – Ionică Bizău Oct 08 '15 at 14:48
0

Actually, I just needed to contact Bitbucket support (they replied in less than an hour). They did:

 git gc

from their side and my remote repo is back to the local repo size.

user123892
  • 1,243
  • 3
  • 21
  • 38