18

So I have mistakenly committed my secrets.yml in previous commits (yikes!!) and I want to clean my git commit history of it. It seems the fasted simplest way is to use BFG. I have downloaded the jar file and installed the Java is requires, and I have created a mirror of my repo - but when I run the command as per the docs:

java -jar bfg.jar --delete-files secrets.yml my-repo.git

I get the error Unable to access jarfile bfg.jar

The docs are plenty pretty, but doesn't specify if there is something I should have done to install this or get it running, or linked to my mirror repo, and I am plenty confused. I watched a youtube tutorial that says I should create a symbolic link using a command like;

ln -s ~/bfg-1.11.6.jar /usr/local/bin/bfg

to run the bfg, except I am uncertain what the second part of that command is referring to( /usr/local/bin/bfg ) or where it should be pointing in my case, as that does not work for me as-is. I have the jar file saved in my user root directory. Do I need to move it? How do I run BFG on my mirror repo, and should I be inside my mirror app's directory when I run it?Or do I run it from outside of the app?

Lazy Badger
  • 94,711
  • 9
  • 78
  • 110
HolyMoly
  • 2,020
  • 3
  • 22
  • 35

3 Answers3

10

2015: From the documentation, it should be:

java -jar bfg.jar <options> yourrepo

Try and use the full path of the jar if you have an error like "Unable to access jarfile bfg.jar": /home/user/path/to/bfg.jar.

If the jars are configured to be run with java, then /usr/local/bin/bfg would be the path of the symlink referencing the right bfg jar.

The alternative is described in "Remove sensitive data"

git filter-branch --force --index-filter \
'git rm --cached --ignore-unmatch Rakefile' \
--prune-empty --tag-name-filter cat -- --all

Or (update Nov. 2017), as commented by JeremyDouglass,

If you download the latest (e.g. from 1.12.16, the bfg-1.12.6.jar) and you only want to use standard commands in a local staging directory only (no path, no symlink), then you can simply rename the jar:

mv bfg-1.12.16.jar bfg.jar 
java -jar bfg.jar --delete-files bad.txt repo.git

2019-2020: more recently, you would now use You should use git filter-repo (that I mentioned here).

Install it first. (python3 -m pip install --user git-filter-repo)

Then, using a path-based filter:

git filter-repo --path secrets.yml --invert-paths HEAD

HEAD means it will change only your current branch commits.
Remove HEAD and it will go over all your commits in all your branches.

VonC
  • 1,262,500
  • 529
  • 4,410
  • 5,250
  • 1
    Thank you VonC, I eventually just abandoned it and did it with `git filter-branch` as per the git docs, I should have tried the full path, and it woulda been great had they covered errors a little in the docs, but such is life. I got the job done regardless - but I will try the full path next time I try using bfg! thanks again ;) – HolyMoly Sep 07 '15 at 18:46
  • 2
    @HolyMoly Glad you made it work. I have edited the answer to include the `git filter-branch` alternative. – VonC Sep 07 '15 at 20:08
  • thanks for the update with the `git filter-branch` info - I am sure that will help others who stumble upon this (and they seem to be still finding their way here!) ;) – HolyMoly May 08 '17 at 16:46
  • 1
    If you download the latest (e.g. `bfg-1.11.6.jar`) and you only want to use standard commands in a local staging directory only (no path, no symlink), then you can simply rename the jar: `mv bfg-1.11.6.jar bfg.jar` then `java -jar bfg.jar --delete-files bad.txt repo.git`. – JeremyDouglass Nov 01 '17 at 08:13
  • 1
    @JeremyDouglass Thank you. I have included your comment in the answer for more visibility. And used 1.12.16 as the latest ;) – VonC Nov 01 '17 at 08:22
  • mv: rename bfg-1.12.16.jar to bfg.jar: No such file or directory – Leonif Feb 13 '20 at 09:19
  • @Leonif Did you download bfg-1.12.16.jar first? – VonC Feb 13 '20 at 09:20
  • yes. I downloaded via homebrew. I have solved this via git filter – Leonif Feb 13 '20 at 09:28
  • 1
    @Leonif Then go the the folder where you have downloaded it, and the mv command will work. – VonC Feb 13 '20 at 09:28
  • Is it ok to run these commands in the local repo directory? – Cin88 Nov 26 '20 at 01:24
  • Also, stupid question but in the `git filter-branch` command, why are there forward-slashes? I hope you guys respond, I know this post is from a while back – Cin88 Nov 26 '20 at 01:27
  • 1
    @Cin88 The forward slashes allow to split a long command over multiple lines. Examples: https://stackoverflow.com/a/18599906/6309, https://superuser.com/q/508507/141. It is a "continuation character": https://askubuntu.com/a/1199918/5470 – VonC Nov 26 '20 at 06:49
  • 1
    @Cin88 And yes, those commands (the old, now obsolete `java -jar bfg.jar` or `git filter-branch`) or the now current solution `git filter-repo` are supposed to be run in the local repo directory. I would make a backup of the repository folder first, in case the filter is not satisfactory. – VonC Nov 26 '20 at 06:59
  • Thank you! I couldn't figure out how to use `git filter-repo` so I went ahead and used `git filter-branch` last night. Everything luckily worked fine – Cin88 Nov 26 '20 at 15:40
4

For me, the bfg jar had version in its name (bfg-1.14.0.jar). I renamed the jar to bfg.jar and proceeded with the command and it worked. I know, silly mistake. But hey!

Leena
  • 703
  • 1
  • 12
  • 21
3

Once you've installed BFG (e.g. using brew install BFG), you can call the BFG command in terminal.
Here is the command line help for BFG:

bfg
bfg 1.13.0
Usage: bfg [options] [<repo>]

  -b, --strip-blobs-bigger-than <size>
                           strip blobs bigger than X (eg '128K', '1M', etc)
  -B, --strip-biggest-blobs NUM
                           strip the top NUM biggest blobs
  -bi, --strip-blobs-with-ids <blob-ids-file>
                           strip blobs with the specified Git object ids
  -D, --delete-files <glob>
                           delete files with the specified names (eg '*.class', '*.{txt,log}' - matches on file name, not path within repo)
  --delete-folders <glob>  delete folders with the specified names (eg '.svn', '*-tmp' - matches on folder name, not path within repo)
  --convert-to-git-lfs <value>
                           extract files with the specified names (eg '*.zip' or '*.mp4') into Git LFS
  -rt, --replace-text <expressions-file>
                           filter content of files, replacing matched text. Match expressions should be listed in the file, one expression per line - by default, each expression is treated as a literal, but 'regex:' & 'glob:' prefixes are supported, with '==>' to specify a replacement string other than the default of '***REMOVED***'.
  -fi, --filter-content-including <glob>
                           do file-content filtering on files that match the specified expression (eg '*.{txt,properties}')
  -fe, --filter-content-excluding <glob>
                           don't do file-content filtering on files that match the specified expression (eg '*.{xml,pdf}')
  -fs, --filter-content-size-threshold <size>
                           only do file-content filtering on files smaller than <size> (default is 1048576 bytes)
  -p, --protect-blobs-from <refs>
                           protect blobs that appear in the most recent versions of the specified refs (default is 'HEAD')
  --no-blob-protection     allow the BFG to modify even your *latest* commit. Not recommended: you should have already ensured your latest commit is clean.
  --private                treat this repo-rewrite as removing private data (for example: omit old commit ids from commit messages)
  --massive-non-file-objects-sized-up-to <size>
                           increase memory usage to handle over-size Commits, Tags, and Trees that are up to X in size (eg '10M')
  <repo>                   file path for Git repository to clean
Tom
  • 16,842
  • 17
  • 45
  • 54
Pradeep Kachhawaha
  • 6,589
  • 2
  • 14
  • 13