12

I am missing a bunch of LFS objects in my git repository*, both on the clients and the server. I am aware that those objects are lost and that's okay. Unfortunately this means that git lfs fetch --all or even git lfs push --all origin will fail.

I would like to purge the "broken pointers" from the repository, either by replacing the binary with a dummy text file or by removing the pointer altogether. I am also aware that this involves rewriting history and that's also fine.

What would be the best way to proceed?

*To clarify, I am missing some LFS files on the server but not all of them and not for all revisions of those files.

For example, I have foo.png that has been modified in 3 commits:

  • foo.png (Version 1, LFS Sha: 03cfd743)
  • foo.png (Version 2, LFS Sha: 661f0797)
  • foo.png (Version 3, LFS Sha: 5fa2f122)

The LFS server no longer has foo.png version 2, so I would like to remove that commit from history. Unfortunately git lfs does not tell me which commit is broken, it simply tells me that 661f0797 is missing.

(For the record, I have found the missing files so I no longer have this issue but the solution should still be interesting!)

Nepoxx
  • 4,849
  • 5
  • 42
  • 61
  • I've tried using BFG to delete files with broken pointers (`java -jar ../bfg-1.13.0.jar -D "broken.file"`), while that works, it completely wipes the history of that particular file even though only one commit might be "broken". I've also tried removing the specific commit with `git rebase -i brokencommit~1` and `drop`ing said commit, but then LFS complains that some files should have been pointers and `git lfs migrate import --everything --fixup` doesn't see this issue. In other words, it's a mess. – Nepoxx Apr 04 '19 at 13:12

5 Answers5

2

if you're still looking for a way to see what commit is affected, get the missing LFS object ID and use this command to find the commit:

git log --source --all -p  -S <LFS-OBJ-ID>
Valentin Dumitru
  • 185
  • 1
  • 10
1

This script worked for me - it removes all blobs that look like LFS files.

git filter-repo --force --blob-callback "
if blob.data.split(b'\n', 1)[0] == b'version https://git-lfs.github.com/spec/v1':
  blob.skip()
"

Explanation:

  • git filter-repo rewrites git history
  • --force is needed to skip safety checks
  • if blob.data.split(b'\n', 1)[0] == b'version https://git-lfs.github.com/spec/v1': blob.skip() This script checks if the first line of the file contains an LFS signature and skips adding this file in such a case.
nikicat
  • 298
  • 3
  • 10
0

You can untrack files, the manual is here.

Examples:

Just a single file:

git lfs untrack "/path/to/my-file.gif"

With wildcard you can catch several files by one command:

git lfs untrack "*.gif"
David
  • 5,882
  • 3
  • 33
  • 44
  • The problem in this case is that I don't necessarily want to untrack those files (and knowing which files are broken is somewhat difficult since the errors are based on the SHA256s). `my-file.gif` might only be broken in commit 5, but fine in commit 1,2,3,4,6,7,8. Ideally I would only want to drop commit 5. – Nepoxx Apr 16 '19 at 13:05
  • Mhm, that's changing the question somehow. About reading encrypted errors there is probably something to find here on SO, even perhaps not directly related to LFS. – David Apr 16 '19 at 13:15
  • question is if it's enough to provide empty or small dummy-files or if the files are still verified by file-hash. – David Apr 16 '19 at 13:19
  • I have tried to parse the output from git-lfs and simply replace the missing files by dummy files, unfortunately git-lfs verifies the files and the hashes do not match. For instance, if git-lfs complains that file '661f0797' is missing, I created a file in `.git/lfs/objects/66/1f/661f0797` but git-lfs sees that the hash of that file is not `661f0797` – Nepoxx Apr 16 '19 at 13:45
  • Yes, I expected that. And possible that '661f0797' is not the full hash, so even if you create a file with that hash, it's still not right perhaps. But I remarked your edit, that you found the files ;-) – David Apr 16 '19 at 13:48
  • You had to find the mapping for files `foo.png (Version 2, LFS Sha: 661f0797)` to resolve the error-messages. But possible is that there is no mapping-table but the hashes are calculated on the fly. – David Apr 16 '19 at 13:49
  • Wait - 661f0797 could be in the data-table and you could look it up - no hidden datastore, just a simple table. – David Apr 16 '19 at 13:55
  • 1
    `661f0797` is indeed not a full hash because sha256 hashes are too long and would have added too much noise here :) – Nepoxx Apr 16 '19 at 14:31
-2

Make git reindex and creat new entries to the files, please read the following

git reset [-q] [<tree-ish>] [--] <paths>...
       This form resets the index entries for all <paths> to their state at <tree-ish>. (It does not
       affect the working tree or the current branch.)

       This means that git reset <paths> is the opposite of git add <paths>.

       After running git reset <paths> to update the index entry, you can use git-checkout(1) to check
       the contents out of the index to the working tree. Alternatively, using git-checkout(1) and
       specifying a commit, you can copy the contents of a path out of a commit to the index and to the
       working tree in one go.

source : man git-reset

A.HEDDAR
  • 299
  • 2
  • 4
  • 2
    This is meaningless. How do you use this? What is the context? How do you reference an lfs hash? Does this hook into LFS? – mathtick Aug 25 '20 at 16:53
-3

Have you tried a simple (A backup is recommended before using the following command) : 

rm -f .git/index
git reset
A.HEDDAR
  • 299
  • 2
  • 4