How to clean repo, if staged files marked as modified?
After
git reset --hard
I get
Encountered 7 file(s) that should have been pointers, but weren't:
Running git clean -fdx (which deletes everything, including untracked directories and anything in .gitignore) doesn't help, either.
Like Travis Heeter mentioned in his answer, Try the following command sequence:
git lfs uninstall
git reset --hard
git lfs install
git lfs pull
In case if this is not working (because this was not working for me), the following hack may work:
git rm --cached -r .
git reset --hard
git rm .gitattributes
git reset .
git checkout .
This worked for me!
I had this exact error with some files stored with git-LFS and solved it the same way I've solved a linending induced borked index .
Clear the cache and do a hard reset:
git rm --cached -r .
git reset --hard
This was significantly faster than a fresh clone for me due to the huge git-LFS files in my repo.
Since git lfs 2.5.0, there is a new command available that makes this easier (docs):
git lfs migrate import --no-rewrite "broken file.jpg" "another broken file.png" ...
This "migrates" files to git lfs which should be in lfs as per .gitattributes, but aren't at the moment (which is the reason for your error message).
--no-rewrite prevents git from applying this to older commits, it creates a single new commit instead.
Use -m "commitmessage" to set a commitmessage for that commit.
The problem comes from the mismatch beetween the filetypes marked as to be tracked by git LFS in the .gitattributes and some matching files already under conventional non-LFS version control.
So the simplest workaround here is to just remove the .gitattributes file for a moment:
git rm .gitattributes
git reset .
git checkout .
Afterwards you can checkout any other branch.
One more advice: When adding a new filetype to git LFS, prefer not to do this by hand by modifying the .gitattributes but e.g. by running:
git lfs track PATTERN
where PATTERN is the pattern for matching files, e.g *.so
This way all already non-LFS versioned files matching the new tracking-pattern will be marked dirty and can be simlpy added, i.e. converted to git LFS (file pointers).
This can happen when you do a checkout that contains files which should have been been tracked by LFS as specified in .gitattributes but somehow they've been committed directly instead. Most likely you have another program managing your repository such as a git GUI or IDE.
This can be frustrating because these files appear out of nowhere and prevent you from making checkouts. As soon as you stash your changes they return! If you get stuck in this situation, a quick fix is to commit these changes on a temporary branch so that you can checkout again.
To genuinely fix this problem, make sure you've committed the files as LFS pointers. This should be as simple as using git add. Check your work using git lfs status before committing. git lfs ls-files will show what files LFS is managing.
git lfs status is misleading since it reads Git LFS objects to be committed when it really lists all changes. Files that you expect to be tracked by LFS should read something like (LFS: c9e4f4a) or (Git: c9e4f4a -> LFS: c9e4f4a) and not (Git: c9e4f4a).
By way of example, I found this to be a problem when adding image assets through Xcode 9.2 where I added "CalendarChecked.png" which it automatically added:
$ git status
Changes to be committed:
(use "git reset HEAD <file>..." to unstage)
new file: Example/Assets.xcassets/CalendarChecked.imageset/CalendarChecked.png
Changes not staged for commit:
(use "git add <file>..." to update what will be committed)
(use "git checkout -- <file>..." to discard changes in working directory)
modified: Example/Assets.xcassets/CalendarChecked.imageset/CalendarChecked.png
$ git lfs status
Git LFS objects to be committed:
Example/Assets.xcassets/CalendarChecked.imageset/CalendarChecked.png (Git: c9e4f4a)
Git LFS objects not staged for commit:
Example/Assets.xcassets/CalendarChecked.imageset/CalendarChecked.png (File: c9e4f4a)
$ git add Example/Assets.xcassets/CalendarChecked.imageset/CalendarChecked.png`
$ git lfs status
Git LFS objects to be committed:
Empty/Empty/Assets.xcassets/CalendarChecked.imageset/CalendarChecked.png (LFS: c9e4f4a)
Git LFS objects not staged for commit:
$
Neither of these solutions worked for me, but I pieced together a few sources to finally get all this fixed.
Push any changes you don't want to lose
If you can... If not, or if you don't care about your changes, press on.
Stop Everything
SourceTree, any servers, file explorers and browsers. Sometimes this stuff won't work if it's being used somewhere else. When in doubt, stop it - with this it's better to overkill.
Also, go into Task Manager, force quit any bash.exe processes. Git Bash tends to hold files open after you close the window.
Open a Command Window (or Terminal)
cd to your local repo.
Uninstall lfs
> git lfs uninstall
Then it'll say something like:
Hooks for this repository have been removed.
Global Git LFS configuration has been removed.
Reset
> git reset --hard
It'll go through a lot of output probably...
Reinstall lfs
> git lfs install
This may again say it found files that should have been pointers but weren't. That's OK, keep going!
Pull with lfs
> git lfs pull
Hopefully pulling with lfs will overwrite the files that got borked.
A few of my sources said at this point their repo was working again, but not me personally. You can open SourceTree or whatever to check if you want, but you may have to start from the top if it didn't work.
Migrate
The core issue here is that lfs, instead of downloading large files like audio, video, images - anything larger than 1Mb - it just points to them on a server. This is useful if you have a bunch of large files, you're not pulling down all that stuff. So your local repo is smaller and nimbler. However, through circumstances I'm not sure about, it seems possible to corrupt the pointers. I'm sure this is an issue that the lfs people are aware of and are working on, but for now we have to work it out ourselves.
What we've done so far is
lfslfsSo now we have all these things in our folder that are either files or pointers to files, and lfs needs to figure out if any files should be pointers and vise versa. And hopefully by performing the steps above we deleted the corrupted pointers. So we're going to perform migrate to kick off the procedure that goes through the files on the repo, and if they're greater than 1Mb, lfs is going to replace them with a pointer.
> git lfs migrate
More Errors
Here's a point at which others have stopped and said they were working again, but not me. I got an error:
Error in git rev-list... exit status 128 fatal: bad revision '...
v1.0.0'
@guneyozsan over at a github help page, posted this final piece to the puzzle, even though it didn't fix his issue.
> git lfs migrate info --include-ref=v1.0.0
Notice the version matches the version that errored - v1.0.0. You will need to replace v1.0.0 with whatever version you got in your error.
I haven't found a source on why this error occurs but my guess is that the lfs version number generated by migrate on your local repo doesn't match the source version. For me, all this started when SourceTree crashed during a push and I forced a machine reboot, and when that happens, lfs doesn't know how to deal with it, so it just gets stuck in this loop where it's trying to update, but it can't read the corrupted data. Hence the lengthy troubleshooting.
Stash and Pull
When you open SourceTree, you'll probably see that it wants to add all your files back. Don't do that. Stash, then pull.
And boom, the horror is hopefully over. If not, this git hub page or this one may help you more, but this is what worked for me.
Ensure you have git lfs version 2.5 or higher installed (download here).
Check you are using the git lfs version you downloaded (2.7.2 for me):
>git lfs version
git-lfs/2.7.2
Run:
git lfs migrate import --fixup --everything
Pull your branch and fix any merge conflicts.
Found in this github comment.
run
git add --renormalize .
and commit those changes. It is safe to do even when another user is doing the same on another branch, since the LFS pointer is derived from the hash of the file. It might also catch some files with wrong line endings.
One possible cause for this error is due to git LFS-related changes to .gitattributes that impacts already added files in a repo.
(I'm not sure of the exact steps to reproduce, but the issue seemed to occur when I touched a file that was newly affected by the .gitattributes that was previously committed as a non-LFS file that should now be an LFS file. The issue seemed aggravated by switching branches, or at least made it impossible to switch branches until the issue was resolved.)
In this case, I used the steps below to prevent this error from occurring repeatedly.
git statusFrom Ratata Tata's answer in How to make change to .gitattributes take effect)
git rm --cached -r .
git add -A
Warning: make sure in step 2 that there was nothing to commit, as the steps above will add any files that were not previously versioned
git status (it should only have modified relevant files to become LFS pointers, i.e. files that can potentially cause the "encountered files that should have been pointers" error) and commit changes
This just shows one more time what a pile of dog**** GIT-LFS is.
You can get into this situation if:
common-base-branch and not in LFSlfs-branch based on common-base-branch, the file was moved to LFSnon-lfs-branch also based on common-base-branch, the file was modified.Or alternatively:
common-base-branchlfs-branch based on common-base-branch, the file was added to LFSnon-lfs-branch also based on common-base-branch, the file was added (but not to LFS.In both cases, when you try to merge non-lfs-branch into lfs-branch, you get this kind of error.
You may ask why this should even happen in the first place, but the answer is that a lot of software is developed by more than once person (which is why you have version control systems like GIT in the first place), and people don't always talk to each other, or LFS is introduced later in the history of a project in a feature branch, while "normal" development still goes on in other branches.
This is a legitimate merge conflict situation, not a bug or corrupted working directory or anything (as some of the other answers suggest). GIT-LFS just handles it poorly.
What you want to do now is to make sure the right version of the conflicted files goes into GIT-LFS, so you may want to choose an answer to this question which does just that... (TODO: Insert link to at least one answer that works)
The accepted answer worked for me but it would only work when I manually typed the commands, I put sleeps between each command and now it works as a bash script:
git rm --cached -r .
sleep 1
git reset --hard
sleep 1
git rm .gitattributes
sleep 1
git reset .
sleep 1
git checkout .
If you just want to get away from that bad commit, you can go back to master by
git reset --soft origin/master
git reset --hard
Then you are free from the nasty 7 non-LFS files:-)
Same as @John Kugelman above, but I put it in a alias because I had to do it a bunch of times.
git rm --cached -r . > /dev/null && git reset --hard > /dev/null && git rm .gitattributes > /dev/null && git reset . && git checkout . > /dev/null
The following process will add a commit which replaces all binary files that should be lfs pointers with lfs pointers.
Clean working copy completely. Together with the force add below this prevents any files getting added or removed due to .gitignore patterns.
git clean -dfx
git reset --hard
git checkout -- .
Add remove for everything to staging area. Working copy will not be touched.
git rm --cached -r .
Readd all files from working copy again. This will basically undo the previous command but will reevaluate lfs filters. Use -f to ignore .gitignore. All files present were previously checked in and should get added again.
git add -f .
You staging area now should only contain the binary files that previously raised the 'should have been pointers' error.
git commit -m "moved files to lfs"
Here's the problem I ran into:
Say you created a branch, and you somehow committed files as non-LFS. So then you tried to correct it by later committing the LFS version of the files on the same branch. However, now you can't rebase or squash because you'll keep running into this "files that should have been pointers but weren't" error in the middle of the rebase.
Solve using git reset --soft: https://stackoverflow.com/a/5201642/2516916
In my case it was one file under lfs rules (I'm assuming it was checked in without lfs installed or something).
So I found it's extension in the .gitattributes file and commented this line out like
#*.7z filter=lfs diff=lfs merge=lfs -text
saved this .gitattributes then git status showed no problems.
After that I've uncommented this line (removed #), saved .gitattributes and git status still shows no problems.
What helped me, without touching the whole repo, was the git restore command which was added in git 2.23
git restore --source=HEAD --staged --worktree -- affected_files
Execute this command a few times until all of the warnings are gone.
none of the above commands worked for me, I found an answer here
git status -s | cut -c 4- | xargs git update-index --assume-unchanged
rm .git/index && git reset
That's because the files are not tracked by LFS, but they matches some .gitattributes files' description.
For example,
server/.gitattributes:
conf/** filter=lfs diff=lfs merge=lfs -text
However, client.gflags matches the server/.gitattributes description, and git will pull it from LFS, but it doesn't have the LFS info, and the error will be thrown up.
Find the .gitattributes file whose description hit the Encoutered file, delete the wrong description or optimize some wildcard character match.
Optimize the example above, server/.gitattributes:
conf/client.conf filter=lfs diff=lfs merge=lfs -text
In addition to the karyon's answer, after performing migrate to fix the local state:
git lfs migrate import --no-rewrite "broken file.jpg" "another broken file.png" ...
You have and extra commit which you probably want to get rid of.
Considering .gitattributes is already fixed on you main / master branch, one can just rebase the local branch onto the one containing the fix, usually:
git rebase upstream/master
And this extra commit will be dropped automatically, since it is meaningless at this point.
When it is obviously an error that shows up out of nowhere, this is what we do in our team:
Disable lfs for that specific type file (modifying .gitattributes or via SourceTree menus)
The change will dissapear and you will see a change on .gitattributes instead
Remove the problem:
3.1 One solution is to execute git reset --hard. Another way, discard changes. Sometimes the file will not come up again.
3.2.1 If the previous solution doesn't work, repeat 1 and 2. Then make sure that this branch you are in (A) has already commited and pushed everything except those annoying files. Then commit your change, but not push.
3.2.2: Go to another branch (B)
3.2.3: Remove that local branch (A) where you performed the commit to .gitattributes, forcing the removal even when it says there's a commit that hasn't been pushed. It will forget that commit (it can afterwards be removed via GC or whatever but it's not a big deal if the file that has the error is not huge)
3.2.4: Checkout the branch A again. It will download the previous status of the repository without the annoying files and LFS settings set up the correct way.
That always works!
Loads of answer on this with multiple steps to fix.
If you are just in a broken state, like stuck on a branch because you can't reset/discard the changes to the problem file(s): deleting the .gitattributes file maybe enough to allow you to make your next git move. Once you have made your git move you may then have to restoring the .gitattributes file but at least you are unstuck.
I wished I had known this before trying all the above. It is a low risk option to try at least.
Here with this embellishment that doesn't require reinstalling the lfs hooks or obscuring any .gitattributes files:
git -c filter.lfs.smudge= -c filter.lfs.clean= reset --hard
