Why does git keep messing with my line endings?

Question

I'm on Windows and I have core.autocrlf disabled:

$ git config core.autocrlf; git config --global core.autocrlf
false
false

Now, I would assume, that git does not mess with any line endings, but some files in my repo keep causing issues.

I just cloned a fresh copy of the repo, and git is already telling me that there are unstaged changes. When I git diff -R, I can see that CRLF line endings have been added to a file:

diff --git b/nginx-1.11.1/contrib/geo2nginx.pl a/nginx-1.11.1/contrib/geo2nginx.pl
index bc8af46..29243ec 100644
--- b/nginx-1.11.1/contrib/geo2nginx.pl
+++ a/nginx-1.11.1/contrib/geo2nginx.pl
@@ -1,58 +1,58 @@
-#!/usr/bin/perl -w
-
-# (c) Andrei Nigmatulin, 2005
+#!/usr/bin/perl -w^M
+^M
+# (c) Andrei Nigmatulin, 2005^M

I don't understand where these line endings come from, but I'm also unable to "revert" this change. When I checkout the file again, it will still be modified afterwards:

$ git checkout -f nginx-1.11.1/contrib/geo2nginx.pl
$ git status
On branch dev
Your branch is up-to-date with 'origin/dev'.
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

        modified:   nginx-1.11.1/contrib/geo2nginx.pl

no changes added to commit (use "git add" and/or "git commit -a")

This makes no sense to me, so I just run dos2unix on the file:

$ dos2unix nginx-1.11.1/contrib/geo2nginx.pl
dos2unix: converting file nginx-1.11.1/contrib/geo2nginx.pl to Unix format...

Now there surely shouldn't be any changes, right? But the file is still being shown as modified in git status and git diff will still report CRLF line endings in the working copy.

When I now stage and commit the file, the resulting file will have LF line endings, even though the diff showed CRLF line endings.

I don't have a global .gitattributes (git config --global core.attributesfile does not output anything). And the .gitattributes in the project has * text eol=lf set (full .gitattributes).

What is going on here and how can I resolve this?

Reproduce the issue

I can repro this issue with an open source project I'm maintaining:

$ git clone git@github.com:fairmanager/fm-log.git
Cloning into 'fm-log'...
remote: Counting objects: 790, done.
remote: Total 790 (delta 0), reused 0 (delta 0), pack-reused 790
Receiving objects: 100% (790/790), 201.71 KiB | 138.00 KiB/s, done.
Resolving deltas: 100% (418/418), done.
Checking connectivity... done.

$ cd fm-log/
$ git status
On branch master
Your branch is up-to-date with 'origin/master'.
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

        modified:   .idea/dictionaries/OliverSalzburg.xml
        modified:   .idea/inspectionProfiles/Project_Default.xml
        modified:   .idea/inspectionProfiles/profiles_settings.xml

no changes added to commit (use "git add" and/or "git commit -a")

Shot in the dark: does `git ls-tree HEAD:nginx-1.11.1/contrib` show two versions of `geo2nginx.pl` that differ only in case? — torek, Aug 22 '16 at 11:17

torek · Accepted Answer · 2016-08-22T11:46:39.763

Interesting: with the newly added-to-question repo I tried cloning it and, on BSD, see the same thing:

$ git clone git@github.com:fairmanager/fm-log.git
Cloning into 'fm-log'...
remote: Counting objects: 790, done.
remote: Total 790 (delta 0), reused 0 (delta 0), pack-reused 790
Receiving objects: 100% (790/790), 201.71 KiB | 0 bytes/s, done.
Resolving deltas: 100% (418/418), done.
Checking connectivity... done.
$ cd fm-log/
$ git status
On branch master
Your branch is up-to-date with 'origin/master'.
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

    modified:   .idea/dictionaries/OliverSalzburg.xml
    modified:   .idea/inspectionProfiles/Project_Default.xml
    modified:   .idea/inspectionProfiles/profiles_settings.xml

no changes added to commit (use "git add" and/or "git commit -a")

Trying to see what's going on:

$ git diff
warning: CRLF will be replaced by LF in .idea/dictionaries/OliverSalzburg.xml.
[snip]

It seems that HEAD:.idea/ files actually have carriage-returns in them (and the last line has no newline, hence the prompt right after the close angle bracket):

$ git show HEAD:.idea/dictionaries/OliverSalzburg.xml | vis
<component name="ProjectDictionaryState">\^M
  <dictionary name="OliverSalzburg">\^M
    <words>\^M
      <w>colorizer</w>\^M
      <w>multiline</w>\^M
    </words>\^M
  </dictionary>\^M
</component>$

The work-tree version likewise has carriage returns (no cut and paste but vis shows the same \^M line endings).

So what has happened in this case, at least, is that due to the .gitattributes setting of:

$ head -2 .gitattributes 
# In general, use LF for text
* text eol=lf

Git will convert these files to LF-only during commit, vs the HEAD version that contains CR-LF endings. This is what git status is saying here.

Commenting out the * text eol=lf in .gitattributes makes the status go away (since the files won't be converted), though of course .gitattributes is now marked as modified. Interestingly, putting the attribute line back again, the status goes completely silent: it's necessary to force git checkout to replace the work-tree version to get the status back (e.g., manually rm -rf .idea and check out again, or git reset --hard).

(Presumably—I'm guessing a bit here—the index entry gets marked specially during git checkout when Git notices that the work-tree and HEAD version differ. This makes Git inspect the file closely. Modifying .gitattributes and running an internal diff via git status probably un-marks the index entries. This part is pure speculation meant to explain the weird behavior of git status...)

I guess I really hadn't fully understood line ending transformations and how they are applied, especially on commit. In the past, I would stage and commit these three files from the example, which would commit LF line endings, but leave me with CRLF in my working copy. Upon changing branches, this would cause further trouble and the CRLF would end up in the repo again at some point. — Oliver Salzburg, Aug 22 '16 at 11:50
It's never been completely clear to me either; I think at various points the mechanism has changed in subtle ways. Git's smudge and clean filters have similar issues (Git will sometimes run a smudge or clean filter multiple times, and `git merge` has an option to run attributes over all three versions of each file since old commits' attributes may differ). — torek, Aug 22 '16 at 11:58
@OliverSalzburg Just to be sure, that is *exactly* the conclusion I mentioned in my answer, is it not? — VonC, Aug 22 '16 at 12:28
@VonC Yes. It just took me a while to understand the relevance of the `.gitattributes` in the project. I previously assumed that it has no relevance at all if `autocrlf` is not enabled. — Oliver Salzburg, Aug 22 '16 at 12:51
@OliverSalzburg And yet, the first thing I mentioned in my answer is the .gitattributes. And I mentioned the xml file of your repo being committed with crlf a full minute *before* torek (whose answers I always admire) chimed in. — VonC, Aug 22 '16 at 12:53
@VonC I'm not quite sure what we're discussing here, but I'm assuming it's that I gave torek the accept instead of you. If that is the case, then I apologize, but when I read this answer, the underlying concepts became clear to me, which was not the case at the time I read yours. I then did not compare timestamps but instead upvoted both answers and marked this one as accepted, as it seemed to explain the problem more thoroughly. — Oliver Salzburg, Aug 22 '16 at 13:15
@OliverSalzburg no problem: coming from torek, I am not surprised. — VonC, Aug 22 '16 at 13:16

score 2 · Answer 2 · edited May 23 '17 at 10:33

2

If the local setting git config core.autocrlf is indeed to false, then those eol changes must come from a .gitattributes file (see man page).

Look for one in your local repo, which would include eol rules (like * text=auto eol=crlf I mentioned here).

Or check if you have a global gitattributes file.

git config --global core.attributesfile

Regarding fairmanager/fm-log, you can see one of the "modified" files .idea/dictionaries/OliverSalzburg.xml added in commit e6f823b with crlf at the end.

Since the .gitattributes has * text eol=lf rule, the working directory checkout blobs with lf eol, hence the git diff.

edited May 23 '17 at 10:33

Community

1
1

answered Aug 22 '16 at 10:53

VonC

1,262,500
529
4,410
5,250

Sorry for not mentioning it before. The project actually has a `.gitattributes`, but it doesn't include what you mentioned. I updated the question. – Oliver Salzburg Aug 22 '16 at 11:12
@OliverSalzburg it could .gitattributes in one of the subfolders – VonC Aug 22 '16 at 11:13
Good point. There aren't any along the path of the file in question though. – Oliver Salzburg Aug 22 '16 at 11:16
@OliverSalzburg Maybe there is a directive at the system level? (at the git installation path). What version of Git are you using? – VonC Aug 22 '16 at 11:18
`git --version` prints `git version 2.9.3.windows.1` I'm not aware of any other directives at the system level. Git installs into `%localappdata%\Programs\Git\mingw64\bin`, which does not include any `.gitattributes` – Oliver Salzburg Aug 22 '16 at 11:20
Does `git diff -R --` also show the file as changed? – rustyx Aug 22 '16 at 11:22
@RustyX Yes, that's how I'm seeing the CRLF. I added information to the question about a GitHub repo where I can repro the issue right now. – Oliver Salzburg Aug 22 '16 at 11:25
@OliverSalzburg All I can say if that those files were somehow committed with crlf. Considering your .gitattributes, it makes sense they are impacted. See my edited answer. – VonC Aug 22 '16 at 11:40

Why does git keep messing with my line endings?

Reproduce the issue

2 Answers2

Linked

Related