How to normalize working tree line endings in Git?

Question

I have cloned a repository that had inconsistend line endings. I have added a .gitattributes that sets the text attribute for the files I want to normalize. Now when I commit changes I get the message:

warning: CRLF will be replaced by LF in FILE.
The file will have its original line endings in your working directory.

How can I make git normalize my working copy of the file for me? Preferably I would like git to normalize the entire working tree.

For people coming to this question in 2019, note that the second answer (by @philippn) is now better than the currently accepted one — CharlieB, Jan 21 '19 at 11:55

John Szakmeister · Answer 1 · 2019-04-23T09:52:42.183

133

For those using v2.16 or better, you can simply use:

git add --renormalize .  # Update index with renormalized files
git status               # Show the files that will be normalized
git commit -m "Introduce end-of-line normalization"

These directions are straight out of the gitattributes. For older versions, the docs (prior to v2.12) provide a different answer:

rm .git/index     # Remove the index to force git to
git reset         # re-scan the working directory
git status        # Show files that will be normalized
git add -u
git add .gitattributes
git commit -m "Introduce end-of-line normalization"

Do this sequence after you have edited .gitattributes.

Update

It appears some users have had trouble with the above instructions. Updated docs for gitattributes (2.12 to 2.14) shows a new set of instructions (after editing the .gitattributes files):

git read-tree --empty   # Clean index, force re-scan of working directory
git add .
git status        # Show files that will be normalized
git commit -m "Introduce end-of-line normalization"

Thanks to @vossad01 for pointing this out.

Also, with either solution the files in your working copy still retain their old line endings. If you want to update them, make sure your working tree is clean and use:

git rm --cached -r .
git reset --hard

Now the line endings will be correct in your working tree.

edited Apr 23 '19 at 09:52

answered Mar 26 '13 at 20:30

John Szakmeister

44,691
9
89
79

This worked, thank you! I was looking through the gitattributes documentation but it was so much to read so I sort of glossed over it and couldn't find what I was looking for. I feel stupid now. – user11171 Mar 26 '13 at 23:42
No worries, it happens! It's not as easy to find as it probably should be. :-) – John Szakmeister Mar 27 '13 at 10:03
17

This works when adding `.gitattributes` for the first time or changing the setting, but it doesn't work if the working tree has a different EOL (at least with MsysGit it doesn't). For that, it appears that `git rm --cached -r .` and then `git reset --hard` worked (warning: destroys your working tree). From https://help.github.com/articles/dealing-with-line-endings. – waddlesplash Feb 14 '14 at 21:03
1

Has something changed with Git that has broken this? I have used this many times, but now am in a repository where it is not working (git 2.13.0, linux). Instead I had to do `git add .` after removing the index (being mindful of anything in the working directory that was previously untracked). – vossad01 Jun 03 '17 at 11:49
@vossad01 Can you explain what failed in the process? I don't see a reason why the current answer shouldn't work. I can certainly update the answer with the new procedure too--I just want to understand the issue a little more. – John Szakmeister Nov 27 '17 at 15:59
@jszakmeister It does not normalize any files, see my previous comment about wondering if something changed with Git that broke it. I don't know (and am not motivated enough to research) what changed, but I have now bounced between enough different environments to be confident something has changed causing new directions to be needed. – vossad01 Nov 27 '17 at 16:08
@vossad01 Interesting. I guess I'm going to have to play with it more and see. Thanks for the heads up. – John Szakmeister Nov 27 '17 at 16:34
@vossad01 So I've tried both ways and the old way still works fine. In fact, with both ways I end up with exactly the same results. I don't know what you did, but the one requirement for `git add -u` to work is that the file must have been versioned already (it must be part of the repo). So maybe that where things fail? I tried this with 2.15.0, so it's possible there was some other bug affecting this for some short time too. – John Szakmeister Nov 28 '17 at 00:38
Only the second method worked for me. Came here as other guides all have the first method, which wasn't working, but git read-tree --empty method was the trick – Peter Apr 08 '18 at 04:31
5

Be cautious with the `git read-tree --empty; git add .` variant. Any tracked files that would be ignored by .gitignore will be deleted when using it. The documentation now recommends `git add --renormalize .` – vossad01 Jun 10 '18 at 14:41
Is there a setting I need to change somewhere? I have ran these commands multiple times now and it says succesfull in bash, but when I open the file in notepad++ with lineendings showing, it still shows CRLF. – Roger Far Mar 18 '21 at 00:10
@RogerFar what OS are you running on? Windows? What does `git config --get core.autocrlf` say? – John Szakmeister Mar 18 '21 at 23:49
It says `false`. But even when it "tries" to convert line endings it too quick, like 5 seconds for 1000 files. In the end I used notepad++ to do a global search and replace. – Roger Far Mar 19 '21 at 17:40
@RogerFar Do you have a `.gitattributes` set up? I've never seen this be a real problem, even under Windows. It sounds like you've missed a step or there is something environmental going on. And 5 seconds for 1,000 is quite possible. The conversion is pretty darn easy, and Git is awfully efficient in what it does. Sorry it was an issue for you though. – John Szakmeister Mar 19 '21 at 23:02
I did setup .gitattributes yeah, but it's strange. It did commit "something" but a git blame didn't show it changed line endings at all, neither did notepad++. – Roger Far Mar 21 '21 at 17:20

score 120 · Accepted Answer · edited Nov 28 '19 at 01:46

120

With Git client 2.16 and higher there is now a much simpler way to do this. Just use:

git add --renormalize .

Note: it's better to do this with a clean workspace. For details, see:

edited Nov 28 '19 at 01:46

ryenus

15,711
5
56
63

answered Jun 01 '18 at 13:54

philippn

1,850
1
14
11

5

I just tried this on Git Bash for Windows and it staged every file in the repo for commit. – william.berg May 29 '19 at 12:45
1

I have a error message: `error: unknown option 'renormalize'` – Ryan Nghiem Jul 25 '19 at 09:48
@RyanNghiem What does `git --version` say? I would guess it's simply older than 2.16 and you need to upgrade. HTH – philippn Jul 29 '19 at 07:26
4

@cja "Clean workspace" as in: the command `git status` returns something like `nothing to commit, working tree clean` – philippn Jul 05 '20 at 17:36
7

This just doesn't do anything at all for me. :-/ – Timmmm Oct 01 '20 at 15:38
2

Doesnt work. Downvote – wilmol Dec 03 '21 at 02:42
1

@wilmol works like a charm for those who read the first 4 words of the answer and not just the copy paste the command. And it helps to have an up up to date version of git. – Martin Jun 02 '22 at 13:00

gavenkoa · Answer 3 · 2017-08-07T12:25:15.700

12

Alternative approach (differs only in command used)

Make sure you have no any pending changes in repository:

$ git status
$ git stash

Modify .gitattributes so CRLF interpretation will changed:

$ echo "*.txt  text" >>.gitattributes
$ git commit -m "Made .txt files a subject to CRLF normalization." -- .gitattributes

Remove data from index and refresh working directory:

$ git rm --cached -r .
$ git reset --hard

Review CRLF fixes that Git proposes:

$ git ls-files --eol
$ git status
$ git diff

Agree with Git decision:

$ git add -u
$ git commit -m "Normalized CRLF for .txt files"

Reload changes as if clean clone was done:

$ git rm --cached -r .
$ git reset --hard

edited Aug 07 '17 at 12:25

answered Aug 07 '17 at 12:11

gavenkoa

45,285
19
251
303

1

Note: the `--eol` option to `git ls-files` is new in Git 2.8 (not that new any more, but some people are still suffering with Git 1.7, even late in 2018!). – torek Sep 21 '18 at 18:28
2

`git ls-files --eol` helps verify that your line endings in all files are as intended, thanks! – forumulator Mar 05 '21 at 02:32

score 4 · Answer 4 · edited Jul 06 '15 at 09:00

4

The .gitattributes settings will only affect new commits. If this repository has no history published (no others depending on it), you might want to go through the whole history. In Unix/Linux, you can use dos2unix(1) to fix all files in combination with find(1), and using the history rewriting of filter-branch (see the discussion in the git book) you can even clean up the full history of the project.

Use with utmost care, on a fresh clone. Get in contact with anybody who might have a clone, and advise them what you want to do.

edited Jul 06 '15 at 09:00

Joffrey

32,348
6
68
100

answered Mar 26 '13 at 19:42

vonbrand

11,412
8
32
52

4

I would generally advise against this solution, unless a perfect history is important. There are so many ways to shoot oneself in the foot by rewriting the history. Normalize the files in the repo in a single commit, and let .gitattributes do the work from there on out. – angularsen May 03 '16 at 21:14
1

@angularsen, if it is important to publish a clean, useable history... that is why I advise to work on a clean, new clone and (re)publish that once it is checked sane. – vonbrand Jun 01 '20 at 05:02

score 2 · Answer 5 · answered Jun 28 '19 at 11:02

The * text=auto option in .gitattributes leaves the Git repository in an 'illegal state' if it contains files with CRLF (Windows) line endings which are now marked as text (see https://marc.info/?l=git&m=154484903528621&w=2). The standard renormalize option does not work correctly with the LFS filters, so the instructions in the other answers or for example at https://help.github.com/en/articles/dealing-with-line-endings, do not work correctly. Instead these steps worked for us:

Situation:

On Windows
Git repository contained files with both CR and CRLF line endings
Added * text=auto to .gitattributes (so not depended on user having set core.crlf=auto on Windows)
Also changed the -crlf to -text for LFS tracked files, not sure that is needed.
1. Create a new branch from the branch with the line ending problem (assuming no uncommitted changes there): git checkout -b feature/doing-stuff-fix-eol
2. Remove the LFS filters from .gitattributes (replace all 'filter=lfs diff=lfs merge=lfs ' with nothing)
3. Commit and push: git commit -a -m "Disable LFS filters for EOL fix"
4. Move to non-git folder
5. Uninstall LFS globally: git lfs uninstall
6. Create a new repository clone: git clone -b feature/doing-stuff-fix-eol [remote repository url] fix-eol
7. Normalize the line endings: git add --renormalize . (note the dot to renormalize all files)
8. Check only the correct files normalized. It should not include files normally handled by LFS!
9. Commit and push (save the hash): git commit -m "Fix line endings"
10. Move to non-git folder
11. Install LFS globally: git lfs install
12. Go to original repository clone and pull
13. Checkout your original branch: git checkout feature/doing-stuff
14. Cherry pick the eol fix commit and push: git cherry-pick [hash]
15. Delete the eol branch and push
16. Delete the eol repository clone (or keep around if you need to fix more branches)

score 1 · Answer 6 · answered Feb 21 '22 at 18:09

I had to re-clone repo with this flag -c core.autocrlf=false, and it worked, no other config required.

Like this:

git clone -c core.autocrlf=false https://github.com/any-repo.git

Our project initially was made with LF on Mac, but on Windows it was automatically converted to CRLF. We use eslint and it underlined every line of code, until I re-cloned it.

Winand · Answer 7 · 2023-06-23T13:01:16.787

I've had CRLF line endings in working tree and LF in repository, because project folder was just copied from a Windows machine to a Linux one with Git 1.8.3. This is how I've reset line endings to LF:

git status returns a list of all changed files
git diff ignores line endings and returns files which are "really" changed
comm -3 compares lists and returns files from git status list which are not in git diff list
git checkout resets these files to HEAD

comm -3 <(git status --porcelain --untracked-files=no | cut -c4- | sort) <(git diff --name-only | sort) | xargs git checkout HEAD --

score 0 · Answer 8 · answered Jun 01 '20 at 03:25

0

The merge.renormalize configuration setting may be useful.

answered Jun 01 '20 at 03:25

Solomon Ucko

5,724
3
24
45

How to normalize working tree line endings in Git?

8 Answers8

Update

Alternative approach (differs only in command used)

Linked