Can't get Git to correctly handle CRLF problems with Eclipse launch configs (XML files)

Question

We have a mixed team with some people using Windows and others using Linux. We have configured the IDE (Eclipse) to use LF as line ending for source files which works well.

But we also share launch configs. These are XML files and Eclipse ignores the project settings for them. Instead, it always uses the platform's line ending when writing the file.

To solve this, we have these lines in .gitattributes:

**/*            eol=lf
**/*.launch     text

My understanding of this configuration is "when Git does a checkout of any file with the extension .launch, no matter where in the tree, it will convert the line endings to the platform's default (no matter what they were in the Git repo)". See the docs on github:

text

This setting tells git to always normalize the files specified. When committed they are stored with LF, on checkout they are converted to the OS's native line endings.

Only it doesn't work. I'm still seeing people committing files where every line changed; diff -R (as per this answer) shows that Git created a file with CRLF on my Linux box.

git checkout -- server.launch doesn't change anything.

What is going on here?

Is there a way to tell Git to simply ignore any line ending changes in some files?

Answer 1

The root cause of your problem is Eclipse. Eclipse uses the JGit Java library to interface with Git repositories. Unfortunately, there is an open bug (#342372) to add .gitattributes file support to the JGit library. So, though your .gitattributes settings will work when using the Git client, they are ignored by Eclipse. See this SO answer for further reading.

To validate the above, below is a Bash test script that will show JGit does not use the .gitattributes settings. You can download a self-contained JGit script here. Ultimately, the eol attribute is respected by the Git client and is not respected by the JGit client.

#!/bin/bash

# Make sure we have all the required programs.
for p in git jgit od; do
    [[ ! -x `which $p` ]] && {
        echo "Could not find executable program '$p'!"
        exit 1
    }
done

# Let's see what is being executed.
set -x

# Create a temporary Git repo.
mkdir temprepo
cd temprepo
git init

# Change core.safecrlf config so we can commit text files with non-native EOLs.
git config core.safecrlf warn

# Create the .gitattributes file.  This file will be used in all recursive
# sub-directories.  A sub-directory can have its own .gitattributes file which
# would override the settings in any parent directories.
echo -en '* eol=lf\n*.launch text\n' >.gitattributes
git add .gitattributes
git commit -m 'Add .gitattributes file.'

# Let's create some test files with different EOLs.
echo -en 'foo\rbar\r' >cr.launch
echo -en 'foo\nbar\n' >lf.launch
mkdir a
echo -en 'foo\r\nbar\r\n' >a/crlf.launch
git add cr.launch lf.launch a/crlf.launch
git commit -m 'Add {cr,lf,crlf}.launch files.'

# Let's see what is actually stored in the Git repo.
git show HEAD:cr.launch | od -A x -t x1z -w16
git show HEAD:lf.launch | od -A x -t x1z -w16
git show HEAD:a/crlf.launch | od -A x -t x1z -w16

# The commit would not have changed the working directory *.launch files.
# Let's inspect the file contents.
od -A x -t x1z -w16 cr.launch
od -A x -t x1z -w16 lf.launch
od -A x -t x1z -w16 a/crlf.launch

# Now let's change the *.launch files in the working directory to have EOLs as
# per Git's settings.
rm -f cr.launch lf.launch a/crlf.launch
git checkout -- cr.launch lf.launch a/crlf.launch

# Now let's inspect the file contents again.
od -A x -t x1z -w16 cr.launch
od -A x -t x1z -w16 lf.launch
od -A x -t x1z -w16 a/crlf.launch

# Let's change the EOL setting and checkout the *.launch files again to see how
# they are affected.
echo -en '* eol=crlf\n*.launch text\n' >.gitattributes
git commit -m 'Change EOL attribute.' .gitattributes
rm -f cr.launch lf.launch a/crlf.launch
git checkout -- cr.launch lf.launch a/crlf.launch
od -A x -t x1z -w16 cr.launch
od -A x -t x1z -w16 lf.launch
od -A x -t x1z -w16 a/crlf.launch

# Remove the *.launch files.
rm -f cr.launch lf.launch a/crlf.launch

# Now, let's checkout the *.launch files using JGit.  Due to a bug in JGit (the
# Java Git library Eclipse uses), the .gitattributes settings are ignored;
# checked out *.launch files will not have CRLF EOLs.
jgit reset --hard HEAD
od -A x -t x1z -w16 cr.launch
od -A x -t x1z -w16 lf.launch
od -A x -t x1z -w16 a/crlf.launch

Answer 2

As far as I can see there are two problems here:

1. The file name should be .gitattributes (with a leading s).

2. The **/ prefix is not understood by git. Simply use *.launch to match all files in all subdirectories and /*.launch to only match files in the top level directory.

The following shell script demonstrates the correct use of the text attribute:

#!/bin/bash

set -ex

rm -rf q22629396
mkdir q22629396
cd q22629396

git init

echo '*.txt text' > .gitattributes
git add .gitattributes
git commit -m 'added .gitattributes'

echo -e 'line with LF' > test.txt
echo -e 'line with CRLF\r' >> test.txt
echo -e 'line with LF' >> test.txt
echo -e 'line with CRLF\r' >> test.txt

mkdir subdir
echo -e 'line with LF' > subdir/test.txt
echo -e 'line with CRLF\r' >> subdir/test.txt
echo -e 'line with LF' >> subdir/test.txt
echo -e 'line with CRLF\r' >> subdir/test.txt

git add test.txt subdir/test.txt
git commit -m 'added test.txt and subdir/test.txt'

git show HEAD:test.txt | hexdump -c
git show HEAD:subdir/test.txt | hexdump -c

(Tested with git version 1.7.9.5 on Ubuntu 12.04.)

Additional Notes:

The text attribute only corrects line endings when checking in files, not when checking them out. And eol=lf only prevents automatic conversion to CRLF on checkout, not automatically convert CRLF to LF on checkout. So if the files have CRLF in them in the repository, you have to re-commit with LF.

If you check out a file with CRLF and have set the text attribute on that file, then I should be marked as "modified" in git status right from the beginning. Now you can run git commit on that file, which will change the line ending in the repository, but not in the working copy.

Resetting the line ending on the file in the working copy after that is hard. Git now replaces CRLF in the file with LF before comparing it to the index or the repository. Because of this git does not think the file is different from the committed version and thus does not do anything when you use commands such as git checkout -f. The only thing that worked in my tests was removing the file locally and checking it out again: rm test.txt; git checkout -- test.txt