8

A friend and I were working on the same .cs file at the same time and when there's a merge conflict git points out there's a conflict but the file isnt loaded with the usual "HEAD" ">>>" stuff because the .cs files were binary files. So we added numerous things (*.cs text and so on)- to our .gitattributes file to make git treat it as a text file which didnt work.

Thats when we realized that git could diff other .cs files and just not this one. The reason for that is because its in unicode encoding as it contains some chinese characters.

So how do we make git diff or merge files that are in utf-16 or utf-8 format?

The furstrating thing is that if i push, gitlab shows exactly whats different. So I dont get how git can diff on the server but not with bash.

user1879789
  • 292
  • 3
  • 8
  • 1
    I believe the following question is discussing the same issue: http://stackoverflow.com/questions/777949/can-i-make-git-recognize-a-utf-16-file-as-text – Nils Magne Lunde Aug 07 '13 at 19:39
  • Yes, I tried that. However the diff displays like this "@@@@@@@" so its not much help and it wont merge either – user1879789 Aug 07 '13 at 20:12
  • A work-around might be to use a different diff/merge tool. Have you tried that? – Nils Magne Lunde Aug 07 '13 at 20:27
  • Git does not play nice with UTF-16, in my experience it thinks text in UTF-16 is actually binary. You're better off using UTF-8. –  Aug 07 '13 at 21:04

1 Answers1

4

I had a similar problem with *.rc files for a c++ project and found the best way to solve this is to use git's smudge and clean filters to store everything in the repository as utf-8, and then convert to utf-16 when writing to the working directory.

This way everything git does such as diffs, merges or whatever will work on the utf8 text without issue, but your working copy will have utf16, which will keep visual studio happy.

To configure this, make sure you're using a version of git that has iconv available (the most recent versions of msysgit do) and add the following to your ~/.gitconfig file:

[filter "utf16"]
    clean = iconv -f utf-16le -t utf-8
    smudge = iconv -f utf-8 -t utf-16le
    required

Then in your .gitattributes file add:

*.rc filter=utf16
resource.h filter=utf16

If you already have existing files in utf16 stored as binary, then you need to remove them from the repository and re-add them.

git rm --cached <names-of-utf16-files>
git commit -am "removed utf16 files"
git add <names-of-utf16-files>
git commit -am "added utf16 files as utf8"

And now everything should work.

Imron
  • 782
  • 5
  • 5
  • 7
    Be careful with this solution - .gitconfig is 100% user-dependent. I recommend using it to *filter* what _you_ see and not *change* what is stored. As in, if someone else checks this out (or if you do in a year or two) without that .gitconfig line - *your code won't compile*. It's better to use [diff "utf16"] than filter, because the .rc *must* be stored as UTF-16 in the repository. – Mahmoud Al-Qudsi Jun 04 '16 at 15:48