git commit leaves changes in the index

Question

The Problem

I'm attempting to commit some changes, but they're being left behind, for reasons I do not understand:

I take a look a the current repo state:

% git status
On branch new-master
Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

    new file:   __init__.py
    new file:   api/__init__.py
    new file:   api/api.py
    new file:   api/common.py
    new file:   api/error.py

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

    modified:   ../status/server.py
    modified:   __init__.py

Untracked files:
  (use "git add <file>..." to include in what will be committed)

    ../env/
    ../foo.db

Messy, but what is in the index is what I want to commit. (Note: there are changes that aren't staged: I am not interested in committing those. The changes under "Changes to be committed" are the changes I desire to be committed.) I commit it:

% git commit -m "Why won't you commit?"
[new-master a4dbe36] Why won't you commit?

And immediately re-run status, to witness nothing has changed:

% git status
On branch new-master
Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

    new file:   __init__.py
    new file:   api/__init__.py
    new file:   api/api.py
    new file:   api/common.py
    new file:   api/error.py

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

    modified:   ../status/server.py
    modified:   __init__.py

Untracked files:
  (use "git add <file>..." to include in what will be committed)

    ../env/
    ../foo.db

If I look at the last commit, we can see that nothing made it in:

% git log -n1 -p
commit a4dbe36e77a9f3b04c2ef33de75e28b6aa7cac0c
Author: Me <john.smith@example.com>
Date:   Fri May 2 11:52:06 2014 -0700

    Why won't you commit?

What is going on here? Why aren't these changes committed?

A minimal testcase!

mkdir bar && cd bar
git init
echo "data." > some_existing_file
git add some_existing_file
git commit -m "Initial commit."

That's just set up. Now for the real fun. I get funky behavior with --intent-to-add (I may just not understand what it does), but this reproduces my full problem:

mkdir subdir && mkdir subdir/another
cd subdir
echo "foodata" > a_file
echo "more data" > another/other_data
git add --intent-to-add a_file
git add another/other_data

And then attempt to commit:

git commit -m "Why isn't this committing anything?"

Potentially use outputs

I don't have any custom hooks (that I'm aware of):

% ls "`git rev-parse --git-dir`"/hooks
applypatch-msg.sample     pre-applypatch.sample     pre-rebase.sample
commit-msg.sample         pre-commit.sample         prepare-commit-msg.sample
post-update.sample        pre-push.sample           update.sample

If it matters, the directory the added files are in is new as well, and does not exist in the preceeding commit. (I was in the directory itself when I ran git status, see the .. in the not-staged-for-commit section.)

git diff HEAD a4dbe36e77a, as requested, does not output anything.

git show:

% git show a4dbe36e77a
commit a4dbe36e77a9f3b04c2ef33de75e28b6aa7cac0c
Author: Me <john.smith@example.com>
Date:   Fri May 2 11:52:06 2014 -0700

    Why won't you commit?

Also, the same, in raw:

% git show --format=raw a4dbe36e77a
commit a4dbe36e77a9f3b04c2ef33de75e28b6aa7cac0c
tree ca9326ccba91dda7540198c082db6e6ab3fb097a
parent 1842d56d5a47ff33bd420a5014f208c85acc5a1f
author Me <john.smith@example.com> 1399056726 -0700
committer Me <john.smith@example.com> 1399056726 -0700

    Why won't you commit?

An ls-tree of that tree:

% git ls-tree --full-tree ca9326ccba91dda7540198c082db6e6ab3fb097a
100644 blob fab05d167a3f8ef9ddf00d7af09e00eea03f1d28    requirements.txt
040000 tree 0d9742d9c1728495d600bde05a7f276f3c65d96d    status

For brevity: git diff --staged shows a diff of changes: this is the diff that I desire to be committed, but that git commit isn't committing. (git diff --cached also shows this diff.) git commit -v also shows this diff, but typing a message does not result in a commit.

The only really weird thing about this branch is that it was an orphaned branch — master contains a WIP, but messy version of this code. I was hoping to clean it up, but retain master because I'm human, and worry I'll delete something. However, this isn't the first commit. (The first commit has actual changes, too.) It's only now something's gone wrong.

Potential clue / something isn't right here:

EDIT @torek had me run this:

% git write-tree
ca9326ccba91dda7540198c082db6e6ab3fb097a

This should turn the index's contents into a tree (which could then be attached to a commit object, and thus a commit is born). However, that hash is the same as HEAD. Given that there are changes in my index, one would expect that the tree ID would have to be different. Specifically, these commands show something is odd:

% git write-tree
ca9326ccba91dda7540198c082db6e6ab3fb097a
% git diff --staged --stat
 wsgi_util/__init__.py     |  0
 wsgi_util/api/__init__.py |  0
 wsgi_util/api/api.py      | 69 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 wsgi_util/api/common.py   | 84 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 wsgi_util/api/error.py    | 82 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 235 insertions(+)
% git write-tree
ca9326ccba91dda7540198c082db6e6ab3fb097a
% git show --format=raw HEAD
commit fe28d60c21e998104e5967faf7af3bf203cd4b26
tree ca9326ccba91dda7540198c082db6e6ab3fb097a
parent f86425decfcb3410d5bc90da6ce6146e04775953
author Me <john.smith@example.com> 1399059521 -0700
committer Me <john.smith@example.com> 1399059521 -0700

    What is going on here?

git ls-files -s, which seems relevant:

100644 fab05d167a3f8ef9ddf00d7af09e00eea03f1d28 0   requirements.txt
100644 e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 0   status/__init__.py
100644 923b97d0d9e6c68e77751bb4e7fb14c41fa9d71a 0   status/api/__init__.py
100644 75f064fc9c170b56545c6b92958b38c10260beec 0   status/api/message.py
100644 07cf4dc004231945847e9f93e3de214dd02c3661 0   status/api/message_db.py
100644 78cda37dcc7a4b4f34089b2d843a9abfcd4118cf 0   status/server.py
100644 bee7c1eb7b92ccf6b78c7dfc5f36e71b77ec5792 0   status/util.py
100644 e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 0   wsgi_util/__init__.py
100644 e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 0   wsgi_util/api/__init__.py
100644 e92f248874be5beec7b18228d4bf0d062b508916 0   wsgi_util/api/api.py
100644 68a5a62034d52684222c7075f59f2927c1d8695e 0   wsgi_util/api/common.py
100644 28d9add6dae84ddab293e3a44cac4824f8d2d482 0   wsgi_util/api/error.py

Another clue?

Because this was getting significantly in the way, I did a git reset --mixed HEAD. I then proceeded to recreate the index. One of the files, wsgi_util/__init__.py isn't empty, but I wish to commit it (for now) as an empty file: there is no useful code in there. To do this, I have thus far run git add --intent-to-add wsgi_util/__init__.py. I cannot commit this index.

However, I discovered that if I instead replace it with an empty file, and then do a normal git add, the resulting index will commit.

What does `git commit -v` show? Does it say it will commit the new files, or does it show an empty list? — John Kugelman, May 02 '14 at 19:06
@JohnKugelman: It displays what I would expect to be committed. (Several files, and their bodies as diffs. (all +, since they're new files.)) — Thanatos, May 02 '14 at 19:12
It would be interesting to see a `git diff HEAD a4dbe36e77a` and a `git show a4dbe36e77a` too — Renato Zannon, May 02 '14 at 19:13
@RenatoZannon: I've added those to the question, as well as an `ls-tree` of the tree object attached to the commit `a4dbe3`. — Thanatos, May 02 '14 at 19:19
@mustard: `git diff --cached` displays a diff that matches my expectations of what should be committed. — Thanatos, May 02 '14 at 19:24
Presumably no stray env variables like GIT_INDEX_FILE either? (Although if it's consistently set even *that* should work.) — torek, May 02 '14 at 19:29
@torek: Nope `echo $GIT_INDEX_FILE` doesn't show anything, and `env` doesn't show that. I also opened a new terminal, and tried committing from there, but with no luck. — Thanatos, May 02 '14 at 19:35
Hm, well, we could try some low level plumbing stuff. `git write-tree` should turn the current index into a tree and give you back the tree id, which you can then look at with `git ls-tree` to see if it looks right. If so the problem is that `git commit` isn't writing that tree, somehow. — torek, May 02 '14 at 19:36
@mustard: I did a reset to the last non-empty commit, but no dice. (I didn't do a reset before posting this, so the `a4dbe` commit's parent is also an empty commit, which was my first attempt.) — Thanatos, May 02 '14 at 19:39
@torek: Now that gives an interesting output, and I think you're onto something. `git write-tree` outputs `ca9326ccba91dda7540198c082db6e6ab3fb097a`, *which is the same tree objects as HEAD*, which I posted above. — Thanatos, May 02 '14 at 19:42
Are there, by any chance, any symlinks in `..` ? Or `git rev-parse --show-cdup` ? — CB Bailey, May 02 '14 at 19:43
OK, so, writing a tree based on the current index produces the wrong tree ... meaning either write-tree is broken, or the current index is screwed up somehow. — torek, May 02 '14 at 19:43
@CharlesBailey: No symlinks. `git rev-parse --show-cdup` outputs a single empty line. `git ls-files -s` looks right to me; I've added it to the question. — Thanatos, May 02 '14 at 19:50
@Alex yea, you can see in the first code block that he had changes staged. — Sam, May 02 '14 at 19:57
@Alex yes, at one point, I did run `git add`. @jeremy I don't understand your question: a few of the files are empty, a few are not. None are committed. — Thanatos, May 02 '14 at 20:01
@torek (@here) Can `git --intent-to-add` produce this behavior? I have a file I added in that manner (just to have it be empty: I do not care for the contents currently); resetting the index and not running `git --intent-to-add` causes the commit to be created. — Thanatos, May 02 '14 at 20:09
@Thanatos: git --intent-to-add explanation from https://www.kernel.org/pub/software/scm/git/docs/git-add.html: **Record only the fact that the path will be added later. An entry for the path is placed in the index with no content.** The expectation is that these files will explicitly be added later when they are ready to be committed — mustard, May 02 '14 at 20:25
@mustard "An entry for the path is placed in the index with no content." I had interpreted this to mean that it was equivalent to adding an empty file. (roughly equivalent to `mv file file.bak && touch file && git add file && mv file.bak file`) This seems to not be the case. Where is this "intent" flag stored, and how can I see it? (And why does `git` not error on this?) — Thanatos, May 02 '14 at 20:28
(I've added a minimal example to the question that reproduces the behavior I'm seeing.) — Thanatos, May 02 '14 at 20:28
@Thanatos - I was able to reproduce this behavior by trying to add a file using the intent flag. My guess is - because of the flag - the file is never added as part of index. The index is different from head - since it is the proposed snapshot. But because the files are never moved to index - the commit is empty. But I cannot find a reference to support that yet. — mustard, May 02 '14 at 20:42
Interesting. I'd have to go dig into the source to see precisely what `--intent-to-add` does, but it's definitely the culprit. See git's `cache-tree.c` around line 358. If `CE_INTENT_TO_ADD` is set the tree entries get stripped out of the tree written by `write-tree`. — torek, May 02 '14 at 20:42
@torek: See my minimal testcase: I'm witnessing not only the `--intent-to-add`'d entries getting let out, but other, normal entries as well. (That the only thing that `if` does is set `to_invalidate`, which is a function-level variable, makes me suspicious.) — Thanatos, May 02 '14 at 20:47
The local `to_invalidate` causes another flag to get set that in turn finally results in that `CE_INTENT_TO_ADD` flag stored in the cache itself. Later reads of the cache see the flag and strip out stuff. I didn't read on to figure out what gets stripped when, why, etc., but clearly that's source of the problem. — torek, May 02 '14 at 20:49
Oops, sorry, was reading too many things simultaneously, there's two different locals, one in `git add` and one in `cache-tree.c` itself. Anyway, the one in cache-tree causes it->entry_count to be set to -1, which winds up stripping out not only the file, but also apparently all the other files in the directory. This is probably a bug... — torek, May 02 '14 at 21:12
See also [Can I use git diff on untracked files?](https://stackoverflow.com/a/857696/456814). It appears that one of the uses of "intent to add" is to add untracked files to diff output. Not sure of such files are meant to be staged for commit though. — , May 02 '14 at 21:17
@Cupcake: I used `--intent-to-add` because my understanding of it (at the time) as that it added a zero length file to the index (and I wanted a zero-length file committed). However, as torek's comments point out, there is more to it than that. — Thanatos, May 02 '14 at 21:42
Just to check, you don't have `GIT_WORK_TREE` or `GIT_DIR` environment variables set to any particular value, right? — VonC, May 03 '14 at 04:49

score 1 · Answer 1 · answered May 03 '14 at 18:39

The documentation states this about --intent-to-add

Record only the fact that the path will be added later. An entry for the path is placed in
the index with no content. This is useful for, among other things, showing the unstaged
content of such files with git diff and committing them with git commit -a

You can read for yourself here. https://www.kernel.org/pub/software/scm/git/docs/git-add.html

From my understanding of the documentation and experimenting myself, you need to use git commit -a to add the files.