git add adding ignored files

Question

I'm trying to remove a previously tracked directory from git, which works, but it's being added back with each subsequent git add ., git add -A, etc. Here's what I've done:

Add to .gitignore in root of project:

node_modules

Run the following:

git rm -r --cached node_modules
git commit -a -m "removed node_modules"
git push origin master

So far so good, this removes the directory from the remote repository. The problem is when I later run git status it tells me the node_modules directory is untracked and keeps adding it back on future commits.

What am I missing and/or how do I find the root of my problem?

From here:

The git add command will not add ignored files by default. ... The git add command can be used to add ignored files with the -f (force) option.

Additional information from comments:

I am tracking .gitignore file.

git check-ignore node_modules/ returns node_modules/ as expected.

No use of submodules.

Update:

I've created a sample that appears to replicate the issue following the steps above:

https://github.com/awhitehouse104/SampleRepo

Resolution:

To summarize the answer and comments from below, the issue was in the encoding of my .gitignore file. I had used echo 'node_modules' > .gitignore to create the file on windows 8 and it came out as UTF-16 with BOM (according to answer below). After a few google searches, it seems this is the default encoding with powershell and I can confirm that saving as UTF-8 seems to have resolved the issue.

tldr; Probably don't use this method of creating .gitignore files or be prepared to change the encoding

echo 'node_modules' > .gitignore

Answer 1

You probably have a negative rule (include-again rule, the one that starts with an !) in your .gitignore file somewhere after the node_modules line.

git check-ignore has a bug/ambiguity in the docs. You expect that if git check-ignore node_modules/ prints node_modules/, then node_modules/ is ignored. But actually it prints a pathname if that pathname matches any ignore pattern - positive or negative. The only way to be sure is to use the -v (--verbose) option, which will make git check-ignore print the matching pattern.
Moreover, if git check-ignore -v says a directory is ignored, it doesn't necessarily mean that all files in that directory are ignored. Example repo:

/
    .git/
    .gitignore
    node_modules/
        bar
        foo

$ cat .gitignore 
/node_modules/*
!/node_modules/foo

$ git check-ignore -v node_modules/
.gitignore:1:/node_modules/*    node_modules/
             ^ positive pattern => ignored

$ git check-ignore -v node_modules/foo
.gitignore:2:!/node_modules/foo node_modules/foo
             ^ negative pattern => not ignored

$ git add -A

$ git status
# On branch master
# Changes to be committed:
#   (use "git reset HEAD <file>..." to unstage)
#
#   new file:   node_modules/foo
#

So if git check-ignore -v node_modules/ says node_modules/ is ignored, do git add -A node_modules/ and then run git check-ignore -v --no-index against individual files that got added, to discover why they were added.

---

Update: I didn't expect that: your .gitignore file is in "UTF-16 with BOM (byte order mark)" encoding:

$ cat .gitignore | hexdump -vC
00000000  ff fe 6e 00 6f 00 64 00  65 00 5f 00 6d 00 6f 00  |..n.o.d.e._.m.o.|
00000010  64 00 75 00 6c 00 65 00  73 00 0d 00 0a 00        |d.u.l.e.s.....|

That's why git probably can't handle it. Save the file in UTF-8 without BOM, that should fix the problem. But I also suggest filing a bug report against git check-ignore - in this corner case its output is clearly not consistent with what git actually ignores.

Answer 2

Add the file to git ignore, then

git update-index --assume-unchanged <file>

Answer 3

You can do

git check-ignore -v --no-index path/with/unexpected/result

to see why git add did or didn't add that path.

git check-ignore docs.

In particular, the point is you want to check what's actually getting added, not a directory.

further, do find . -name .git. Submodules are nested repos, .gitmodules and the submodule command are handy but they're just there to help with them.

Answer 4

Here is what I do to ignore node_modules folder after I tracked it.

1. Create the .gitignore file and add node_modules to it.

2. Commit the .gitignore file. After this point, whatever you update in the node_modules folder won't appear in git status.

3. But this does not delete what we already have on the repo under the node_modules folder. So now we need to remove whatever we have committed previously.

   For this, use git rm --cached node_modules -r

4. Now git status will show that the files are deleted.

5. Use git commit -m "node_modules removed" command with any message.

Now everything should be removed from the repo and future changes will not be tracked.

Answer 5

Another approach if you don't want to use git rm --cached

rm -Rf node_modules
git add -u
git commit -m "stop tracking node_modules"

npm install
# done

Also note the distinction between node_modules and node_modules/ which you seem to have correct. (Thanks umläute for the note on this)

Answer 6

gitignore - Specifies intentionally untracked files to ignore.

$HOME/.config/git/ignore, $GIT_DIR/info/exclude, .gitignore

Each line in a gitignore file specifies a pattern. When deciding whether to ignore a path, Git normally checks gitignore patterns from multiple sources, with the following order of precedence, from highest to lowest (within one level of precedence, the last matching pattern decides the outcome):

To ignore entire directory you should use /**,

A trailing /** matches everything inside. For example, abc/** matches all files inside directory "abc", relative to the location of the .gitignore file, with infinite depth.

Or

You can ignore the entire directory by adding this line to your root .gitignore file:

/Dir_Name

Instead, you can add a /logs/.gitignore file containing this:

[^.]*

The directory will remain in your repo, but all files inside /directory will be ignored. Easy!

The steps you need to follows are,

1. Remove it from the project directory (without actually deleting it):

   git rm --cached folder/*

2. If you don't already have a .gitignore, you can make one right inside of your project folder:
3. project/.gitignore. Put folder/* (or any of the pattern you think better from above listing) in the .gitignore Commit:
4. git commit -m "message".
5. Push your change to github.

Example to exclude everything except a specific directory foo/bar (note the /* - without the slash, the wildcard would also exclude everything within foo/bar):

$ cat .gitignore
# exclude everything except directory foo/bar
/*
!/foo
/foo/*
!/foo/bar

Answer 7

I created a repository to try and duplicate your issue, and I got the first answer from http://www.stackoverflow.com/questions/11451535/gitignore-not-working to work.

Here's my repo if you are curious: https://github.com/IAMZERG/so_project_gitignore

Try adding this to the .gitignore instead:

**/directory_to_remove

After that, run git rm --cached directory_to_remove -r

git status should show that you deleted a bunch of files in the directory you are trying to remove. Then, commit and push to your remote, and everything should be golden... Maybe?

Answer 8

I've found a way to add files to .gitignore even if they have already been commited once. It's a bit brutal but it works great.

1. Add the different files to your .gitignore

2. Move these files out from your git project

3. Rescan & StageChanged, these operation will tell you that these files have been deleted

4. Commit

5. Move back your files to your project, they won't be tracked anymore !

Answer 9

The origin has to be bare. Here is a little bash to demonstrate it. Feel free to edit it, if it does not represent your use case.

#!/bin/bash

rm -rf /tmp/git_test
mkdir /tmp/git_test/
git init --bare /tmp/git_test/primary_repo

# populate repo
cd /tmp/git_test/
git clone ./primary_repo ./secondary_repo
cd secondary_repo
echo hi_bob > some_content
mkdir node_modules
echo hi_dave > node_modules/moduleA
git add .
git commit -m "populated primary"
git push

# do the removal in tertiary 
cd /tmp/git_test/
git clone ./primary_repo ./tertiary_repo
cd tertiary_repo
echo node_modules >> .gitignore
git add .gitignore
git rm -r --cached node_modules
git commit -a -m "removed node_modules"
git push origin master
rm -r node_modules
echo --------------------------------
echo git status:
git status

Answer 10

I had a similary issue. Making sure my encoding was ANSI and the line endings were Unix(LF) fixed my issue.