Git nothing added to commit but untracked files present - even with those files added to gitignore

Question

I have untracked files present that I need in the local directory, but wish not to commit to the remote repo (e.g. obj files, mtl files, .vs folder and so on). I have this .gitignore file I use:

## Ignore Visual Studio temporary files, build results, and
## files generated by popular Visual Studio add-ons.
##
## Get latest from https://github.com/github/gitignore/blob/master/VisualStudio.gitignore

# User-specific files
*.suo
*.user
*.userosscache
*.sln.docstates
*.obj
*.png
*.log
*.blend
*.blend1
*.mtl
*.jpg
*.jpeg
*.opendb


# User-specific files (MonoDevelop/Xamarin Studio)
*.userprefs

# Build results
[Dd]ebug/
[Dd]ebugPublic/
[Rr]elease/
[Rr]eleases/
x64/
x86/
bld/
[Bb]in/
[Oo]bj/
[Ll]og/

# Visual Studio 2015/2017 cache/options directory
.vs/
# Uncomment if you have tasks that create the project's static files in wwwroot
#wwwroot/

# Visual Studio 2017 auto generated files
Generated\ Files/

# MSTest test Results
[Tt]est[Rr]esult*/
[Bb]uild[Ll]og.*

# NUNIT
*.VisualState.xml
TestResult.xml

# Build Results of an ATL Project
[Dd]ebugPS/
[Rr]eleasePS/
dlldata.c

# Benchmark Results
BenchmarkDotNet.Artifacts/

# .NET Core
project.lock.json
project.fragment.lock.json
artifacts/

# StyleCop
StyleCopReport.xml

# Files built by Visual Studio
*_i.c
*_p.c
*_h.h
*.ilk
*.meta
*.obj
*.iobj
*.pch
*.pdb
*.ipdb
*.pgc
*.pgd
*.rsp
*.sbr
*.tlb
*.tli
*.tlh
*.tmp
*.tmp_proj
*.log
*.vspscc
*.vssscc
.builds
*.pidb
*.svclog
*.scc

# Chutzpah Test files
_Chutzpah*

# Visual C++ cache files
ipch/
*.aps
*.ncb
*.opendb
*.opensdf
*.sdf
*.cachefile
*.VC.db
*.VC.VC.opendb

# Visual Studio profiler
*.psess
*.vsp
*.vspx
*.sap

# Visual Studio Trace Files
*.e2e

# TFS 2012 Local Workspace
$tf/

# Guidance Automation Toolkit
*.gpState

# ReSharper is a .NET coding add-in
_ReSharper*/
*.[Rr]e[Ss]harper
*.DotSettings.user

# JustCode is a .NET coding add-in
.JustCode

# TeamCity is a build add-in
_TeamCity*

# DotCover is a Code Coverage Tool
*.dotCover

# AxoCover is a Code Coverage Tool
.axoCover/*
!.axoCover/settings.json

# Visual Studio code coverage results
*.coverage
*.coveragexml

# NCrunch
_NCrunch_*
.*crunch*.local.xml
nCrunchTemp_*

# MightyMoose
*.mm.*
AutoTest.Net/

# Web workbench (sass)
.sass-cache/

# Installshield output folder
[Ee]xpress/

# DocProject is a documentation generator add-in
DocProject/buildhelp/
DocProject/Help/*.HxT
DocProject/Help/*.HxC
DocProject/Help/*.hhc
DocProject/Help/*.hhk
DocProject/Help/*.hhp
DocProject/Help/Html2
DocProject/Help/html

# Click-Once directory
publish/

# Publish Web Output
*.[Pp]ublish.xml
*.azurePubxml
# Note: Comment the next line if you want to checkin your web deploy settings,
# but database connection strings (with potential passwords) will be unencrypted
*.pubxml
*.publishproj

# Microsoft Azure Web App publish settings. Comment the next line if you want to
# checkin your Azure Web App publish settings, but sensitive information contained
# in these scripts will be unencrypted
PublishScripts/

# NuGet Packages
*.nupkg
# The packages folder can be ignored because of Package Restore
**/[Pp]ackages/*
# except build/, which is used as an MSBuild target.
!**/[Pp]ackages/build/
# Uncomment if necessary however generally it will be regenerated when needed
#!**/[Pp]ackages/repositories.config
# NuGet v3's project.json files produces more ignorable files
*.nuget.props
*.nuget.targets

# Microsoft Azure Build Output
csx/
*.build.csdef

# Microsoft Azure Emulator
ecf/
rcf/

# Windows Store app package directories and files
AppPackages/
BundleArtifacts/
Package.StoreAssociation.xml
_pkginfo.txt
*.appx

# Visual Studio cache files
# files ending in .cache can be ignored
*.[Cc]ache
# but keep track of directories ending in .cache
!*.[Cc]ache/

# Others
ClientBin/
~$*
*~
*.dbmdl
*.dbproj.schemaview
*.jfm
*.pfx
*.publishsettings
orleans.codegen.cs

# Including strong name files can present a security risk
# (https://github.com/github/gitignore/pull/2483#issue-259490424)
#*.snk

# Since there are multiple workflows, uncomment next line to ignore bower_components
# (https://github.com/github/gitignore/pull/1529#issuecomment-104372622)
#bower_components/

# RIA/Silverlight projects
Generated_Code/

# Backup & report files from converting an old project file
# to a newer Visual Studio version. Backup files are not needed,
# because we have git ;-)
_UpgradeReport_Files/
Backup*/
UpgradeLog*.XML
UpgradeLog*.htm
ServiceFabricBackup/
*.rptproj.bak

# SQL Server files
*.mdf
*.ldf
*.ndf

# Business Intelligence projects
*.rdl.data
*.bim.layout
*.bim_*.settings
*.rptproj.rsuser

# Microsoft Fakes
FakesAssemblies/

# GhostDoc plugin setting file
*.GhostDoc.xml

# Node.js Tools for Visual Studio
.ntvs_analysis.dat
node_modules/

# Visual Studio 6 build log
*.plg

# Visual Studio 6 workspace options file
*.opt

# Visual Studio 6 auto-generated workspace file (contains which files were open etc.)
*.vbw

# Visual Studio LightSwitch build output
**/*.HTMLClient/GeneratedArtifacts
**/*.DesktopClient/GeneratedArtifacts
**/*.DesktopClient/ModelManifest.xml
**/*.Server/GeneratedArtifacts
**/*.Server/ModelManifest.xml
_Pvt_Extensions

# Paket dependency manager
.paket/paket.exe
paket-files/

# FAKE - F# Make
.fake/

# JetBrains Rider
.idea/
*.sln.iml

# CodeRush
.cr/

# Python Tools for Visual Studio (PTVS)
__pycache__/
*.pyc

# Cake - Uncomment if you are using it
# tools/**
# !tools/packages.config

# Tabs Studio
*.tss

# Telerik's JustMock configuration file
*.jmconfig

# BizTalk build output
*.btp.cs
*.btm.cs
*.odx.cs
*.xsd.cs

# OpenCover UI analysis results
OpenCover/

# Azure Stream Analytics local run output
ASALocalRun/

# MSBuild Binary and Structured Log
*.binlog

# NVidia Nsight GPU debugger configuration file
*.nvuser

# MFractors (Xamarin productivity tool) working folder
.mfractor/

# Local History for Visual Studio
.localhistory/

And as you see, it is clearly stated that I want git to ignore the files tracked. Yet I get this message:

git status On branch default_shader Untracked files: (use "git add ..." to include in what will be committed)

    .gitignore
    .vs/
    3dengine_headeronly.sln
    3dengine_headeronly/
    CMakeLists.txt
    Debug/
    Tests/
    cmake-build-debug/
    packages/
    resources/

I followed this SO post on the exact same problem and their solution was to use a .gitignore file. In my case it didn't work: whenever I use git add . or git add -A all files get added, regardless of their presence in .gitignore.

Note I get this message I quoted after git rm -r --cached . to unstage my previously added unwanted files.

How to get around the issue?

Answer 1

Use this :

git add --all

Or this :

git add -A

Answer 2

The list of untracked files reads (this is just a straight quote of your own quote):

.gitignore
.vs/
3dengine_headeronly.sln
3dengine_headeronly/
CMakeLists.txt
Debug/
Tests/
cmake-build-debug/
packages/
resources/

Now, note that there are only three names here that do not end with a slash:

1. .gitignore

   You probably should add and commit this file, so you probably do want it to be untracked at the moment, until you add and commit it.

2. 3dengine_headeronly.sln

   This file is not listed as "do not auto-add nor complain about untracked-ness". Check it out yourself: search through your .gitignore contents for anything resmbling 3dengine_headeronly.sln. One entry that comes close is the last line in this group:

   # JetBrains Rider
   .idea/
   *.sln.iml
   

   which lists *.sln.iml as "do not auto-add nor complain". But 3dengine_headeronly.sln ends with .sln, not .sln.iml. Another is the line *.sln.docstates, but again, that ends with .sln.docstates, and the file's actual name ends with just .sln.

3. CMakeLists.txt

   As before, there is no entry that would make Git shut up about this.

That leaves unexplained the entries ending with slashes. The problem with diagnosing this is that Git has summarized the contents of those directories, rather than listing the individual files. One can only explain this if one has the actual file names, which requires running git status with the -u or --untracked-files= flag, and giving this flag the word all to tell Git to list each untracked file individually.

Without this flag ... well, let's say, for instance, that Tests contains:

Tests/a.b
Tests/c.d
Tests/e.obj

One of these three files matches the *.obj pattern, but the other two match no pattern, so Git will complain that Tests/a.b and Tests/c.d are untracked. But without -uall, the way Git will complain about this is to say that Tests/ is untracked. This is true even if every file within Tests/ is marked as "don't complain" except for one file in Tests or some subdirectory within Tests: Git will summarize the one file by printing Tests/ because there are no tracked files within Tests/, but there is at least one untracked file within Tests/ that Git has not been told "shut up about this file".

There is one that is pretty mysterious, and that one is Debug/. You do have this line in your .gitignore:

[Dd]ebug/

This should cause all files within Debug and any subdirectory inside Debug to be not-complained-about. Using -uall will probably shed some light on what is going on, though. (See below for why this one is such a mystery.)

Long: how git status scans the work-tree

It's important to realize that Git never tracks a directory itself. What listing a directory in .gitignore does is give Git permission not to examine the directory.

To understand this we need to define what it means to track a file in the first place, and how Git examines your work-tree when it wants to collect up a list of untracked files for git status to complain about.

A tracked file, in Git, is simply any file whose name is currently listed in Git's index. To see which files are currently in the index, you can run git ls-files --stage (but note that in a big directory this tends to list a lot of files). These are the files that will be in the next commit you make. Initially, the index is full of all the files that were in the commit you most recently checked out. The next commit will contain all those same files. If you overwrite one of the index copies, the next commit will contain the updated index copy—and that's what git add does, when you git add a file that's already in the index.

Hence, an untracked file is simply any file that exists in the work-tree, but that is not currently in the index. Using git rm—with or without --cached—you can remove any file from the index, if it's currently in the index. Once it's gone from the index, if you've kept the work-tree copy, it has become untracked.

What git status does is to run two comparisons—two git diffs, in effect. The first one compares the current commit frozen files to the index's unfrozen (but Git-ified) files: whatever is the same, Git says nothing about, and whatever is different, Git says: file with pathname P is staged for commit. This is way better than listing every file in the index: it tells you about file P only if it's going to be different in the next commit.

The second comparison is the tricky one, and is where .gitignore comes in. To run the second comparison, Git compares the Git-ified but not-yet-frozen copy of a file that's in the index with the regular, non-Git-ified copy in the work-tree. When the two are different, Git tells you that this file is not staged for commit. You can run git add to copy the work-tree copy over top of the index copy, so that the index and work-tree match—and now, presumably, the HEAD commit and index copies of the file differ, so that the next git status will say staged for commit.

That works great for files that are in both the index and the work-tree, but fails to alert you to files that you forgot to git add to copy into the index. So, having collected up the list of comparison results, Git also has to scan through the work-tree to find every file that's actually there. Any file that is in the work-tree, but does not have a corresponding copy in the index, is untracked.

This scanning is one of the slower parts of git status (though the actual speed depends a lot on the speed of your operating system). Git starts by reading through every file and sub-directory ("folder") name within the top level work-tree directory. It has to read this directory entirely, of course, because it's the top level of the work-tree. Any file name in this directory represents a file in the work-tree, and the file is either tracked—in the index—or not. If the file is untracked, Git will complain about it unless you also suppress the complaint, using a .gitignore entry.

For directories found in the top level, though, git status does not check whether the directory is in the index, because directories are never in the index. Instead, Git just looks inside the directory to see what files and directories it contains. Any files within that directory have to be checked to see if they're untracked, and if so, whether git status should gripe about them. If the directory contains its own sub-directories, those sub-directories must be scanned in the same way, recursively.

What listing a directory in .gitignore does is give Git permission to not look inside it. Let's consider the Debug/ directory specifically, for instance. Let's assume that there are no tracked files inside Debug/, and that you have run git status.

Git will start by reading the top level directory. In this directory, it finds files named .gitignore, 3dengine_headeronly.sln, and CMakeLists.txt. Those files are not in the index and not listed as a name or pattern in .gitignore, so git status will complain about them (at the end, when it lists untracked files). But it also finds .vs/, 3dengine_headeronly/, Debug/, Tests/, cmake-build-debug/, packages/, and resources/.

Now, if there are some tracked files within (say) packages, Git's going to have to descend into packages anyway, to compare the index copies of those files with their work-tree copies. But we've assumed here that there are no tracked files in Debug. So git status should be able to run through the .gitignore entries and see that [Dd]ebug (Git removes the trailing slash for comparison purposes) matches Debug. This would permit git status to skip over Debug entirely, not reading it at all.

The fact that Debug/ comes out at the end, with just the trailing slash, means that Git must have opened and scanned the Debug directory. The only reason Git would do that is if there is at least one tracked file inside Debug. Hence:

git ls-files --stage | grep Debug/

will probably show at least one tracked file. Even so, it's not clear why the [Dd]ebug/ rule did not match the untracked files contained in the directory, so getting a list of the exact names, and using git --no-index check-ignore -v on any tracked files inside Debug/, might be helpful.

(This optimization, where git status sometimes doesn't open a directory at all, is especially tricky, and is where a lot of .gitignore issues start. There's not much to be done about this until and unless someone teaches Git better ways to skip, or not skip, entire directories. However, using git status -uall definitely helps some of the other tricky cases.)

Answer 3

In my case it was only a bug. When I pushed to github I have seen that commit isn't actually empty. Make sure to add all modified files to commit.