Even though you create a project under your "default repo location" this does not mean a Git repo will be created for you. You need to create (or clone) a Git repository BEFORE creating a project/solution, or use the "Add to Source Control" option to ensure a new repo is created for you.
The "default repo location" is not a Git repo, and a solution folder is not (by default) a Git repo.
The real purpose of VS's "default git repo location" is so that VS tooling can locate any repositories you have cloned locally. It will search the immediate folder you specify and (I believe) any sub-folders. Any folders recognized as Git repository roots will then appear in the list of Repos in the 'connect' page of Visual Studio UI.
If you want to use the command-line this page on git-scm.com has a small snippet which shows how you could create a folder and initialize it as a git repo (locally.) Here is an example showing how to initialize your solution folder as a new, local Git repository from powershell (with msysgit installed):
set-alias git "C:\Program Files (x86)\Git\bin\git.exe"
cd "C:\_Projects\HelloGitWorld\"
git init
git add .
git status
From there you can launch visual studio, load your project/solution and see that git integration is now working as intended. The same set of commands can be executed from a CMD shell, but you will probably have to specify the full path to git, or just use git bash (which may be obtuse for you if you're not accustomed to using unix shells.)
If you don't want to use a command-line tool you can use Visual Studio Tools for Git and select "New.." from the "Connect to Team Projects" panel (thanks to Edward Thomson for pointing this out), or use a third party tool such as TortoiseGit. This will simplify the creation of a git repository without requiring you to understand how Git works, nor how to work with a command-line. For most simply scenarios you can rely entirely on Visual Studio Tools for Git. No need for anything third-party.
I use visualstudio.com and github.com for project hosting, as well as bare (private) repositories on a secure SAN at my home (since, in the end, you can't trust anyone anywhere with your private works. I don't care what people say.) You might find this article on saintsjd.com helpful in deciding if a bare repo is what you're really after, and this guide on stackoverflow to see how to create one yourself.
Let me know if I can clarify anything, or if you need some examples.