In your config file you can define configSource:
<configuration>
<appSettings configSource="filepath1.config" />
<connectionStrings configSource="filepath2.config" />
<!--etc-->
</configuration>
Put the configurations that you need to keep private in a separate config file, then exclude them in your .gitignore.
Keep in mind that this will ignore the whole section and overwrite it with the context you have in the referenced file.
You can also do Configuration Transform, which allows you to only overwrite a small set of variables in sections. For example:
In your main Web.config:
<configuration>
<appSettings>
<add key="Key1" value="Something I dont't Care"/>
<add key="Key2" value="Something dummy"/>
</appSettings>
</configuration>
And in your Web.Release.config:
<configuration xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform">
<appSettings>
<add key="Key2" value="Something I want to keep secret"
xdt:Transform="SetAttributes" xdt:Locator="Match(key)" />
</appSettings>
</configuration>
In this case the "Key2" value that you want to keep private will be in a separate file, and you can exclude the Web.Release.config through .gitignore.
Also there's another approach that I never tried, which can also overwrite config using external file.