To solved this I choose the XML transformation route using the VS plug-in slowcheetah (idea taken here)
With this tool you create build configuration for each of your environments (ex: dev, QA, PP) and generate the XML transform of the app.config.
Replace the required section Example:
<configuration xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform">
<SqlUnitTesting xdt:Transform="Replace">
<DataGeneration ClearDatabase="true" />
<ExecutionContext Provider="System.Data.SqlClient" ConnectionString="Data Source=sqspsqlxx;Initial Catalog=IC_RMI_Result;Integrated Security=True;Pooling=False;MultipleActiveResultSets=False"
CommandTimeout="30" />
<PrivilegedContext Provider="System.Data.SqlClient" ConnectionString="Data Source=sqpsqlxx;Initial Catalog=IC_RMI_Result;Integrated Security=True;Pooling=False;MultipleActiveResultSets=False"
CommandTimeout="30" />
</SqlUnitTesting>
For TFS integration there is probably a way to use the out of box nuget restore but I simply added the package to source control.
At teamp build queue (or harcode in your build deinition) simply specify the build configuration you want to target (dev, qa, PP).
(note: my team build agent uses windows crendentials so I made sure the account has the required rights on the target DBs)