System.Configuration.ConfigurationManager.ConnectionStrings retrieves old value at runtime

Question

In my dotnetcore winforms application I am able to get the connection string correctly from app.config when running in the IDE

  var connectionString = System.Configuration.ConfigurationManager.ConnectionStrings["ConnectionString"].ConnectionString;

For run time, I thought I would just replace the connection string in the MyApp.Win.config file. However the connection string is being pulled from MyApp.Win.dll.config

The docs are here.