Thanks to WCF Client without app.config/web.config by Kishore Gorjala, I eliminated all reliance on an app.config as follows:
EndpointAddress endpointAddress = new EndpointAddress("http://myServiceURL.com");
WSHttpBinding serviceBinding = new WSHttpBinding();
serviceBinding.ReceiveTimeout = new TimeSpan(0, 0, 120);
MyServiceClient myClient = new MyServiceClient(serviceBinding, endpointAddress);
According to this blog, you might want to try BasicHttpBinding instead of the WSHttpBinding as well.
This technique is also mentioned on the blog Minimal WCF server/client WITHOUT app.config.
Experimental evidence: This worked perfectly - and no more app.exe.config to worry about.