Creating robust HTTP connection for dummy users with WinINET

Question

I'm making a program which downloads a simple file from internet on Windows, using Wininet family API because I want to utilize its IE-compatible proxy behavior. As you all know, current IE has several proxy settings: auto-detect (WPAD), auto-configure (PAC), manually single URL, proxy servers per protocol, socks, direct, ... For most users, the "direct download" works fine; however for some users (especially those behind firewall / NAT), they always need special proxy settings when making connections.

It's painful to write code to handle all these cases so I hope WinINET with InternetOpen (INTERNET_OPEN_TYPE_PRECONFIG) can help me. It does for most users, however I still find some users complaining connection failure. These user may have very special network environments (eg, need username/password auth for proxy) and direct connection does not work for them.

Sometimes dummy users had wrong configuration, and I'd like wininet to try "all" possible proxy settings for me; unfortunately the INTERNET_OPEN_TYPE_PRECONFIG will only try the one that user configured, not "every possible proxy settings".

So my question is, how do I make a program with most strong ability to workaround all http connection (especially for proxy configuration) for dummy users (i.e, they don't understand how to configure their system)? Is there any suggested way to make HTTP connections without the need to take care of proxy stuff? (i.e., a "super" connection solver which will try all possible proxy settings), or if there's any method to tell WinINET to enable all its proxy settings to create connection?

Answer 1

A reasonable algorithm would be this:

1. try a connection using the user's current proxy settings (the ones that IE uses). These are the ones most likley to be working. See WinHttpGetIEProxyConfigForCurrentUser() on MSDN to get these settings, or INTERNET_OPEN_TYPE_PRECONFIG in WinINet.

2. if that fails, try a direct connection.

3. if that fails, try a WPAD autodetection using WinHttpGetProxyForUrl. (WPAD is the automated way that clients can find a Proxy Auto-Config (PAC) file on a network, typically a corporate network.) You'll want to select the option to check both DHCP and DNS for the proxy file. WinHTTP AutoProxy Functions on MSDN contains a code sample using this API and lots of supporting info. If this succeeds, you'll get back the proxy info you can then plug into your HTTP download code. AFAIK, there is not an equivalent WinINet option to do a new WPAD detection automatically-- only WinHTTP.

4. Next you can try to look for Firefox proxy settings. This SO answer details information about the location and format of these settings so you can access them programmatically.

5. if that still fails, ask the user (in your UI) to check that their internet connection is actually connected. it's actually far more common for users to be disconnected than for all of the above proxy-detection steps to fail. ask the user to open up their browser to make sure they can get to an internet site-- if the steps above fail, their browser is likley unable to see the web. Once the user claims they're connected, repeat steps 1-3.

6. if all of above fail, the you really have little choice but to ask the user to manually specify their proxy info. You'll probably want to clone the proxy UI of IE or firefox, and turn those UI settings into the appropriate parameters to configure proxy options when you download that file.

Note that there's no magic to proxy detection-- your choices are use the user's settings, use the default computer settings, use a direct connection, use WPAD to find a PAC file, or ask the user. There is, unfortunately, no way to "make HTTP connections without the need to take care of proxy stuff".

BTW, although you could use WinHTTP only for proxy detection and then use WinINet for fetching the file, I'd suggest using WinHTTP for both parts. When you want the most flexibility and control over your HTTP interaction (as well as better stability than WinINet), WinHTTP is preferable.

UPDATE 2:

J.J. had a good idea above-- I added a step for checking Firefox proxy settings. Since oddball proxies are generally found in corporate environments (which tend to standardize on IE) it's somewhat less likley that this will help, but it's worth a try.

UPDATE: I just edited the section above in response to Francis's correct comment: WinHTTP is not the "successor" to WinINet in a strict sense (meaning 100% of the features of WinINet are avilable in WinHTTP). Rather, WinHTTP is designed for apps which are using HTTP for data-interchange and don't care about integrating with Interet Explorer's cache, cookies, dial-up UI, etc.

Server apps definitely fall into this category (WinHTTP, unlike WinINet, is safe for use in server apps) but lots of client software use it too, for example the Windows Update client on every modern PC. In general, WinHTTP is preferable for a client app which needs to download files over HTTP and needs more fine-grained control over the networking and wants to control its own UI.

WinHTTP is more hands-on (for example, you need to make one API call to get proxy info and another to use that proxy info) but that extra degree of control allows you to do things you can't do with WinINet, like forcing ignoring the user's proxy settings and trying a new WPAD detection.

Answer 2

The best description I've found of the process IE itself uses is in How the Windows Update client determines which proxy server to use to connect to the Windows Update Web site. Combine the sequence when using IE to access Windows Update with your knowledge of WinINet or WinHTTP WinHttpGetIEProxyConfigForCurrentUser, WinHttpDetectAutoProxyConfigUrl, WinHttpGetProxyForUrl, and WinHttpGetDefaultProxyConfiguration to get your application connected.

Here's the sequence from the KB article:

1. If the user is configured to automatically detect settings, look for a PAC file using WPAD.
2. No PAC file found? If the user has the IE automatic configuration script set, use that PAC file.
3. Still no PAC file found? Look for a proxy. If the user has manually configured IE proxy settings, use them.
4. No IE proxy configured? Check the default proxy config.
5. No default proxy configured? Use a direct connection.