11

I want to make a web request from one of available IP addresses on server so I use this class:

public class UseIP
{
    public string IP { get; private set; }

    public UseIP(string IP)
    {
        this.IP = IP;
    }

    public HttpWebRequest CreateWebRequest(Uri uri)
    {
        ServicePoint servicePoint = ServicePointManager.FindServicePoint(uri);
        servicePoint.BindIPEndPointDelegate = new BindIPEndPoint(Bind);
        return WebRequest.Create(uri) as HttpWebRequest;
    }

    private IPEndPoint Bind(ServicePoint servicePoint, IPEndPoint remoteEndPoint, int retryCount)
    {
        IPAddress address = IPAddress.Parse(this.IP);
        return new IPEndPoint(address, 0);
    }
}

Then:

UseIP useIP = new UseIP("Valid IP address here...");
Uri uri = new Uri("http://ip.nefsc.noaa.gov");
HttpWebRequest request = useIP.CreateWebRequest(uri);
// Then make the request with the specified IP address

But the solution just works the first time!

Xaqron
  • 29,931
  • 42
  • 140
  • 205
  • Yes, I want to rapidly change my IP address. What approach should I go? – Xaqron May 24 '11 at 18:05
  • Are you getting any exceptions? – alexD May 24 '11 at 18:18
  • You could try to make the Binding only the first time and keep it in a static or instance variable? – Cilvic May 24 '11 at 18:22
  • @alexD: No. Just the first bound IP works. Subsequent instances of `UseIP` class will use the same IP address. – Xaqron May 24 '11 at 18:24
  • The bounty will be awarded for a solution to "changing IP address of web requests frequently". Something like `UseIP` class which works. – Xaqron May 26 '11 at 23:18
  • I am not sure what is wrong. Running the code sample in a test project (translated to VB.NET) and it worked as expected. The IP address returned by the CreateWebRequest method has the updated IP address. I didn't post my sample as an answer as I didn't seem to have done anything special. – Frazell Thomas May 27 '11 at 00:10
  • @Frazell: Try changing the IP address without closing the application. On subsequent changes the application uses the first IP instead of IP you mentioned. – Xaqron May 27 '11 at 13:56

4 Answers4

17

A theory:

HttpWebRequest relies on an underlying ServicePoint. The ServicePoint represents the actual connection to the URL. Much in the same way your browser keeps a connection to a URL open between requests and reuses that connection (to eliminate the overhead of opening and closing the connection with each request), ServicePoint performs the same function for HttpWebRequest.

I think that the BindIPEndPointDelegate that you are setting for the ServicePoint is not being called on each use of HttpWebRequest because the ServicePoint is reusing the connection. If you could force the connection to close, then the next call to that URL should cause the ServicePoint to need to call BindIPEndPointDelegate again.

Unfortunately, it doesn't appear that the ServicePoint interface gives you the ability to directly force a connection to close.

Two solutions (each with slightly different results)

1) For each request, set HttpWebRequest.KeepAlive = false. In my test, this caused the Bind delegate to get called one-for-one with each request.

2) Set the ServicePoint ConnectionLeaseTimeout property to zero or some small value. This will have the effect of periodically forcing the Bind delegate to be called (not one-for-one with each request).

From the documentation:

You can use this property to ensure that a ServicePoint object's active connections do not remain open indefinitely. This property is intended for scenarios where connections should be dropped and reestablished periodically, such as load balancing scenarios.

By default, when KeepAlive is true for a request, the MaxIdleTime property sets the time-out for closing ServicePoint connections due to inactivity. If the ServicePoint has active connections, MaxIdleTime has no effect and the connections remain open indefinitely.

When the ConnectionLeaseTimeout property is set to a value other than -1, and after the specified time elapses, an active ServicePoint connection is closed after servicing a request by setting KeepAlive to false in that request.

Setting this value affects all connections managed by the ServicePoint object.

public class UseIP
{
    public string IP { get; private set; }

    public UseIP(string IP)
    {
        this.IP = IP;
    }

    public HttpWebRequest CreateWebRequest(Uri uri)
    {
        ServicePoint servicePoint = ServicePointManager.FindServicePoint(uri);
        servicePoint.BindIPEndPointDelegate = (servicePoint, remoteEndPoint, retryCount) =>
        {
            IPAddress address = IPAddress.Parse(this.IP);
            return new IPEndPoint(address, 0);
        };

        //Will cause bind to be called periodically
        servicePoint.ConnectionLeaseTimeout = 0;

        HttpWebRequest req = (HttpWebRequest)WebRequest.Create(uri);
        //will cause bind to be called for each request (as long as the consumer of the request doesn't set it back to true!
        req.KeepAlive = false;

        return req;
    }
}

The following (basic) test results in the Bind delegate getting called for each request:

static void Main(string[] args)
    {
        //Note, I don't have a multihomed machine, so I'm not using the IP in my test implementation.  The bind delegate increments a counter and returns IPAddress.Any.
        UseIP ip = new UseIP("111.111.111.111");

        for (int i = 0; i < 100; ++i)
        {
            HttpWebRequest req = ip.CreateWebRequest(new Uri("http://www.yahoo.com"));
            using (WebResponse response = req.GetResponse())
            {
            }
        }

        Console.WriteLine(string.Format("Req: {0}", UseIP.RequestCount));
        Console.WriteLine(string.Format("Bind: {0}", UseIP.BindCount));
    }
Joe Enzminger
  • 11,110
  • 3
  • 50
  • 75
  • Summary - if you want each request to come from a new IP address, make sure HttpWebRequest.KeepAlive is false for each request. Performance will suffer because you are opening and closing connections with each request. Use ConnectionLeaseTimeout if you want to occassionally force a new IP address to be used for a given URI. – Joe Enzminger May 29 '11 at 00:10
  • @Joe: I have tested setting `KeepAlive` of `HttpWebRequest` to `false`. Some websites refuse to service such a client. – Xaqron May 30 '11 at 20:02
  • Ok - if that is the case, you have two options: Use ConnectionLeaseTimeout=0. You will occasionally reuse IP addresses, but about 60% (from my test) of your requests will call the Bind delegate. If that is not acceptable, then using HttpWebRequest wont' work for you. You need to write your own version of a web client that sends a KeepAlive header but closes the connection after the request. You can do this using the System.Net.Socket class. – Joe Enzminger May 30 '11 at 23:00
  • @Joe: What about cheating the remote server. Using `HttpWebServer` with no `KeepAlive` but set the http header manually? – Xaqron May 31 '11 at 01:07
  • @Xaqron: When I tried this (by using req.Headers.Add("Connection", "Keep-Alive");), it throws an exception saying you must use the appropriate property instead. I don't think it's possible to set this particular header manually. I find it odd that some servers would reject requests with no Keep-Alive header - as far as the HTTP protocol is concerned they shouldn't be counting on it. It takes me back to the suggestion that to get the behavior you need you are probably going to have to write your own implementation of HttpWebRequest. – Joe Enzminger May 31 '11 at 04:29
  • @Joe: Using reflection there's a hack to set those headers. – Xaqron May 31 '11 at 04:33
1

Problem may be with the delegate getting reset on each new request. Try below:

//servicePoint.BindIPEndPointDelegate = null; // Clears all delegates first, for testing
servicePoint.BindIPEndPointDelegate += delegate
    {
        var address = IPAddress.Parse(this.IP);
        return new IPEndPoint(address, 0);
    };

Also as far as I know, the endpoints are cached so even clearing the delegate may not work in some cases and they may get reset regardless. You may unload/reload the app domain as the worst case scenario.

Teoman Soygul
  • 25,584
  • 6
  • 69
  • 80
0

I like this new class UseIP.

There is a point at Specify the outgoing IP Address to use with WCF client about protecting yourself from IPv4/IPv6 differences.

The only thing that would need to change is the Bind method to be like this:

private IPEndPoint Bind(ServicePoint servicePoint, IPEndPoint remoteEndPoint, int retryCount)
{
    if ((null != IP) && (IP.AddressFamily == remoteEndPoint.AddressFamily))
        return new IPEndPoint(this.IP, 0);
    if (AddressFamily.InterNetworkV6 == remoteEndPoint.AddressFamily)
        return new IPEndPoint(IPAddress.IPv6Any, 0);
    return new IPEndPoint(IPAddress.Any, 0);
}

re: the Bind method being called multiple times.

What works for me is to remove any delegate link before I add it.

ServicePoint servicePoint = ServicePointManager.FindServicePoint(uri);
servicePoint.BindIPEndPointDelegate -= this.Bind;   // avoid duplicate calls to Bind
servicePoint.BindIPEndPointDelegate += this.Bind;

I also like the idea of caching the UseIP objects. So I added this static method to the UseIP class.

private static Dictionary<IPAddress, UseIP> _eachNIC = new Dictionary<IPAddress, UseIP>();
public static UseIP ForNIC(IPAddress nic)
{
    lock (_eachNIC)
    {
        UseIP useIP = null;
        if (!_eachNIC.TryGetValue(nic, out useIP))
        {
            useIP = new UseIP(nic);
            _eachNIC.Add(nic, useIP);
        }
        return useIP;
    }
}
Community
  • 1
  • 1
Jesse Chisholm
  • 3,857
  • 1
  • 35
  • 29
  • Oops. Sorry. I changed the Type of the property **IP** to be **IPAddress** so I wouldn't have to parse it every time. I forgot to mention that. – Jesse Chisholm Jan 28 '13 at 02:14
0

I have changed your example a little and make it work on my machine:

public HttpWebRequest CreateWebRequest(Uri uri)
{
    HttpWebRequest wr = WebRequest.Create(uri) as HttpWebRequest;
    wr.ServicePoint.BindIPEndPointDelegate = new BindIPEndPoint(Bind);
    return wr;
}

I did that because:

  • I think the call to FindServicePoint actually does the request using the "default" ip, without even calling the binding delegate, to the URI you have specified. In my machine, at least, the BindIPEndPointDelegate was not called in the way you have presented (I know the request was made because I didn't set the Proxy and got a proxy authentication error);
  • In the documentation of ServicePointManager, it states that "If there is an existing ServicePoint object for that host and scheme, the ServicePointManager object returns the existing ServicePoint object; otherwise, the ServicePointManager object creates a new ServicePoint object" witch would probably return always the same ServicePoint if the URI was the same (maybe explaining why the subsequent calls is happening in the same EndPoint).
  • In this way we can be sure that, even when the URI has already been requested, it will use the desired IP instead of using some previous "caching" of ServicePointManager.
Vladimir
  • 264
  • 1
  • 3
  • I have tried this before. If you change the IP address rapidly you will find it doesn't work. The problem is delegate should be `static` so you cannot connect the same `Uri` from different IP addresses at the same time. – Xaqron May 27 '11 at 16:52
  • re: **probably always return the same ServicePoint for the same Uri.** True!. The oddity is that it returns the same ServicePoint for _any_ Uri that addresses the same remote IPAddress. E.g. **http :// 1.2.3.4/FirstTarget** and **http :// 1.2.3.4/SecondTarget** return the same ServicePoint. – Jesse Chisholm Jan 28 '13 at 01:37