0

In my python script, I am fetching pages but I already know the IP of the server. So I could save it the hassle of doing a DNS lookup, if I can some how pass in the IP and hostname in the request.

So, if I call http://111.111.111.111/ and then pass the hostname in the HOST attribute, I should be OK. However the issue I see is on the server side, if the user looks at the incomming request (ie REQUEST_URI) then they will see I went for the IP.

Anyone have any ideas?

Wizzard
  • 12,582
  • 22
  • 68
  • 101
  • 1
    probably http://stackoverflow.com/questions/2236498/tell-urllib2-to-use-custom-dns and http://stackoverflow.com/questions/4555026/repeated-host-lookups-failing-in-urllib2 – okm Mar 25 '12 at 09:41

2 Answers2

0

First, the main idea is suspicious. Well, you can "know" IP of the server but this knowledge is temporary and its correctness time is controlled by DNS TTLs. For stable configuration, server admin can provide DNS record with long TTL (e.g. a few days) so DNS request will be always fulfilled using the nearest caching resolver or nscd. For changing configuration, TTL can be reduced to a few seconds or ever to 0 (means no caching), and it can be useful for some kind of load balancers. You try to organize your own resolver cache which is TTL ignorant, and this can lead to requests to non-functioning or wrong servers, with incorrect contents. So, I suggest not to do this.

If you are strictly sure you shall do this and you can't use external tools as custom resolver or even /etc/hosts, try to install custom "opener" (see urllib2.build_opener() function in documentation) which overrides DNS lookup. However I didn't do this ever, the knowledge is only on documentation read just now.

Netch
  • 4,171
  • 1
  • 19
  • 31
0

You can add the ip address mapping to the hosts file.

zenpoy
  • 19,490
  • 9
  • 60
  • 87