0

As the title says, if I do resp=httpx.get(url) how do I get the IP from which the url was resolved?

There's This question about how to do it with requests but the same methods don't appear to exist in the httpx objects.

Dean MacGregor
  • 11,847
  • 9
  • 34
  • 72
  • Do you really need to do this via httpx, or would just `socket.gethostbyname(...)` do? – AKX Oct 03 '22 at 10:40
  • @AKX The site I'm downloading from has a load balancer so the server, and by extension IP, requests are served by isn't always the same. Can we know for certain that, in serial calls to `socket.gethostbyname` and then `httpx.get`, they'll be the same server or is there a chance they'd be different? – Dean MacGregor Oct 03 '22 at 10:48
  • We can't know for sure, and proxies etc. will muddle the waters further. See my answer for an ugly incantation though ;) – AKX Oct 03 '22 at 10:51

2 Answers2

4

Empirically, you can do it with an exquisitely ugly incantation like as follows.

Do note that this involves digging into no less than 6 underscore-private attribute of the response stream, so it could stop working at any time when you upgrade httpx. (This worked for me with httpx 0.23.0.)

import httpx

with httpx.Client() as hc:
    r = hc.get("https://httpbin.org/get")
    peername = r.stream._stream._httpcore_stream._stream._connection._network_stream._sock.getpeername()
    print(r.json())
    print(peername)

This prints out

{'args': {}, 'headers': {'Accept': '*/*', 'Accept-Encoding': 'gzip, deflate', 'Host': 'httpbin.org', 'User-Agent': 'python-httpx/0.23.0', 'X-Amzn-Trace-Id': 'Root=...'}, 'origin': '....', 'url': 'https://httpbin.org/get'}
('44.207.168.240', 443)
AKX
  • 152,115
  • 15
  • 115
  • 172
  • How did you come up with that so fast? Are you a contributor to httpx and knew where to look right away? – Dean MacGregor Oct 03 '22 at 13:42
  • No - I just used a debugger to dig into the responses object since I knew it would have to contain a reference to the socket, or the data wouldn't be readable via it :) – AKX Oct 03 '22 at 14:35
0

I was looking to solve the same problem. I was concerned about forward the compatibility of the "exquisitely ugly incantation" provided in the answer here. I'm using httpx with Trio, and realized I can use the Trio socket implementation to use an async version of sockets to answer this question. see: https://trio.readthedocs.io/en/stable/reference-io.html#trio.socket.getaddrinfo

I posted this question to the httpx Q&A and got this answer: https://github.com/encode/httpx/discussions/2633#discussioncomment-5439764 In short, access to this information is provided through httpcore. There is an open issue to update the documentation.

import httpx

with httpx.Client() as client:
    with client.stream("GET", "https://httpbin.org/get") as response:
        network_stream = response.extensions["network_stream"]
        server_addr = network_stream.get_extra_info("server_addr")
        print(response.json())
        print(server_addr)

read: https://www.encode.io/httpcore/extensions/#extra-network-information

EGarbus
  • 136
  • 4