How to configure urllib3 company proxy for contextily?

Question

my end goal is to retrive a tile map using contextily and ploting (from this example):

import geopandas
import contextily as ctx
df = geopandas.read_file(geopandas.datasets.get_path('nybb'))
df = df.to_crs(epsg=3857)
ax = df.plot(figsize=(10, 10), alpha=0.5, edgecolor='k')
ctx.add_basemap(ax)

I am using Anaconda behind a company proxy. Proxies are set correctly in the .condarc file:

proxy_servers:
  http: http://myproxy.com:8080
  https: https://myproxy.com:8080

Package install works fine, and conda config --show does indeed detect the proxy configuration.

However, I get the following urllib3 error after ctx.add_basemap(ax):

urllib3.exceptions.NewConnectionError: <urllib3.connection.VerifiedHTTPSConnection object at 0x000001EDD50E66D8>: Failed to establish a new connection

What I don't understand is that despite the proxy configuration, I need to specify the proxy setting to have a basic urllib3 request work.

If no proxy is specified:

http = urllib3.PoolManager()
r = http.request('GET', 'http://httpbin.org/robots.txt')
Out: NewConnectionError('<urllib3.connection.HTTPConnection object at 0x0000012FE62B0F98>: Failed to establish a new connection:

But specifying the proxy works:

proxy = urllib3.ProxyManager('http://myproxy.com:8080')
proxy.request('GET', 'http://google.com/')
Out: <urllib3.response.HTTPResponse at 0x12fe6280048>

I would have supposed setting the proxy in Anaconda would handle this for any web related request.

I thought of trying to set the proxy info definitely for urllib3, as I understand is done through urllib.request.install_opener(opener) in https://stackoverflow.com/a/36881923/12356293, but such an 'opener' doesn't seem to exist for urllib3.

In the end:

• How to specify proxy information using contextily?
• Why setting the proxy information in Anaconda is not enough for urllib3?

Any help is appreciated, thanks.

Answer 1

I had a similar problem while using contextily behind a company proxy. I managed to get it working by modifiying the code of the tile.py file of contextily. First of all, contextily doesn't use urllib3, it uses pythons request module. This might explain, why your proxy information in Anaconda is not working.

To get the proxy working, I used the answer from this question: How to handle proxies in urllib3

I added at the top:

from urllib3 import ProxyManager, make_headers

default_headers = make_headers(proxy_basic_auth='myusername:mypassword')
http = ProxyManager("https://myproxy.com:8080/", proxy_headers=default_headers)

to the tile.py file.

In the _fetch_tile() method I changed

with io.BytesIO(request.content) as image_stream:

to

with io.BytesIO(request.data) as image_stream:

since I am using urllib3 now. And in the _retryer() method i changed

request = requests.get(tile_url, headers={"user-agent": USER_AGENT})
request.raise_for_status()

to

request = http.request("GET", tile_url)
#request.raise_for_status()

You should probably also adapt the exception clause, to be on the safe side. With these changes I got it working for me. I hope this does work for you too.

Kind Regards J.P.