urllib3 HTTPResponse.read() returns empty bytes

Question

I'm trying to read a website's content but I get an empty bytes object, b''.

import urllib3
from urllib3 import PoolManager
urllib3.disable_warnings()
https = PoolManager()

r = https.request('GET', 'https://minemen.club/leaderboards/practice/')

print(r.status)
print(r.read())

When I open the URL in a web browser I see the website, and r.status is 200 (success).

Why does r.read() not return the content?

I'd skip this line `urllib3.disable_warnings()` and see if you get any useful message. — norok2, May 04 '20 at 13:02
There is no warning in this case, but you're right to say that the warnings should not be disabled, especially when things don't go as expected. — Keldorn, May 04 '20 at 13:07

Keldorn · Answer 1 · 2023-02-23T08:04:54.587

8

What makes you think it is wrong? Try the following, you'll have much more output:

print(r.data)

Check HTTPResponse to see how to use the r object you got.

edited Feb 23 '23 at 08:04

answered May 04 '20 at 13:04

Keldorn

1,980
15
25

1

The link to HTTPResponse seems to be dead. Can you please update? Thanks – chizou Feb 25 '21 at 17:55
1

Updated Link: https://urllib3.readthedocs.io/en/stable/reference/urllib3.response.html# – Matt Binford Feb 22 '23 at 18:29

score 1 · Answer 2 · edited Jul 05 '22 at 09:28

This is how urllib3.response.HTTPResponse.read is supposed to work.

It is explained for example here by one of the contributors to urllib3:

This is about documentation. You cannot use read() by default, because by default all the content is consumed into data. If you want read() to work, you need to set preload_content=True on the call to urlopen. Want to give that a try?

So you can simply use r.data.

urllib3 HTTPResponse.read() returns empty bytes

2 Answers2