0

I want to set the headers in urllib.request to get rid off the http.client.BadStatusLine in a way

headers = ['Content-length']=str(len(bytes(body, 'utf-8')))
req = urllib.request.Request(url, bytes(body, 'utf-8'), headers)

Source : BadStatusLine exception raised when returning reply from server in Python 3

What does body refers to in hearders?

Community
  • 1
  • 1
user6575792
  • 69
  • 1
  • 7
  • Possible duplicate of [Set the header in urllib.request python 3](http://stackoverflow.com/questions/38418477/set-the-header-in-urllib-request-python-3) – user94559 Jul 17 '16 at 06:53

1 Answers1

0

The correct syntax is headers['Content-length']=str(len(bytes(body, 'utf-8')))

Guess there is a typo. The Content length header denotes the size of the html response or otherwise by the server. For example If the server is requested an html page then it returns the size of the html content.

For example when you request the page : http://www.york.ac.uk/teaching/cws/wws/webpage1.html
(It is simple page to demo, it has no extra resources), The server response headers are shown below. You can see this for any page in the dev console in chrome or firefox , Under "network" tab and "all" headers.

Accept-Ranges:bytes
Cache-Control:max-age=300
Connection:Keep-Alive
Content-Encoding:gzip
Content-Length:1957
Content-Type:text/html
Date:Sun, 17 Jul 2016 06:48:41 GMT
Expires:Sun, 17 Jul 2016 06:53:41 GMT
Keep-Alive:timeout=4, max=100
Server:Apache/2.2.22 (Ubuntu)
Vary:Accept-Encoding

The content-Length header shows a length of 1957 bytes. If you go the page and check the source out by pressing Ctrl-U. You will observe that the size of the text is much more. But the text received is compressed. So head on to http://www.txtwizard.net/compression , And put the source text. You will observer that the compressed size is 1957 bytes (Encoding is also mentioned as Content-Encoding:Gzip in the headers, and the site is configured for gz compression.

So, as for the code, body is the response of the server html or otherwise (for eg a file, or image or anything as denoted by the content-type header.)

Another example : https://github.com/python/cpython/blob/master/Lib/http/server.py#L752
this is the python3 code for SimpleHTTPResponseHandler, that can be used to write custom handlers for the inbuilt python server.

The variable encoded is the string that contains the html response encoded as a binary file to be passed to the do_GET function. Check out that the Content-Length header is set to the size of encoded string.

prateeknischal
  • 752
  • 4
  • 12