0

Is it possible to get the 'content-length' of a web page if the server doesn't include content-length as a header (even if the whole page is downloaded) without downloading the whole content?

I am doing some scraping using python requests and only need to classify data based on size of response.

Below is the response of the server for that matter.

HTTP/1.1 200 OK
Cache-Control: no-cache,private
Content-Type: text/html
Server: Microsoft-IIS/7.2
X-Powered-By: ASP.NET
Date: Fri, 04 Jul 2017 11:28:40 GMT
Connection: close
jonrsharpe
  • 115,751
  • 26
  • 228
  • 437
kappa101
  • 71
  • 1
  • 7
  • 3
    `len(response.content)`? By the time you've got the response, you *have* downloaded the whole content. – jonrsharpe Jul 07 '17 at 12:51
  • the content page is large. For this i need to download the whole content. – kappa101 Jul 08 '17 at 09:06
  • If you're making a GET you're doing that anyway. Maybe try a HEAD (see https://stackoverflow.com/questions/3854842/content-length-header-with-head-requests) but if the server doesn't provide the Content-Length with it you're out of options. – jonrsharpe Jul 08 '17 at 10:23

0 Answers0