1

I've been experimenting with different Python modules such as pycurl and requests but still unable to get curl -v <URL> output as shown below.

DESIRED OUTPUT (In Python code)

C:\>curl -v http://example.com/
*   Trying 93.184.216.34...
* TCP_NODELAY set
* Connected to example.com (93.184.216.34) port 80 (#0)
> GET / HTTP/1.1
> Host: example.com
> User-Agent: curl/7.52.1
> Accept: */*
>
< HTTP/1.1 200 OK
< Cache-Control: max-age=604800
< Content-Type: text/html; charset=UTF-8
< Date: Mon, 21 Jan 2019 00:34:32 GMT
< Etag: "1337+ident"
< Expires: Mon, 28 Jan 2019 00:34:32 GMT
< Last-Modified: Fri, 09 Aug 2013 23:54:35 GMT
< Server: ECS (sjc/4E29)
< Vary: Accept-Encoding
< X-Cache: HIT
< Content-Length: 1270
<
<!doctype html>
<html>
... input truncated ...
</html>
* Curl_http_done: called premature == 0
* Connection #0 to host example.com left intact

C:\>

Since this is in Windows, I don't want to use os.system and subprocess modules as curl.exe is not there by default.

Here are my attempts ... But I still did not get similar output as produced by curl -v

>>> import requests
>>> requests.get("http://example.com").content
>>> requests.get("http://example.com").text

>>> import pycurl                        
>>> c = pycurl.Curl()                    
>>> c.setopt(c.URL, 'http://example.com')
>>> c.perform()
  • See https://stackoverflow.com/questions/10588644/how-can-i-see-the-entire-http-request-thats-being-sent-by-my-python-application – Selcuk Jan 21 '19 at 01:17
  • If you want your output to be exactly like `curl` use `curl`. And even Windows can run subprocesses. If you actually want something else: please explain. – Klaus D. Jan 21 '19 at 03:00

2 Answers2

2

If you are looking for information about Last-Modified, Cache-Control etc, in requests you can check out headers.

>>> import requests
>>> req = requests.get("http://example.com")
>>> req.headers
{'Content-Encoding': 'gzip', 
'Accept-Ranges': 'bytes', 
'Cache-Control': 'max-age=604800', 
'Content-Type': 'text/html; charset=UTF-8', 
'Date': 'Mon, 21 Jan 2019 01:13:53 GMT', 
'Etag': '"1541025663"', 
'Expires': 'Mon, 28 Jan 2019 01:13:53 GMT', 
'Last-Modified': 'Fri, 09 Aug 2013 23:54:35 GMT', 
'Server': 'ECS (dca/24A0)', 
'Vary': 'Accept-Encoding', 
'X-Cache': 'HIT', 
'Content-Length': '606'}
Serge
  • 1,027
  • 9
  • 8
1

Without any third-party packages (e.g. requests), in Python 3.7.2:

>>> import urllib.request
>>> dict(urllib.request.urlopen("http://example.com").headers)
{'Accept-Ranges': 'bytes', 'Cache-Control': 'max-age=604800', 'Content-Type': 'text/html; charset=UTF-8', 'Date': 'Mon, 21 Jan 2019 02:52:13 GMT', 'Etag': '"1541025663+gzip"', 'Expires': 'Mon, 28 Jan 2019 02:52:13 GMT', 'Last-Modified': 'Fri, 09 Aug 2013 23:54:35 GMT', 'Server': 'ECS (sjc/4E8B)', 'Vary': 'Accept-Encoding', 'X-Cache': 'HIT', 'Content-Length': '1270', 'Connection': 'close'}

docs

Cole
  • 1,715
  • 13
  • 23