4

Is there a way to automatically follow redirects using the http.client module:

def grab_url(host, path = '/'):
  class Data: pass
  result = Data()
  try:
    con = http.client.HTTPConnection(host)
    con.request('GET', path)
    response = con.getresponse()
    if response.status == 200:
      result.content = response.read().decode('utf-8')
      result.headers = response.getheaders()
  catch Exception as e:
    print(e)
  return result

The above works as long as the request returns a http response of 200, but I don't know how to handle redirects like 301?

Using pyCurl I would simply set the FOLLOWLOCATION to True:

def grab_url(host, path = '/'):
  buffer = BytesIO()
  c = pyCurl.Curl()
  c.setopt(c.FOLLOWLOCATION, True)
  c.setopt(c.URL, host + path)
  c.setopt(c.WRITEDATA, buffer)
  c.perform()
  status = c.getinfo(c.RESPONSE_CODE)
  if status == 200:
    return buffer.getvalue().decode('iso-8859-1')
Cyclonecode
  • 29,115
  • 11
  • 72
  • 93
  • https://stackoverflow.com/questions/20475552/python-requests-library-redirect-new-url?answertab=votes#tab-top ? i will suggest you to use requests instead if you can – toheedNiaz Apr 15 '18 at 19:35
  • @toheedNiaz yes I know I could use curl or requests but is this not possible using the http.client module? – Cyclonecode Apr 15 '18 at 19:42
  • https://stackoverflow.com/a/110547/2699643 here is something that explains what you want – toheedNiaz Apr 15 '18 at 19:48

0 Answers0