57

When would someone use httplib and when urllib?

What are the differences?

I think I ready urllib uses httplib, I am planning to make an app that will need to make http request and so far I only used httplib.HTTPConnection in python for requests, and reading about urllib I see I can use that for request too, so whats the benefit of one or the other?

jahmax
  • 8,181
  • 7
  • 26
  • 25
  • Remember that in Python 3.x `urllib` and `urllib2` have been **unified** into a single module `urllib`. So 2.x `urllib` and 3.x `urllib` are NOT the same modules. – treecoder Sep 20 '11 at 09:57
  • 2
    For those with the same question (coming in from Google), know that all of these answers are great. But also know that the answer to "Which library you should use" largely depends on what is important to you: do you want a friendly API OR something that automates settings and makes a GET very simple to do? Are you free to make your own library choices, OR is having a library "built in" (ie, you'd have to support users who may not have 'httplib2' installed) something which is important to you? There's a case for many libraries because the built-in Python module can't satisfy them all. – Scott Prive Jul 22 '13 at 14:00
  • Try [requests](http://pypi.python.org/pypi/requests), the very simple and powerful module based on urllib2- docs [here](http://docs.python-requests.org/en/latest/index.html). – Ambyte Sep 20 '11 at 12:17

6 Answers6

48

urllib (particularly urllib2) handles many things by default or has appropriate libs to do so. For example, urllib2 will follow redirects automatically and you can use cookiejar to handle login scripts. These are all things you'd have to code yourself if you were using httplib.

Robus
  • 8,067
  • 5
  • 47
  • 67
19

I would like to say something about urllib, urllib2, httplib and httplib2.

The main difference between urllib* and httplib* is that:

httplib and httplib2 handles HTTP/HTTPs request and response directly and give you more space to do your own job.

urllib and urllib2 are build upon httplib, they are more abstract and powerful, but sometimes won't fulfill your specified need about some HTTP related operations.

And for httplib and httplib2, I'd say that they are both HTTP client library. However httplib2 is much more powerful and have much more features than httplib.

As for urllib and urllib2, quote from this link:

urllib and urllib2 are both Python modules that do URL request related stuff but offer different functionalities. Their two most significant differences are listed below:

  • urllib2 can accept a Request object to set the headers for a URL request, urllib accepts only a URL. That means, you cannot masquerade your User Agent string etc.
  • urllib provides the urlencode method which is used for the generation of GET query strings, urllib2 doesn't have such a function. This is one of the reasons why urllib is often used along with urllib2.

I would recommend my personal blog Httplib Httplib2 Urllib Urllib2-what’s the Difference.

Hope it helps:-)

evandrix
  • 6,041
  • 4
  • 27
  • 38
Lihang Li
  • 361
  • 2
  • 5
10

urllib/urllib2 is built on top of httplib. It offers more features than writing to httplib directly.

however, httplib gives you finer control over the underlying connections.

Corey Goldberg
  • 59,062
  • 28
  • 129
  • 143
6

If you're dealing solely with http/https and need access to HTTP specific stuff, use httplib.

For all other cases, use urllib2.

Matt Joiner
  • 112,946
  • 110
  • 377
  • 526
5

If you need high level stuff like Caching, Keep-Alive, Compression or Authentication, tryhttplib2

Cees Timmerman
  • 17,623
  • 11
  • 91
  • 124
optixx
  • 2,110
  • 3
  • 16
  • 16
1

For those folks moving things up to Py3 (and for some reason cannot or have not refactored to use the awesome requests module), this is a good transition between versions:

try:
    import http.client as httplib
except ImportError:
    import httplib

Works in both Python version sets.

Cometsong
  • 568
  • 9
  • 21