84

I am trying to use urllib2 to open url and to send specific cookie text to the server. E.g. I want to open site Solve chess problems, with a specific cookie, e.g. search=1. How do I do it?

I am trying to do the following:

import urllib2
(need to add cookie to the request somehow)
urllib2.urlopen("http://chess-problems.prg")
starball
  • 20,030
  • 7
  • 43
  • 238
Oleg Tarasenko
  • 9,324
  • 18
  • 73
  • 102

5 Answers5

113

Cookie is just another HTTP header.

import urllib2
opener = urllib2.build_opener()
opener.addheaders.append(('Cookie', 'cookiename=cookievalue'))
f = opener.open("http://example.com/")

See urllib2 examples for other ways how to add HTTP headers to your request.

There are more ways how to handle cookies. Some modules like cookielib try to behave like web browser - remember what cookies did you get previously and automatically send them again in following requests.

rll
  • 5,509
  • 3
  • 31
  • 46
Messa
  • 24,321
  • 6
  • 68
  • 92
  • 9
    If you have multiple cookies, you need to join them in a single header value, separated by a semicolon. For example, if you have a dictionary of cookie values called `cookievals`, then use `opener.addheaders.append(('Cookie', "; ".join('%s=%s' % (k,v) for k,v in cookievals.items())))`. – Greg Glockner Feb 04 '15 at 04:22
  • What if they're using other params to urlopen? – Andrew May 08 '17 at 19:48
56

Maybe using cookielib.CookieJar can help you. For instance when posting to a page containing a form:

import urllib2
import urllib
from cookielib import CookieJar

cj = CookieJar()
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
# input-type values from the html form
formdata = { "username" : username, "password": password, "form-id" : "1234" }
data_encoded = urllib.urlencode(formdata)
response = opener.open("https://page.com/login.php", data_encoded)
content = response.read()

EDIT:

After Piotr's comment I'll elaborate a bit. From the docs:

The CookieJar class stores HTTP cookies. It extracts cookies from HTTP requests, and returns them in HTTP responses. CookieJar instances automatically expire contained cookies when necessary. Subclasses are also responsible for storing and retrieving cookies from a file or database.

So whatever requests you make with your CookieJar instance, all cookies will be handled automagically. Kinda like your browser does :)

I can only speak from my own experience and my 99% use-case for cookies is to receive a cookie and then need to send it with all subsequent requests in that session. The code above handles just that, and it does so transparently.

Morten Jensen
  • 5,818
  • 3
  • 43
  • 55
  • 6
    @PiotrDobrogost you are right, I do not send a specific cookie with this code :) I am assuming I will receive one when POSTing, and the `CookieJar` instance will handle it for me on all relevant domains from then on. – Morten Jensen Dec 03 '12 at 21:26
  • 2
    The OP clearly states *(…) to send specific cookie text (…)* so this is not an answer. – Piotr Dobrogost Aug 20 '17 at 20:37
13

You might want to take a look at the excellent HTTP Python library called Requests. It makes every task involving HTTP a bit easier than urllib2. From Cookies section of quickstart guide:

To send your own cookies to the server, you can use the cookies parameter:

>>> cookies = dict(cookies_are='working')

>>> r = requests.get('http://httpbin.org/cookies', cookies=cookies)
>>> r.text
'{"cookies": {"cookies_are": "working"}}'
Morten Jensen
  • 5,818
  • 3
  • 43
  • 55
Piotr Dobrogost
  • 41,292
  • 40
  • 236
  • 366
  • 1
    It's based on `urllib3`, it supports pooling of connections like urllib3, and persists cookies, headers - see [`Requests Session`](http://docs.python-requests.org/en/latest/user/advanced/#session-objects). It's beautiful! – Serge S. Jan 27 '13 at 18:41
5

Use cookielib. The linked doc page provides examples at the end. You'll also find a tutorial here.

Marcelo Cantos
  • 181,030
  • 38
  • 327
  • 365
  • In the tutorial they are saving cookie after receiving it from server isn't it? – Oleg Tarasenko Jul 26 '10 at 12:42
  • They are both saving cookies retrieved from the server and returning them back to the server, since this is usually how cookies work. If you want to do something out of the ordinary, then you'll have to dig a little deeper into the library. I'm pretty sure it'll support whatever you want to do. – Marcelo Cantos Jul 26 '10 at 12:49
3

This answer is not working since the urllib2 module has been split across several modules in Python 3. You need to do 

from urllib import request
opener = request.build_opener()
opener.addheaders.append(('Cookie', 'cookiename=cookievalue'))
f = opener.open("http://example.com/")
Solal
  • 611
  • 2
  • 9
  • 26