Fetching cookies from a file with python

Question

I'm using mechanize and python to log into a site. I've created two functions. The first one logs in and the second one searches the site. How exactly do I store the cookies from the login so when I come to searching I have a cookie.

Current code.

import mechanize
import cookielib

def login(username, password):
    # Browser
    br = mechanize.Browser()

    # Cookie Jar
    cj = cookielib.LWPCookieJar()
    br.set_cookiejar(cj)
    cj.save('cookies.txt', ignore_discard=False, ignore_expires=False)
    # Rest of login

def search(searchterm):

    # Browser
    br = mechanize.Browser()

    # Cookie Jar
    cj = cookielib.LWPCookieJar()
    br.set_cookiejar(cj)
    cj.load('cookies.txt', ignore_discard=False, ignore_expires=False)
    # Rest of search

I read through the cookielib info page but there aren't many examples there and I haven't been able to get it working. Any help would be appreciated. Thanks

is your goal getting the search results or saving the cookie? — Otto Allmendinger, Feb 08 '12 at 12:31

bigato · Answer 1 · 2013-05-27T11:21:26.043

You have to login first before you can save the cookies:

   # Cookie Jar
   cj = cookielib.LWPCookieJar()
   br.set_cookiejar(cj)

   # Do the login ...

   # Only now you have something to save:
   cj.save('cookies.txt', ignore_discard=False, ignore_expires=False)

Edit: Just to add to the answer, I am dealing at the moment with authentication at a website where I had to change both the options to ignore_discard=True, ignore_expires=True, in both the save and the load methods. Otherwise it would not work, because only one of the three cookies I was receiving would be saved. I looked into Firefox and it is saving all three cookies also.

score 3 · Accepted Answer · answered Feb 08 '12 at 12:30

You need to use the same browser instance, obviously:

def login(browser, username, password):
  # ...

def search(browser, searchterm):
  # ...

br = mechanize.Browser()
cj = cookielib.LWPCookieJar()
br.set_cookiejar(cj)
cj.load('cookies.txt', ignore_discard=False, ignore_expires=False)
login(br, "user", "pw")
search(br, "searchterm")

Now that you have common context, you should probably make a class out of it:

class Session(object):
  def __init__(browser):
    self.browser = browser

  def login(user, password):
    # ... can access self.browser here

  def search(searchterm):
    # ... can access self.browser here

br = mechanize.Browser()
cj = cookielib.LWPCookieJar()
br.set_cookiejar(cj)
cj.load('cookies.txt', ignore_discard=False, ignore_expires=False)
session = Session(br)
session.login("user", "pw")
session.search("searchterm")

Fetching cookies from a file with python

2 Answers2