I am using requests to fetch a webpage and I would like to save the cookies in a Netscape-style cookies.txt file. How can I achieve this? I have tried the following:
import requests
import cookielib
url = 'http://www.yahoo.com'
ua = ("Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:21.0) "
"Gecko/20100101 Firefox/21.0")
headers = {'user-agent': ua}
mcj = cookielib.MozillaCookieJar()
requests.get(url, headers=headers, cookies=mcj)
mcj.save('cookies.txt')
print open('cookies.txt').read()
but it just creates a cookie file with no cookies, so the output is:
# Netscape HTTP Cookie File
# http://www.netscape.com/newsref/std/cookie_spec.html
# This is a generated file! Do not edit.