0

I have a code where in a loop I would call many requests for web scraping

import requests

requests = requests.Session()

for x in range(0,5):
    cookies = {
        'key': 'iteration: '+ str(x)
    }
    response = requests.get('A site', cookies=cookies).text
    
    response = requests.get('A site/proxies', cookies=cookies).text
    print(requests.cookies)

I have tried this but the cookies are not showing up

renea
  • 1
  • 2
  • You are missing what session cookies are supposed to be used for. They are for accepting and storing cookies the server sends *you* and then automatically sending those cookies along during the next request to a matching domain. – luk2302 Nov 04 '22 at 08:21
  • @luk2302 yes I have a matching domain like asite.com and another is asite.com/proxy. Can session cookies be used to also send and then store it? – renea Nov 04 '22 at 08:23
  • Does this answer your question? [How to add a cookie to the cookiejar in python requests library](https://stackoverflow.com/questions/17224054/how-to-add-a-cookie-to-the-cookiejar-in-python-requests-library) – luk2302 Nov 04 '22 at 08:25
  • yes! @luk2302 thank you, it's just what it is cookiejar – renea Nov 04 '22 at 08:34

0 Answers0