26

I would like to integrate python Selenium and Requests modules to authenticate on a website.

I am using the following code:

import requests
from selenium import webdriver

driver = webdriver.Firefox()
url = "some_url" #a redirect to a login page occurs
driver.get(url) #the login page is displayed

#making a persistent connection to authenticate
params = {'os_username':'username', 'os_password':'password'}
s = requests.Session()
resp = s.post(url, params) #I get a 200 status_code

#passing the cookies to the driver
driver.add_cookie(s.cookies.get_dict())

The problem is that when I enter the browser the login authentication is still there when I try to access the url even though I passed the cookies generated from the requests session.

How can I modify the code above to get through the authentication web page?

Dharman
  • 30,962
  • 25
  • 85
  • 135
AntoG
  • 833
  • 1
  • 10
  • 16
  • These are two different clients with two different sessions- you cannot mix them in this way. Try complete script on `requests` or complete on `selenium`. Also you might need to try https://pypi.python.org/pypi/selenium-requests/ – Andersson Feb 07 '17 at 10:57
  • I thought passing the cookies would have fixed the problem as I open the browser with the cookies generated by the requests session. Is it not the case then? – AntoG Feb 07 '17 at 11:20
  • Try to update `headers` with same `User-Agent` info, e.g. `"User-Agent": "Mozilla/5.0"` – Andersson Feb 07 '17 at 11:22
  • I got the same issue – AntoG Feb 07 '17 at 12:46
  • My guess is you're sending POST request to a wrong URL. Lots of websites use a different URL (compared to the login landing URL) for authentication – Shane Feb 07 '17 at 13:55
  • this is not the case as `resp.text` prints out the correct page I would like to reach. I think there might be an error in the parameter passed to `add_cookie`. – AntoG Feb 08 '17 at 13:12

2 Answers2

40

I finally found out what the problem was. Before making the post request with the requests library, I should have passed the cookies of the browser first. The code is as follows:

import requests
from selenium import webdriver

driver = webdriver.Firefox()
url = "some_url" #a redirect to a login page occurs
driver.get(url)

#storing the cookies generated by the browser
request_cookies_browser = driver.get_cookies()

#making a persistent connection using the requests library
params = {'os_username':'username', 'os_password':'password'}
s = requests.Session()

#passing the cookies generated from the browser to the session
c = [s.cookies.set(c['name'], c['value']) for c in request_cookies_browser]

resp = s.post(url, params) #I get a 200 status_code

#passing the cookie of the response to the browser
dict_resp_cookies = resp.cookies.get_dict()
response_cookies_browser = [{'name':name, 'value':value} for name, value in dict_resp_cookies.items()]
c = [driver.add_cookie(c) for c in response_cookies_browser]

#the browser now contains the cookies generated from the authentication    
driver.get(url)
AntoG
  • 833
  • 1
  • 10
  • 16
  • 1
    How would that look the other way around? Creating the connection with the driver and pass the auth cookies to the driver? – dan_0 Aug 18 '21 at 13:53
1

I had some issues with this code because its set double cookies to the original browser cookie (before login) then I solve this with cleaning the cookies before set the login cookie to original. I used this command:

driver.delete_all_cookies()
Charlles Pimenta
  • 11
  • 2