Selenium No Element Found Exception even after copying element from source code

Question

I am trying to simply login to this page to access LexisNexis. Here is my code:

from selenium import webdriver
browser = webdriver.Firefox(executable_path='/usr/local/bin/geckodriver')
type(browser)

browser.get('https://sfx.carli.illinois.edu/sfxuiu?url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&ctx_enc=info:ofi/enc:UTF-8&ctx_ver=Z39.88-2004&rfr_id=info:sid/sfxit.com:azlist&sfx.ignore_date_threshold=1&rft.object_id=63750000000001351&svc.fulltext=yes')
linkElem2 = browser.find_element_by_link_text('LEXIS NEXIS DATABASES') 
type(linkElem2)
linkElem2.click()
alert = browser.switch_to.alert
alert.accept()

loginElem = browser.find_element_by_id('j_username')
loginElem.send_keys('username')
passElem = browser.find_element_by_id('j_password')
passElem.send_keys('password')
passElem.submit()

And here is the html source:

div class="form_row"

label for="j_username"
Enter your strongNetID:/strong/label

input id="j_username" name="j_username" autocomplete="OFF" autocapitalize="off" autocorrect="off" value="" type="text"
/div

I am not sure what I am doing wrong :/

theGuy · Answer 1 · 2018-08-23T19:46:47.027

0

Clicking on that link opens a new tab, so you need to switch to new tab before locating username and password fields. Try this (indentation may be broken):

from selenium import webdriver
import time
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
browser = webdriver.Firefox(executable_path='/usr/local/bin/geckodriver')
browser.get('https://sfx.carli.illinois.edu/sfxuiu?url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&ctx_enc=info:ofi/enc:UTF-8&ctx_ver=Z39.88-2004&rfr_id=info:sid/sfxit.com:azlist&sfx.ignore_date_threshold=1&rft.object_id=63750000000001351&svc.fulltext=yes')
linkElem2 = browser.find_element_by_link_text('LEXIS NEXIS DATABASES') 
linkElem2.click()
alert = browser.switch_to.alert  
alert.accept()
time.sleep(5) #to add 5 sec wait, you can use other wait methods for window handle. e.g. https://stackoverflow.com/questions/26641779/python-selenium-how-to-wait-for-new-window-opens
browser.switch_to.window(browser.window_handles[1])
WebDriverWait(browser, 10).until(EC.presence_of_element_located((By.ID, 'j_username')))
loginElem = browser.find_element_by_id('j_username')
loginElem.send_keys('username')
passElem = browser.find_element_by_id('j_password')
passElem.send_keys('password')
passElem.submit()

you will need following imports

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

edited Aug 23 '18 at 19:46

answered Aug 23 '18 at 18:06

theGuy

683
5
15

Thank you! I am still getting errors though. first it said name 'driver' not defined, so I used browser instead. Then it said list index out of range, so I changed it to 0. Then it says name 'wait' not defined, so I used: – D. Shaat Aug 23 '18 at 18:18
from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC from selenium.common.exceptions import TimeoutException – D. Shaat Aug 23 '18 at 18:18
correct, You will need imports for wait method, please see my updated answer – theGuy Aug 23 '18 at 18:23
I used the imports, but I keep getting Timeout Exception. I tried 10,20,30,60 but still not working :/ – D. Shaat Aug 23 '18 at 18:26
I am not sure if this means it still can't find the element. Maybe the element is hidden? But I am not sure how to make it so that python can find it!! – D. Shaat Aug 23 '18 at 18:27
No, its not hidden. Are you accepting alert? I was trying in chrome earlier and now I just tried in FF and found that you have to accept alert. Can you try with that? – theGuy Aug 23 '18 at 18:28
Yup, accepting alert and also not switching tabs because that gave me an error – D. Shaat Aug 23 '18 at 18:29
No, you will have to switch to a different tab, try my updated answer again. – theGuy Aug 23 '18 at 18:33
Just tried it, getting this error: browser.switch_to.window(browser.window_handles[1]) #// to switch to new tab IndexError: list index out of range – D. Shaat Aug 23 '18 at 18:35
instead of browser.switch_to.window(browser.window_handles[1]) can you try browser.switch_to_window(driver.window_handles[-1]) // this would get the lastest window. Also, try to add some wait before searching for new tab – theGuy Aug 23 '18 at 18:54
yes, it did. It was just a matter of adding some wait before searching for new tab as I mentioned in my above comment. I have updated my answer. – theGuy Aug 23 '18 at 19:24
Now I am getting this error: ConnectionResetError: [Errno 54] Connection reset by peer – D. Shaat Aug 23 '18 at 19:39
This is part of the output I see: File "/Users/danashaat/anaconda/lib/python3.6/http/client.py", line 297, in begin version, status, reason = self._read_status() File "/Users/danashaat/anaconda/lib/python3.6/http/client.py", line 258, in _read_status line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1") File "/Users/danashaat/anaconda/lib/python3.6/socket.py", line 586, in readinto return self._sock.recv_into(b) ConnectionResetError: [Errno 54] Connection reset by peer – D. Shaat Aug 23 '18 at 20:01
It doesn't show me exactly which line is producing this error but I do get to the login page before getting the error – D. Shaat Aug 23 '18 at 20:02
none of the code you posted in error is in the stack trace. The problem in your code where ever you are using read() method. – theGuy Aug 23 '18 at 20:16
I don't really know what that means!! – D. Shaat Aug 23 '18 at 20:31

Sers · Accepted Answer · 2018-08-23T23:15:10.050

1. Problem "LEXIS NEXIS DATABASES" link open new window and you have to switch to it to login, Handle multiple windows in Python:

from selenium import webdriver

browser = webdriver.Firefox(executable_path='/usr/local/bin/geckodriver')
browser.implicitly_wait(5)

browser.get('https://sfx.carli.illinois.edu/sfxuiu?url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&ctx_enc=info:ofi/enc:UTF-8&ctx_ver=Z39.88-2004&rfr_id=info:sid/sfxit.com:azlist&sfx.ignore_date_threshold=1&rft.object_id=63750000000001351&svc.fulltext=yes')
browser.find_element_by_link_text('LEXIS NEXIS DATABASES').click()

browser.close()
browser.switch_to.window(browser.window_handles[0])

browser.find_element_by_id('j_username').send_keys('username')
passElem = browser.find_element_by_id('j_password')
passElem.send_keys('password')
passElem.submit()

2. Problem Link you share in comments is have error when you open for the first time(new browser profile). All you need is open page and click "Login with your Illinois NetID" to open login page successfully. This way much better, because you don't need to handle windows, check code below:

from selenium import webdriver

browser = webdriver.Firefox(executable_path='/usr/local/bin/geckodriver')
browser.implicitly_wait(5)

browser.get('https://compass2g.illinois.edu/webapps/login/')
browser.find_element_by_link_text('Login with your Illinois NetID').click()

browser.implicitly_wait(5)

browser.find_element_by_id('j_username').send_keys('username')
passElem = browser.find_element_by_id('j_password')
passElem.send_keys('password')
passElem.submit()

Still not working :/ I even ran the exact code from the book automate the boring stuff, the one to login to email, and it gives the same error! Element not found!! — D. Shaat, Aug 23 '18 at 20:32
https://shibboleth.illinois.edu/idp/profile/SAML2/POST/SSO?execution=e1s1 — D. Shaat, Aug 23 '18 at 21:44
Great! This helped a lot!!!!! I had to change the passElem line to look like the browser.find_element_by_id line, but that was it. Thanks again, I spent HOURS trying to figure this out — D. Shaat, Aug 24 '18 at 00:02

Selenium No Element Found Exception even after copying element from source code

2 Answers2