Why does selenium throw element not found error even when the element is present?

Question

This is what i wrote, its a simple program to login to my instagram account, credentials changed:

from selenium import webdriver
import time
import os

class InstaBot:

    def __init__(self, username=None, password=None):

        self.username = username
        self.password = password

        self.driver = webdriver.Chrome('./chromedriver.exe')

        self.login()

    def login(self):

        self.driver.get(r'https://www.instagram.com/accounts/login/')

        # login_btn = self.driver.find_element_by_xpath('//*[@id="react-root"]/section/main/div/article/div/div[1]/div/form/div[3]') # login button xpath changes after text is entered, find first

        self.driver.find_element_by_name('username').send_keys(self.username)
        self.driver.find_element_by_name('password').send_keys(self.password)

        # login_btn.click()

if __name__ == '__main__':

    ig_bot = InstaBot(username='temp', password='tempp')

I don't know why it throws error. Please help.

Traceback (most recent call last):
  File "instabot.py", line 30, in <module>
    ig_bot = InstaBot(username='temp', password='tempp')
  File "instabot.py", line 15, in __init__
    self.login()
  File "instabot.py", line 23, in login
    self.driver.find_element_by_name('username').send_keys(self.username)
  File "C:\Program Files\Python37\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 496, in find_element_by_name
    return self.find_element(by=By.NAME, value=name)
  File "C:\Program Files\Python37\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 978, in find_element
    'value': value})['value']
  File "C:\Program Files\Python37\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "C:\Program Files\Python37\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"[name="username"]"}
  (Session info: chrome=86.0.4240.183)

Chrome does open up the login page but fails to find the element.

PS: You could have guessed that I am new to it

Edit :

I figured that when i use time.sleep for some seconds, it works properly.... but i dont see it as the best way to fix it, so kindly see to it and suggest me something, or this is how it actually works...

score 0 · Accepted Answer · answered Nov 08 '20 at 09:04

I found the answer, YESSS!!

I should have waited for the page to load before searching for the elements, they are of two types:

1 explicit waits

2 implicit waits

These are actually documented here : https://selenium-python.readthedocs.io/waits.html

The following should be added before searching for the elements:

WebDriverWait(self.driver, 2).until(EC.presence_of_element_located((By.PARTIAL_LINK_TEXT, "Sign up")))

For this to work some import lines are required at the top, ie

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

Edit1 : The one I used earlier is the implicit and the one used now is explicit

Why does selenium throw element not found error even when the element is present?

1 Answers1