Selenium Automation to Select a li Menu Item in An ul with Python and Selenium

Question

So I am having an overall issue where I am trying to automate the login, menu-item selection, and then pre-populate the fields to initialize a file download. I have been able to auto login, many thanks to the answer provided within another post. However, I have not been successful in trying to navigate to a specific menu item and select that item to take me to a new page. If I go to that page directly, I am logged out and have to log back in. There is a spot called "Custom Reports", this is what I need to click on to take me to another page. If the li objects had IDs it would make this so much easier, however there are no IDs as one could imagine. Here is the source for the Webpage I am trying to automate, follow the link to it that I have stored in github.

https://github.com/Richard-Barrett/ITDataServicesInfra/blob/master/Python/Collegeboard/TSI/tsi_pagesource

I am trying to click on this element

                  <li ng-show="core.acl['244']!='N'" ng-class="{active:isLocation('/customReports')}">
                     <a href="#/customReports" translate='reports.CustomReports.title'></a>
                  </li>

As you can imagine, I have not been able to do so. Here is the code I am using to pull out the information so far. Login Automation works.

#!/bin/python
# ===========================================================
# Created By: Richard Barrett
# Organization: DVISD
# DepartmenT: Data Services
# Purpose: Test Score & 3rd Party Website Data Pull Automation
# Date: 01/20/2020
# ===========================================================

import selenium
import os
import unittest
import requests
import time
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support.select import Select

#Variable Definitions NEED TO MAKE
#find_elements_by_name
#find_elements_by_xpath
#find_elements_by_link_text
#find_elements_by_partial_link_text
#find_elements_by_tag_name
#find_elements_by_class_name
#find_elements_by_css_selector


#URL Variables
login_url = 'https://www.accuplacer.org/'
redirect_url = 'https://www.accuplacer.org/api/home.html#/'
reports_scheduler_url = 'https://www.accuplacer.org/api/home.html#/reportScheduler'
custom_reports_url = 'https://www.accuplacer.org/api/home.html#/customReports'

#WebDriver Path
browser = webdriver.Chrome("C:\Program Files (x86)\Google\Chrome\chromedriver.exe")

#Parent URL
browser.get("https://www.accuplacer.org")

# Authentication
# Credentials NEEDS TO BE ENCRYPTED AND NOT BAKED INTO THE SCRIPT NEEDS UNIT TEST
username = browser.find_element_by_id("login")
password = browser.find_element_by_id("password")
username.send_keys("##############")
password.send_keys("##############")
#browser.send_keys(Keys.ENTER)
#browser.send_keys(Keys.RETURN)

# Work on Notifications Window Clause
# Need a step for when notifications appear to click "Submit" option
# <button type="button" ng-show="updateWhatsNewFlag == 'failure'" translate="submit" class="btn btn-sm btn-success ng-scope" ng-click="updateShowWhatsNewFlag()" style="">Submit</button>

# Need a step to close if the notification popup window is present
# <button type="button" class="btn btn-default ng-scope" data-dismiss="modal" ng-click="clearMessages()" translate="common.btn.close">Close</button>

# Authentication submit.click()
#browser.find_element_by_css_selector('.btn.btn-lg.btn-primary').click()
element = WebDriverWait(browser, 20).until(
        EC.element_to_be_clickable((By.CSS_SELECTOR, ".btn.btn-lg.btn-primary.pull-left")))
element.click();

#Navigate to CustomReports XPATH=//*[@id="leftNav"]/ul/li[11]/ul/li[9]/a
#browser.find_element_by_xpath('//*[@id="leftNav"]/ul/li[11]').click()


# Make the report by selecting objects
#obj_report = Select(browser.find_element_by_name("Reports"))
element = WebDriverWait(browser, 20).until(
        EC.element_to_be_clickable((By.CSS_XPATH, "*[@id="leftNav"]/ul/li[11]/ul/li[9]/a")))
element.click();

Everytime I try to us the xpath it doesn't give me anything and basically says it doesn't exist.

I recently restructured the repository for the page source and selenium recording: https://github.com/Richard-Barrett/ITDataServicesInfra/blob/master/Python/Selenium/Workflows/SeIDE_TSI_WORKFLOW — R. Barrett, Jan 24 '20 at 17:50

undetected Selenium · Accepted Answer · 2020-01-23T22:04:14.563

1

The desired element is an Angular element so to locate and click() on the element you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following Locator Strategies:

Using CSS_SELECTOR:

WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "li>a[href$='customReports'][translate*='CustomReports']"))).click()

Using XPATH:

WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.XPATH, "//li/a[contains(@href,'customReports') and contains(@translate, 'CustomReports')]"))).click()

Note : You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

edited Jan 23 '20 at 22:04

answered Jan 23 '20 at 21:42

undetected Selenium

183,867
41
278
352

Added those in ```DevTools listening on ws://127.0.0.1:60753/devtools/browser/8cb1948b-10e1-46bd-ad25-e4fcf8ff49e6 Traceback (most recent call last): File ".\collegboard_tsi_export.py", line 69, in custom_report = WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.XPATH, "//li[contains(@ng-class, 'customReports')]/a[contains(@href,'customReports')]"))) File "C:\Python38\lib\site-packages\selenium\webdriver\support\wait.py", line 80, in until raise TimeoutException(message, screen, stacktrace) selenium.common.exceptions.TimeoutException: Message:``` – R. Barrett Jan 23 '20 at 21:53
@R.Barrett Checkout the updated answer and let me know the status. – undetected Selenium Jan 23 '20 at 22:04
Looking into it. – R. Barrett Jan 23 '20 at 22:15
I had already tried it with the naming convention browser instead of the driver. Just tried it still no luck. I did record a Selenium Recording Test and saved the output here https://github.com/Richard-Barrett/ITDataServicesInfra/blob/master/Python/Selenium/TSI_CustomReport_Element_Identification – R. Barrett Jan 23 '20 at 22:18
@R.Barrett There was a change in the xpath and cssSelector as well, let me know the status. – undetected Selenium Jan 24 '20 at 05:39
I was not able to to use either command, no selection is occurring. – R. Barrett Jan 24 '20 at 14:36
I found the solution, I had to use LINK_TEXT ```element = WebDriverWait(browser, 20).until( EC.element_to_be_clickable((By.LINK_TEXT, "Reports"))) element.click(); element = WebDriverWait(browser, 20).until( EC.element_to_be_clickable((By.LINK_TEXT, "Custom Reports"))) element.click();``` – R. Barrett Jan 24 '20 at 16:12

Muzzamil · Answer 2 · 2020-01-24T14:46:01.660

0

Please try below code using xpath with webdriver explicit wait.

element = WebDriverWait(browser, 20).until(
EC.element_to_be_clickable((By.XPATH, "//a[contains(@href, '/customReports')]")))
element.click();

you need below import

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

OR

Not recommended but a working solution in most cases (try to click with Java script but without wait as it can be fail on wait )

element=browser.find_element(By.XPATH, "//a[contains(@href, '/customReports')]")
browser.execute_script("arguments[0].click();", element)

edited Jan 24 '20 at 14:46

answered Jan 23 '20 at 22:12

Muzzamil

2,823
2
11
23

I tried the following but we are using 3 positional arguments instead of 2 ```PS C:\Users\richard.barrett\Git\ITDataServicesInfra\Python\Collegeboard\TSI> python .\collegboard_tsi_export.py DevTools listening on ws://127.0.0.1:59923/devtools/browser/49c4c813-8147-4d85-adae-41b085059367 Traceback (most recent call last): File ".\collegboard_tsi_export.py", line 64, in EC.element_to_be_clickable(By.XPATH,"//a[contains(@href, '/customReports')")) TypeError: __init__() takes 2 positional arguments but 3 were given``` – R. Barrett Jan 24 '20 at 14:24
@R.Barrett Can you try last solution with Java script – Muzzamil Jan 24 '20 at 14:30
@R.Barrett, As I am typing from mobile so I missed the 1 () in that line. Now you can try it. – Muzzamil Jan 24 '20 at 14:46
I found the solution ```element = WebDriverWait(browser, 20).until( EC.element_to_be_clickable((By.LINK_TEXT, "Reports"))) element.click(); element = WebDriverWait(browser, 20).until( EC.element_to_be_clickable((By.LINK_TEXT, "Custom Reports"))) element.click();``` – R. Barrett Jan 24 '20 at 16:11
I had to use LINK_TEXT – R. Barrett Jan 24 '20 at 16:11
Well. I write xpath based on Linktext/Href. Enjoy – Muzzamil Jan 24 '20 at 16:13
`Linktext` which resembles the `innerText` and `href` both are HTML attributes but are quite different. – undetected Selenium Jan 24 '20 at 23:06
You are right. But InnerText don’t play any vital role it is just alias for any but If any don’t have href you can’t call it LinkText. HREF is only attribute who make any anchor LinkText. – Muzzamil Jan 24 '20 at 23:33
@Muzzamil Incorrect, DOM events can even do the job. – undetected Selenium Jan 25 '20 at 10:30
@DebanjanB Right, I think you are talking about onClick() event but that part is belong to JS. Don’t take it as arguments, I am seeing in healthy discussion can help to learn new thing. – Muzzamil Jan 25 '20 at 10:35

score 0 · Answer 3 · answered Jan 24 '20 at 16:10

So I ended up finding the solution. The solution is that I had to first navigate to reports and then move on the child item of custom reports. I used Link_Text instead.

element = WebDriverWait(browser, 20).until(
        EC.element_to_be_clickable((By.LINK_TEXT, "Reports")))
element.click();

element = WebDriverWait(browser, 20).until(
                EC.element_to_be_clickable((By.LINK_TEXT, "Custom Reports")))
element.click();

I appreciate all of the help from you guys @DebanjB and @Muzzamil, I doubt without their guidance I would not have been able to find out how to do this! Much appreciated you two.

Selenium Automation to Select a li Menu Item in An ul with Python and Selenium

3 Answers3