Working with modals using selenium and python

Question

<div id="pageHeader" class="mediumActivityList hasBanner hasMainNav hasSubNav"><div id="rwdNavBlock"><div id="rwdNavLogo" class="golfLogo"></div><div id="rwdNav" role="navigation"><ul><li aria-haspopup="true" class="topnav_item"><a href="#" class=""><span class="topnav_item">Tee Times</span></a><ul style=""><li aria-haspopup="false" class=""><a href="Member_select" class=".ftSlotConflitInit"><span class="">Make, Change, or View Tee Times</span></a></li><li aria-haspopup="false" class=""><a href="Member_sheet?index=0" class=".ftSlotConflitInit"><span class="">Today's Tee Sheet</span></a></li><li aria-haspopup="false" class=""><a href="Member_teelist" class=".ftSlotConflitInit"><span class="">My Tee Times / Calendar</span></a></li><li aria-haspopup="false" class=""><a href="Member_teelist_list" class=".ftSlotConflitInit"><span class="">My Tee Times / List</span></a></li><li aria-haspopup="false" class=" lastItem"><a href="Member_joinMe_list" class=".ftSlotConflitInit"><span class="">Looking for Players to Join</span></a></li></ul></li>
<li aria-haspopup="false" class="topnav_item"><a href="Member_events" class=".ftSlotConflitInit"><span class="topnav_item">Events</span></a></li>
<li aria-haspopup="true" class="topnav_item"><a href="#" class=""><span class="topnav_item">Lessons</span></a><ul style=""><li aria-haspopup="false" class=""><a href="Member_lesson" class=".ftSlotConflitInit"><span class="">Individual Lessons</span></a></li><li aria-haspopup="false" class=""><a href="Member_lesson?group=yes" class=".ftSlotConflitInit"><span class="">Group Lessons</span></a></li><li aria-haspopup="false" class=""><a href="Member_lesson?all_search" class=".ftSlotConflitInit"><span class="">All Open Pro Lessons</span></a></li><li aria-haspopup="false" class=""><a href="Member_lesson?bio=yes" class=".ftSlotConflitInit"><span class="">View Pros' Bios</span></a></li><li aria-haspopup="false" class=" lastItem"><a href="Member_teelist_list?mode=lesson" class=".ftSlotConflitInit"><span class="">My Scheduled Lessons</span></a></li></ul></li>
<li aria-haspopup="true" class="topnav_item"><a href="#" class=""><span class="topnav_item">Handicaps</span></a><ul style=""><li aria-haspopup="false" class=" lastItem"><a href="Member_handicaps?todo=view" class=".ftSlotConflitInit"><span class="">View Handicaps and Scores</span></a></li></ul></li>
<li aria-haspopup="true" class="topnav_item"><a href="#" class=""><span class="topnav_item">Search</span></a><ul style=""><li aria-haspopup="false" class=""><a href="Member_searchmem" class=".ftSlotConflitInit"><span class="">Other Members' Tee Times</span></a></li><li aria-haspopup="false" class=""><a href="Member_searchpast?subtee=cal" class=".ftSlotConflitInit"><span class="">Your Past Tee Times - This Calendar Year</span></a></li><li aria-haspopup="false" class=""><a href="Member_searchpast?subtee=year" class=".ftSlotConflitInit"><span class="">Your Past Tee Times - Past 12 Months</span></a></li><li aria-haspopup="false" class=" lastItem"><a href="Member_searchpast?subtee=forever" class=".ftSlotConflitInit"><span class="">Your Past Tee Times - Since Inception</span></a></li></ul></li>
<li aria-haspopup="true" class="topnav_item"><a href="#" class=""><span class="topnav_item">Email</span></a><ul style=""><li aria-haspopup="false" class=""><a href="Member_email" class=".ftSlotConflitInit"><span class="">Send Email</span></a></li><li aria-haspopup="false" class=" lastItem"><a href="Member_email?manage_distribution_lists" class=".ftSlotConflitInit"><span class="">Manage Distribution Lists</span></a></li></ul></li>
<li aria-haspopup="false" class="topnav_item"><a href="Member_partner" class=".ftSlotConflitInit"><span class="topnav_item">Partners</span></a></li>
<li aria-haspopup="false" class="topnav_right_item profileNavItem"><a href="Member_services" class=".ftSlotConflitInit"><span class="topnav_right_item profileNavItem">Settings</span></a></li>
<li aria-haspopup="false" class="topnav_right_item helpNavItem"><a href="Member_help_rwd" class=".ftSlotConflitInit"><span class="topnav_right_item helpNavItem">Help</span></a></li>
<li aria-haspopup="false" class="topnav_right_item lastItem"><a href="Logout?u=311&amp;ftc=0&amp;ext=0&amp;cl=GJONAS74912" class=".ftSlotConflitInit"><span class="topnav_right_item">Exit</span></a></li></ul></div><div id="rwdNav2" role="navigation"><a id="rwdNavButton" href="#rwdNav2" title="Menu" class="ftB-36-White ftB-36-Menu"><span></span><span>Menu</span></a><ul class="ui-accordion ui-widget ui-helper-reset" role="tablist"><li aria-haspopup="true" class="topnav_item "><a href="#" class="ui-accordion-header ui-state-default ui-corner-all ui-accordion-icons" role="tab" id="ui-id-1" aria-controls="ui-id-2" aria-selected="false" aria-expanded="false" tabindex="0"><span class="ui-accordion-header-icon ui-icon ui-icon-triangle-1-e"></span><span class="topnav_item">Tee Times</span></a><ul class="ui-accordion-content ui-helper-reset ui-widget-content ui-corner-bottom" id="ui-id-2" aria-labelledby="ui-id-1" role="tabpanel" aria-hidden="true" style="display: none;"><li aria-haspopup="false" class=""><a href="Member_select" class=".ftSlotConflitInit"><span class="">Make, Change, or View Tee Times</span></a></li><li aria-haspopup="false" class=""><a href="Member_sheet?index=0" class=".ftSlotConflitInit"><span class="">Today's Tee Sheet</span></a></li><li aria-haspopup="false" class=""><a href="Member_teelist" class=".ftSlotConflitInit"><span class="">My Tee Times / Calendar</span></a></li><li aria-haspopup="false" class=""><a href="Member_teelist_list" class=".ftSlotConflitInit"><span class="">My Tee Times / List</span></a></li><li aria-haspopup="false" class=" lastItem"><a href="Member_joinMe_list" class=".ftSlotConflitInit"><span class="">Looking for Players to Join</span></a></li></ul></li>
<li aria-haspopup="false" class="topnav_item"><a href="Member_events" class="ui-accordion-header ui-state-default ui-corner-all ui-accordion-icons noChildren .ftSlotConflitInit" role="tab" id="ui-id-3" aria-selected="false" aria-expanded="false" tabindex="-1"><span class="ui-accordion-header-icon ui-icon ui-icon-triangle-1-e"></span><span class="topnav_item">Events</span></a></li>
<li aria-haspopup="true" class="topnav_item "><a href="#" class="ui-accordion-header ui-state-default ui-corner-all ui-accordion-icons" role="tab" id="ui-id-4" aria-controls="ui-id-5" aria-selected="false" aria-expanded="false" tabindex="-1"><span class="ui-accordion-header-icon ui-icon ui-icon-triangle-1-e"></span><span class="topnav_item">Lessons</span></a><ul class="ui-accordion-content ui-helper-reset ui-widget-content ui-corner-bottom" id="ui-id-5" aria-labelledby="ui-id-4" role="tabpanel" aria-hidden="true" style="display: none;"><li aria-haspopup="false" class=""><a href="Member_lesson" class=".ftSlotConflitInit"><span class="">Individual Lessons</span></a></li><li aria-haspopup="false" class=""><a href="Member_lesson?group=yes" class=".ftSlotConflitInit"><span class="">Group Lessons</span></a></li><li aria-haspopup="false" class=""><a href="Member_lesson?all_search" class=".ftSlotConflitInit"><span class="">All Open Pro Lessons</span></a></li><li aria-haspopup="false" class=""><a href="Member_lesson?bio=yes" class=".ftSlotConflitInit"><span class="">View Pros' Bios</span></a></li><li aria-haspopup="false" class=" lastItem"><a href="Member_teelist_list?mode=lesson" class=".ftSlotConflitInit"><span class="">My Scheduled Lessons</span></a></li></ul></li>
<li aria-haspopup="true" class="topnav_item "><a href="#" class="ui-accordion-header ui-state-default ui-corner-all ui-accordion-icons" role="tab" id="ui-id-6" aria-controls="ui-id-7" aria-selected="false" aria-expanded="false" tabindex="-1"><span class="ui-accordion-header-icon ui-icon ui-icon-triangle-1-e"></span><span class="topnav_item">Handicaps</span></a><ul class="ui-accordion-content ui-helper-reset ui-widget-content ui-corner-bottom" id="ui-id-7" aria-labelledby="ui-id-6" role="tabpanel" aria-hidden="true" style="display: none;"><li aria-haspopup="false" class=" lastItem"><a href="Member_handicaps?todo=view" class=".ftSlotConflitInit"><span class="">View Handicaps and Scores</span></a></li></ul></li>
<li aria-haspopup="true" class="topnav_item "><a href="#" class="ui-accordion-header ui-state-default ui-corner-all ui-accordion-icons" role="tab" id="ui-id-8" aria-controls="ui-id-9" aria-selected="false" aria-expanded="false" tabindex="-1"><span class="ui-accordion-header-icon ui-icon ui-icon-triangle-1-e"></span><span class="topnav_item">Search</span></a><ul class="ui-accordion-content ui-helper-reset ui-widget-content ui-corner-bottom" id="ui-id-9" aria-labelledby="ui-id-8" role="tabpanel" aria-hidden="true" style="display: none;"><li aria-haspopup="false" class=""><a 
span></a></li>
<li aria-haspopup="false" class="topnav_right_item helpNavItem"><a href="Member_help_rwd" class="ui-accordion-header ui-state-default ui-corner-all ui-accordion-icons noChildren .ftSlotConflitInit" role="tab" id="ui-id-14" aria-selected="false" aria-expanded="false" tabindex="-1"><span class="ui-accordion-header-icon ui-icon ui-icon-triangle-1-e"></span><span class="topnav_right_item helpNavItem">Help</span></a></li>
<li aria-haspopup="false" class="topnav_right_item lastItem"><a href="Logout?u=311&amp;ftc=0&amp;ext=0&amp;cl=GJONAS74912" class="ui-accordion-header ui-state-default ui-corner-all ui-accordion-icons noChildren .ftSlotConflitInit" role="tab" id="ui-id-15" aria-selected="false" aria-expanded="false" tabindex="-1"><span class="ui-accordion-header-icon ui-icon ui-icon-triangle-1-e"></span><span class="topnav_right_item">Exit</span></a></li></ul><a id="rwdHelpButton" href="Member_help_rwd" title="Menu" class="ftB-36-White ftB-36-Help ftB-36-Right helpTopic ft-helpLink .ftSlotConflitInit"><span></span><span>Help</span></a></div></div><div class="banner_container"><div class="rwd_banner"><div class="banner_title"><a id="title" href="Member_announce" class=".ftSlotConflitInit">The Club at Mediterra</a></div><div class="welcome_member"><span>Welcome,</span> <span>Stephen Smith</span></div><div id="weather"><a href="https://wwwa.accuweather.com/forecast.asp?partner=&amp;zipcode=34110" 
<div class="slot_container" data-
</div>
<div id="footer">
<div class="modal_list pleaseWait_container"></div>
<div class="ui-dialog ui-widget ui-widget-content ui-corner-all ui-front newpropushbelowmenu ui-dialog-buttons" tabindex="-1" role="dialog" aria-describedby="ui-id-16" aria-labelledby="ui-id-17" style="height: auto; width: 650px; top: 391px; left: 418px; display: block; z-index: 101;">
<div class="ui-dialog-titlebar ui-widget-header ui-corner-all ui-helper-clearfix">
<span id="ui-id-17" class="ui-dialog-title">Notice</span>
<button type="button" class="ui-button ui-widget ui-state-default ui-corner-all ui-button-icon-only ui-dialog-titlebar-close" role="button" title="Close" style="display: none;">
<span class="ui-button-icon-primary ui-icon ui-icon-closethick">
</span>
<span class="ui-button-text">Close</span>
</button>
</div>
<div class="modal_list slotPageLoadNotification_container ui-dialog-content ui-widget-content" id="ui-id-16" style="width: auto; min-height: 97px; max-height: 924px; height: auto;">
<div class="sub_instructions">
Sorry, the time you requested is not available. Would you like an alternate time?<br>
<br>
The next available tee time is: <b>3:23 PM <br><br><br>Select "Continue" to use this alternate time.</b>
</div>
</div>
<div class="pleaseWaitActivityIndicator" style="position: absolute; inset: 0px; display: none;">
</div><div class="ui-dialog-buttonpane ui-widget-content ui-helper-clearfix">
<div class="ui-dialog-buttonset">
<button type="button" class="ui-button ui-widget ui-state-default ui-corner-all ui-button-text-only" role="button">
<span class="ui-button-text">Go Back</span>
</button>
<button type="button" class="ui-button ui-widget ui-state-default ui-corner-all ui-button-text-only" role="button">
<span class="ui-button-text">Continue</span>
</button>
</div>
</div>
</div>
<div class="ui-widget-overlay ui-front" style="z-index: 100;"></div></body></html>

I am trying to access a modal that is displayed under certain circumstances. I have read everything I can find, watched youtube videos on the subject, and experimented heavily with no success. It seems to me that the elements inside of the modal are not accessible - I have tried waiting, sleeping, etc. I have also tried switching to logic. I have attached a copy of the relevant HTML and the XPATH address for both elements. Please note that the elements are next to each other - the second one immediately follows the first. I am able to access the first one (/html/body/div[4]) using find element by xpath(I get an message typed out), but i am unable to access the second ([5]) **

        br.find_element(By.XPATH, "/html/body/div[4]")
        print(thread, "found modal list please wait container")
        sleep(5)
        br.find_element(By.XPATH, "/html/body/div[5]")
        print(thread, "line after modal list please wait container")

**using virtually the same location (slight change in xpath address). If someone can steer me in the right direction I'd appreciate it.

Thanks for reading.

Answer 1

Absolute xpath like:

/html/body/div[4]

and

/html/body/div[5]

which has the complete path beginning from the root to the element which we want to identify, would always be brittle as if there is any change in attributes beginning from the root to the element, our absolute xpath will become invalid.

---

Solution

Instead of absolute xpath, you can use a logical xpath as follows:

br.find_element(By.XPATH, "//div[@id='footer']/div[@class='modal_list pleaseWait_container']")
print(thread, "found modal list please wait container")
sleep(5)
br.find_element(By.XPATH, "//div[@id='footer']/div[@class='ui-dialog ui-widget ui-widget-content ui-corner-all ui-front newpropushbelowmenu ui-dialog-buttons']")
print(thread, "line after modal list please wait container")

---

Additionally, as the elements are within a Modal ideally you need to induce WebDriverWait for the presence_of_element_located() and you can use the following locator strategies:

WebDriverWait(br, 20).until(EC.presence_of_element_located((By.XPATH, "//div[@id='footer']/div[@class='modal_list pleaseWait_container']")))
print(thread, "found modal list please wait container")
WebDriverWait(br, 20).until(EC.presence_of_element_located((By.XPATH, "//div[@id='footer']/div[@class='ui-dialog ui-widget ui-widget-content ui-corner-all ui-front newpropushbelowmenu ui-dialog-buttons']")))
print(thread, "line after modal list please wait container")

• Note : You have to add the following imports :

  from selenium.webdriver.support.ui import WebDriverWait
  from selenium.webdriver.common.by import By
  from selenium.webdriver.support import expected_conditions as EC