Index out of range error: using Selenium to iterate page clicks in a list of elements over multiple webpages

Question

I'm using Selenium to iterate over a list of elements, on consecutive pages of a website. These elements all have the same class name.

Here's my code:

#set up driver
driver = webdriver.Chrome(executable_path='/Applications/chromedriver')

#set variable for webpages to be iterated over
page = 'https://www.bandsintown.com/?came_from=257&page='
urlBucket = []
for i in range (0,3):
    uniqueUrl = page + str(i)
    urlBucket.append(uniqueUrl)

#  iterate over urls    
for i in urlBucket:
    drivers = driver.get(i)
    allelements = len(driver.find_elements_by_class_name('eventList-5e5f25ca'))
    for index in range((allelements)-1):
        driver.find_elements_by_class_name("eventList-5e5f25ca")[index].click()
  
    print allelements

It's giving me a IndexError: list index out of range error. I am trying to figure out why this is, as I already have the range set to be the length of the elements list minus 1.

Answer 1

When you click first time to the eventList-5e5f25ca element, new page opens where no any element with eventList-5e5f25ca class. You get error in the driver.find_elements_by_class_name("eventList-5e5f25ca")[index] line because list of elements is empty and you try to click to the element with index 1.

If you want to get all events, first collect links and then open them in separate loop:

base_url = 'https://www.bandsintown.com/?came_from=257&page='

events = []
for i in range(1, 4):
    driver.get(base_url + str(i))

    # get events links
    event_list = driver.find_elements_by_css_selector('div[class^=eventList-] a[class^=event-]')
    # collect href attribute of events in even_list
    events.extend(list(event.get_attribute("href") for event in event_list))

print(len(events))

# iterate throw all events and open them.
for event in events:
    driver.get(event)

You can also avoid using for i in range(1, 3): loop by clicking on View All button.
If your goal to scrape data and not testing, you can do it much faster using scrape libraries like requests, beautifulsoup, scrapy and ect. All data comes as json using simple request.