Selenium about how to get the new_driver after click the more_button?

Question

Q: I'm using Selenium to get a page with contents, and after I click the more button,the page outputs more content,and how I get the new page through webdriver?

some codes like this:

def parase_questions(self):
    driver = self.login()
    driver.implicitly_wait(2)
    more_btn = driver.find_element_by_css_selector(".zg-btn-white.zg-r3px.zu-button-more")
    more_btn.click()
    # should I do something to get the new driver ?
    print driver.page_source
    question_links = driver.find_elements_by_css_selector('.question_link')
    print len(question_links)

Answer 1

If I understand you correctly, after you click the More button, there are more elements with question_link class loaded. You would need a way to wait for the question links to be loaded.

Here is one idea - a custom Expected Condition that would help you to wait until there are more than N number of elements:

from selenium.common.exceptions import StaleElementReferenceException
from selenium.webdriver.support import expected_conditions as EC

class wait_for_more_than_n_elements(object):
    def __init__(self, locator, count):
        self.locator = locator
        self.count = count

    def __call__(self, driver):
        try:
            count = len(EC._find_elements(driver, self.locator))
            return count > self.count
        except StaleElementReferenceException:
            return False

Usage:

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait

driver = self.login()
driver.implicitly_wait(2)

question_links = driver.find_elements_by_css_selector('.question_link')

more_btn = driver.find_element_by_css_selector(".zg-btn-white.zg-r3px.zu-button-more")
more_btn.click()

# wait
wait = WebDriverWait(driver, 10)
wait.until(wait_for_more_than_n_elements((By.CSS_SELECTOR, ".question_link"), len(question_links))

# now more question links were loaded, get the page source
print(driver.page_source)