1

I'm trying to scrape the "10 Year Australian bond" prices table on this website: https://www2.asx.com.au/markets/trade-our-derivatives-market/derivatives-market-prices/bond-derivatives

The following code works fine:

url='https://www2.asx.com.au/markets/trade-our-derivatives-market/derivatives-market-prices/bond-derivatives'
driver.get(url)
time.sleep(2)
id='onetrust-accept-btn-handler'
driver.find_element_by_id(id).click()
time.sleep(2)
driver.find_element_by_xpath('/html/body/div[1]/div/div[3]/div/div[1]/div/ul/li[3]').click()
id='tab-panel_17'
time.sleep(2)
tbl = driver.find_element_by_id(id).get_attribute('outerHTML')
soup = BeautifulSoup(tbl, 'html.parser')
aus_10y_future = pd.read_html(str(soup))[0]

In order to click on the "10 Year Australian bond" tab, I tried to use relative xpath insted of absolute one. enter image description here

So, insted of

driver.find_element_by_xpath('/html/body/div[1]/div/div[3]/div/div[1]/div/ul/li[3]').click()

I tried:

driver.find_element_by_xpath('//*[contains(@class_name,"cmp-tabs__tabtitle")]/li[3]').click()

but I get an error. What am I doing wrong? Thanks

younggotti
  • 762
  • 2
  • 15

2 Answers2

1

class_name is not a valid element attribute. It should be class.
Also I see you are using wrong element.
This should work

driver.find_element_by_xpath("//ul[contains(@class,'cmp-tabs__tablist')]//li[3]").click()
Prophet
  • 32,350
  • 22
  • 54
  • 79
1

A css selector is far better locator than xpath, You can read about it more here

You can furthur use this css_selector :

li[data-f2-context-code='XT']

in code :

driver.find_element_by_css_selector("li[data-f2-context-code='XT']").click()

or if you can construct a way better xpath like this :

//li[@data-f2-context-code='XT']

and use it like this :

driver.find_element_by_xpath("//li[@data-f2-context-code='XT']").click()
cruisepandey
  • 28,520
  • 6
  • 20
  • 38
  • 1
    I'm a newbie so I wasn't aware of css_selector. Performance differences are not an issue for me (my web scraping activity is very light) but it's good to have another tool. Thanks – younggotti Aug 13 '21 at 11:39