Selenium/Python: Find element with no other attributes

Question

I want to recover a number that is located in the following table: the site

<table class="table table-hover table-inx">
 <tbody><tr>
  </tr>
  <tr>
  </tr>
  <tr>
  </tr>
  <tr>
  <td class=""><label for="RentNet">Miete (netto)</label></td>
  <td>478,28 €</td>
  </tr>
  <tr>
  </tr>
  <tr>
  </tr>
  <tr>
  <td class=""><label for="Rooms">Zimmer</label></td>
  <td>4</td>
  </tr>
  </tbody></table>

I suppose this strange format happens because the table entries are optional. I get to the table with driver.find_element_by_css_selector("table.table.table-hover") and I see how one could easily iterate through the <tr> tags. But how do I find the second <td> holding the data, in the <tr> with the <label for="Rooms"> ? Is there a more elegant way than "find the only td field with a one-digit number" or load the detail page?

This similar question didn't help me, because there the tag in question has an id

EDIT:

I just found out about a very helpful cheat sheet for Xpath/CSS selectors posted in an answer to a related question: it contains ways to reference child/parent, next table entry etc

Answer 1

You can select the appropriate td tag using driver.find_element_by_xpath(). The XPath expression that you should use is as follows:

`'//label[@for="Rooms"]/parent::td/following-sibling::td'`

This selects the label tag with for attribute equal to Rooms, then navigates to its parent td element, then navigates to the following td element.

So your code will be:

elem = driver.find_element_by_xpath(
     '//label[@for="Rooms"]/parent::td/following-sibling::td')

An example of the XPath expression in action is here.

Answer 2

With xpath, you can create a search for an element that contains another element, like so:

elem = driver.find_element_by_xpath('//tr[./td/label[@for="Rooms"]]/td[2]')

The elem variable will now hold the second td element within the "Rooms" label row (which is what you were looking for). You could also assign the tr element to the variable, and then work with all of the data in the row since you know the cell structure (if you would like to work with the label and data).

Answer 3

Have you tried xpath? Firebug is a great tool for copying xpaths. It will use indices to select the element you want. It's especially useful when your element has no name or ID.

Edit: not sure why I was down voted? I went on the site and found the XPath Firebug gave me:

/html/body/div[2]/div[7]/div[2]/div[3]/div/div[1]/div/div[3]/div[3]/div/table/tbody/tr[7]/td[2]

To get that 4, just:

xpath = "/html/body/div[2]/div[7]/div[2]/div[3]/div/div[1]/div/div[3]/div[3]/div/table/tbody/tr[7]/td[2]"
elem = driver.find_element_by_xpath(xpath)

print elem.text  # prints '4'

And to get all the elements for "rooms", you can simply driver.find_elements_by_xpath using partial xpath, so like this:

xpath = "/div/div[1]/div/div[3]/div[3]/div/table/tbody/tr[7]/td[2]"
elems = driver.find_elements_by_xpath(xpath)  # returns list
for elem in elems:
    print elem.text  # prints '3', '3', '4'

Finally, you might be able to get the data with page source. First, let's make a function that outputs a list of rooms when we input the page source:

def get_rooms(html):
    rooms = list()
    partials = html.split('''<label for="Rooms">''')[1:]
    for partial in partials:
        partial = partial.split("<td>")[1]
        room = partial.split("</td>")[0]
        rooms.append(room)
    return rooms

Once we have that function defined, we can retrieve the list of room numbers by:

html = driver.page_source
print get_rooms(html)

It should output:

["3", "3", "4"]