Getting data from this table html python

Question

I would like to extract data from this table which shows the currency rates.

Visit https://www.iceplc.com/travel-money/exchange-rates

I have tried this approach but its not working

      table_id = driver.find_element(By.ID, 
     'data_configuration_feeds_ct_fields_body0')
      rows = table_id.find_elements(By.TAG_NAME, "tr") # get all of the 
      rows in the table
      for row in rows:

      col = row.find_elements(By.TAG_NAME, "td")[1] #note: index start from 
      0, 1 is col 2
      print(col.text) #prints text from the element

This is the html

    </td>

                    <td valign="top" class="OuterProdCell test">

                                <table class="ProductCell">
                                    <tr>
                                    <td class="rateCountryFlag">
                                        <ul id="prodImages">
                                            <li>
                                                <a href="/travel-money/buy-chilean-peso" title="Buy Chilean Peso" class="flags chilean-peso" ></a>
                                            </li>
                                        </ul>
                                    </td>

                                    <td class="ratesName">
                                    <a href="/travel-money/buy-chilean-peso" title="Buy Chilean Peso">
                                    Chilean Peso</a>
                                    </td>

                                    <td class="ratesClass">
                                    <a  class="orderText" href="/travel-money/buy-chilean-peso" title="Buy Chilean Peso">774.8540</a>
                                    </td>
                                    <td class="orderNow">                                           
                                        <ul id="prodImages">
                                            <li>
                                                <a class="reserveNow" href="/travel-money/buy-chilean-peso" title="Buy Chilean Peso">Order<br/>now</a>
                                            </li>
                                            <li>
                                                <a href="/travel-money/buy-chilean-peso" title="Buy Chilean Peso" class="flags arrowGreen" ></a>
                                            </li>
                                        </ul>
                                    </td>
                                    </tr>
                                </table>

I have also tried the python selenium approach however I can get the currency rate of each one but not the name

             driver.get("https://www.iceplc.com/travel-money/exchange-
             rates")
             rates = driver.find_elements_by_class_name("ratesClass")

             for rate in rates:
             print(rate.text)

Its meant to output the whole table in this format in a line going down — xys234, Apr 26 '17 at 12:31
And where is `Euro 1.146 US Dollar - 1.2536 Australian Dollar - 1.6647` in your html? — Chanda Korat, Apr 26 '17 at 12:34

score 1 · Accepted Answer · edited May 23 '17 at 11:54

If you're just trying to get the exchange rates then you would be better off using api, see this question. Web scraping leaves you vulnerable to changes in the target webpage breaking your code.

If scraping is your goal though, you just need to reuse your selenium approach but search for the "ratesName" class.

For example:

driver.get("https://www.iceplc.com/travel-money/exchange-rates")
rates.append( (driver.find_elements_by_class_name("ratesName"), driver.find_elements_by_class_name("ratesClass")) )

for rate in rates:
print( "Name: %s, Rate: %s" % (rate[0], rate[1]) )

score 1 · Answer 2 · answered Apr 26 '17 at 12:46

By analyzing the structure of the page it is clear that you have to analyze row by row and you have to select the column component you are interested.

For each row extracting the two elements you are interested by using the find_element_by_tag_name and find_element_by_class_name

(documentation here http://selenium-python.readthedocs.io/locating-elements.html)

driver.get("https://www.iceplc.com/travel-money/exchange-rates")
rates=driver.find_elements_by_tag_name('tr')
for i in rates:
        print i.find_element_by_class_name('ratesName').text, i.find_element_by_class_name('ratesClass').text

Output is:

US - Dollar 1.2536
Croatia - Kuna 8.3997
Canada - Dollar 1.7006
Australia - Dollar 1.6647
Euro - 1.1469
...

Getting data from this table html python

2 Answers2