0

For a hobby project I am trying to scrape interest % of a bank website. I used the following code to get to the html tags that contain the % I need.

from bs4 import BeautifulSoup
import requests 

response = requests.get('https://www.rabobank.nl/particulieren/hypotheek/hypotheekrente/rente-annuiteitenhypotheek-en-lineaire-hypotheek/')

soup = BeautifulSoup(response.text, 'html.parser')

soup.find_all(id = "idm1872399280")

Now I want to extract the % per category as stated on the website and write them to a csv file. Would appreciate your help!

Luukv93
  • 339
  • 1
  • 6
  • 19

1 Answers1

2

This seems like a duplicate question. The solution from here is modified to suit your specific question.

Potential solution: 

from bs4 import BeautifulSoup
import requests 

response = requests.get('https://www.rabobank.nl/particulieren/hypotheek/hypotheekrente/rente-annuiteitenhypotheek-en-lineaire-hypotheek/')

soup = BeautifulSoup(response.text, 'html.parser')
table = soup.find("div", attrs={"class": "tpa-table__scrollable-area"})
rows = table.findAll('tr')
data = [[td.findChildren(text=True) for td in tr.findAll("td")] for tr in rows]
data = [[u"".join(d).strip() for d in l] for l in data]

The output of data looks like this

[[u'',
  u'Met NHG of tot en met 67,5% van de marktwaarde*',
  u'Meer dan 67,5% tot en met 90% van de marktwaarde*',
  u'Meer dan 90% van de marktwaarde*'],
 [u'1 jaar', u'1,20%', u'1,40%', u'1,60%'],
 [u'2 jaar', u'1,25%', u'1,45%', u'1,65%'],
 [u'3 jaar', u'1,35%', u'1,55%', u'1,75%'],
 [u'4 jaar', u'1,35%', u'1,55%', u'1,75%'],
 [u'5 jaar', u'1,40%', u'1,60%', u'1,80%'],
 [u'6 jaar', u'1,40%', u'1,60%', u'1,80%'],
 [u'7 jaar', u'1,40%', u'1,60%', u'1,80%'],
 [u'8 jaar', u'1,40%', u'1,60%', u'1,80%'],
 [u'9 jaar', u'1,40%', u'1,60%', u'1,80%'],
 [u'10 jaar', u'1,40%', u'1,60%', u'1,80%'],
 [u'11 jaar', u'1,55%', u'1,75%', u'1,95%'],
 [u'12 jaar', u'1,65%', u'1,85%', u'2,05%'],
 [u'13 jaar', u'1,75%', u'1,95%', u'2,15%'],
 [u'14 jaar', u'1,75%', u'1,95%', u'2,15%'],
 [u'15 jaar', u'1,75%', u'1,95%', u'2,15%'],
 [u'20 jaar', u'1,80%', u'2,00%', u'2,20%'],
 [u'25 jaar', u'2,05%', u'2,25%', u'2,45%'],
 [u'30 jaar', u'2,10%', u'2,30%', u'2,50%']]

One could iterate over data to find the percentages. An example list element can be accessed as follows

In [4]:data[1][3]
Out[5]: u'1,60%'
SidharthMacherla
  • 390
  • 1
  • 8