beautiful soup and requests not getting full page

Question

My code looks like this.

from bs4 import BeautifulSoup
import requests

r  = requests.get("http://www.data.com.sg/iCurrentLaunch.jsp")
data = r.text
soup = BeautifulSoup(data)
n = soup.findAll('table')[7].findAll('table')
for tab in n:
    print tab.findAll('td')[1].text

what I am getting is the property name till IDYLLIC SUITES,after that I get error "list index out of range".What is the problem?

Do you have lxml installed? I cannot reproduce your problem with any of the 3 supported parsers, but some installations of lxml and dependencies have problems. — Martijn Pieters, Sep 20 '14 at 14:19
Does `BeautifulSoup(data, 'html.parser')` work for you then? — Martijn Pieters, Sep 20 '14 at 14:44
Also see [Beautiful Soup 4 find\_all don't find links that Beautiful Soup 3 finds](http://stackoverflow.com/q/17698836) — Martijn Pieters, Sep 20 '14 at 14:45

score 2 · Accepted Answer · answered Sep 20 '14 at 14:36

2

I am not sure what is exactly bothering you. Because when I tried your code (as it is) it worked for me.

Still, try changing the parser, may be to html5lib

So do,

pip install html5lib

And then change your code to,

from bs4 import BeautifulSoup
import requests

r  = requests.get("http://www.data.com.sg/iCurrentLaunch.jsp")
data = r.text
soup = BeautifulSoup(data,'html5lib') # Change of Parser
n = soup.findAll('table')[7].findAll('table')
for tab in n:
    print tab.findAll('td')[1].text

Let me know if it helps

answered Sep 20 '14 at 14:36

Md. Mohsin

1,822
3
19
34

The output is the same for all three parsers. – Martijn Pieters Sep 20 '14 at 14:42
Changing the parser worked for me.Thanks – sameedhayat Sep 20 '14 at 14:44
I don't know whats the problem I tried lxml parser It gave the error of "list out of index" – sameedhayat Sep 20 '14 at 14:45

beautiful soup and requests not getting full page

1 Answers1