Web Scraping with Python3

Question

This code is about web scraping. The problem is when i run this code a get some error whis is "ValueError: Length of passed values is 6, index implies 7 ". How can i fix this error. the values langth is but index length is 7 that's why error occurs.

import requests
from bs4 import BeautifulSoup
import pandas as pd

url="https://en.wikipedia.org/wiki/List_of_Game_of_Thrones_episodes"
page=requests.get(url)
soup=BeautifulSoup(page.text,'html.parser')

table=soup.find('table',{'class':'wikitable plainrowheaders wikiepisodetable'}).tbody

rows=table.find_all('tr')

columns=[v.text.replace('\u200a[20]','') for v in rows[0].find_all('th')]

df=pd.DataFrame(columns=columns)

for record in range(1,len(rows)):
    tds=rows[record].find_all('td')

    if len(tds)==4:
        values=[tds[0].text,tds[1].text,tds[2].text,tds[3].text]
    else:
        values=[td.text for td in tds]

    df=df.append(pd.Series(values,index=columns),ignore_index=True)

    print(df)

Hare is the error in line number 25.The error is given below.

  File "good.py", line 25, in <module>
    df=df.append(pd.Series(values,index=columns),ignore_index=True)
  File "C:\Users\User\AppData\Local\Programs\Python\Python38-32\lib\site-packages\pandas\core\series.py", line 300, in __init__
    raise ValueError(
ValueError: Length of passed values is 6, index implies 7```

how can i solve this problem:::::

score 1 · Answer 1 · answered Dec 19 '19 at 18:06

Did a quick google search and found this. Seems like you are trying refer to an empty dataframe.

I haven't worked with Numpy and hence, I don't know what a dataframe is and what it's supposed to do, but I went and looked at your code and seems like pd.Series(values,index=columns) expects the length of values and columns to be equal.

I checked your code and saw that the lengths aren't same. Length of values is 6, while length of columns is 7. Hence, the error.

Check out REPL HERE. I've put few print statements, enable them and check the values. Also, I added a check if length match or not...when I added this, seems like your code went through just fine.

This may not be the answer you're looking for, but until someones replies, I'd suggest that you look into your values and columns

Web Scraping with Python3

1 Answers1