0

trying to figure out how to loop urls with a "for" loop (or any other loop for that matter)

I'm data scraping Howlongtobeat.com the url's are structured like this:

https://howlongtobeat.com/game.php?id=38050

where only the number at the end of "id=" changes, how do I get the end of the string to change numbers??

page_number = range (38040, 38060)

url = 'https://howlongtobeat.com/game.php?id={page_number}'

this isn't working because I'm not adding to the string

and

url = 'https://howlongtobeat.com/game.php?id=' + page_number

isn't working because I'm getting this errror

 TypeError: must be str, not range

FYI using beautifulsoup and csv writer to scrap the data and write it to a csv

I'm a beginner at this stuff so start from the top

Thanks!!!!!!!

Stephen Rauch
  • 47,830
  • 31
  • 106
  • 135
littlejiver
  • 255
  • 2
  • 13
  • 1
    Possible duplicate of [In Python, how to specify a format when converting int to string?](https://stackoverflow.com/questions/3813735/in-python-how-to-specify-a-format-when-converting-int-to-string) – Stephen Rauch Jun 13 '18 at 00:11
  • well `page_number` is a range, not an integer. You have to loop through it – Cfreak Jun 13 '18 at 00:15

1 Answers1

0
from bs4 import BeautifulSoup

url = 'https://howlongtobeat.com/game.php?id='

for page in range(38040, 38060):
    new_url = url + str(page)
    print(new_url)

Output:

C:\Users\siva\Desktop>python test.py
https://howlongtobeat.com/game.php?id=38040
https://howlongtobeat.com/game.php?id=38041
https://howlongtobeat.com/game.php?id=38042
https://howlongtobeat.com/game.php?id=38043
https://howlongtobeat.com/game.php?id=38044
https://howlongtobeat.com/game.php?id=38045
https://howlongtobeat.com/game.php?id=38046
https://howlongtobeat.com/game.php?id=38047
https://howlongtobeat.com/game.php?id=38048
https://howlongtobeat.com/game.php?id=38049
https://howlongtobeat.com/game.php?id=38050
https://howlongtobeat.com/game.php?id=38051
https://howlongtobeat.com/game.php?id=38052
https://howlongtobeat.com/game.php?id=38053
https://howlongtobeat.com/game.php?id=38054
https://howlongtobeat.com/game.php?id=38055
https://howlongtobeat.com/game.php?id=38056
https://howlongtobeat.com/game.php?id=38057
https://howlongtobeat.com/game.php?id=38058
https://howlongtobeat.com/game.php?id=38059
Siva
  • 509
  • 6
  • 22