-3

What's a simple and performat way to save online published lists of IP addresses like this one in a standard python list? Example

ip_list = ['109.70.100.20','185.165.168.229','51.79.86.174']

HTML parsing library beautifulsoap seems way to sophisticated for the simple structure.

Madamadam
  • 842
  • 2
  • 12
  • 24

1 Answers1

1

Its not that beautifulsoup is too sophisticated, its that the content type is text, not html. There are several APIs for downloading content, and requests is popular. If you use its text property, it will perform any decoding and unzipping needed

import requests
resp = requests.get("https://www.dan.me.uk/torlist/")
ip_list = resp.text.split()
tdelaney
  • 73,364
  • 6
  • 83
  • 116
  • Thanks, your code is working fine. I thought it's not a file of type text because under the hood it's proper HTML … – Madamadam Apr 24 '20 at 22:52
  • Interesting. When i tested with `requests` I got a warning message about too many page accesses so just went off of my original download. I'll check it again. – tdelaney Apr 24 '20 at 23:19
  • I tried it again and it worked. `resp.headers['Content-Type']` was `'text/plain;charset=UTF-8'` and `resp.text` was just text. – tdelaney Apr 24 '20 at 23:26