Python scraping of JSON webpage

Question

I'm trying to scrape a page with this JSON data:

{"supply": 33391639424594933, "circulation": 34239675266895397, "delegations": 1190828, "stake": 24567963450666814, "d": 0, "k": 500, "ADABTC": 2.669e-05, "ADAUSD": 1.17, "ADAEUR": 1.066, "ADAJPY": 143.18, "ADAGBP": 0.88944, "ADACAD": 1.47, "ADAAUD": 1.56, "ADABRL": 5.65, "tokens": 4012794, "policies": 49730, "load_24h": 0.7224179343311306, "load_1h": 0.8698008795204403, "load_5m": 0.923056640625}

All I want is to get the "tokens" value.

Here is my current script:

from bs4 import BeautifulSoup
import requests
import json

url = "https://pool.pm/total.json"
print(url)
page =requests.get(url)
soup = BeautifulSoup(page.content, 'html.parser')
script = soup.find('tokens')

print(script)

But nothing happens when I run this code.

Please format your code properly. If the web page is JSON, read is as JSON, don't try to parse it yourself. See https://docs.python-requests.org/en/latest/user/quickstart/#json-response-content — Thierry Lathuille, Mar 24 '22 at 19:26

score 2 · Accepted Answer · answered Mar 24 '22 at 19:26

2

bs4 can parse HTML pages, not JSON. So you don't need bs4 here but requests:

import requests

r = requests.get('https://pool.pm/total.json')

print(r.json()['tokens'])

answered Mar 24 '22 at 19:26

Freek

146
7

score 0 · Answer 2 · answered Mar 24 '22 at 19:28

0

This will work:

from bs4 import BeautifulSoup
import requests
import json

url = "https://pool.pm/total.json"
print(url)
page =requests.get(url)

script = page.json()['tokens']

print(script)

You don't need bs4 for json.

answered Mar 24 '22 at 19:28

Kawish Qayyum

171
7

Python scraping of JSON webpage

2 Answers2