why does my web scraping code returns []?

Question

I am scraping some data from this web site, https://eleccionesnacionales.corteelectoral.gub.uy/ResumenResultados.htm, I inspected the page and found this

photo

knowing that, i coded this, but its just returns "[]" i ve been with this for like 2 hours but I cant find a solution

import requests
from bs4 import BeautifulSoup

url = requests.get("https://eleccionesnacionales.corteelectoral.gub.uy/ResumenResultados.htm")
soup = BeautifulSoup(url.content, "lxml")

votos = soup.find_all("span", class_="subtotal sin-margin sin-padding ")
print(votos)

Did you inspect the DOM of the page or the actual source code? — Klaus D., Jun 09 '20 at 03:09

score 0 · Answer 1 · answered Jun 09 '20 at 03:47

Use this JSON API. I don't know why this number "1591673487333" but most properly it's his date.

https://eleccionesnacionales.corteelectoral.gub.uy/JSON/ResumenGeneral_D.json?1591673487333

this output of this JSON file https://pastebin.com/yXvkArBv

Use this code to generate number. like this "1591673487333"

import time
millis = int(round(time.time() * 1000))
print(millis)

Example:

import time
import urllib.request, json 

millis = int(round(time.time() * 1000))

with urllib.request.urlopen("https://eleccionesnacionales.corteelectoral.gub.uy/JSON/ResumenGeneral_D.json?" + str(millis)) as url:
    data = json.loads(url.read().decode())
    print(data)
    print(data[0]['EleccionNacional'][0]['LemaNombre']) # Partido Frente Amplio
    print(data[0]['EleccionNacional'][0]['Total']) # 949376

score 0 · Answer 2 · edited Mar 15 '21 at 15:37

This code will give you an output

import requests
from bs4 import BeautifulSoup

url = requests.get("https://eleccionesnacionales.corteelectoral.gub.uy/ResumenResultados.htm")
soup = BeautifulSoup(url.content, "lxml")
# print(soup)
votos = soup.find_all("span", {"class":"subtotal sin-margin sin-padding"})
for span in votos:
    print(span.text)
print(votos)

This is a general procedure for any websites. But the website that you have given is using some template engine and the data is presented as some variable, here as

{{Total | format . 0 }}

The output of the program will be

{{Total | format . 0 }}
{{Total | format . 0 }}
[<span class="subtotal sin-margin sin-padding">{{Total | format . 0 }}</span>, <span class="subtotal sin-margin sin-padding">{{Total | format . 0 }}</span>]

However if you would like to get it, refer this page: How to retrieve the values of dynamic html content using Python

It will use selenium and chrome drivers to get the actual rendered site and use it to scrape data.

why does my web scraping code returns []?

2 Answers2