0

This is my first time developing a website and I am coming across some issues. I have some python code that scrapes some data with beautifulsoup4 and displays the number on my site with flask. However, I found that my site does not automatically update the values at all, and rather only updates when I manually make my host reload.

How can I make it so that my python script "re-scrapes" every time a visitor visits my site, or just every 5 minutes or so? Any help would be greatly appreciated! emphasized text Host- Pythonanywhere

Here is my current backend python code:

import bs4 as bs
import urllib.request
from flask import Flask, render_template

app = Flask(__name__)

link = urllib.request.urlopen('https://www.health.pa.gov/topics/disease/coronavirus/Pages/Cases.aspx')
soup = bs.BeautifulSoup(link, 'lxml')

body = soup.find('body')  # get the body so you can do soup.find_all() inside it
tables = soup.find_all('table')

for table in tables:
    table_rows = table.find_all('tr')
    for tr in table_rows:
        td = tr.find_all('td')
        row = [i.text for i in td]
        if row.count('Bucks') > 0:
            print(row[1])
            # Bucksnum shows the amount of cases in bucks county,
            bucksnum = str(row[1])
            data = bucksnum




# this is the part that connects the flask file to the html file
@app.route("/")
def home():
    return render_template("template.html", data=data)


@app.route("/")
def index():
    return bucksnum


if __name__ == '__main__':
    app.run(host='0.0.0.0')

index()
Xinyun Zhou
  • 93
  • 7

2 Answers2

1

Your application is only gathering the data once, when it starts up. If you want it to grab the data every single time someone visits the page, you could place your code in which you grab and process the table data into the relevant view function indicated by the @app.route('/route') wrapper, and the function will be run every time it's visited.

Mark
  • 41
  • 1
  • 2
0

You would need to use schedulers, check this thread that discusses a similar issue, you can use it to call a certain function that updates your data every interval of time.

Iteeeh
  • 67
  • 6