0

I have a Flask application with a dashboard running and some parts of the dashboard use a date as input form for filter data that comes from a database. But if the user does not enter the date as request arg then the route returns a template rendered with the last available date in the table gps_data.

I have the next route for the dashboard index which works as I want:

import psycopg2 as pg2

@bp.route("/index", methods=["GET", "POST"])
def index():
    date = request.args.get("date")  # date form input
    imei = request.args.get("imei")  # imei form input

    # if the user request 'index' without any date use last date from the table gps_data
    if not date:
        cur = conn.cursor() # psycopg2 cursor to postgresql database
        query = """SELECT
                       max(timestamp_utc::date)
                   FROM
                       gps_data
                   WHERE imei = imei%s
                """
        try:
            # gps_data_last_date
            cur.execute(query, imei)
            date = cur.fetchall()[0][0]
        except Exception as e:
            conn.rollback()
            date = None
            print(e)
        else:
            conn.commit()

        return redirect(url_for("dashboard.index", date=date))

    return render_template(
        "dashboard.html",
        date=date
    )

But the problem is that I do not want to get the max date while the user loads the dashboard page, because the table gps_data is very big and the performance when loading the page could be affected. So I want to use a kind of background task that that updates the max date in a global variable or something similar (once per day for example), avoiding to query the database table gps_data.

I have done research about which is the best way to accomplish this and there seems to be two options:

  1. Adding Celery + Redis, and set up a Celery periodic task in Flask, and then read the date in the view function from Redis.
  2. Add a background thread to Flask like this answer and read the date in the view function from a global variable.
  3. Keep using the SQL table gps_data but add an index on columns timestamp_utc and imei.

Do you know if there is any better or more efficient option than the these two? If there is no other, which one do you think would be the best of these two options?

EDIT: Added third option

Eduardo G
  • 370
  • 4
  • 17
  • 1
    Just because there's a lot of data does not mean you cannot put together a query which will return almost instantly. You're looking for a solution to a problem which doesn't exist. – Teejay Bruno Aug 02 '21 at 01:14
  • Remember that you can't "push" data to a web page. If you want something to update later, you need to have Javascript on the page that either fetches it via AJAX or refreshes. – Tim Roberts Aug 02 '21 at 02:09

0 Answers0