4

I'm trying to make a progress bar reflecting pandas dataframe build progress from sql. Currently I have a table with 9 columns containing 1000 records.

import pandas as pd
import psycopg2 as ps
import pandas.io.sql as psql

conn = ps.connect(user="user", password="password", database="database")
sql = "select * from table"

a = datetime.datetime.now()
df = psql.read_frame(sql, con=conn)
---and blablabla some little functions
b = datetime.datetime.now()
print b-a

instead of getting delta start & end time of the function, I would prefer and it would be nice to show progress bar to end user (just in case the data is getting bigger), so they have idea how long it would take. Is that possible? how to do it?

wrez
  • 159
  • 5
  • 11
  • What progress do you wan't to monitor? Usually, all the job is being done at the DB side, and it is a very tricky task, to monitor an sql execution/result fetching. – alko Jan 12 '14 at 18:38
  • @alko loading data from sql (or files) to pandas dataframe & also putting object to GUI can take a fairly 'long' time, what i'm thinking is: i could make a progress bar to show them how long they gonna wait. – wrez Jan 13 '14 at 02:35
  • 1
    Related http://stackoverflow.com/questions/18603270/progress-indicator-during-pandas-operations-python/18611535#18611535 I guess the difference is how to separate out into multiple sql jobs and concat e.g. WHERE id is within a certain range (but I suspect this will rarely be worth the effort in itself)... – Andy Hayden Jan 13 '14 at 08:19

0 Answers0