1

I'm currently using python to loop through heavy processing tasks on tables in our database. In order to be friendly to other users I want this processing to only happen during nocturnal hours. When the early risers start using the database, I want to be able to program the premature termination of this script.

I'm aware of interrupting cow so I could have something like this:

from interruptingcow import timeout

from dbsettings import dbsetting

con = connect(database=dbset['database'],
              host=dbset['host'],
              user=dbset['user'],
              password=dbset['password'])
cursor = con.cursor()

YEARS = {"2014":range(1, 13),
         "2015":range(1, 13)}

def update_table(yyyymm, con, cursor)
     table = 'public.tablename'+yyyymm
     cursor.execute('Do something on {table}'.format(table))
     con.commit()

try:
    with timeout(57600, exception=RuntimeError):
        for year in YEARS:
            for month in years[year]:
                yyyymm = get_yyyymm(year, month)
                update_table(yyyymm, con, cursor)
except RuntimeError:
    print "It's morning, killing database processing"

But is this a best practice? Is there a better way?

raphael
  • 2,762
  • 5
  • 26
  • 55
  • Possibly implementing something along these lines http://stackoverflow.com/questions/323972/is-there-any-way-to-kill-a-thread-in-python – raphael Aug 10 '16 at 19:46
  • If its required, you can add some persistence and when the program starts again, it does from the last good state. [`Pickle](https://docs.python.org/3/library/pickle.html) could be an option. – Jose Raul Barreras Aug 10 '16 at 20:09
  • @JoseRaulBarreras I'll look into that, at the moment going to try something based on this http://stackoverflow.com/a/14924210/4047679 – raphael Aug 10 '16 at 20:39
  • Running into serializabity issues, pickle might not work? for example: http://stackoverflow.com/questions/12461901/debugging-pickle – raphael Aug 10 '16 at 20:47

0 Answers0