0

I wrote a program that uses threads to keep a connection alive while the main program loops until it either has an exception or is manually closed. My program runs in 1 hour intervals and the timeout for the connection is 20 minutes, thus I spawn a thread for every connection element that exist inside of my architecture. Thus, if we have two servers to connect to it connects to both these serves and stays connected and loops through each server retrieving data.

the program I wrote works correctly, however I can't seem to find a way to handle when the program it's self throws an exception. This is to say I can't find an appropriate way to dispose of the threads when the main program excepts. When the program excepts it will just hang open because of the thread not excepting as well and it won't close correctly and will have to be closed manually.

Any suggestions on how to handle cleaning up threads on program exit?

This is my thread:

def keep_vc_alive(vcenter,credentials, api):
   vm_url =  str(vcenter._proxy.binding.url).split('/')[2]
   while True:
      try:
         logging.info('staying connected %s' % str(vm_url))
         vcenter.keep_session_alive()
      except:
         logging.info('unable to call current time of vcenter %s attempting to reconnect.' % str(vm_url))
         try:
            vcenter = None
            connected,api_version,uuid,vcenter = vcenter_open(60, api, * credentials)
         except:   
            logging.critical('unable to call current time of vcenter %s killing application, please have administrator restart the module.' % str(vm_url))
            break
      time.sleep(60*10)

Then my exception clean up code is as follows, obviously I know.stop() doesn't work, but I honestly have no idea how to do what it is im trying to do.

except Abort:  # Exit without clearing the semaphore
   logging.exception('ApplicationError')
   try:
      config_values_vc = metering_config('VSphere',['vcenter-ip','username','password','api-version'])     
      for k in xrange(0, len(config_values_vc['username'])):   # Loop through each vcenter server
         vc_thread[config_values_vc['vcenter-ip'][k]].stop() 
   except:
      pass
   #disconnect vcenter
   try:
      for vcenter in list_of_vc_connections:
         list_of_vc_connections[vcenter].disconnect()
   except:
      pass
   try:        # Close the db is it is open (db is defined)
      db.close()
   except:
      pass
   sys.exit(1)

except SystemExit:
   raise

except:
   logging.exception('ApplicationError')
   semaphore('ComputeLoader', False)
   logging.critical('Unexpected error: %s' % sys.exc_info()[0])
   raise
Grant Zukel
  • 1,153
  • 2
  • 24
  • 48

2 Answers2

2

Instead of sleeping, wait on a threading.Event():

def keep_vc_alive(vcenter,credentials, api, event):  # event is a threading.Event()
   vm_url =  str(vcenter._proxy.binding.url).split('/')[2]
   while not event.is_set():  # If the event got set, we exit the thread
      try:
         logging.info('staying connected %s' % str(vm_url))
         vcenter.keep_session_alive()
      except:
         logging.info('unable to call current time of vcenter %s attempting to reconnect.' % str(vm_url))
         try:
            vcenter = None
            connected,api_version,uuid,vcenter = vcenter_open(60, api, * credentials)
         except:   
            logging.critical('unable to call current time of vcenter %s killing application, please have administrator restart the module.' % str(vm_url))
            break
      event.wait(timeout=60*10)  # Wait until the timeout expires, or the event is set.

Then, in your main thread, set the event in the exception handling code:

except Abort:  # Exit without clearing the semaphore
   logging.exception('ApplicationError')
   event.set()  # keep_alive thread will wake up, see that the event is set, and exit
dano
  • 91,354
  • 19
  • 222
  • 219
1

The generally accepted way to stop threads in python is to use the threading.Event object.

The algorithm followed usually is something like the following:

import threading
...
threads = []
#in the main program
stop_event = threading.Event()
#create thread and store thread and stop_event together
thread = threading.Thread(target=keep_vc_alive, args=(stop_event))
threads.append((thread, stop_event))
#execute thread
thread.start()
...
#in thread (i.e. keep_vc_alive)
# check is_set in stop_event
while not stop_event.is_set():
    #receive data from server, etc
    ...
...
#in exception handler
except Abort:
    #set the stop_events
    for thread, stop_event in threads:
        stop_event.set()
    #wait for threads to stop
    while 1:
        #check for any alive threads
        all_finished = True
        for thread in threads:
            if thread.is_alive():
                all_finished = False
        #keep cpu down
        time.sleep(1)
tsn
  • 838
  • 9
  • 20