executing a method at the same time in python

Question

In the code below, I want the saveData function to be executed 48 times at the same time. I'm using thread to achieve this, but instead of saving the files, the program prints the elapsed time and exits right after the execution. Why the saveData function is not being executed? How can I accomplish this?

#!/usr/bin/env python
import sys

import numpy as np
import h5py
import scipy
from PIL import Image
import timeit
import thread

import matplotlib.pyplot as plt

def saveImage(array, filename):
  fig=plt.figure(figsize=(4,3))
  ax=fig.add_subplot(1,1,1)
  plt.axis('off')
  p = plt.imshow(array)
  p.set_cmap('gray')
  extent = ax.get_window_extent().transformed(fig.dpi_scale_trans.inverted())
  plt.savefig(filename, bbox_inches=extent) 

def saveData(value1, value2, value3, dset):
  filename = "tomo1_" + str(value1) + ".png" 
  data = dset[value1,:,:]
  saveImage(data, filename)
  filename = "tomo2_" + str(value2) + ".png" 
  data = dset[:,value2,:]
  saveImage(data, filename)
  filename = "tomo3_" + str(value3) + ".png" 
  data = dset[:,:,value3]
  saveImage(data, filename)

def run():

  # Reopen the file and dataset using default properties.
  f = h5py.File(sys.argv[1])
  dset = f[sys.argv[2]]

  dim1 = len(dset)
  dim2 = len(dset[0])
  dim3 = len(dset[0][0])

  slice1 = 0
  slice2 = 0
  slice3 = 0
  factor1 = dim1/48
  factor2 = dim2/48
  factor3 = dim3/48
  tic=timeit.default_timer()
  for i in range(0,48):
    thread.start_new_thread(saveData,(slice1, slice2, slice3, dset))
    slice1 = slice1 + factor1
    slice2 = slice2 + factor2
    slice3 = slice3 + factor3

  toc=timeit.default_timer()
  print "elapsed time: " + str(toc - tic)

if __name__ == "__main__":
    run()

score 2 · Answer 1 · edited May 23 '17 at 10:32

First, it is recommended that you use the friendlier module "threading" rather than the low level module "thread".

Second, you need to wait for the threads to finish their work. If you use the threading.Thread object, it has a "join" method that you can use to make sure that your threads have finished before your code progresses.

Look at this answer for an example:

https://stackoverflow.com/a/11968818/1055722

score 1 · Accepted Answer · edited May 23 '17 at 11:57

The issue at hand, is that your parent thread finishes, but doesn't check if there are child threads left which are still running, which are silently killed in this manner! I would recommend the following approach:

Instead of import thread use import threading

change your thread code around:

thread.start_new_thread(saveData,(slice1, slice2, slice3, dset))

to

threads_running = []          # keep track of your threads
# thread starting loop
for i in xrange(48):     # in this case xrange is what you really want!
    ...                  # do your data preparations here (slice, etc.)
    thread = threading.Thread(target=saveDate,
                              args=(slice1, slice2, slice3, dset))
    thread.start()
    threads_running.append(thread)   # "register" the running thread

# thread waiting to finish loop
while threads_running:            
    thread = thread_lst[i]
    thread.join(0.1)         # wait 0.1 second for thread to finish 
    if thread.is_alive():    # else check next thread
        continue
    else:
        print "Thread %s finished" % threads_running.pop(i)

executing a method at the same time in python

2 Answers2