Multi-threaded tools

Question

I am still pretty new to python scripting and am trying to speed up and smooth out some of my tools by adding multi-threading support. I feel like I may be miss-understanding some of the workflow theory here so feel free to redirect my mindset if I should be doing this another way. Basically what I am trying to do is the following:

==================================================================
###Pseudo-code -- I apologize if this does not follow existing conventions
main_function()
    if do_stuff_function is not running:
        Thread1 = new thread running do_stuff_function

do_stuff_function(args)
    do some stuff
    wait for 60 seconds (possibly using time.sleep())
    end thread (tell main_function that the thread is done)
==================================================================
###Abridged code
def main(self):
      check = True
      index = 0
      While index < 5:
          if check == True:
              check = False
              thread1 = threading.Thread(target=self.doStuff, args=(self, index))



def do_stuff(self, index):
     ###Stuff happens here####

     ###iterate index and somehow return it (possibly a global variable)
     time.sleep(60)
     ###somehow end the thread (still not entirely sure how to do that)

===================================================================

Notes:

-- This tool has a gui and it all tends to lock up and freeze if i run the time.sleep() in the main loop which is why I figured that multithreading would be a good solution (feel free to correct me if that is wrong). Or possibly a different way to wait that doesn't freeze the entire thread while waiting.

-- The program freezes in the while loop, is there a way to do this check without having to loop, possibly something like a callback (like a simulated button press when the do_stuff() function ends)?

-- I am also looking to try and add some error checking to this, so based on the results of do_stuff() return different error codes (just something to keep in mind).

I apologize if that is not enough info; feel free to ask for more specific info if you need it. Really guys, I appreciate all the help I can get, I am just trying to get a solid understanding of this stuff! Thanks!

Threads do not accelerate anything in your code, on the contrary, they slow it down. — michaelmeyer, Jun 25 '13 at 10:18
To build on @doukremt's comment look up python's Global Interpreter Lock (GIL), here is an [article](http://eli.thegreenplace.net/2012/01/16/python-parallelizing-cpu-bound-tasks-with-multiprocessing/) that helped me. — John, Jun 25 '13 at 12:03
can you elaborate on the "Idle event"? whenever I look it up I get the IDLE module. — TheBeardedBerry, Jun 26 '13 at 03:05

score 2 · Accepted Answer · edited May 23 '17 at 12:21

2

threads have some overhead and they all share the same CPU core, threads are good for waiting for user input or downloading files and other things where you are i/o limited, if you need more computing power I would suggest you use the multiprocessing module (see Multiprocessing vs Threading Python)

another problem with threads is that they 'lock up' because of the Global Interpreter Lock, this is because only 1 thread is allowed to write to memory at any one time, the absurd result is that to many threads brings down your program because they are all waiting to access memory

edited May 23 '17 at 12:21

Community

1
1

answered Jun 25 '13 at 15:10

jcr

1,015
6
18

Thank you for your help! I feel like the multiprocessing module is more what I wanted. Is the number of processes dependent on the number of CPU cores? if it is then how do you determine the number of cores the computer has (to allow the tool to adapt to multiple computer configurations)? – TheBeardedBerry Jun 26 '13 at 03:04
@TheBeardedBerry multiprocessing.cpu_count() will do the trick see: [link](http://stackoverflow.com/questions/1006289/how-to-find-out-the-number-of-cpus-in-python) – jcr Jun 26 '13 at 11:32

Multi-threaded tools

1 Answers1