How to limit the number of Thread objects created?

Question

I want in the following code to limit the number of class_f objects running at the same time. i.e. Limit the number of threads working at the same time. How can I do that ?

#!/usr/bin/env python

import random
import time
from threading import Thread

list_num = [1,2,3,4,5,6,7,8,9,10]

class class_f(Thread):

    def  __init__(self,x):
        Thread.__init__(self)
        self.x = x

    def run(self):

        time.sleep(random.randint(0,1))
        res = self.x * self.x
        print str(res)+"\n"


def main():
    for num in list_num:
        c=class_f(num)
        c.start()

if __name__ == "__main__":
    main()

Answer 1

I would recommend using a bounded semaphore. You would implement it like this:

maxThreads = 10
semaphore = threading.BoundedSemaphore(maxThreads)

Then in your __init__ call sempahore.acquire() before the thread starts doing work. When the thread is done call sempahore.release(). Acquire decrements the number stored in the semaphore. If you try to acquire a semaphore with a value of zero it will wait until the semaphore is released again. Take a look at Thread Synchronization Mechanisms in Python by Fredrik Lundh for a more in depth explanation of using semaphores.

Answer 2

You could use the largely undocumented ThreadPool class in multiprocessing.pool because it makes what you want to do easy—as compared to you writing your own (which sounds like what you need).

Here's how to apply it to the code in your question (which I have also modified to conform more closely with the PEP 8 - Style Guide for Python Code recommendations):

from multiprocessing.pool import ThreadPool
import random
from threading import Lock, Thread
import time

MAX_THREADS = 5
threads = []
list_num = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
print_lock = Lock()  # Prevents overlapped printing from threads.

class ClassF(Thread):
    def  __init__(self, x):
        Thread.__init__(self)
        self.x = x

    def run(self):
        time.sleep(random.randint(0, 1))
        res = self.x * self.x
        with print_lock:
            print(str(res) + '\n')


def main():
    pool = ThreadPool(processes=MAX_THREADS)
    for num in list_num:
        pool.apply_async(ClassF(num).start)

    pool.close()  # No more tasks will be submitted.
    pool.join()  # Wait for all threads to exit.

if __name__ == "__main__":
    main()