6

I am trying to use concurrent.futures.ThreadPoolExecutor module to run a class method in parallel, the simplified version of my code is pretty much the following:

class TestClass:

    def __init__(self, secondsToSleepFor):

        self.secondsToSleepFor = secondsToSleepFor


    def testMethodToExecInParallel(self):

        print("ThreadName: " + threading.currentThread().getName())
    

        print(threading.currentThread().getName() + " is sleeping for " + str(self.secondsToSleepFor) + " seconds")
    

        time.sleep(self.secondsToSleepFor)


        print(threading.currentThread().getName() + " has finished!!")


with concurrent.futures.ThreadPoolExecutor(max_workers = 2) as executor:

    futuresList = []

    print("before try")

    try:

         testClass = TestClass(3)        

         future = executor.submit(testClass.testMethodToExecInParallel)

         futuresList.append(future)

    except Exception as exc:

         print('Exception generated: %s' % exc)

If I execute this code it seems to behave like it is intended to. But if I make a mistake like specifying a wrong number of parameters in "testMethodToExecInParallel" like:

 def testMethodToExecInParallel(self, secondsToSleepFor):

and then still submitting the function as:

future = executor.submit(testClass.testMethodToExecInParallel)

or trying to concatenate a string object with an integer object (without using str(.) ) inside a print statement in "testMethodToExecInParallel" method:

def testMethodToExecInParallel(self):

        print("ThreadName: " + threading.currentThread().getName())
        print("self.secondsToSleepFor: " + self.secondsToSleepFor) <-- Should report an Error here

the program doesn't return any error; just prints "before try" and ends execution...

Is trivial to understand that this makes the program nearly undebuggable... Could someone explain me why such behaviour happens?

(for the first case of mistake) concurrent.futures.ThreadPoolExecutor doesn't check for a function with the specified signature to submit and, eventually, throw some sort of "noSuchFunction" exception?

Maybe there is some sort of problem in submitting to ThreadPoolExecutor class methods instead of simple standalone functions and, so, such behaviour could be expected?

Or maybe the error is thrown inside the thread and for some reason I can't read it?

-- EDIT --

Akshay.N suggestion of inserting future.result() after submitting functions to ThreadPoolExecutor makes the program behave as expected: goes nice if the code is correct, prints the error if something in the code is wrong.

I think users must be warned about this very strange behaviour of ThreadPoolExecutor: if you only submit functions to ThreadPoolExecutor WITHOUT THEN CALLING future.result():

  • if the code is correct, the program goes on and behaves as expected
  • if something in the code is wrong seems the program doesn't call the submitted function, whatever it does: it doesn't report the errors in the code
ela
  • 325
  • 2
  • 10

1 Answers1

5

As far as my knowledge goes which is "not so far", you have to call "e.results()" after "executor.submit(testClass.testMethodToExecInParallel)" in order to execute the threadpool . I have tried what you said and it is giving me error, below is the code

>>> import concurrent.futures as cf
>>> executor = cf.ThreadPoolExecutor(1)
>>> def a(x,y):
...     print(x+y)
...
>>> future = executor.submit(a, 2, 35, 45)
>>> future.result()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\username 
\AppData\Local\Programs\Python\Python37\lib\concurrent\futures\_base.py", line 
425, in result
    return self.__get_result()
  File "C:\Users\username
\AppData\Local\Programs\Python\Python37\lib\concurrent\futures\_base.py", line 
384, in __get_result
    raise self._exception
  File "C:\Users\username
\AppData\Local\Programs\Python\Python37\lib\concurrent\futures\thread.py", line 
57, in run
    result = self.fn(*self.args, **self.kwargs)
TypeError: a() takes 2 positional arguments but 3 were given

Let me know if it still doesn't work

SomeGuyOnAComputer
  • 5,414
  • 6
  • 40
  • 72
Akshay Nailwal
  • 196
  • 1
  • 9
  • In that case, I can't understand why first snippet of my code works fine... If calling e.results() is needed to execute the threads, I would expect the program to only print "before try" and exit even in that case... – ela Feb 20 '19 at 07:55
  • 1
    Thank you VERY much! Adding future.result() it seems to work just like expected: goes ok if the code is correct, prints the error if there's something wrong... Anyway I think it would be important to report this strange behaviour to users which are/will be in the same situation as me – ela Feb 20 '19 at 10:15
  • future = executor.submit(f, vars) followed by print(f'{future.result()}') worked for me as well. My thanks to both of you for this solution! – C. Cooney Nov 07 '20 at 16:08