1

How can I get the following to work? The main point is that I want to run a method (and not a function) asynchronously.

from multiprocessing import Pool

class Async:
    def __init__(self, pool):
        self.pool = pool
        self.run()

    def run(self):
        p.apply_async(self.f, (10, ))

    def f(self, x):
        print x*x

if __name__ == '__main__':
    p = Pool(5)
    a = Async(p)
    p.close()
    p.join()

This prints nothing.

Mahdi
  • 1,778
  • 1
  • 21
  • 35

2 Answers2

3

The problem appears to be due to the fact that multiprocessing needs to pickle self.f while bound methods are not picklable. There is a discussion on how to solve the problem here.

The apply_async apparently creates an exception which is put inside the future returned. That's why nothing is printed. If a get is executed on the future, then the exception is raised.

Mahdi
  • 1,778
  • 1
  • 21
  • 35
0

Its definitely possible to thread class methods using a threadpool in python 2 - the following programme did what I would expect.

#!/usr/bin/env python

from multiprocessing.pool import ThreadPool

class TestAsync():
  def __init__(self):
    pool = ThreadPool(processes = 2)

    async_completions = []
    for a in range(2):
      async_completions.append(pool.apply_async(self.print_int, (  a,)))

    for completion in async_completions:
      res = completion.get()
      print("res = %d" % res)

  def print_int(self, value):
    print(value)
    return (value*10)


a = TestAsync()
frankster
  • 1,529
  • 2
  • 16
  • 20
  • Thanks @fankster. It does indeed work. The only problem is, as far as I understand, the ThreadPool will still use one "process" (in other words, will run on only one core). Your answer will be helpful if that is not a concern, but I want to run on multiple cores. – Mahdi Jun 02 '17 at 21:28