0

I'm using the first answer to this question

Overcoming Python's limitations regarding instance methods

to be able to use multiprocess module on methods of one of my own classes.

As an example let's say that I have the following:

from multiprocessing import Pool

def myParallelFunc(my_list, a, b, inst):
    # do something
    return True

def myFunc:
    # instantiate custom class
    my_instance = MyObject()

    pool = Pool()
    pool.map(functools.partial(myParallelFunc, a=5, b=7, inst=my_instance), my_list)

    # SOLUTION!!!
    pool.close()

Now I have another program that calls myFunc let's say 100 times. Every time I call myFunc some memory is occupied and never freed. Is there a way to explicitly free it?

Community
  • 1
  • 1
andrea.ge
  • 1,937
  • 1
  • 18
  • 27

1 Answers1

1

You are creating a new Pool at every call to myFunc. It is not deleted automatically when myFunc exits, because the child processes and related threads remain.

Create a Pool, keep it around for those 100 calls, then .close it

Janne Karila
  • 24,266
  • 6
  • 53
  • 94