Python subclassing multiprocessing.Lock

Question

I'm trying to understand why python can not compile the following class.

class SharedResource(multiprocessing.Lock):
    def __init__(self, blocking=True, timeout=-1):
        # super().__init__(blocking=True, timeout=-1)
        self.blocking = blocking
        self.timeout = timeout
        self.data = {}

TypeError: method expected 2 arguments, got 3

The reason why I'm subclassing Lock my objective is to create a shared list of resource that should be usable only by on process at a time.

this concept will be eventually in a Flash application where the request should not be able to use the resource concurrently

RuntimeError: Lock objects should only be shared between processes through inheritance

class SharedResource():
    def __init__(self, id, model):
        '''
        id: mode id
        model: Keras Model only one worker at a time can call predict
        '''  
        self.mutex = Lock()
        self.id = id
        self.model = model

manager = Manager()
shared_list = manager.list() # a List of models
shared_list.append(SharedResource())

def worker1(l):
    ...read some data
    while True:
        resource = l[0]
        with m:
            resource['model'].predict(...some data)
        time.sleep(60)  


if __name__ == "__main__":
   processes = [ Process(target=worker1, args=[shared_list])]
   for p in processes:
       p.start()
   for p in processes:
       p.join()

is it giving the error for the class definition or when you try to create an object? can you show the call to create an instance of this class? — rite2hhh, Sep 17 '20 at 20:27
`__init__` doesn't have any documented public parameters (there's a `ctx`) -- you're thinking of the `acquire` method -- though you're probably better to have a `Lock` member instead of subclassing — anthony sottile, Sep 17 '20 at 20:27
@AnthonySottile that was my first attempt I've added more details to show why in my case this do not work — Gavello, Sep 17 '20 at 20:55
That inheritance does not refer to subclassing -- you don't want to subclass Lock. It means that subprocesses can inherit the lock from the parent process. Is discussed in https://stackoverflow.com/questions/25557686/python-sharing-a-lock-between-processes — antont, Sep 17 '20 at 21:01

score 0 · Answer 1 · answered Jan 03 '22 at 23:31

The reason you are getting this error is because multiprocessing.Lock is actually a function.

In .../multiprocessing/context.py there are these lines:

def Lock(self):
    '''Returns a non-recursive lock object'''
    from .synchronize import Lock
    return Lock(ctx=self.get_context())

This may change in the future so you can verify this on your version of python by doing:

import multiprocessing
print(type(multiprocessing.Lock))

To actually subclass Lock you will need to do something like this:

from multiprocessing import synchronize
from multiprocessing.synchronize import Lock

# Since Lock is now a class, this should work:
class SharedResource(Lock):
    pass

I'm not endorsing this approach as a "good" solution, but it should solve your problem if you really need to subclass Lock. Subclassing things that try to avoid being subclassed is usually not a great idea, but sometimes it can be necessary. If you can solve the problem in a different way you may want to consider that.

Python subclassing multiprocessing.Lock

1 Answers1

Linked