0

I'm struggling to use Python Multiprocessing Manager objects... I have this kind of structure going on:

from multiprocessing.managers import BaseManager, NamespaceProxy
from multiprocessing import RLock

class Foo(object):

    def __init__(self, a):
        self._a = a
        self._lock_a = RLock()

    @property
    def a(self):
        return self._a

    def update_a(self, new_value):
        with self._lock_a:
            self._a = new_value

    def get_bar(self):
        return Bar(self)

class FooManager(BaseManager): pass

class FooProxy(NamespaceProxy):
    _exposed_ = ('__getattribute__', '__setattr__', '__delattr__', 'a')

FooManager.register("Foo", Foo, FooProxy)


class Bar(object):

    def __init__(self, foo):
        self._foo = foo

    @property
    def a(self):
        return self._foo.a

    def update_a(self, value):
        self._foo.update_a(value)


if __name__ == "__main__":
    fmgr = FooManager()
    fmgr.start()
    foo = fmgr.Foo(5)

    bar = Bar(foo)
    print(bar.a)
    bar.update_a(10)
    print(bar.a)

It runs the first method just fine and prints the 5. However, when I go to update the value I'm getting this error:

RuntimeError: RLock objects should only be shared between processes through inheritance

Now the reason I was using manager was specifically so that calls like this, and potentially from other processes would "funnel" to the same single object. It seems like it's trying to copy/share the managed object here though. Can someone let me know if it's possible to do what I'm trying here and if so how to do this? Or do I have the wrong idea about what a Manager does?

What I want really is for the Bar object(s) to just hold the reference to the single foo manager object. In my actual implementation I have a central object which is basically a giant data manager, and then other objects which use this data and slice it up in different ways. What I want is to be able to share the super-set of data, but allow individual objects (in separate threads) to grab the data through the single manager.

Doug
  • 108
  • 8
  • It actually appears that the error occurs when you call `foo.update_a(...)` in general... not just from the class which holds a `Foo` object. Any ideas? – Doug Oct 12 '17 at 20:11
  • It seems to be issues with the `FooProxy`. If I use it, I get the RLock error, if I register the class without it (and thus skip out exposing the `a` property), it works fine. Seems to be related to my wanting to use @property objects (https://stackoverflow.com/questions/46715204/python-multiprocessing-manager-with-property-objects) – Doug Oct 12 '17 at 20:40

1 Answers1

1

So the work-around I found was to implement get_ functions for each property I wanted exposed, then I wrote a custom register function and AutoProxy methods that linked any properties in the class with the appropriate get_ functions. It would have been great to have been able to do this without writing explicit get_ functions, but this seemed to be the best workaround.

Doug
  • 108
  • 8