How to set the element value of a subscriptable attribute of a proxy object

Question

I used this stackoverflow question to try to implement a manager object that would manage the writing of attributes of a custom object. I know that I need to expose the magic methods like __getattribute__, __setattr__, and __delattr__ by defining _exposed_ in my proxy class. However, when I try to set the value of an element of a subscriptable attribute of the custom object, it remains unchanged.

Viewing the multiprocessing documentation, I can't find the multiprocessing.managers subclass NamespaceProxy—mentioned in the aforementioned post—anywhere. I am able to import it on one hand; however, I have a nagging doubt that it isn't being implemented correctly.

Here is how I've tried to change the value of an element of an array, attributed to an object of a custom class:

from multiprocessing.managers import BaseManager, NamespaceProxy
import numpy as np

class TestClass(object):
   def __init__(self, a):
       '''
       Args:
           a (np.ndarray): the array that needs to be changed  
       '''
       self.a = a

class TestProxy(NamespaceProxy):
    # exposes the magic methods for TestProxy objects needed for setting their attributes
    _exposed_ = ('__getattribute__', '__setattr__', '__delattr__')

class MyManager(BaseManager):
    pass

if __name__ == '__main__':
    MyManager.register('test', TestClass, TestProxy)
    manager = MyManager()
    manager.start()

    arr = np.array([0,0,0,0])
    managed_obj = manager.test(arr)
    managed_obj.a[0] = 1

    print(managed_obj.a)

# Console: [0 0 0 0]

# Expected ouput: [1 0 0 0]

Edit: I was able to change the actual value of a with something like

arr2 = np.array([0])
managed_obj.a = arr2
print(managed_obj.a)

# Console : [0]

However, I still don't know how to change the value of an element of a.

Answer 1

I haven't figured out how (or if) this can be made to work via some kind of generic __setitem__ method — my original goal — but here's how to define a custom method (named my_setitem() below) that appears to be able to change the value of indexed element(s) in the array.

It won't allow you to change the value of a subscriptable attribute of a proxy object directly like you want, but it does show a way to changing them.

I think this will work in Python 3.7, as I tried to avoid doing anything that would only work in v3.8 with one exception, namely this new feature that was added to f-string support was is very handy for debugging — but it should be relatively easy to remove/replace if necessary.

from multiprocessing.managers import BaseManager, NamespaceProxy
import numpy as np

from functools import partial
print = partial(print, flush=True)  # Change default.


class TestClass(object):
    def __init__(self, a):
        '''
        Args:
            a (np.ndarray): the array that needs to be changed
        '''
        print(f'TestClass.__init__ called, {a = }')
        self.a = a

    def my_setitem(self, name, index, value):
        print(f'in TestClass.my_setitem()')
        attr = getattr(self, name)
        attr[index] = value


class TestProxy(NamespaceProxy):
    # exposes the magic functions for TestProxy objects needed for setting their attributes
    _exposed_ = ('__getattribute__', '__setattr__', '__delattr__', 'my_setitem')

    def my_setitem(self, name, index, value):
        print(f'in TestProxy.my_setitem()')
        callmethod = object.__getattribute__(self, '_callmethod')
        return callmethod('my_setitem', (name, index, value))


class MyManager(BaseManager):
    pass


if __name__ == '__main__':
    MyManager.register('Testclass', TestClass, TestProxy)
    manager = MyManager()
    manager.start()

    arr = np.array([0,0,0,0])
    print(f'calling manager.Testclass(arr)')
    managed_obj = manager.Testclass(arr)
    print(f'result: {managed_obj = }')
    print()

#    managed_obj.a[0] = 42  # The problem, doesn't work.
    name = 'a'
    index = 1
#    index = slice(0, 2)  # slices also work
    print(f'executing managed_obj.my_setitem({name=}, {index=}, 42)')
    managed_obj.my_setitem(name, index, 42)

    print(f'result: {managed_obj.a = }')  # -> result: managed_obj.a = array([ 0, 42,  0,  0])

Here's the output it produces (running with Py 3.8.0 on my system):

calling manager.Testclass(arr)
TestClass.__init__ called, a = array([0, 0, 0, 0])
result: managed_obj = <TestProxy object, typeid 'Testclass' at 0x4b3f520>

executing managed_obj.my_setitem(name='a', index=1, 42)
in TestProxy.my_setitem()
in TestClass.my_setitem()
result: managed_obj.a = array([ 0, 42,  0,  0])