3

I am doing some multiprocessing and I need to share an instance between two processes. I have been trying to use the multiprocessing module to try and accomplish this but there isn't much hope to share anything that can't be pickled it seems; so far I have tried to use a manager and create a proxy object to handle my object by following this SO question and this SO question. I understand that sharing mutable object instances isn't exactly python's forte, but what is the easiest way to do this?

To layout my situation more clearly, I am working on UNIX systems exclusively so it uses forks and copy-on-write memory management to my understanding. This object that I need to share is read-only on the main process but read and write on the subprocess. The easiest way I can think of doing this is to just share a reference to the object instance in memory to the subprocess and communicate between processes not to write when the object is being used by the main process.

Community
  • 1
  • 1
user2909415
  • 979
  • 3
  • 10
  • 26
  • possible duplicate of [Multiprocessing Share Unserializable Objects Between Processes](http://stackoverflow.com/questions/21968278/multiprocessing-share-unserializable-objects-between-processes) – User Mar 14 '15 at 07:52
  • What problems are you facing using a manager? – User Mar 14 '15 at 08:19
  • @User I can't use the register method for my classes because they cannot be serialised. – user2909415 Mar 14 '15 at 08:20
  • @User I think I just solved it. I was using a synch manager to register my classes instead of a base manager. I will a few more tests and if the issue is solved I will close the thread. – user2909415 Mar 14 '15 at 08:33
  • @User So it turns out that was the problem. So should I keep this thread with my solution, because I don't think I will be the only person that will try to register a class using a synch manager thinking that it will work because the synch manager is a subclass of base manager. – user2909415 Mar 14 '15 at 08:57
  • 1
    You should answer your own question and accept it as answer. That helps other people find the solution faster. – User Mar 14 '15 at 19:15

1 Answers1

1

I solved this problem myself, but for anyone who comes to this thread with the same problem:

I made the mistake of instantiating a SyncManager() object then calling the BaseManager.register() method assuming this would work since SyncManager is just a subclass of BaseManager(). I didn't look into exactly why this is the case, but I do know the fix, and that is to just instantiate a BaseManager() and register your Python object with that instead.

user2909415
  • 979
  • 3
  • 10
  • 26