0

How can I share a list between two python scripts using mmap but without using files?

My first python script calls the other using subprocess.Popen(["python","child.py"])

I have to call 10 different subprocess and don't want to use a file as it will increase I/O time which is not desirable.

Each of my subprocess will modify this list variable created in the parent process.

Note: I don't want to use multiprocess or threads and require the solution using subprocess.Popen only.

Pynchia
  • 10,996
  • 5
  • 34
  • 43
human torch
  • 303
  • 5
  • 15
  • Do you expect two way changing of the list, I.e. should main process know about list changes in subprocesses and vice versa? – Oleksii M Feb 10 '16 at 07:53
  • Yes. The parent will initialize the list. Child will modify it. Parent will print the updated list. – human torch Feb 10 '16 at 08:09
  • Do you specifically need to use mmap? Otherwise you can use [multiprocessing's Manager](https://docs.python.org/3/library/multiprocessing.html#sharing-state-between-processes) – Pynchia Feb 10 '16 at 08:14
  • @SiHa so far I tried using files and pickle library but that was increasing my I/O time and hence I wanna shift to mmap – human torch Feb 10 '16 at 08:15
  • 3
    I think the base question here is: What have you tried so far? Where is your problem, where does your code not work. You should not expect sombody to place a general tutorial for `mmap` as an answer. And if you have a question/problem why ruling out the obvious `multiprocess`. Sounds like "Help me but only in the way I want you to help me". – jochen Feb 10 '16 at 08:16
  • @Pynchia please read my post completely. – human torch Feb 10 '16 at 08:16
  • @jochen I read online the difference between subprocess and multiprocess and found subprocess to suit my other needs also when compare to multiprocess. I requested specifically for subprocess because otherwise post would have become vast. If it's like I am asking you to help me the way I want it only then it's cause i've read about other ways. I simply wanted to keep this post to the point. – human torch Feb 10 '16 at 08:20
  • 3
    then this looks like an [XYproblem](http://xyproblem.info/). What have you tried so far? – Pynchia Feb 10 '16 at 08:26
  • There are others facing this too – human torch Feb 10 '16 at 08:28
  • related (share array without copying): [Use numpy array in shared memory for multiprocessing](http://stackoverflow.com/a/7908612/4279) – jfs Feb 13 '16 at 14:21

0 Answers0