1

Hello ALL ( and thanks in advance) Problem background:

  1. I have a Flask which loads image (with Pillow) in to the memory
  2. For each request I would like to create a child process which will use loaded into memory Pil object. I'm not sure regarding best way to achieve that but I've tried to use this one https://github.com/luizalabs/shared-memory-dict

Here is parent/child simplified example:

Parent:

   import time
   from shared_memory_dict import SharedMemoryDict
   from PIL import Image

   if __name__ == '__main__':
      print("Starting PARENT")
      source_img = Image.open('d:/input_image.png')
      size = 1024 * 1024 * 16
      smd = SharedMemoryDict(name='shared_mem', size=size)
      smd['source_img'] = source_img
      print("PARENT started!")
      source_img.show()
      time.sleep(99999)

Child:

  from shared_memory_dict import SharedMemoryDict
  
  if __name__ == '__main__':
    print("Starting CHILD")
    size = 1024 * 1024 * 16
    smd = SharedMemoryDict(name='shared_mem', size=size)
    existing_smd = SharedMemoryDict(name='shared_mem', size=size)
    source_img = existing_smd['source_img']
    source_img.show()

Once I run it for the first time (running parent , then the child) - everything works normaly But second child run fails with this error:

Starting CHILD
Traceback (most recent call last):
 File "child_dict.py", line 23, in <module>
 source_img = existing_smd['source_img']
 File "/data/anaconda3/envs/STR_3_8/lib/python3.8/site-packages/shared_memory_dict/dict.py", 
 line 92, in __getitem__
 return self._read_memory()[key]
 KeyError: 'source_img'

So it looks like after running child - the PIL object is removed from SharedMemoryDict.

Any idea how to make same entry in shared memory re-usable by multiple child processes is welcome.

davidism
  • 121,510
  • 29
  • 395
  • 339
Yevgeni Burshtein
  • 81
  • 1
  • 10
  • I think you should better create a function to load the image and cache the result, see for example https://stackoverflow.com/questions/55630814/how-to-cache-flask-function-results – Carlos Horn Jun 12 '22 at 14:09
  • Thanks! Store of an image is just an example , in real case I would like to store in shared memory AI model ( which is loaded only once by Flask) and on each request I want a child process to re-use the model for processing the request. – Yevgeni Burshtein Jun 12 '22 at 14:21
  • This might help you https://stackoverflow.com/questions/32815451/are-global-variables-thread-safe-in-flask-how-do-i-share-data-between-requests – Carlos Horn Jun 12 '22 at 15:40

0 Answers0