2

I have a large data file to load in a list. and use that data in other python programs. eg. I have run one python program A which loads the data from file and load it is a list. when i run another python program B, i require to process the data from the list which was loaded in the first program A. Program A will be run only once. program B will be called as and when it needs to be run. Any idea how to achieve this in python? I have used pickle and loading the pickle file takes few seconds. I want to load the pickle file in program A. and in program B i need to use data from the list which was loaded in program A.

wjandrea
  • 28,235
  • 9
  • 60
  • 81
vvr
  • 21
  • 3
  • Why can't you have the code that would run in program B run in program A instead – Erik McKelvey Dec 07 '21 at 17:06
  • just import program B as a module in A, then call B from A – 2pichar Dec 07 '21 at 17:06
  • 1
    https://docs.python.org/3/library/multiprocessing.shared_memory.html#multiprocessing.managers.SharedMemoryManager.ShareableList maybe? – saquintes Dec 07 '21 at 17:09
  • Thanks for your answers. This is being called from frontend and i'm running this python as an API called from nodeJS. So program B is called as an API. If i have both programs as one, when ever API is called the program A takes long time to process the data. If it is already loaded, then the program B takes only milliseconds. – vvr Dec 08 '21 at 04:43
  • Do you want to download data over a slow connection and then perform the analysis? A database in memory could be good for this. https://stackoverflow.com/questions/3047412/in-memory-database-in-python – David Lukas Dec 13 '21 at 10:18

0 Answers0