I develop a python module, MyModule. The user adds it as a git submodule, and imports multiple classes and functions.
from MyModule.MyModuleSubfolder import ClassA
from MyModule.MyModuleSubfolder import ClassB
from MyModule.MyModuleOtherSubfolder import useful_function
The input the user supplied to useful_function is a very complicated dictionary with fields that may be dictionaries/lists/very large numpy arrays, etc. I want to use dill (or any other alternative) to store this input within useful_function, so I can read it within useful_function for debug.
I implemented something like this (I omit all the files open/close lines, they are irrelevant):
...
input_dict['file_path'] = "folder\input_file.pkl"
...
def useful_function(input_dict,load_flag=False):
if load_flag:
input_dict = dill.load(input_dict['file_path'],...)
else:
dill.dump(input_dict,...)
It works perfectly well when I test it while running MyModule as a standalone project or when the user uses MyModule as a git submodule: both of us can write files and read them.
However, when I try to load the input_file.pkl created by the user (who ran my code as a submodule) when I run it as a standalone project, I get the ModuleNotFoundError: No module named 'MyModule' error, which was discussed in a few similar questions before (such as here). However, the situation is a bit different, so the "mainify" solution is not good for me. First, I do not want to dump any class defined in MyModule, just a (very complicated) dictionary, so in principle there may be a way to store this dictionary while being ignorant of the existence of any imports in the main, but pickle/dill (and other libraries which use/extends pickle) do necessarily save all the imports from main.
Second, I have no control over the the user, which can import any arbitrary number of MyModule files, or even other libraries for which I have no access, so I can't a-priory mainify everything.
Do you have any better solution?
