I am stuck into finding solution to below multiprocessing issue.
I have a class Record in record.py module. The responsibility of record class is to process the input data and save it into a JSON file. The Record class has method put() to update JSON file.
The record class is initialized in the class decorator. The decorator is applied over most of the classes of various sub-modules. Decorator extracts information of each method it decorates and sends data to put() method of Record class. put() method of Record class then updates the JSON file.
The problem is when the different process runs, each process creates its own instance of record object and Json data gets corrupted since multiple processes tries to update the same json file. Also, each process may have threads running that tries to access and update same JSON file. Please let me know how can i resolve this problem.
class Record():
def put(data):
# read json file
# update json file with new data
# close json file
def decorate_method(theMethod):
# Extract method details
data = extract_method_details(theMethod)
# Initialize record object
rec = Record()
rec.put(data)
class ClassDeco(cls):
# This class decorator decorates all methods of the target class
for method in cls(): #<----This is just a pseudo codebase
decorate_method()
@ClassDeco
class Test()
def __init__():
pass
def run(a):
# some function calls
if __name__ == "__main__":
t = Test()
p = Pool(processes=len(process_pool))
p.apply_async(t.run, args=(10,))
p.apply_async(t.run, args=(20,))
p.close()
