0

I have the following function 

def collect_data4evaluation(self, limit, obj_types):
    type_limit = int(round(limit / len(obj_types)))
    D = ReverseSearch.load_unreverted_dict()
    data_set = []
    for obj_type, d in D.items():
        if obj_type in obj_types:
            data_set = data_set + self.collect_dublication(D=d, obj_type=obj_type, type_limit=type_limit)
    random.shuffle(data_set)
    return data_set

It goes through items of dictionary and generates the the list of dictionaries according to obj_type and d of D.items()

I want to use multiprocessing to speed it up. Somthing like this

pool = Pool(cpu_count())
data_set += pool.map(self.collect_dublication, D=d, obj_type=obj_type,  type_limit=type_limit)

But I don't understand how to iterate over dict items using map function

Graygood
  • 363
  • 1
  • 3
  • 18
  • 3
    Does this help you: https://stackoverflow.com/questions/59756754/multiprocessing-pool-with-a-function-that-has-multiple-args-and-kwargs/59775559#59775559 There are some example codes that show how to use the `Pool`. – Amiram Jan 23 '20 at 14:12
  • 1
    Thanks that works fine for me – Graygood Jan 23 '20 at 20:45

0 Answers0