0

I have a list of dictionaries, which contain further dictionaries within it. I am using multiprocessing to process this list.

My understanding is that when you pass an iterable to pool.map(), it basically sends individual elements(in this case a single dictionary) to the function. However, I am getting this error.

'list indices must be integers, not dict'

This is the snippet of code I am using:

pool = multiprocessing.Pool(processes=multiprocessing.cpu_count() - 1)
pool.map(perform_clean, biglist)
pool.close()

here biglist is the [{{}}]. Please tell me if I understanding it wrong.

Paul
  • 5,473
  • 1
  • 30
  • 37
pri
  • 1
  • Possible duplicate of [Python multiprocessing - Passing a list of dicts to a pool](https://stackoverflow.com/questions/43957843/python-multiprocessing-passing-a-list-of-dicts-to-a-pool) – Nathan Nov 02 '18 at 17:45
  • 2
    Can we see perform_clean? And do you get the same error if you call `perform_clean(biglist[0])` outside of multiprocessing? – dwagnerkc Nov 02 '18 at 17:49
  • Post the full stack trace. It should tell you exactly what line is throwing the error. This is almost certain an issue in `perform_clean`. – juanpa.arrivillaga Nov 02 '18 at 18:05
  • 1
    Never mind. The function perform_clean had other functions that were throwing the error. When I ran it outside of multiprocessing, I was able to identify it. Rookie multithreading debugging mistake. Thanks @dwagnerkc – pri Nov 02 '18 at 19:31
  • np. multiprocess debugging is the worst. – dwagnerkc Nov 02 '18 at 20:49

0 Answers0