18

I'm writing an app in Python and I need to run some tasks simultaneously. The module multiprocessing offers the class Process and the concurrent.futures module has the class ProcessPoolExecutor. Both seem to use multiple processes to execute their tasks, but their APIs are different. Why should I use one over the other?

I know that concurrent.futures was added in Python 3, so I guess it's better?

dano
  • 91,354
  • 19
  • 222
  • 219
Daniel Jonsson
  • 3,261
  • 5
  • 45
  • 66
  • Duplicate of http://stackoverflow.com/questions/24896193/whats-the-difference-between-pythons-multiprocessing-and-concurrent-futures but should not be closed because the answer provides details unavailable elsewhere. – max May 06 '17 at 08:02

1 Answers1

14

The motivations for concurrent.futures are covered in the PEP.

In my practical experience concurrent.futures provides a more convenient programming model for long-running task submission and monitoring situations. A program I recently wrote using concurrent.futures involved monitoring a directory for incoming files over a 2-3 hour window, translating each file as it arrives to a task, submitting it and so on. Future objects returned by the ProcessPoolExecutor allow for tracking task status, providing intermediate status reports etc in a convenient way.

iruvar
  • 22,736
  • 7
  • 53
  • 82