0

I need to design a thread pool system, in Python in this case, but I'm more interested in the general methodology.

It has to be something along the lines of https://www.metachris.com/2016/04/python-threadpool/, where threads wait idling until some jobs are pushed into the pool. How that works, using condition variables, is clear to me.

I have one additional requirement though: the jobs I'm pushing into the pool cannot run all in parallel. Each of them has a class (i don't mean the object class here, just a simple integer that somehow classifies the job) and only one job per class can be running at the same time. If a job is pushed having the same class of a job that is currently running, it has to wait in the queue until the latter is done.

I have already modified the mentioned class to do this, but what I achieved is pretty messy and I'm not sure it's reliable, so I would ask what modifications would be suggested or whether I should use a totally different approach. Again: I don't need the code, but rather a description.

Thanks.

SukkoPera
  • 621
  • 4
  • 8
  • This is a reasonable design. For example, the C++ networking library Boost.Asio provides such a mechanism under the name [strand](https://stackoverflow.com/questions/25363977/what-is-the-advantage-of-strand-in-boost-asio). I am not aware of universal term for it. What exactly is your question here though? How to implement this mechanism, or just whether it's a good design from an interface perspective? – ComicSansMS May 29 '18 at 09:37
  • My question is: what would be the "cleanest" set of modifications I should carry out to the mentioned Python module (which is pretty much the standard way of implementing a thread pool) to make it only run one job per class at a time? – SukkoPera May 29 '18 at 09:40
  • Doesn't the GIL already take care of this for you? – matt May 29 '18 at 09:53
  • As far as I understand no, the jobs are I/O bound and thus a thread will be interrupted and another one will take over, possibly running two jobs of the same class at the same time. – SukkoPera May 29 '18 at 10:18

0 Answers0