5

Say I start 10 process in a loop using Process() but I only have 8 cores available. How does python handle this?

Nitro
  • 1,263
  • 4
  • 16
  • 37
  • 1
    I don't think Python cares. It depends on how your OS handles; most likely by alternating processes. –  Oct 11 '16 at 00:41
  • 1
    have a look at this http://stackoverflow.com/questions/36795086/on-what-cpu-cores-are-my-python-processes-running – xCodeZone Oct 11 '16 at 00:49
  • https://github.com/python/cpython/blob/master/Lib/multiprocessing/process.py – enderland Oct 11 '16 at 00:50
  • 1
    Your computer is **always** running more processes than cores (except if you use a supercomputer...) and the OS handles this quite fine... the OS will just schedule those processes like all others. – Bakuriu Oct 11 '16 at 16:23

1 Answers1

4

While best practice is to use as many threads as you have virtual cores available, you don't have to stick to that. Using less means you could be under-utilizing your available processor capacity. Using more means you'll be over-utilizing your available processor capacity.

Both these situations mean you'll be doing work at a slower rate than would otherwise be possible. (Though using more threads than you have cores has less of an impact than using fewer threads than your have cores.)

Ouroborus
  • 16,237
  • 4
  • 39
  • 62
  • Hi. Quick clarification on this. Say I have a dual core machine and I spin up three processes using `Process()`, each of which runs a master function that runs a sub-function in a continuous, never ending loop. The loops run every 60 seconds so there's down time between loops. Will the OS be able to allocate compute resource between the different processes or does one process get orphaned because the other two processes are running in perpetual loops? – Jossy Aug 05 '22 at 02:51
  • 1
    @Jossy Processes are usually swapped out rapidly by the OS using a technique called "preemptive multitasking". Assuming all your processes have the same priority, and you don't have enough cpu at the moment, they'll share cpu as equally as is possible. – Ouroborus Aug 05 '22 at 03:05