Why are threads spread between CPUs?

Question

I am trying to get my head around threading vs. CPU usage. There are plenty of discussions about threading vs. multiprocessing (a good overview being this answer) so I decided to test this out by launching a maximum number of threads on my 8 CPU laptop running Windows 10, Python 3.4.

My assumption was that all the threads would be bound to a single CPU.

EDIT: it turns out that it was not a good assumption. I now understand that for multithreaded code, only one piece of python code can run at once (no matter where/on which core). This is different for multiprocessing code (where processes are independent and run indeed independently).
While I read about these differences, it is one answer which actually clarified this point.

I think it also explains the CPU view below: that it is an average view of many threads spread out on many CPUs, but only one of them running at one given time (which "averages" to all of them running all the time).

It is not a duplicate of the linked question (which addresses the opposite problem, i.e. all threads on one core) and I will leave it hanging in case someone has a similar question one day and is hopefully helped by my enlightenment.

The code

import threading
import time


def calc():
    time.sleep(5)
    while True:
        a = 2356^36

n = 0
while True:
    try:
        n += 1
        t = threading.Thread(target=calc)
        t.start()
    except RuntimeError:
        print("max threads: {n}".format(n=n))
        break
    else:
        print('.')

time.sleep(100000)

Led to 889 threads being started.

The load on the CPUs was however distributed (and surprisingly low for a pure CPU calculation, the laptop is otherwise idle with an empty load when not running my script):

Why is it so? Are the threads constantly moved as a pack between CPUs and what I see is just an average (the reality being that at a given moment all threads are on one CPU)? Or are they indeed distributed?

Why would you expect threads to be consolidated to a single core? The purpose of threads is to have multiple executions of code in flight at once, and a single core cannot do more than one thing at once. Spreading them out is the only way to actually get parallelism. — GManNickG, Feb 06 '16 at 07:45
Possible duplicate of [Python threads all executing on a single core](http://stackoverflow.com/questions/4496680/python-threads-all-executing-on-a-single-core) — , Feb 06 '16 at 08:02
@JarrodRoberson: my case is actually the opposite of the question you linked to. But it indirectly helped me to understand the crucial part: that with multithreading only one piece of python code can run at once (no matter the core it is on). Which is not the case for multiprocessing (where processes are independent and therefore no need for GIL) — WoJ, Feb 06 '16 at 08:40
This is a specific limitation of Python regarding threads. This is not the same using other languages. See "Global Interpreter Lock" in python documentation, as Rolf Schorpion mentions. — neodelphi, Feb 06 '16 at 11:01

Rolf Schorpion · Accepted Answer · 2016-02-06T11:13:30.233

As of today it is still the case that 'one thread holds the GIL'. So one thread is running at a time.

The threads are managed on the operating system level. What happens is that every 100 'ticks' (=interpreter instruction) the running thread releases the GIL and resets the tick counter.

Because the threads in this example do continuous calculations, the tick limit of 100 instructions is reached very fast, leading to an almost immediate release of the GIL and a 'battle' between threads starts to acquire the GIL.

So, my assumption is that your operating system has a higher than expected load , because of (too) fast thread switching + almost continuous releasing and acquiring the GIL. The OS spends more time on switching than actually doing any useful calculation.

As you mention yourself, for using more than one core at a time, it's better to look at multiprocessing modules (joblib/Parallel).

Interesting read: http://www.dabeaz.com/python/UnderstandingGIL.pdf

score -1 · Answer 2 · answered Feb 06 '16 at 07:59

Um. The point of multithreading is to make sure they work gets spread out. A really easy cheat is to use as many threads as you have CPU cores. The point is they are all independent so they can actually run at the same time. If they were on the same core only one thread at a time could really run at all. They'd pass that core back and forth for processing at the OS level.

Your assumption is wrong and bizarre. What would ever lead you to think they should run on the same CPU and consequently go at 1/8th speed? As the only reason to thread them is typically to get the whole batch to go faster than a single core alone.

In fact, what the hell do you think writing parallel code is for if not to run independently on several cores at the same time? Like this would be pointless and hard to do, let's make complex fetching, branching, and forking routines to accomplish things slower than one core just plugging away at the data?

*The point is they are all independent so they can actually run at the same time.* No they cannot. Now that I red some less excited answers I understand that with multithreading only ONE piece of python code can run at a given moment (due to GIL), they are just switched between to give an impression of concurrency. So no real concurrency. Multiprocessing (as in "many independent processes") does allow concurrent run (as the processes are independent - which is not the case for threads). — WoJ, Feb 06 '16 at 08:38
This answer is - to reuse the phrase - "wrong and bizarre". Maybe stick to answering questions in languages you know? — Voo, Feb 06 '16 at 12:19

Why are threads spread between CPUs?

2 Answers2