Threads seem to alternate rather than run parallel

Question

Here is a little program that runs two for loops on separate threads:

#include <iostream>
#include <thread>

void print_one() {
for (int i = 0; i < 1000; i++) {
    std::cout << i << "\n";
    }
}

void print_two() {
for (int i = 0; i > -1000; i--) {
    std::cout << i << "\n";
    }
}

int main(){
    std::thread t1(print_one);
    std::thread t2(print_two);

    t1.join();
    t2.join();
}

The program follows a pattern in which one for loop runs about 30-400 times (usually about 30-200), after which the other loop gets its "turn" and does the same. The behavior continues until the program exits.

I left mutex out on purpose to show that it's not about locks. In all examples that I've seen on Youtube, the loops usually alternate after every 1-3 iterations. On my PC it's as if two threads simply can't run simultaneously, and one thread has to take time off while the other one is working. I'm not experiencing any performance issues in general, so it doesn't seem like hardware failure to me.

A thread not doing anything while another one has time to execute std::cout and add a new line hundreds of times in a row just doesn't sound right to me. Is this normal behavior?

I have a Ryzen 5 1600 processor, if that makes any difference.

Answer 1

A thread not doing anything while another one has time to execute std::cout and add a new line hundreds of times in a row just doesn't sound right to me. Is this normal behavior?

There is no other sensible possibility here. Each thread only needs to do a single integer increment before it needs access to the terminal, something that only one thread can access at a time.

There are only two other possibilities, and they're both obviously terrible:

1. A single core runs the two threads, switching threads after every single output. This provides atrocious performance as 90+% of the time, the core is switching from one thread to the other.

2. The threads run on two cores. Each core does a single increment, then waits for the other core to finish writing to the terminal, and then writes to the terminal. Each core spends at least half its time waiting for the other core to release the terminal. This takes up two cores and provides fairly poor performance because the threads are spending a lot of time stopping and starting each other.

What you are seeing is the best possible behavior. The only sensible thing to do is to allow each thread to run long enough that the cost of switching is drowned out.

Answer 2

Answer to the first part:

The behaviour you observed is a normal behaviour.

Each thread gets a specific amount of time (to be executed on the core) that is dynamically decided by the OS, based on current situations.

Answer to the second part:

The cout operation is buffered.

Even if the calls to write (or whatever it is that accomplishes that effect in that particular implementation) are guaranteed to be mutually exclusive, the buffer might be shared by the different threads. This will quickly lead to corruption of the internal state of the stream. [1]

That is why you see only newline for some time.

The solution to this issue is using printf(). It's behaviour is atomic.

EDITS:

The point made about the printf() function that its behaviour is atomic is based on my experiments. The code at [2] demonstrates that the printf() function is atomic in nature.

You can actually test it.

References:

1. https://stackoverflow.com/a/6374525/7422352
2. https://www.researchgate.net/publication/351979208_Code_to_demonstrate_sequential_execution_of_2_parallel_regions_created_using_OpenMP_API