std::cout from multiple threads

Question

I have an application in which I perform costly calculations in parallel worker threads. For simplicity, I write results to stdout directly from these threads.

This worked fine until I changed a few things in an attempt to make the code run faster. First, I replaced std::endl with "\n" to prevent a flushing after every line. And I added the following lines to the init part of my main program:

    std::cin.tie(nullptr);
    std::ios_base::sync_with_stdio(false);

The basic structure of the worker thread code looks like this:

while(true) {
    // get data from job queue, protected by unique_lock on std::mutex

    // process the data

    // print results
    {
        std::lock_guard<std::mutex> lk(outputMutex_);
        std::cout << "print many results" << "\n"; // was originally std::endl
    }
}

Since this "optimization", the output of the workers occasionally "mixes". i.e. the mutex does not serve its intended purpose.

Why is this happening? My understanding is that there is just a single stdout stream buffer, and that the data arrives in the corresponding buffer in sequence, even if the output is not flushed from this buffer before releasing the mutex. But that does not seem to be the case...

(I realize that maybe it would be nicer to have the output generated in a separate thread, but then I'd need to pass back these results using another queue, which did not seem necessary here)

Update: Maybe my post was not clear enough. I do not care about the sequence of the results. The problem is that (for the example above) instead of this:

print many results
print many results
print many results

I sometimes get:

print many print many results
results
print many results

And the outputMutex_ is a static member that is shared by all worker threads.

You're making unsequenced calls that modify the same resource. The result is, iirc, undefined so it makes no difference, \n or std::endl. — bipll, Jun 12 '20 at 15:45
Does this answer your question? [Synchronizing STD cout output multi-thread](https://stackoverflow.com/questions/9332263/synchronizing-std-cout-output-multi-thread) — underscore_d, Jun 12 '20 at 15:49
@bipll: I am not worried about the sequence of the results, but the results are not printed as entity despite protection by the mutex — user52366, Jun 12 '20 at 16:36
@underscore-d: no, not really. My output should be protected by the mutex, and I'd like to avoid the overhead of moving these simple results to a separate "output" thread, which also requires synchronization. — user52366, Jun 12 '20 at 16:40
@user52366 and `outputMutex_` is unique across all the workers and they all lock the same mutex? — bipll, Jun 12 '20 at 17:35
The code snippets in the question should work just fine, so the problem must be in the code that isn't shown. — Pete Becker, Jun 12 '20 at 17:44
@bipll: yes, the mutex is unique and used across all workers, and nobody else is writing any output. The code really did work before the optimization, with literally billions of lines written without this problem. — user52366, Jun 12 '20 at 17:47

crsn · Answer 1 · 2020-06-12T20:55:20.490

1

you are accessing cout by multiple threads. The access to its queue is protected by the mutex, but it needs to be flushed. That doesn't happen automatically (always at least :) )

std::endl flushes cout, '\n' doesn't

or, you can tell cout to flush, with std::flush:

https://en.cppreference.com/w/cpp/io/manip/flush

try:

while(true) {
    // get data from job queue, protected by unique_lock on std::mutex

    // process the data

    // print results
    {
        std::lock_guard<std::mutex> lk(outputMutex_);
        std::cout << "print many results" << "\n"; // not flushed
        std::cout << std::flush; // flushed!
        std::cout << "print something else" << std::endl; // flushed!
    }
}

more on: https://stackoverflow.com/a/22026764/13735754

edited Jun 12 '20 at 20:55

answered Jun 12 '20 at 20:48

crsn

609
4
11

This is indeed my observation, but what I don't understand is why the flushing matters. This would imply that there are several buffers (one per thread), and not just one per stream as I assumed. The problem would then arise because content is buffered between worker iterations, and mixed in the output when flushed. If there is a single buffer, then no mixing can occur because the content is delivered by line, and the delivery of each line is protected by the mutex. – user52366 Jun 13 '20 at 04:52
Flushing basically refers to cout buffer, and the flush action implies that the string will be printed, and removed from the memory. The problem you see, is because you are protecting the access to cout, but you are not telling cout to flush. Flush will happen anyway. When it will happen is not up to you if you don't flush. Eg. it may happen when cout finishes its internal buffer. – crsn Jun 13 '20 at 08:53
It might seem odd, but the reason for this is that there are situation in which you want the comfort of calling std::cout, without having to pay the ridiculously high computational price of writing for real to cout. – crsn Jun 13 '20 at 08:56
I understand that the time when the stdout buffer is flushed is out of my control. But what I don't understand is why the buffer (there is only one) would be filled out of sequence... Unless there would be multiple buffers, one per thread... but that is not the case, I think. – user52366 Jun 13 '20 at 11:28
let's try to make an example: – crsn Jun 13 '20 at 12:11
This answer does not address the question of why flushing is necessary when a lock is in place. If each thread adds a line of unprinted text to the buffer, it should not matter who flushes the buffer or when it is flushed. As long as both threads don't try to write to the head of the buffer at the same time, which is prevented using the lock, the text should not be jumbled (though the order of the lines will be non-deterministic). So why do lines still get jumbled? Is the buffer abstraction for std::cout incorrect? – nullUser Dec 30 '21 at 23:20

std::cout from multiple threads

1 Answers1