1

I've got a fairly simple situation: I have a high speed producer of data in one thread that generates a buffer with [varying length] elements in it. Once this buffer is filled, I have a consumer that writes it to disk.

Right now, the producer thread, which needs to get back to producing ASAP, spins in place if the buffer hasn't been written by the consumer thread yet:

int volatile datatogo=0; // global with starting condition

while(datatogo != 0) // spin, buffer not yet written out
{
    if (recflag == 0) return; // recording is terminated
}
// clipped code fills buffer, then:
datatogo = lengthoffill;

...and in the other thread, the buffer writer does this:

    while(recflag)
    {
        if (datatogo)
        {
            if (m_File.write(sbuffer,datatogo) == -1)
            {
                recflag=0; // bail NOW
            }
            datatogo = 0; // buffer transferred
        }
        usleep(100); // 100 uS
    }

The net effect of this is that the write-to-disk consumer gives up the CPU when it has completed a write, knowing that the producer will have to take some time to actually fill the buffer. With the consumer asleep when there is no data, the CPU is available to the producer. The consumer sleeps for 100 uS, checks for data, and goes back to sleep if there is none, so it doesn't do much in that state except sleep.

However, because the sleep time is arbitrary, this is unlikely to ever be optimum; even if I carefully tune it to work on my machine (8 cores, 3 GHz), it'll act differently on another machine. There will be times when data is ready to be written, and the consumer has just gone to sleep, tossing the entire 100 uS out the window, so to speak. I also have qualms about the resolution and reliability of such a small timing window -- and larger windows won't work.

So. There are multiple mechanisms in Qt to control access to things, which is essentially what I want to do. But I don't understand which one (if any) will do what I want to do, which is:

1) have the consumer sleep until the buffer is full, then write and go back to sleep, or until the job is stopped so it can close the file (it can check which when woken up)

2) while the producer sleeps only if, and while, the buffer is being written out, otherwise fills the buffer and goes back to producing its content.

I need as much CPU available as possible -- this is a software defined radio application, and there is data flying everywhere, multiple FFTs running, all manner of graphics hoo-ha happening, etc. Spin time is werry, werry bad.

Can some kind soul point me to the ideal Qt mechanism to use? I find the Qt docs on QMutex, QWaitCondition, QSemaphore to be... somewhat opaque.

fyngyrz
  • 2,458
  • 2
  • 36
  • 43

1 Answers1

5

Marking an integer variable volatile is not sufficient for the guarantee you want in a multithreaded program:

Why is volatile not considered useful in multithreaded C or C++ programming?

If you like getting down to the nuts and bolts, there actually atomic integers and pointers you can use:

http://doc.qt.io/archives/qt-4.7/qatomicint.html

http://doc.qt.io/archives/qt-4.7/qatomicpointer.html

But you really do want to use Qt's threading primitives. Spinning/sleeping is wasteful compared to using mutexes, semaphores, and wait conditions. These are fairly general concepts you'll find in any thread library, so I'd look around for good explanations of them. Not easy stuff to wrap one's head around at first, but one needs to know it!

The producer/consumer example for QWaitCondition should be illustrative for your scenario:

http://doc.qt.io/qt-4.8/qt-threads-waitconditions-example.html

Community
  • 1
  • 1
  • Stupid 5 min edit limit "atomic integers and pointers" - great tip. I know volatile isn't enough, although it was working (other than the stated issues) in my case. The "explanation" on the QT page isn't worth a plugged nickle. It doesn't even once explain what they DO, it just uses them. While adding an unexplained mutex in there as well, just to make sure I was completely confused. That was my point when I called them "somewhat opaque": these examples do not teach. They are, perhaps, boilerplate, but without understanding, they're not even good boilerplate. – fyngyrz Apr 18 '12 at 02:23
  • Qt docs are quite good compared to competition. Yet they don't go into detail about how to tell time or what a clock is, despite having date/time classes. Threading is a lot harder to explain, so unsurprising they treat the topic as a reference and superficial beyond that. (They emphasize peculiarities of QThread and signal/slots...stuff people who already know how to write safe threading algorithms couldn't know just by looking at the method definitions.) If you have specific questions about the example code, you can always post them here...several producer/consumer Qs already! (at right) – HostileFork says dont trust SE Apr 18 '12 at 05:43
  • I'm not talking about explaining threading. I'm talking about things like "when you lock a mutex, the following occurs: X. When you unlock a mutex, the following occurs: Y. When we say "Block", we mean that when another call is made, Z occurs, until the mutex is unlocked, when A occurs." This is simply explaining what the function of the provided API resource is, and there is no excuse for not providing it -- docs without it are shite, no matter who makes them. And saying "good compared to the competition" is meaningless. The idea isn't to measure against another failure. – fyngyrz Apr 29 '12 at 16:20