what is the fastest way to notify another thread that data is available? any alternativies to spinning?

Question

One my thread writes data to circular-buffer and another thread need to process this data ASAP. I was thinking to write such simple spin. Pseudo-code!

    while (true) {
        while (!a[i]) {
            /* do nothing - just keep checking over and over */
        }
        // process b[i]
        i++;
        if (i >= MAX_LENGTH) {
            i = 0;
        }
    }

Above I'm using a to indicate that data stored in b is available for processing. Probaly I should also set thread afinity for such "hot" process. Of course such spin is very expensive in terms of CPU but it's OK for me as my primary requirement is latency.

The question is - am I should really write something like that or boost or stl allows something that:

1. Easier to use.
2. Has roughly the same (or even better?) latency at the same time occupying less CPU resources?

I think that my pattern is so general that there should be some good implementation somewhere.

upd It seems my question is still too complicated. Let's just consider the case when i need to write some items to array in arbitrary order and another thread should read them in right order as items are available, how to do that?

upd2

I'm adding test program to demonstrate what and how I want to achive. At least on my machine it happens to work. I'm using rand to show you that I can not use general queue and I need to use array-based structure:

#include "stdafx.h"
#include <string>
#include <boost/thread.hpp>
#include "windows.h" // for Sleep


const int BUFFER_LENGTH = 10;
int buffer[BUFFER_LENGTH];
short flags[BUFFER_LENGTH];

void ProcessorThread() {
    for (int i = 0; i < BUFFER_LENGTH; i++) {
        while (flags[i] == 0);
        printf("item %i received, value = %i\n", i, buffer[i]);
    }
}


int _tmain(int argc, _TCHAR* argv[])
{
    memset(flags, 0, sizeof(flags));
    boost::thread processor = boost::thread(&ProcessorThread);
    for (int i = 0; i < BUFFER_LENGTH * 10; i++) {
        int x = rand() % BUFFER_LENGTH;
        buffer[x] = x;
        flags[x] = 1;
        Sleep(100);
    }
    processor.join();
    return 0;
}

Output:

item 0 received, value = 0
item 1 received, value = 1
item 2 received, value = 2
item 3 received, value = 3
item 4 received, value = 4
item 5 received, value = 5
item 6 received, value = 6
item 7 received, value = 7
item 8 received, value = 8
item 9 received, value = 9

Is my program guaranteed to work? How would you redesign it, probably using some of existent structures from boost/stl instead of array? Is it possible to get rid of "spin" without affecting latency?

Answer 1

This is what Condition Variables were designed for. std::condition_variable is defined in the C++11 standard library.

What exactly is fastest for your purposes depends on your problem; You can attack it from several angles, but CVs (or derivative implementations) are a good starting point for understanding the subject better and approaching an implementation.

Answer 2

If the consuming thread is put to sleep it takes a few microseconds for it to wake up. This is the process scheduler latency you cannot avoid unless the thread is busy-spinning as you do. The thread also needs to be real-time FIFO so that it is never put to sleep when it is ready to run but exhausted its time quantum.

So, there is no alternative that could match latency of busy spinning.

(Surprising you are using Windows, it is best avoided if you are serious about HFT).

Answer 3

Consider using C++11 library if your compiler supports it. Or boost analog if not. And in your case especially std::future with std::promise.

There is a good book about threading and C++11 threading library:

Anthony Williams. C++ Concurrency in Action (2012)

Example from cppreference.com:

#include <iostream>
#include <future>
#include <thread>

int main()
{
// future from a packaged_task
std::packaged_task<int()> task([](){ return 7; }); // wrap the function
std::future<int> f1 = task.get_future();  // get a future
std::thread(std::move(task)).detach(); // launch on a thread

// future from an async()
std::future<int> f2 = std::async(std::launch::async, [](){ return 8; });

// future from a promise
std::promise<int> p;
std::future<int> f3 = p.get_future();
std::thread( [](std::promise<int>& p){ p.set_value(9); }, 
             std::ref(p) ).detach();

std::cout << "Waiting..." << std::flush;
f1.wait();
f2.wait();
f3.wait();
std::cout << "Done!\nResults are: "
          << f1.get() << ' ' << f2.get() << ' ' << f3.get() << '\n';

}

Answer 4

If you want a fast method then simply drop to making OS calls. Any C++ library wrapping them is going to be slower.

e.g. On Windows your consumer can call WaitForSingleObject(), and your data-producing thread can wake the consumer using SetEvent(). http://msdn.microsoft.com/en-us/library/windows/desktop/ms687032(v=vs.85).aspx

For Unix, here is a similar question with answers: Windows Event implementation in Linux using conditional variables?

Answer 5

Do you really need threading?

A single threaded app is trivially simple and eliminates all the issues with thread safety and the overhead of launching threads. I did a study of threaded vs non threaded code to append text to a log file. The non threaded code was better in every measure of performance.