5

For some silly reason, there's a piece of hardware on my (GNU/Linux) machine that can only communicate a certain occurrence by writing a value to memory. Assume that by some magic, the area of memory the hardware writes to is visible to a process I'm running. Now, I want to have a thread within that process keep track of that value, and as soon as possible after it has changed - execute some code. However, it is more important to me that the thread not waste CPU time than for it to absolutely minimize the response delay. So - no busy-waiting on a volatile...

How should I best do this (using modern C++)?

Notes:

  • I don't mind a solution involving atomics, or synchronization mechanisms (in fact, that would perhaps be preferable) - as long as you bear in mind that the hardware doesn't support atomic operations on host memory - it performs a plain write.
  • The value the hardware writes can be whatever I like, as can the initial value in the memory location it writes to.
  • I used C++11 since it's the popular tag for Modern C++, but really, C++14 is great and C++17 is ok. On the other hand, even a C-based solution will do.
user1643723
  • 4,109
  • 1
  • 24
  • 48
einpoklum
  • 118,144
  • 57
  • 340
  • 684
  • How does the hardware modify the memory value? Memory mapped I/O? DMA? – 1201ProgramAlarm Mar 25 '18 at 22:54
  • @1201ProgramAlarm: Try not making assumptions about that. But - in my case it's probably DMA; but it may even be DMA to elsewhere and then some kernel driver copies to the memory location I know. – einpoklum Mar 25 '18 at 23:05
  • 2
    I asked because if it is one of those, the memory is updated using the bus and the CPU isn't aware of it until you ask for the data, so you'd have to use some sort of polling loop. If a kernel drive is updating it, there _might_ be a way to be notified when the page changes but even that might be too frequent. – 1201ProgramAlarm Mar 25 '18 at 23:13
  • @1201ProgramAlarm: Assume I can't put anything in kernel space, and the driver won't cater to my needs. – einpoklum Mar 26 '18 at 00:04
  • The problem with portable C++ code here is that it tends to assume everything is C++. In this case, the writer isn't. This is a poster child for `volatile`; a new value appears out of the blue as far as C++ is concerned. Any alternative will be non-portable. – MSalters Mar 26 '18 at 00:31
  • 2
    This reminds me what I once read about Linux and `fork()`. Roughly: all memory pages are set to read-only. First write access forces an exception which causes copying of page and the write access is repeated with success. I'm not sure whether this helps for your problem or even is usable for your specific situation. Just the thing came in my mind when reading this: an exception on write attempt. – Scheff's Cat Mar 26 '18 at 05:55
  • @Scheff that's a sound idea, but unfortunately the kernel does not work the same way as user space: https://brennan.io/2016/11/03/kernel-dev-ep2/, — if you somehow make kernel code SIGSEGV, it will simply suspend your process without a way out. [userfaultfd](http://man7.org/linux/man-pages/man2/userfaultfd.2.html) looks promising, but it does not seem to report accesses by kernel code. – user1643723 Mar 26 '18 at 06:33
  • 1
    @MSalters: I didn't say the code had to be portable... – einpoklum Mar 26 '18 at 07:55
  • @einpoklum what about semaphore? – user2284570 Dec 10 '20 at 23:29
  • @user2284570: What about it? – einpoklum Dec 10 '20 at 23:32
  • @einpoklum using it for your problem. Since you can suspend a thread until a specific region of memory get a specific value. – user2284570 Dec 10 '20 at 23:35
  • @user2284570: But the hardware can't "post" on the semaphore. And trying to notice when the hardware wrote in order to them post on the semaphore leaves us with the original problem. – einpoklum Dec 10 '20 at 23:37
  • @einpoklum no, you only asks the process. Because that s correct, the hardware doesn t need to acquire the semaphore in order to write in ram. You just require the address to be fixed. – user2284570 Dec 10 '20 at 23:41
  • Could you imagine using the same mechanisms as *Watchpoints* inside debugger. I do not know exactly how it works under the hood but on modern CPUs there are specific debug registers where you can set the address on which to break when it is read ou written ( this is just an idea, unfortunately I do not have a MCVE that would implement that) – NGI Dec 23 '20 at 08:37
  • Did you have a look at [Is it possible to set a gdb watchpoint programmatically?](https://stackoverflow.com/q/8941711/3972710). Both answers from [Will Hawkins](https://stackoverflow.com/a/47118320/3972710) and [tinytaro](https://stackoverflow.com/a/47118320/3972710) seem convincing (I played both) as long as you may grant priviledges (or maybe setcap it) and you accept receiving signals – NGI Dec 23 '20 at 19:25

2 Answers2

1

So, the naive thing to do would be non-busy sleeping, e.g.:

volatile int32_t* special_location = get_special_location();
auto polling_interval_in_usec = perform_tradeoff_between_accuracy_and_cpu_load();
auto polling_interval = std::chrono::microseconds(polling_interval_in_usec);

while(should_continue_polling()) {
    if (*special_location == HardwareIsDone) { 
         do_stuff();
         return;
    }
    std::this_thread::sleep_for(polling_interval); 
}
einpoklum
  • 118,144
  • 57
  • 340
  • 684
-1

This is usually done via std::condition_variable.

... as long as you bear in mind that the hardware doesn't support atomic operations on host memory - it performs a plain write.

Implementations of std::atomic may fall back to mutexes in such cases

UPD - Possible implementation details: assuming you have some data structure in a form of:

struct MyData {
    std::mutex mutex;
    std::condition_variable cv;
    some_user_type value;
};

and you have an access to it from several processes. Writer process overrides value and notifies cv via notify_one, reader process waits on cv in a somewhat similar to busy wait manner, but thread yields for the wait duration. Everything else I could add is already present in the referred examples.

Andrei R.
  • 2,374
  • 1
  • 13
  • 27
  • Please be more concrete. How will this work in my case? – einpoklum Mar 26 '18 at 07:54
  • It seems you've misunderstood my question. The writer is not a process, it's hardware. it can't do anything other than a simple write to a location in memory. So - unless I'm mistaken, your suggestion can't work. – einpoklum Mar 26 '18 at 09:02
  • @einpoklum, you specified `The value the hardware writes can be whatever I like`, and I assumed it is running c++ code. Otherwise, your options are limited to busy-waiting and checking on timeout, and you mentioned both. – Andrei R. Mar 26 '18 at 09:31