Strange behavior when calling std::invoke(std::forward(...)) with address-sanitization in a std::thread with a std::ref

Question

Problem

I am trying to pass a lambda-closure to std::thread that calls arbitrary closed-over function with arbitrary closed-over arguments.

template< class Function, class... Args > 
std::thread timed_thread(Function&& f, Args&&... args) {
  // Regarding capturing perfectly-forwarded variables in lambda, see [1]
  auto thread_thunk = ([&] {
    std::cout << "Start thread timer" << std::endl;
    // Regarding std::invoke(_decay_copy(...), ...), see (3) of [2].
    // Assume no exception can be thrown from copying.
    std::invoke(_decay_copy(std::forward<Function>(f)),
                _decay_copy(std::forward<Args>(args)...));
  }
}

int main() {
  int i = 3;
  std::thread t = timed_thread(&print_int_ref, std::ref(i));
  t.join()
  return 0;
}

/*
[1]: https://stackoverflow.com/questions/26831382/capturing-perfectly-forwarded-variable-in-lambda
[2]: https://en.cppreference.com/w/cpp/thread/thread/thread
*/

I use std::forward so that r-value references and l-value references get forwarded (dispatched correctly).
Because std::invoke and the lambda create temporary data-structures, the caller must wrap references in std::ref.

The code appears to work, but causes stack-use-after-scope with address sanitization. This is my primary confusion.

Suspects

I think this may be related to this error, but I do not see the relation since I am not returning a reference; The reference to i should be valid for the duration of main's stack-frame which should outlast the thread because main joins on it. The reference is passed by copies (std::reference_wrapper) into the thread_thunk.

I suspect args... cannot be captured by reference, but then how should it be captured?

A secondary confusion: changing {std::thread t = timed_thread(blah); t.join();} (braces to force destructor) to timed_thread(blah).join(); incurs no such problem, even though to me they appear equivalent.

Minimal example

#include <functional>
#include <iostream>
#include <thread>

template <class T>
std::decay_t<T> _decay_copy(T&& v) { return std::forward<T>(v); }

template< class Function, class... Args > 
std::thread timed_thread(Function&& f, Args&&... args) {
  // Regarding capturing perfectly-forwarded variables in lambda, see [1]
  auto thread_thunk = ([&] {
    std::cout << "Start thread timer" << std::endl;
    // Regarding std::invoke(_decay_copy(...), ...), see (3) of [2].
    // Assume no exception can be thrown from copying.
    std::invoke(_decay_copy(std::forward<Function>(f)),
                _decay_copy(std::forward<Args>(args)...));
    std::cout << "End thread timer" << std::endl;
  });

  /* The single-threaded version code works perfectly */
  // thread_thunk();
  // return std::thread{[]{}};

  /* multithreaded version appears to work
     but triggers "stack-use-after-scope" with ASAN */
  return std::thread{thread_thunk};
}

void print_int_ref(int& i) { std::cout << i << std::endl; }

int main() {
  int i = 3;

  /* This code appears to work
     but triggers "stack-use-after-scope" with ASAN */
  // {
  //   std::thread t = timed_thread(&print_int_ref, std::ref(i));
  //   t.join();
  // }

  /* This code works perfectly */
  timed_thread(&print_int_ref, std::ref(i)).join();
  return 0;
}

Compiler command: clang++ -pthread -std=c++17 -Wall -Wextra -fsanitize=address test.cpp && ./a.out. Remvoe address to see it work.

ASAN backtrace

Sam Varshavchik · Accepted Answer · 2020-06-15T03:41:53.523

1

Both versions appear to be undefined behavior. It is potluck whether the undefined behavior will be caught by the sanitizer. It is fairly likely that even the allegedly working version will also trip the sanitizer, if the program is rerun sufficient amount of times. The bug is here:

std::thread timed_thread(Function&& f, Args&&... args) {
  // Regarding capturing perfectly-forwarded variables in lambda, see [1]
   auto thread_thunk = ([&] {

The closure uses the captured args by reference.

As you know, the parameters to timed_thread go out of scope and get destroyed when timed_thread returns. That's their scope. That's how C++ works.

But you have no guarantees, whatsoever, that this closure gets executed by the new execution thread and references the captured, by reference, all the args..., before they vanish in a puff of smoke here:

return std::thread{thread_thunk};

Unless this new thread manages to execute the code inside thread_hunk, that references the captured, by reference args..., it will end up accessing after this function returns, and this results in undefined behavior.

edited Jun 15 '20 at 03:41

answered Jun 15 '20 at 03:39

Sam Varshavchik

114,536
5
94
148

I am confused why that is not undefined behavior in [this question\(https://stackoverflow.com/questions/26831382/capturing-perfectly-forwarded-variable-in-lambda), which says to capture by reference. – charmoniumQ Jun 15 '20 at 03:41
1

@charmoniumQ: In that question, the lambda is being executed inside the function. With `std::thread`, the lambda is being executed at some unknown point in time. – Dietrich Epp Jun 15 '20 at 03:42
Because it is not a new execution thread, and `lambda()` returns and does its job before the function returns, and all the function parameters go out of scope and get destroyed. – Sam Varshavchik Jun 15 '20 at 03:42
@charmoniumQ: Notably, this is why `std::thread`'s constructors perform a DECAY_COPY on the function arguments *before* starting up the new thread. It takes nothing by reference, though you can use `std::ref` to override this. – Nicol Bolas Jun 15 '20 at 03:45
Then how should one forward the reference into the lambda without copies? Should I make a new question asking that, now that I've gotten the dangling-`ref`s straightened out? – charmoniumQ Jun 15 '20 at 03:46
1

You can't. Not in this case, that involves a new execution thread. There's a long, sad trail of questions on stackoverflow.com about attempting to pass parameters to new execution threads without making copies. This is only possible if the parent execution thread itself directly starts the execution thread,***without any intermediate functions calls or wrappers like the one that's shown here*** and waits for the execution thread to use the captured-by-reference parameters. This is fundamental to C++. – Sam Varshavchik Jun 15 '20 at 03:47
I did not see any UB in the working version (which got edited out). – afic Jun 15 '20 at 03:50
... and you can forward as many references to lambdas as you want, as long as you ensure, by whatever means that work for you, that the lambdas will use those references before the referenced objects get destroyed in the parent execution thread. – Sam Varshavchik Jun 15 '20 at 03:50
@afic Is what you are referring to still present in the "Minimal example" link? I though it was too verbose to list entirely in this question. – charmoniumQ Jun 15 '20 at 03:52
@charmoniumQ That link gives my a 502 error. In general I don't follow external links in questions. – afic Jun 15 '20 at 03:55

score 0 · Answer 2 · answered Jun 15 '20 at 03:44

0

The object being used after its lifetime is the std::ref(i). Follow the references. The function takes the std::ref by reference, the lambda captures by reference, the lambda is copied into the newly created thread which copies the reference to the std::ref(i).

The working version is working because the lifetime of std::ref(i) ends at the semicolon, and the thread is joined before then.

answered Jun 15 '20 at 03:44

afic

500
4
13

All mentioning of the working version in the question got edited out, see the edit history. – afic Jun 15 '20 at 03:47
I moved the minimal example back into the SO post for posterity. – charmoniumQ Jun 15 '20 at 04:02

Strange behavior when calling std::invoke(std::forward(...)) with address-sanitization in a std::thread with a std::ref

Problem

Suspects

Minimal example

2 Answers2