Unable to use std::bind with variadic template parameters

Question

I'm working on implementing a wrapper for std::thread that will allow me retrieve arbitrary return values after the thread is finished executing. While I am using C++11, I am using an older ARM architecture that does not support fully support atomic int's, which means that I can't use std::future, std::promise, std::packaged_task, and much of the stl threading functionality (I do get std::thread at least). I am testing with gcc 4.8.4.

While working on my implementation, I ran into this bug, which makes it impossible for me capture variadic template parameters with a lambda. Unfortunately, I can not upgrade my compiler to 4.9 at the moment.

I'm attempting to implement a workaround using std::bind, but am running into quite a few issues. I'm unsure if these are compiler bugs or implementation errors on my part. Here is the source:

#include <iostream>
#include <memory>
#include <thread>

#include <unistd.h>
#include <pthread.h>

class ConcurrentTaskBase
{
public:
   ConcurrentTaskBase(int priority, const std::function<void()>& runTask)
      : m_thread(),
        m_active(true)
   {
      auto wrap = [this](int priority, const std::function<void()>& runTask)
      {
         //Unrelated pthread stuff that I commented out
//         sched_param param{priority};
//
//         int err = pthread_setschedparam(pthread_self(), SCHED_RR, &param);
//         if (err)
//            cout << "failed to set new priority: " << err << endl;

         runTask();
      };
      m_thread = std::thread(wrap, priority, runTask);
   }

   virtual ~ConcurrentTaskBase(void)
   {
      waitForCompletion();
   }

   void waitForCompletion(void)
   {
      if (m_active)
      {
         m_thread.join();
         m_active = false;
      }
   }

private:
   std::thread m_thread;
   bool m_active;
};

template<class R, class... ArgTypes>
class ConcurrentTask;

template<class R, class... ArgTypes>
class ConcurrentTask<R(ArgTypes...)> : public ConcurrentTaskBase
{
public:
   ConcurrentTask(int priority, const std::function<R(ArgTypes...)>& task, ArgTypes&&... args)
      : ConcurrentTaskBase(priority, bindTask(task, std::forward<ArgTypes>(args)...))
   {}

   std::shared_ptr<R> getReturn(void) noexcept
   {
      waitForCompletion();
      return m_storage;
   };

private:
   static std::function<void(void)> bindTask(const std::function<R(ArgTypes...)>& task, ArgTypes&&... args)
   {
      auto action = [task](ArgTypes&&... args) -> void
      {
         //Eventually m_storage = std::make_shared<R>(task(std::forward<ArgTypes>(args)...)); after bugs are fixed
         task(std::forward<ArgTypes>(args)...);
         return;
      };
      std::function<void(void)> bound = std::bind(action, std::forward<ArgTypes>(args)...);
      return bound;
   };

   std::shared_ptr<R> m_storage;
};

int testFunction(int val)
{
   std::cout << "Was given " << val << std::endl;
   return val + 10;
}

int main()
{
   ConcurrentTask<int(int)> task(20, testFunction, 5);
//   shared_ptr<int> received = task.getReturn();
//   testFunction(*received);
   return 0;
}

And here is my compiler output:

16:31:00 **** Incremental Build of configuration Debug for project TestLinuxMint ****
make all 
Building file: ../src/TestLinuxMint.cpp
Invoking: GCC C++ Compiler
g++ -std=c++0x -O0 -g3 -Wall -pthread -c -fmessage-length=0 -MMD -MP -MF"src/TestLinuxMint.d" -MT"src/TestLinuxMint.o" -o "src/TestLinuxMint.o" "../src/TestLinuxMint.cpp"
../src/TestLinuxMint.cpp: In instantiation of ‘static std::function<void()> ConcurrentTask<R(ArgTypes ...)>::bindTask(const std::function<_Res(_ArgTypes ...)>&, ArgTypes&& ...) [with R = int; ArgTypes = {int}]’:
../src/TestLinuxMint.cpp:58:84:   required from ‘ConcurrentTask<R(ArgTypes ...)>::ConcurrentTask(int, const std::function<_Res(_ArgTypes ...)>&, ArgTypes&& ...) [with R = int; ArgTypes = {int}]’
../src/TestLinuxMint.cpp:91:53:   required from here
../src/TestLinuxMint.cpp:76:90: error: conversion from ‘std::_Bind_helper<false, ConcurrentTask<R(ArgTypes ...)>::bindTask(const std::function<_Res(_ArgTypes ...)>&, ArgTypes&& ...) [with R = int; ArgTypes = {int}]::__lambda1&, int>::type {aka std::_Bind<ConcurrentTask<R(ArgTypes ...)>::bindTask(const std::function<_Res(_ArgTypes ...)>&, ArgTypes&& ...) [with R = int; ArgTypes = {int}]::__lambda1(int)>}’ to non-scalar type ‘std::function<void()>’ requested
       std::function<void(void)> bound = std::bind(action, std::forward<ArgTypes>(args)...);
                                                                                          ^
make: *** [src/TestLinuxMint.o] Error 1

16:31:01 Build Finished (took 319ms)

The issue seems to be on line 76, where there is a failed conversion from std::bind(*) to std::function<void(void)>. This code is definitely still under development, but I need to get past this issue to move forward. I've looked at multiple other posts here on SO, but all of them seem to be able to use std::bind on variadic template parameters without issue.

SOLUTION

Here is the final solution (as pertaining to this question) that I came up with thanks to kzraq and this post.

Source:

#include <iostream>
#include <memory>
#include <utility>
#include <vector>
#include <thread>
#include <type_traits>
#include <typeinfo>
#include <tuple>
#include <memory>

//------------------------------------------------------------------------------------------------------------
template <std::size_t... Ints>
struct idx_sequence
{
   using type = idx_sequence;
   using value_type = std::size_t;
   static constexpr std::size_t size() noexcept { return sizeof...(Ints); }
};

//------------------------------------------------------------------------------------------------------------
template <class Sequence1, class Sequence2>
struct _merge_and_renumber;

template <std::size_t... I1, std::size_t... I2>
struct _merge_and_renumber<idx_sequence<I1...>, idx_sequence<I2...> >
   : idx_sequence<I1..., (sizeof...(I1)+I2)...>
{
};

//------------------------------------------------------------------------------------------------------------
template <std::size_t N>
struct make_idx_sequence : _merge_and_renumber<make_idx_sequence<N/2>, make_idx_sequence<N - N/2> >
{
};
template<> struct make_idx_sequence<0> : idx_sequence<> { };
template<> struct make_idx_sequence<1> : idx_sequence<0> { };

//------------------------------------------------------------------------------------------------------------
template<typename Func, typename Tuple, std::size_t... Ints>
auto applyImpl(Func&& f, Tuple&& params, idx_sequence<Ints...>)
   -> decltype(f(std::get<Ints>(std::forward<Tuple>(params))...))
{
    return f(std::get<Ints>(std::forward<Tuple>(params))...);
};

template<typename Func, typename Tuple>
auto apply(Func&& f, Tuple&& params)
   -> decltype(applyImpl(std::forward<Func>(f),
               std::forward<Tuple>(params),
               make_idx_sequence<std::tuple_size<typename std::decay<Tuple>::type>::value>{}))
{
    return applyImpl(std::forward<Func>(f),
                     std::forward<Tuple>(params),
                     make_idx_sequence<std::tuple_size<typename std::decay<Tuple>::type>::value>{});
};

class ConcurrentTaskBase
{
public:
    ConcurrentTaskBase(int priority, const std::function<void()>& task)
        : m_thread(),
          m_active(true)
    {
        auto wrap = [this](int priority, const std::function<void()>& task)
        {
           //Unrelated pthread stuff that I commented out
           sched_param param{priority};

           int err = pthread_setschedparam(pthread_self(), SCHED_RR, &param);
           if (err)
              std::cout << "failed to set new priority: " << err << std::endl;

           task();
        };
        m_thread = std::thread(wrap, priority, task);
    }

    virtual ~ConcurrentTaskBase(void)
    {
        waitForCompletion();
    }

    void waitForCompletion(void)
    {
        if (m_active)
        {
            m_thread.join();
            m_active = false;
        }
    }

private:
    std::thread m_thread;
    bool m_active;
};

template<class R, class... ArgTypes>
class ConcurrentTask;

template<class R, class... ArgTypes>
class ConcurrentTask<R(ArgTypes...)> : public ConcurrentTaskBase
{
public:
    ConcurrentTask(int priority, const std::function<R(ArgTypes...)>& task, ArgTypes&&... args)
    : ConcurrentTaskBase(priority, bindTask(task, std::forward<ArgTypes>(args)...))
    {}

    std::shared_ptr<R> getReturn(void) noexcept
    {
        waitForCompletion();
        return m_storage;
    }

private:
    std::function<void(void)> bindTask(const std::function<R(ArgTypes...)>& task, ArgTypes&&... args)
    {
        auto params = std::make_tuple(args...);
        return [this, task, params](){m_storage = std::make_shared<R>(apply(task, params));};
    };
    std::shared_ptr<R> m_storage;
};

template<class... ArgTypes>
class ConcurrentTask<void(ArgTypes...)> : public ConcurrentTaskBase
{
public:
    ConcurrentTask(int priority, const std::function<void(ArgTypes...)>& task, ArgTypes&&... args)
       : ConcurrentTaskBase(priority, bindTask(task, std::forward<ArgTypes>(args)...))
    {}

private:
    std::function<void(void)> bindTask(const std::function<void(ArgTypes...)>& task, ArgTypes&&... args)
    {
        auto params = std::make_tuple(args...);
        return [this, task, params](){apply(task, params);};
    };
};

// Example stuff
struct MyStruct
{
    int x;
    int y;
};
int testFunction(MyStruct val)
{
    std::cout << "X is " << val.x << " Y is " << val.y << std::endl;
    return val.x + 10;
}

void printMe(int x)
{
    std::cout << "Printing " << x << std::endl;
}
int main()
{
    ConcurrentTask<int(MyStruct)> task(20, testFunction, {5, -21});
    std::shared_ptr<int> received = task.getReturn();
    std::cout << "Return value is " << *received << std::endl;

    ConcurrentTask<void(int)> voidTask(25, printMe, -123);
    return 0;
}

Answer 1

As a guess, bind presumes it can be called repeatedly (esp when called in an lvalue context!), so does not turn rvalue parameters into rvalue parameters to its bound function as rvalue parameters. Which your code demands. That lambda is not perfect forwarding!

You are also capturing const& std::functions by reference in lambdas, which just invites dangling reference hell. But that is a runtime problem. As a general rule never & capture unless lifetime of lambda and all copies ends in the current scope; definitely don't do it during prototyping even if "certain" it won't be a problem.

I would consider writing a weak version of std::apply and index_sequences and packing the arguments into a tuple then doing your apply to unpack into the target callable. But that is a bias, dunno if ideal.

Answer 2

This is more or less what Yakk wrote about. Maybe I do not understand your idea well enough, but to me it seems that you've overengineered it and you're using std::function too early. Also, ArgTypes&& won't be a list of forwarding/universal references, since they're not deduced in bindTask.

The following compiles successfully on gcc 4.8.2:

Get your own integer_sequence for C++11. Courtesy of Xeo.

Write apply to apply tuple parameters to a function (maybe this could be improved):

template<typename Func, typename Tuple, unsigned int... is>
auto apply_impl(Func&& f, Tuple&& params, seq<is...>)
    // -> decltype(f(std::get<is>(std::forward<Tuple>(params))...)) // C++11 only
{
  using std::get; // enable ADL-lookup for get in C++14
  return f(get<is>(std::forward<Tuple>(params))...);
}

template<typename Func, typename Tuple>
auto apply(Func&& f, Tuple&& params)
    // -> decltype(apply_impl(std::forward<Func>(f), std::forward<Tuple>(params),
    //    GenSeq<std::tuple_size<typename std::decay<Tuple>::type>::value>{}))
    // C++11 only
{
    return apply_impl(std::forward<Func>(f), std::forward<Tuple>(params),
        GenSeq<std::tuple_size<typename std::decay<Tuple>::type>::value>{});
}

Simplify your bindTask (though at this point I'd keep it as a template):

auto params = make_tuple(args...);
std::function<void(void)> bound = [task,params]{ apply(task, params); };
return bound;

In C++14 do [task=std::move(task),params=std::move(params)] to avoid needless copies.