11

I started playing with Boost::Range in order to have a pipeline of lazy transforms in C++. My problem now is how to split a pipeline in smaller parts. Suppose I have:

int main(){
  auto map = boost::adaptors::transformed; // shorten the name
  auto sink = generate(1) | map([](int x){ return 2*x; })
                          | map([](int x){ return x+1; })
                          | map([](int x){ return 3*x; });
  for(auto i : sink)
    std::cout << i << "\n";
}

And I want to replace the first two maps with a magic_transform, i.e.:

int main(){
  auto map = boost::adaptors::transformed; // shorten the name
  auto sink = generate(1) | magic_transform()
                          | map([](int x){ return 3*x; });
  for(auto i : sink)
    std::cout << i << "\n";
}

How would one write magic_transform? I looked up Boost::Range's documentation, but I can't get a good grasp of it.

Addendum: I'm looking to write a class like this:

class magic_transform {
    ... run_pipeline(... input) {
        return input | map([](int x){ return 2*x; })
                     | map([](int x){ return x+1; });
};
Community
  • 1
  • 1
bruno nery
  • 2,022
  • 2
  • 20
  • 31
  • 1
    What is the return type of `generate`? I am worried that your `sink` contains a reference to a temporary that expires at the semicolon. – Mankarse Nov 05 '12 at 23:46
  • `generate`'s return type is `boost::iterator_range`, please check http://liveworkspace.org/code/841508d3b54bed4181d4e9fb6058200f for details. – bruno nery Nov 05 '12 at 23:56
  • Magic transform does not get an input parameter. You wrote it as a nullary function. – Yakk - Adam Nevraumont Nov 06 '12 at 00:35
  • OK, it would be more a class than a function. See above. – bruno nery Nov 06 '12 at 00:39
  • Your needs are still vague. Is it the pipe syntax you want? A transform around chained lamdas gives the functionality and external use of |. Do you actually want | within your magic_transform, and if so why? – Yakk - Adam Nevraumont Nov 06 '12 at 02:36
  • Yes, I want | within my magic_transform - I'm designing a program that consists mostly of a pipeline, and the pipeline can get complex. I want to be able to break it into smaller pipes :) – bruno nery Nov 06 '12 at 15:59

2 Answers2

6

The most difficult problem is figuring out the return type in the code. decltype and lambdas do not mix well (see here), so we have to think of an alternative way:

auto map = boost::adaptors::transformed;

namespace magic_transform
{
   std::function<int(int)> f1 = [](int x){ return 2*x; };
   std::function<int(int)> f2 = [](int x){ return x+1; };
   template <typename Range>
   auto run_pipeline(Range input) -> decltype(input | map(f1) | map(f1))
   {
        return input | map(f1) | map(f2);
   }
}

...
auto sink = magic_transform::run_pipeline(generate(1))
                          | map([](int x){ return 3*x; });

The simple solution is stick the lambdas into std::function so we can use decltype to deduce the return type. I used namespace magic_transform in the example, but you could adapt this code into a class if you would like also. Here is a link adapting your code to the above.

Also, using std::function might be overkill here. Instead you could just declare two normal functions instead (example).

I was also experimenting with boost::any_range, there seems to be some incompatibilities with C+11 lambdas, etc. The closest I could get was the following (example):

auto map = boost::adaptors::transformed;
using range = boost::any_range<
               const int,
               boost::forward_traversal_tag,
               const int&,
               std::ptrdiff_t
               >;

namespace magic_transform
{
    template <typename Range>
    range run_pipeline(Range r)
    {
        return r | map(std::function<int(int)>([](int x){ return 2*x; }))
             | map(std::function<int(int)>([](int x){ return x+1; }));
    }
}

int main(){
  auto sink = magic_transform::run_pipeline(boost::irange(0, 10))
                          | map([](int x){ return 3*x; });
  for(auto i : sink)
    std::cout << i << "\n";
}
Community
  • 1
  • 1
Jesse Good
  • 50,901
  • 14
  • 124
  • 166
  • Before asking here, I tried `decltype` and found it too verbose for this application (you end up writing the pipeline twice). The `any_range`, OTOH, allows you to only specify input and output, correct? – bruno nery Nov 06 '12 at 16:29
  • Now I see, `any_range` specifies the output (value and reference). Interesting. – bruno nery Nov 06 '12 at 17:10
  • Also, you can use `any_range` for the parameter too. E.g.: `range run_pipeline(range r)` defines a pipeline that acts on a range of ints and returns a range of ints. – bruno nery Nov 06 '12 at 17:45
  • @brunonery: Yes, I couldn't get `any_range` to work with your `generate(1)` example though. – Jesse Good Nov 06 '12 at 21:42
  • @brunonery: I found out you need to change the iterator tag to `std::forward_iterator_tag`, [here is a working example](http://liveworkspace.org/code/97445258abe19613c766a921e1adbab1). – Jesse Good Nov 07 '12 at 07:54
1

I think will work:

auto magic_transform()->decltype(boost::adaptors::transformed(std::function<int(int)>())
{
    std::function<int(int)> retval = [](int x){ return [](int x){ return x+1; }(2*x);
    return boost::adaptors::transformed(retval);
}

but it probably isn't what you are looking for. :) (Jokes in the above code: chained lambdas for 2*x+1, use of decltype on basically the implementation to find the return type),

Looking at the source code for http://www.boost.org/doc/libs/1_46_1/boost/range/adaptor/transformed.hpp, the type that magic_transform wants to return is boost::range_detail::transform_holder<T>, where T is the type of the function.

When you do it on the stack with lambdas, T ends up being some very narrow type. If you want to pass around abstract transformations without exposing all of the details, using std::function<outtype(intype)> may be reasonable (there will be a small run-time overhead).

Hope that works.

Yakk - Adam Nevraumont
  • 262,606
  • 27
  • 330
  • 524
  • 1
    Yes, chaining the lambdas wasn't what I was looking for :), please check my update. I could actually use `auto` and `decltype`, but I wanted something more generic. – bruno nery Nov 05 '12 at 23:56
  • So you want some way to pipe (well combine in order) transforms together and have the result be a transform? – Yakk - Adam Nevraumont Nov 06 '12 at 00:37
  • Not only transforms, but eventually filters and other range adaptors. – bruno nery Nov 06 '12 at 00:40
  • Range adaptor's result type is documentated. for example, `transformed` returns boost::transformed_range http://www.boost.org/doc/libs/1_52_0/libs/range/doc/html/range/reference/adaptors/reference/transformed.html – Akira Takahashi Nov 06 '12 at 01:16