Metaprogramming in C++ and in D

Question

The template mechanism in C++ only accidentally became useful for template metaprogramming. On the other hand, D's was designed specifically to facilitate this. And apparently it's even easier to understand (or so I've heard).

I've no experience with D, but I'm curious, what is it that you can do in D and you cannot in C++, when it comes to template metaprogramming?

Answer 1

The two biggest things that help template metaprogramming in D are template constraints and static if - both of which C++ could theoretically add and which would benefit it greatly.

Template constraints allow you to put a condition on a template that must be true for the template to be able to be instantiated. For instance, this is the signature of one of std.algorithm.find's overloads:

R find(alias pred = "a == b", R, E)(R haystack, E needle)
    if (isInputRange!R &&
        is(typeof(binaryFun!pred(haystack.front, needle)) : bool))

In order for this templated function to be able to be instantiated, the type R must be an input range as defined by std.range.isInputRange (so isInputRange!R must be true), and the given predicate needs to be a binary function which compiles with the given arguments and returns a type which is implicitly convertible to bool. If the result of the condition in the template constraint is false, then the template won't compile. Not only does this protect you from the nasty template errors that you get in C++ when templates won't compile with their given arguments, but it makes it so that you can overload templates based on their template constraints. For instance, there's another overload of find which is

R1 find(alias pred = "a == b", R1, R2)(R1 haystack, R2 needle)
if (isForwardRange!R1 && isForwardRange!R2
        && is(typeof(binaryFun!pred(haystack.front, needle.front)) : bool)
        && !isRandomAccessRange!R1)

It takes exactly the same arguments, but its constraint is different. So, different types work with different overloads of the same templated function, and the best implementation of find can be used for each type. There's no way to do that sort of thing cleanly in C++. With a bit of familiarity with the functions and templates used in your typical template constraint, template constraints in D are fairly easy to read, whereas you need some very complicated template metaprogramming in C++ to even attempt something like this, which your average programmer is not going to be able to understand, let alone actually do on their own. Boost is a prime example of this. It does some amazing stuff, but it's incredibly complicated.

static if improves the situation even further. Just like with template constraints, any condition which can be evaluated at compile time can be used with it. e.g.

static if(isIntegral!T)
{
    //...
}
else static if(isFloatingPoint!T)
{
    //...
}
else static if(isSomeString!T)
{
    //...
}
else static if(isDynamicArray!T)
{
    //...
}
else
{
    //...
}

Which branch is compiled in depends on which condition first evaluates to true. So, within a template, you can specialize pieces of its implementation based on the types that the template was instantiated with - or based on anything else which can be evaluated at compile time. For instance, core.time uses

static if(is(typeof(clock_gettime)))

to compile code differently based on whether the system provides clock_gettime or not (if clock_gettime is there, it uses it, otherwise it uses gettimeofday).

Probably the most stark example that I've seen where D improves on templates is with a problem which my team at work ran into in C++. We needed to instantiate a template differently based on whether the type it was given was derived from a particular base class or not. We ended up using a solution based on this stack overflow question. It works, but it's fairly complicated for just testing whether one type is derived from another.

In D, however, all you have to do is use the : operator. e.g.

auto func(T : U)(T val) {...}

If T is implicitly convertible to U (as it would be if T were derived from U), then func will compile, whereas if T isn't implicitly convertible to U, then it won't. That simple improvement makes even basic template specializations much more powerful (even without template constraints or static if).

Personally, I rarely use templates in C++ other than with containers and the occasional function in <algorithm>, because they're so much of a pain to use. They result in ugly errors and are very hard to do anything fancy with. To do anything even a little bit complicated, you need to be very skilled with templates and template metaprogramming. With templates in D though, it's so easy that I use them all the time. The errors are much easier to understand and deal with (though they're still worse than errors typically are with non-templated functions), and I don't have to figure out how to force the language into doing what I want with fancy metaprogramming.

There's no reason that C++ couldn't gain much of these abilities that D has (C++ concepts would help if they ever get those sorted out), but until they add basic conditional compilation with constructs similar to template constraints and static if to C++, C++ templates just won't be able to compare with D templates in terms of ease of use and power.

Answer 2

I believe nothing is better qualified to show the incredible power (TM) of the D template system than this renderer I found years ago:

Yes! This is actually what is generated by the compiler ... it is the "program", and quite a colourful one, indeed.

Edit

The source seems to be back online.

Answer 3

The best examples of D metaprogramming are D standard library modules that make heavy use of it vs. C++ Boost and STL modules. Check out D's std.range, std.algorithm, std.functional and std.parallelism. None of these would be easy to implement in C++, at least with the kind of clean, expressive API that the D modules have.

The best way to learn D metaprogramming, IMHO, is by these kinds of examples. I learned largely by reading the code to std.algorithm and std.range, which were written by Andrei Alexandrescu (a C++ template metaprogramming guru who has become heavily involved with D). I then used what I learned and contributed the std.parallelism module.

Also note that D has compile time function evaluation (CTFE) which is similar to C++1x's constexpr but much more general in that a large and growing subset of functions that can be evaluated at runtime can be evaluated unmodified at compile time. This is useful for compile-time code generation, and the generated code can be compiled using string mixins.

Answer 4

Well in D you can easily impose static constraints on template parameters and write code depending on the actual template argument with static if.
It's possible to simulate that for simple cases with C++ by using template specialization and other tricks (see boost) but it's a PITA and very limited cause the compiler doesn't expose many details about types.

One thing C++ really just can't do is sophisticated compile time code generation.

Answer 5

Here's a piece of D code that does a custom-made map() which returns its results by reference.

It creates two arrays of length 4, maps each corresponding pair of elements to the element with the minimum value, and multiplies it by 50, and stores the result back into the original array.

Some important features to note are the following:

• The templates are variadic: map() could take any number of arguments.

• The code is (relatively) short! The Mapper structure, which is the core logic, is only 15 lines -- and yet it can do so much with so little. My point isn't that this is impossible in C++, but that certainly isn't as compact and clean.

---

import std.metastrings, std.typetuple, std.range, std.stdio;

void main() {
    auto arr1 = [1, 10, 5, 6], arr2 = [3, 9, 80, 4];

    foreach (ref m; map!min(arr1, arr2)[1 .. 3])
        m *= 50;

    writeln(arr1, arr2); // Voila! You get:  [1, 10, 250, 6][3, 450, 80, 4]
}

auto ref min(T...)(ref T values) {
    auto p = &values[0];
    foreach (i, v; values)
        if (v < *p)
            p = &values[i];
    return *p;
}

Mapper!(F, T) map(alias F, T...)(T args) { return Mapper!(F, T)(args); }

struct Mapper(alias F, T...) {
    T src;  // It's a tuple!

    @property bool empty() { return src[0].empty; }

    @property auto ref front() {
        immutable sources = FormatIota!(q{src[%s].front}, T.length);
        return mixin(Format!(q{F(%s)}, sources));
    }

    void popFront() { foreach (i, x; src) { src[i].popFront(); } }

    auto opSlice(size_t a, size_t b) {
        immutable sliced = FormatIota!(q{src[%s][a .. b]}, T.length);
        return mixin(Format!(q{map!F(%s)}, sliced));
    }
}


// All this does is go through the numbers [0, len),
// and return string 'f' formatted with each integer, all joined with commas
template FormatIota(string f, int len, int i = 0) {
    static if (i + 1 < len)
        enum FormatIota = Format!(f, i) ~ ", " ~ FormatIota!(f, len, i + 1);
    else
        enum FormatIota = Format!(f, i);
}

Answer 6

I wrote up my experiences with D's templates, string mixins, and template mixins: http://david.rothlis.net/d/templates/

It should give you a flavour of what is possible in D -- I don't think that in C++ you can access an identifier as a string, transform that string at compile time, and generate code from the manipulated string.

My conclusion: Extremely flexible, extremely powerful, and usable by mere mortals, but the reference compiler is still somewhat buggy when it comes to the more advanced compile-time metaprogramming stuff.

Answer 7

String manipulation, even string parsing.

This is a MP library that generates recursive decent parsers based on grammars defined in strings using (more or less) BNF. I haven't touched it in years but it used to work.

Answer 8

in D you can check the size of a type and the available methods on it and decide which implementation you want to use

this is used for example in the core.atomic module

bool cas(T,V1,V2)( shared(T)* here, const V1 ifThis, const V2 writeThis ){
    static if(T.sizeof == byte.sizeof){
       //do 1 byte CaS
    }else static if(T.sizeof == short.sizeof){
       //do 2 byte CaS
    }else static if( T.sizeof == int.sizeof ){
       //do 4 byte CaS
    }else static if( T.sizeof == long.sizeof ){
       //do 8 byte CaS
    }else static assert(false);
}

Answer 9

Just to counter the D ray tracing post, here is a C++ compile time ray tracer (metatrace):

(by the way, it uses mostly C++2003 metaprogramming; it would be more readable with the new constexprs)

Answer 10

There are quiet a few things you can do in template metaprogramming in D that you cannot do in C++. The most important thing is that you can do template metaprogramming WITHOUT SO MUCH OF A PAIN!