35

I've been reviewing the draft version of the C++11 standard. Specifically the section on lambdas, and I am confused as to the reasoning for not introducing polymorphic lambdas.

For example, amongst the 100001 ways polymorphic lambdas could be used, I had hoped we could use code such as the following:

template<typename Container>
void foo(Container c)
{
    for_each(c.begin(), c.end(), [](T& t) { ++t; });
}

What were the reasons:

  • Was it that the committee ran out of time?

  • That polymorphic lambdas are too hard to implement?

  • Or perhaps that they are seen as not being needed by the PTB?

Note: Please remember the example above is not the only one, and it is only provided as a guide to the types of code. Answers that solely concentrate on providing a workaround for the above piece of code will not be considered as valid!

Related sources:

Community
  • 1
  • 1
  • 11
    damn, what a messed up syntax. – Femaref Jan 10 '11 at 01:38
  • 23
    whats wrong with the syntax? its actually quite nice. –  Jan 10 '11 at 01:46
  • 4
    @Femaref: I agree. Although, each of the three sets of parentheses are actually there for a reason: the first (`[]`) contains the list of variables to close over and/or which ones *not* to close over as well as *how* to capture them (by-reference or by-value), the second (`()`) contains the parameter list, the third (`{}`) contains the body. But, for example, something like `λt → ++t` (as in Haskell) instead of `[](T& t) { ++t; }` would be *much* nicer. – Jörg W Mittag Jan 10 '11 at 01:50
  • 3
    @Jorg: The term is not "close over" it is called the capture list. Essentially the ability to capture variables either be reference or copy from the enclosing scope, to be used in the lambda. - I personally think the syntax is quite nice. –  Jan 10 '11 at 01:52
  • 7
    @Dominar That's what "close over" means. http://en.wikipedia.org/wiki/Closure_(computer_programming) – etarion Jan 10 '11 at 01:54
  • 3
    Jörg: I don't disagree that each of those characters have a reason - I think it's just bloaty, but that's probably caused by the fact that it's trying to patch an already moth-ridden dress which wasn't looking good to begin with. – Femaref Jan 10 '11 at 01:57
  • @etarion: Well, usually you say "close over the environment". Most languages do not allow you to explicitly specify *which parts* of the environment and *how* to "close over", they always close over the *entire* environment, so there's no fully established term for what tho call it when you "close over" or "capture" *individual variables* as opposed to the entire environment. – Jörg W Mittag Jan 10 '11 at 01:59
  • 1
    I hate the C++0x's lambda syntax as well, but the syntax is at least consistent with the style of C/C++. Haskell's lambda is much nicer, but it would look rather odd. – kirakun Jan 10 '11 at 02:12
  • 5
    @Kirakun: It would be an interesting experiment to remove everything that has been made redundant by later extensions (e.g. remove all forms of initialization except the uniform initialization syntax), keep the *abstract* syntax for that non-redundant subset of C++ identical to what it is today, but design a new *concrete* syntax more along the lines of Scala and/or Cobra and/or Ruby (depending on whether you prefer braces, indentation or keywords). I bet you can get some rather nice looking language that is 100% isomorphic to C++. – Jörg W Mittag Jan 10 '11 at 02:25
  • 5
    The thing about the C++ syntax for lambdas is that it's pretty much unavoidable. In a GC'ed language you can get away with a regular closure that just captures everything by reference. In C++, you pretty much need to be able to specify whether a variable should be captured by reference or value. If it wasn't for that requirement, the `[]`'s could easily be eliminated. – jalf Jan 10 '11 at 02:29
  • 4
    I seem to recall that litb once mentioned that lambdas were required to be monomorphic because otherwise they didn't play nice with Concepts. (which seems like a surprisingly silly argument, and is just extra ironic now that concepts have been dropped) Maybe he can enlighten us on the details when he sees this question though. – jalf Jan 10 '11 at 02:36
  • 1
    @Dominar: now that you've linked [Can lambda functions be templated?](http://stackoverflow.com/questions/3575901/can-lambda-functions-be-templated), what does your question add that isn't already answered there? – Ken Bloom Jan 10 '11 at 03:03
  • 11
    Meh. I can live without it. `[](decltype(*begin) t) { ++t; }` – Jon Purdy Jan 10 '11 at 03:11
  • 2
    @jalf, now that concepts have been dropped, it may be that they didn't add template lambdas because they want to be able to add concepts back into the language next time around. – Ken Bloom Jan 10 '11 at 04:29
  • 1
    There are exactly 33 ways polymorphic lambdas can be used? – loneboat Jan 10 '11 at 04:30
  • @Ken: maybe. (Or just that they dropped concepts to save time, and so it'd be silly to spend additional time re-adding features like this once concepts were out) But it seems to me that the notion of concepts is crippled if it can't express polymorphic lambdas. Perhaps, they should instead have focused on coming up with more expressive concepts. It seems like they were just too infatuated with concepts and unwilling to see that if it can't express popular language features, then it's not the popular language feature that needs to be fixed, but the concepts specification. – jalf Jan 10 '11 at 10:05
  • then again, this is based on a half-remembered comment/answer by litb I read on SO months ago. So take it for what it's worth. ;) – jalf Jan 10 '11 at 10:05

5 Answers5

51

The reason we don't have polymorphic lambdas is explained pretty well in this posting.

It has to do with the concepts feature that was pulled from C++11: essentially, polymorphic lambdas are ordinary, unconstrained function templates and we didn't know how to typecheck a concept-constrained template that used an unconstrained template. However, solving that problem turns out to be easy as shown here(dead link), so I don't think there's any obstacle remaining.

The link to cpp-next is dead; the relevant info can be found here

Community
  • 1
  • 1
Dave Abrahams
  • 7,416
  • 5
  • 31
  • 19
  • 2
    Note that our [proposal](http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2012/n3418.pdf) for polymorphic lambdas was looked upon favorably by the Evolution Working Group in Portland, so if we refine the proposal according to those comments, I think we'll see the feature in C++2014. – Dave Abrahams Nov 17 '12 at 02:58
  • 1
    The second link is dead. – senbrow Nov 05 '14 at 22:15
16

Since the argument, c, meets the STL requirements for a container, you should be able to use something like

template<typename Container>
void foo(Container c)
{
    for_each(c.begin(), c.end(),[](typename Container::reference t) { ++t; });
}

I'll also showcase John Purdy's comment above, which is another way to get the typename you want in this lambda:

template<typename Container>
void foo(Container c)
{
   for_each(c.begin(),c.end(),[](decltype(*c.begin()) t) { ++t; });
}

(Yes, Dominar, I know you don't like this answer, because it doesn't answer your question, but I'm willing to bet that the next person who comes along asking this question is going to be looking for a way to make their code work, so it does make sense to have some techniques around where the question is relevant.)

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
Ken Bloom
  • 57,498
  • 14
  • 111
  • 168
  • 7
    Ken: Not sure what you're on about as the compilers already do something very similar: http://codepad.org/BoaD4Mhi –  Jan 10 '11 at 01:45
  • 13
    Who are the people up-voting this answer? This is NOT a correct answer, and will NOT be chosen as the correct answer. –  Jan 10 '11 at 01:46
  • 1
    @Dominar: You're right. I think my mind has been corrupted by Scala generics or something. I can't quite figure out the mental gymnastics to understand what the C++ compiler does to get that right. – Ken Bloom Jan 10 '11 at 01:48
  • 1
    @Dominar: I've removed the part of the answer that was wrong. Someone else will have to explain the theory behind the design. (I think they're up-voting me because the answer is a practical way to make your code work.) – Ken Bloom Jan 10 '11 at 01:49
  • 5
    @Ken: In fact it can, the ground work for such type deduction was set back in the 03 standard. Please remove your answer as I'm only looking for correct answers, I'd hate to send people on a wild-goose-red-herring chase :D –  Jan 10 '11 at 01:50
  • @sepp2k: I think his complaint is that C++ doesn't give you syntax to introduce the T in a lambda. – Ken Bloom Jan 10 '11 at 01:50
7

It's probably because there already is a syntax for doing that, and the purpose of lambdas is to introduce a much simpler syntax that covers most cases. When you try to cover all cases (what if you wanted the auto-generated functor to inherit a particular base class?), you lose the comparative advantages (simplicity and terseness) of the lambda.

I really don't like the proposed syntax. Is T a keyword? Do all identifiers for which name lookup fails get turned automatically into template typename arguments? That prevents you from detecting misspellings, which IMO is a BAD idea:

for_each(c.begin(),c.end(),[](iterater& t) { ++t; });
// programmer misspelled "iterator" and now has a polymorphic lambda, oops

It also introduces action-at-a-distance behavior, if the named type get introduced in some header file somewhere, the meaning changes suddenly. Also really BAD.

Well, since it's supposed to create a template, we could borrow the existing syntax:

for_each(c.begin(),c.end(),[]template<typename T>(T& t) { ++t; });

This is unambiguous and now allows non-type template arguments (useful for accepting arrays by reference), but is really unwieldy. At this point you're better off writing out the functor by hand, it'll be much easier to understand.

However, I think a simple syntax is possible using the auto keyword:

for_each(c.begin(),c.end(),[](auto& t) { ++t; });

This next section incorrectly assumes that the template parameter appears on the functor type rather than its operator()():

But now you have a problem that for_each infers a typename template argument, not a template template argument. Type inference isn't possible in that context.

In the current proposal, lambdas have type, even if it's an unmentionable (other than decltype) type. You'd have to lose that feature in order to accommodate inference at the call-site.

Example showing that the issue is NOT a shortcoming of lambdas, it's simply a non-deducible context:

#include <vector>
#include <algorithm>
#include <iterator>

int main(void)
{
    using namespace std;
    vector<int> a(10);
    vector<int> b(10);
    vector<int> results;

    transform(a.begin(), a.end(), b.begin(), back_inserter(results), min<int>);
}

The template type parameter to std::min must be explicitly specified. Lambdas are no different from using existing functors in this regard.

EDIT: Ok, now that I realize we aren't suggesting that the lambda generate a template functor type, but a single non-template functor type which implements a templated function application operator (operator()()), I agree that the compiler should be able to generate such a thing. I propose that using the auto keyword here would be a good simple syntax for requesting that.

However, I'm not really happy with auto either. What about lambdas with multiple parameters:

[](auto& x, auto& y){ return x + y; }
//becomes
template<typename T1, typename T2>
auto operator()(T1& x, T2& y) -> decltype(x + y) { return x + y; }

Ok, that works well enough, but what if we wanted two parameters but only one type argument:

[](auto& x, decltype(x)& y){ return x + y; }
//becomes
template<typename T1>
auto operator()(T1& x, T1& y) -> decltype(x + y) { return x + y; }

Seems ok, but I find the syntax misleading. The syntax suggests that the type parameter is inferred from the first actual parameter, and the second parameter is coerced to the same type, but actually both actual parameters are considered equal during type inference.

Perhaps it's best that this case be limited to one lambda parameter per type argument, and if you want something more constrained, write the functor yourself. This seems to me to be a good compromise between flexibility and power vs keeping the syntax simple.

Ben Voigt
  • 277,958
  • 43
  • 419
  • 720
  • 1
    Interesting conclusions, but I don't think they are generally valid please read the following: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2002/n1375.html As I mentioned at the bottom of the question, the example is just an example. but +1 for being a far better answer than whats already there. –  Jan 10 '11 at 02:36
  • So just to clarify, the `(auto& t)` syntax doesn't actually work, but you think that the C++ standards committee should have made it work because it captures this really reasonable use case without the lambda syntax getting too bad. – Ken Bloom Jan 10 '11 at 02:36
  • @Ken: No, I think `auto` shows that we wouldn't have to sacrifice simple syntax, but then I go on to argue that we would have to sacrifice function arguments having type (current you can have a template argument which is itself a template, but you can't have a function argument of that type, you have to make a concrete type by supplying a template parameter list to the template template argument). And my fingers are getting tired of typing the word "template". – Ben Voigt Jan 10 '11 at 02:45
  • @Ben: look at my second answer [below](http://stackoverflow.com/questions/4643039/c0x-and-the-lack-of-polymorphic-lambdas-why/4643298#4643298) for a way that this could be made to work. You're arguing exactly the part I deleted from my [first](http://stackoverflow.com/questions/4643039/c0x-and-the-lack-of-polymorphic-lambdas-why/4643063#4643063) answer, which Dominar demonstrated was wrong. – Ken Bloom Jan 10 '11 at 02:48
  • @Dominar: That's dynamic polymorphism, which I actually did address in my answer (second sentence: "what if you wanted the auto-generated functor to inherit a particular base class?"). Your entire question here is about static polymorphism, which is totally incompatible. Brandishing a proposal for dynamic polymorphism does nothing to substantiate your claim that static polymorphism is feasible, and it certainly say anything about the particular context you chose, which is in fact non-deducible, as you can easily prove by trying to pass a functor template. – Ben Voigt Jan 10 '11 at 02:50
  • 3
    If they had something in the standard library like: `template using id = T;` then you could do `[](auto& x, std::id& y)` to stop deduction. I think it's still viable, just needs more utility functionality. Roger Pate and I discussed this a while ago before he left. With the new language, you could actually get rid of explicit template syntax, and just use `auto` in the parameter type. (Any function that did so had an implicit `template `.) It would simplify templates greatly. – GManNickG Jan 10 '11 at 16:45
  • 1
    @GMan: Wouldn't that actually require something like `std::id`? Getting ugly, but perhaps necessary. And I don't think that `auto` could replace explicit template notation in the general case, but it sure would be a nice shorthand to simplify writing a significant fraction of template functions. – Ben Voigt Jan 10 '11 at 18:40
  • 1
    Why not just add more brackets? `[](T& x, T& y){x++; y--;}` – Ken Bloom Jan 11 '11 at 14:11
  • @DaveAbrahams: I would love to see a syntax for polymorphic lambdas that solves the issues I pointed out in my answer. Any ideas? – Ben Voigt Dec 21 '11 at 19:33
  • @BenVoigt: I think your answer got it right when it said you can write something more verbose if you really need more constraint. Lambdas are only as useful as they are terse. Still can't beat Boost.Lambda/Boost.Phoenix syntax for most jobs. – Dave Abrahams Dec 28 '11 at 20:42
3

Well, now that you've linked n1968, the answer to your question is apparent. It's found in section 5.1 of the proposal.

Ben Voigt
  • 277,958
  • 43
  • 419
  • 720
  • 3
    True. But jeez, I can't say I agree with the reasoning. I'm starting to wonder if the addition of concepts is at all desirable. It was supposed to improve template error messages, not prevent implementation of intuitive and useful language features. – jalf Jan 10 '11 at 10:16
  • 2
    @jalf: I'm pretty sure that's why they died. Concepts were enormously complex, it would be like learning a second language on top of the first. – GManNickG Jan 10 '11 at 16:08
  • @GMan: But... I thought the official word was that they aren't actually dead, they just missed the deadline. Although the originally proposed syntax may very well be dead. – Ben Voigt Jan 10 '11 at 16:11
  • @Ben: Sorry, ambiguous wording on my part. "died" meant "didn't make it [into the new standard]". – GManNickG Jan 10 '11 at 16:42
  • @GMan: Phew, that was a close one. Now I still have new C++ features to look forward to in another decade. – Ben Voigt Jan 10 '11 at 18:11
  • 1
    Yeah, but I'm starting to wonder if they're dead *enough*. The more I learn about the concepts proposal, the more I feel it was just misguided. Too big and too ambitious, sure, but also compromising a lot of valuable aspects of the C++ language, possibly making generic code *harder* to write. So if and when Concepts get resurrected, I'm hoping they'll take a big step back and really reconsider what they're trying to achieve. And @Ben, I think they've said they're aiming for a 5-year'ish schedule going forward, so you might get new features in less than a decade. ;) – jalf Jan 10 '11 at 23:53
  • I think it's interesting that concepts started out as just "a way to optionally clean up template error messages", and then became a way to enable additional control over overloading, an even an attempt at making C++'s type system compete with Haskell for strictness and theoretical completeness. – jalf Jan 10 '11 at 23:56
  • Bjarne Stroustrup discusses the state of conecpts in an interview (featured on Slashdot today) at [CodeGuru](http://www.codeguru.com/cpp/misc/article.php/c18357__2/An-Interview-with-C-Creator-Bjarne-Stroustrup.htm). He says "A minor adjustment to what we had for C++0x is not enough". – Ken Bloom Jan 11 '11 at 21:34
  • @jalf: concepts were never just "a way to optionally clean up template error messages," even if that's how they were sold to the public initially. – Dave Abrahams Dec 28 '11 at 20:43
-2

The following (your comment to my other answer above) works:

#include <algorithm>
#include <vector>

struct foo
{
   template<typename T>
   void operator()(T& t)
   {
      ++t;
   }
};

int main()
{

   std::vector<int> v;
   std::for_each(v.begin (),v.end(),foo());

   return 0;
}

But the following does not:

#include <algorithm>
#include <vector>

template<typename T>
struct foo
{
   void operator()(T& t)
   {
      ++t;
   }
};

int main()
{

   std::vector<int> v;
   std::for_each(v.begin (),v.end(),foo()); // <-- the syntax for foo here 
                                            //     is kinda fictitious

   return 0;
}

Probably the C++ committee saw lambdas as being more similar to the second example than the first. (Though I haven't figured out clever way to define a lambda in which this would make a difference. Anyone got any crazy ideas?)

Ken Bloom
  • 57,498
  • 14
  • 111
  • 168
  • @Dominic: You just commented with a link to another question, where'd it go? @Ken: Yes, case #2 not working is exactly what I'm getting at. I hadn't considered that `operator()` would be templated rather than the functor type. Yeah, I guess that could work. – Ben Voigt Jan 10 '11 at 02:53
  • Here's the question he referred to http://stackoverflow.com/questions/3083498/problem-using-templated-predicate-in-stl-algorithms (it's not an exact duplicate of this question, but it is related). – Ken Bloom Jan 10 '11 at 02:54
  • 9
    @Ken: becuase foo() is an inline instantiation, you need to specialize it at instantiation - there is nothing crazy about it, if you did foo() it would work. http://codepad.org/VtLmqNlW –  Jan 10 '11 at 02:54
  • 2
    @Ben: I deleted it because it wasn't exactly like the code Ken had, I thought something that was exactly like what Ken had would be better as he seems to only understand very narrow/strict definitions of problems. –  Jan 10 '11 at 02:56
  • Dominar: but the compiler can't figure out the `` part on it's own, and it needs you to tell it that part. (Hence, to define a comparable lambda you'd have to do what I said in my [first answer](http://stackoverflow.com/questions/4643039/c0x-and-the-lack-of-polymorphic-lambdas-why/4643063#4643063)) – Ken Bloom Jan 10 '11 at 02:57
  • @Ken: DO NOT change the question! if you do it again, I will report you to the moderators. –  Jan 10 '11 at 02:58
  • @Ken: Thats fine, that has nothing to do with the question. In short none of your answers so far have ANYTHING to do with the question. –  Jan 10 '11 at 02:58
  • @Dominar: Chill out. And if you haven't read the site FAQ yet, please read it. In short, the moderators really don't care if I edit your question, so long as I'm not vandalizing it or perverting its intent. They envision the site as being kinda like wikipedia, and if someone can your question better and clearer, then more power to them. http://stackoverflow.com/faq – Ken Bloom Jan 10 '11 at 03:09
  • 7
    @Ken: That's fine for people who understand and know what the question is about, you have proven that you have grasped neither. So it makes sense for you not to edit, and as to why I made the original comment, by using std::iterator_traits you can deduce type via the value_type, so your change really added nothing and infact added even more confusion. The best thing you could have done from the start was to delete your answer, stay quite follow the question and let others educate you. –  Jan 10 '11 at 03:39
  • @Dominar: we can see the edit - don't need to be a moderator for that - and Ken Bloom didn't change your answer. He just pointed out the bit that isn't actually C++. As for understanding the question, your answers don't convince me that you do. Lambda's are anonymous functions, not objects. – MSalters Jan 10 '11 at 11:05
  • 1
    @MSalters: The edit being discussed is on the question not the answer. And lambdas most definitely are syntactic sugar for objects implementing `operator()()`. – Ben Voigt Jan 10 '11 at 15:04
  • 4
    @MSalters: I think you should learn to read the question, just out of curiosity, you wouldn't happen to be one of the puppets voting on Ken Blooms answers? for someone with 27k points, you don't seem to be clued in on the subject matter. –  Jan 11 '11 at 06:23