0

I'm using variadic templates to capture static type information in Isis2, an atomic multicast library (isis2.codeplex.com). Some Isis2 events are delivered via upcall. For example if you code

Group g("myGroup");
g.Handlers[UDPATE] += [](string& name, Foo& f) { ... your code };
....
g.OrderedSend(UPDATE, "John Doe", new Foo(...));

then on receiving a multicast in group g carrying an update with a string and a Foo object in it, Isis2 would construct a local instance of the Foo object and then upcall to this lambda with the appropriate arguments.

So here's my puzzle. I have the variadic code to scan the arguments to OrderedSend and can capture the static type information needed to build my messages. I end up passing the real OrderedSend method a one-dimensional array of arguments, each with its type, a pointer or safe reference to the data or object, and for an object, the address of a marshaling method. But to use a variadic template to scan the lambda, I need to look at "inner argument list" of the function, in the sense that the object being added to the vector of handlers is a lambda: the type_traits methods will just say that it is an object of type "function". I'm after the string and Foo types, from the argument list of the lambda. But type_traits.h lacks anything for accessing the argument list, as far as I can see.

A GCC-11 specific option is to unmangle the typeid and parse the resulting string. But is there a variadic template feature that would let me get to the argument list of the lambda at compile time?

Ken Birman
  • 1,088
  • 8
  • 25
  • So, jargon. an "upcall" is just a "callback" with extra jargon? What is the type of `g.Handlers[UPDATE]`? Is `Group` your type, or Isis? What is the "real OrderedSend"? There is one OrderedSend in your example code, is that one real or not? Where, in the above code, do you "scan the lambda"? I mean, you have the types you are *calling* the lambda with, why do you think you need the signature of the lambda? Why do you think having (the? an?) argument list for a lambda would help in any way? – Yakk - Adam Nevraumont Jun 16 '15 at 17:39
  • Oh, you are the *author* of Isis2. So I guess the distinction between "your code" and "isis2 code" is a bit arbitrary. But I glanced at the `.h` file for Isis2, and it contains no `Group` type, nor `OrderedSend` method. There does seem to be some in the C# code base, are you trying to write a C++ version of the current C# library, and your code above is an example of what you **want** to work in C++? That does explain the `new Foo` being bound to a `Foo&` strangeness (pointers and references are distinct types, and `string&` won't bind to `"hello"`). What information does `UPDATE` contain? – Yakk - Adam Nevraumont Jun 16 '15 at 17:41
  • So, all messages sent through `UPDATE` must have the same signature, right? And all listeners must be compatible with that signature? Why are you deducing the signature at point of use (both message production and consumption), then causing run-time errors if they don't match, instead of defining the message syntax in one spot (say, where `UPDATE` is defined), and at point of use generating compile-time errors if the sender sends incompatible types, or if the listener registers an incompatible callback? – Yakk - Adam Nevraumont Jun 16 '15 at 17:49
  • @Yakk: just a callback. G.Handlers is an array of std::vector. I declare them as (void*)() because I don't know the types people will actually register. As for OrderedSend, the real method wants a vector of arguments, and their types. I use the variadic temp,ate to generate this (in C# this is what you get with varargs) – Ken Birman Jun 17 '15 at 02:11
  • @Yakk: how did you "look at the .h file for Isis2"? I haven't posted it. There is a C# library, is this what you mean? Yes, I'm translating it to C++ with the goal of dropping it onto an RDMA enabled NIC. The card can't run Mono, so I need to shift from C# to C++. Anyhow, RDMA wants to do memory-memory DMA at optical line speeds, and with managed memory I obviously can't, which also argues for using C++ for this new version... – Ken Birman Jun 17 '15 at 02:17
  • @Yakk: a compile time error would be preferable, but I think impossible without writing my own pre-processor, and I think that would make the system harder to use. I like the simplicity of giving people one file and saying: link against this, or add it to your project. I'm old school; I find that the world has become overly complex. So I'm trying to keep things simple even in C++11. Distributed fault-tolerant secure computing for the masses..... But the exception (on a mismatch) occurs in the sender, when he calls g.OrderedSend but there is no corresponding handler. – Ken Birman Jun 17 '15 at 02:20
  • PS: the French ATC system has run on my old Isis Toolkit, a similar library, for 20 years, and the NYSE ran on it for a decade. Oracle uses it deep in their network manager and has for ages. So those are examples of who uses a library like this. Their code + my code tells the planes where it is safe to fly.... Isis2 is being used initially in a control system for the bulk (high voltage) smart power grid, but the library is a general system. You could use it for anything. – Ken Birman Jun 17 '15 at 02:26

2 Answers2

1
template<class Sig>
struct MessageName {
  std::string name;
  MessageName() = delete;
  MessageName( std::string o ):name(o) {}
  MessageName(MessageName&&)=default;
  MessageName(MessageName const&)=default;
  MessageName& operator=(MessageName&&)=default;
  MessageName& operator=(MessageName const&)=default;
};

// trait to determine if some args are compatible:
template<class Sig, class...Ts>
struct is_compatible : std::false_type {};
template<>
struct is_compatible<void()> : std::true_type {};

template<class A0, class...Args, class T0, class...Ts>
struct is_compatible<void(A0, Args...), T0, Ts...>:
  std::integral_constant<bool,
    std::is_convertible<T0, A0>::value
    && is_compatible< void(Args...), Ts... >::value
  >
{};
struct HandlerMap {
  template<class Sig>
  void add_handler(
    MessageName<Sig> msg,
    block_deduction< std::function<Sig> > handler
  )
  {
    // ...
  }
  template<class Sig, class...Ts>
  typename std::enable_if<is_compatible<Sig, Ts...>::value>::type
  send_message( MessageName<Sig> msg, Ts&&... ts )
  {
    // ...
  }
};

The UPDATE token should be of type MessageName. All MessageNames must claim a signature associated with them.

MessageName< void(std::string const&, Foo const&) > UPDATE{"update"};

like the above.

Then, when you add a handler, the call to add_handler will check the assigned function against the required signature, and give you a std::function.

Similarly, when you send a message, the types passed can be checked against the signature. You should even convert the arguments into each of the signature's argument types in the body of the function.

This moves as much of the type checking as possible to compile time, which is good C++ style.

Yakk - Adam Nevraumont
  • 262,606
  • 27
  • 330
  • 524
0

No, it's impossible. What if the object is not a lambda but a struct with overloads? Or a polylambda? You cannot assume that the function object has one and only one signature- there are lots of ways to get more than one.

Here's a simple example:

struct fun {
    int i;
    void operator()(int x) {}
    void operator()(float x) {} 
};

There's nothing uber-complex or non-POD about this struct or any of it's arguments.

Puppy
  • 144,682
  • 38
  • 256
  • 465
  • Well, in practice I wouldn't be able to handle such cases and would give an error message: anything you can put into a message needs to have a conversion to POD, and a conversion back to the runtime form, and those kinds of uber-complex objects wouldn't support that sort of conversion in and out. Most message libraries have similar limitations (like Google ProtoNetbufs). I'm fine with having a way to say "can't do that" for those cases. But if we're talking about a string or an int or a Foo (and the Foo implements a serialization interface), why shouldn't it be possible to figure that out? – Ken Birman Jun 16 '15 at 14:54
  • There's nothing ubercomplex about the object or it's arguments. They can all still be POD and still be impossible. – Puppy Jun 16 '15 at 14:58
  • Since these systems deliver events by upcall, the object being added to the handler list is an event handler -- a callable method. So it is definitely a lambda. I work by exact match, not a sophisticated runtime polymorphic match, and I do this even in C# where one can be much fancier. So I'm not asking for the ultimate in reflection here. Just a form of very limited reflection with exact type matches... – Ken Birman Jun 16 '15 at 14:59
  • @KenBirman: You cannot know that it's a lambda... a callable method may be any struct with the appropriate operator() overload. There is *no* semantic difference between a lambda and an arbitrary callable object. They are completely indistinguishable. Lambdas are just syntactic sugar and have no special semantics at all. – Puppy Jun 16 '15 at 14:59
  • @Puppy Documentation for a method may state that only lambdas are supported, not regular functions or other callable objects, even if they aren't specifically rejected and not even detectable in general. But polymorphic lambdas, as you mention in your answer, are a good point. –  Jun 16 '15 at 15:01
  • OK, I grant your point. But now that we agree that the most general case won't be solvable, are you saying that the simplest ones aren't either? I don't see any real reason that, in principle, one couldn't get argument type information at compile time and loop over it... – Ken Birman Jun 16 '15 at 15:07
  • @hvd: You can document that, but it's thoroughly pointless to do so since it's both unenforcable and *utterly meaningless*. – Puppy Jun 16 '15 at 15:08
  • @KenBirman: You can solve it for some simple cases... but I don't recall how offhand. Also, the typeid does not contain this information, it only contains the mangled *name* of the type, which does not include any member functions that may or may not be on it (like operator()). – Puppy Jun 16 '15 at 15:09
  • In what sense is it utterly meaningless? I'm trying to support the rough equivalent of a mouse-click upcall for a GUI. For my purposes, something limited can be very powerful. An Isis2 user writes 1000's of lines of code and maybe 20 of them are concerned with sending multicasts and receiving them. Those 20 can be limited and the result isn't meaningless at all! They end up with a fault-tolerant replicated object with strong replication semantics... I would argue that this is very meaningful. – Ken Birman Jun 16 '15 at 15:11
  • @Puppy: type_traits lets me get all sorts of information at compile time. That's what I'm after: compile time type information... (I don't want C++ to behave like C#... your answer is kind of a C# reflection answer...) – Ken Birman Jun 16 '15 at 15:12
  • @Puppy: for your example, they would register each different event handler (hence the handlers would need some kind of name with which to distinguish them). – Ken Birman Jun 16 '15 at 15:14
  • @KenBirman: There is no meaningful difference between a lambda and any random callable object. That is literally how they are defined. They are some callable object. That's what it says in the specification for how lambdas behave. It's meaningless to only accept lambdas because there is no semantic difference between them and any other callable object. – Puppy Jun 17 '15 at 11:32
  • @Puppy, yes, understood. I agree. My point is just that I don't actually need to support dynamic reflection on a list of callable methods in such an object; I can limit each such object to having just one callable method. But in fact a Cornell PhD student just showed me yesterday that one actually CAN reflect both on the compile-time list of callable methods, and on their argument lists, using fancier variadic template constructions. I'll leave yours marked as an answer but as far as I can tell from what Matthew showed me (plus from Yakk's answer), there are at least two ways to do it! – Ken Birman Jun 17 '15 at 13:31