Dynamically creating function parameters that are perfectly forwarded in C++

Question

I have a scenario that poses a problem for which I haven't found a good solution as of yet.

In essence, I need to pass arguments to a C++ variadic templated function. That function has the following generalized signature/form:

template <typename... Options>
ReaderType GetAList(std::string const& name, Options&&... options)
{ /*... do stuff ...*/ }

I have read several StackOverflow Q&As regarding perfect forwarding, variadic templates, dynamic parameter lists, etc. Specifically these:

C++ generic function call with varargs parameter

Dynamically creating a C++ function argument list at runtime

Create function call dynamically in C++

C++ forwarding reference and r-value reference

And those are but a handful of the StackOverflow threads (if you will) that I've read regarding the subject. I also found an interesting article on the LinkedIn website regarding variadic templates and functions:

Modern C++ and Variadic Functions: How to Shoot Yourself in the Foot and How to Avoid It

The point of mentioning this is that I've attempted to do the homework, but it hasn't yielded fruit just yet. Thus I decided to ask a question.

The issue at work here is that the options aren't known until runtime. Thus I must dynamically add parameters to the GetAList method invocation. In the above examples, the parameters were known at compile-time, such as these:

interceptor(gWrappers[1], 10.12, 12.10); // <--- example 1: params are given statically at compile time
func_map["printall"]("iic",5,10,'x');    // <--- example 2: same approach

That won't work in my scenario, because I just don't know which options the user will select from those that are available. So I must take a look at which options they want to use and form a parameter list dynamically.

In other examples of dynamic parameter generation (which were good examples, I mean, I did learn something from reading those threads), the lynch-pin was centered around the parameter type. That is, an intermediary method was being used in which you pass a function that you want to invoke as the first parameter, then a list or a vector of the arguments. Here is a quick snippet of that:

template <typename Ret, typename...Arg>
Ret call (Ret (*func)(Arg...), std::list<boost::any> args)
{ /*...do stuff... */ }

int foo(int x, double y, const std::string& z, std::string& w)
{ /*...more stuff...*/ }

int main ()
{
    std::list<boost::any> args;
    args.push_back(1);
    args.push_back(4.56);
    const std::string yyy("abc");
    std::string zzz("123");
    args.push_back(std::cref(yyy));
    args.push_back(std::ref(zzz));
    call(foo, args);
}

Ultimately with this type of approach, the parameter argument types are deduced via the non-templated function definition (which is the function foo in the above example). Cool.

However, I can't do that, because the second parameter of the GetAList method is a parameter pack containing forward references, which are r-values. I did peek into the source for these methods (such as GetAList) and the options values are forwarded further down into their implementation (via std::forward).

I did consider using intermediate methods that would have a matching parameter list to all of the possible permutations of the available options for a given method, such as GetAList. Those intermediate methods would then invoke the corresponding method, such as GetAList. That would work in the above model.

However, the number of matching methods would climb rather steeply with an increase in the number of available options for a given method. That is, you could end up with 25, 100 or more methods to cover all options permutations. That would be almost as ugly as an attempt to solve this with macro magic.

And so, I am attempting to avoid the entire "if these parameters then call this method, if those parameters then call this other one, etc" type of solution. Instead, I have been hoping to find some type of C++ syntactical magic wand that I can wave to solve the issue. So far to no avail, thus the posting of my question.

For completion, a typical invocation of the GetAList method (and its siblings) would look like this:

ReaderType myReader = client->GetAList( someName, 
    optionsNamespace::PostName(std::string("whatever") ),
    optionsNamespace::MaxInList(1) );

And it is those last two parameters that I just don't know until runtime. Thus I must dynamically create them during the method invocation.

Finally, it might be noteworthy (well not really, but in the interest of completion) that the call stack involved here contains a C-to-C++ bridge in which the information regarding which options that the end user has selected come to my portion of the stack in the form of a C varargs list. However, that list content may need transcoding, so I cannot use that content as-is. Thus I pre-process the varargs content, transcode if necessary, and save the results in an "options block" for lack of a better term. That options block is then processed in the C++ portion of the stack that I am also responsible for; and that is where this magic needs to occur. I need to sift through the options block, determine which options were selected, retrieve their values and use those values to create options objects within the appropriate method invocation statement (such as GetAList).

Having read that, I hope it makes sense.

Anyway, I am continuing to experiment, to attempt to find an answer. However, if any of you have input/feedback, I'd be interested in reading it.

EDIT

A couple of replies asked for clarity regarding my question. Apologies for anything that was unclear. The question I have is: how do you dynamically create a parameter list at runtime that can be sent to a C++ function that has a variadic template?

As I mentioned, the typical use-case scenario in regard to the GetAList method is to specify which option objects you want statically. That means that your code invokes the method with the arg list "hard-coded" into that method invocation. Like so:

ReaderType myReader = client->GetAList( someName, 
    optionsNamespace::PostName(std::string("whatever") ), // <--- This is option object #1
    optionsNamespace::MaxInList(1) );                     // <--- This is option object #2

I need to be able to invoke the GetAList method shown above, but the option arguments cannot be "hard-coded" directly into that invocation as shown above (option objects #1 and #2). I must dynamically create the parameter list of option objects at runtime, based on the options that the end user selected.

Initially I explored the vector approach. So the idea was to sift through the option list, create the option object and push it onto the end of the vector simultaneously. Then invoke an intermediate function that would expand the vector into a comma-separated list of parameters that could be passed to GetAList. However there were problems with that. The primary one being the determination of the parameter types. As stated in my original explanation, the types in the examples I read about were deduced by examining the end function that was to be invoked (foo in the example above). However GetAList does not have a parameter list from which the parameter types can be deduced, like foo has. So I looked into how I could use std::forward to just forward the objects from the intermediate function to GetAList, but nothing I came up with worked either.

Does that provide clarity? I can attempt it again if not. Thank you for the replies thus far, I appreciate it!

Answer 1

Definitely impossible.

a brief question: the count of arguments is known at runtime, but the function call should know it at the compile-time. how to solve it?

the usual solution is trying to build the all possible calls and then choose one (similar to std::visit), which you mention at first. but it's awful and sometimes not able to reach the purpose, if using boost::any or std::any but you don't know all the type it may have.

maybe you will think of variadic functions or virtual functions. but neither of them can pass the gap between runtime and compile-time. variadic functions, just get some additional informations from the arguments or something else and then determinate what to do. and if something is dependent on type, it should still rely on a big switch-statement and then choose one of branches.

if you really want to do that, maybe you have to make some compromises:

1. Firstly, you must know all possible types of you arguments, and give each of them an unique ID.
2. Secondly, build a mapping from all combinations of IDs to all possible function calls (remember to make invalid function calls compilable). you can use switch-statement (sometimes not useful enough), virtual function (mapping IDs to objects derived from a base class) or a big mapping table.
3. Thirdly, maybe you should create some helper-classes (such as std::ref) to keep the invariance of value-category and reference.
4. Finally, when you get the arguments, store them appropriately and get the ID combination, and then do the actual function call. in this case you get the return value as an indeterminate type. if it's useful and actually polymorphic, you have to build a return type mapping table as well, and then use a big switch-statement at the real using position. this process is similar to do 1-2-3 again with function type void(std::any).

Factly, this is similar to what std::variant and std::visit do. so you can also create a wrapper of std::variant< *all possible types* > and then use std::visit.