creating va_list dynamically in GCC - can it be done?

Question

my problem with vsprintf is that I can not obtain input arguments directly, I have to first get inputs one by one and save them in void**, then pass this void** to vsprintf(), it is all fine for windows, but when I come to 64bit linux, gcc cannot compile because it is not allowed to convert from void** to va_list, Is there anyone that can give me some help how I should do this under linux?

Can I create va_list dynamically in GCC?

void getInputArgs(char* str, char* format, ...)
{
    va_list args;
    va_start(args, format);
    vsprintf(str, format, args);
    va_end(args);
}  

void process(void)
{
    char s[256];
    double tempValue;
    char * tempString = NULL;
    void ** args_ptr = NULL;
    ArgFormatType format;   //defined in the lib I used in the code
    int numOfArgs = GetNumInputArgs();  // library func used in my code

    if(numOfArgs>1)
    {
        args_ptr = (void**) malloc(sizeof(char)*(numOfArgs-1));
        for(i=2; i<numOfArgs; i++)
        {
            format = GetArgType();    //library funcs

            switch(format)
            {
                case ArgType_double:
                    CopyInDoubleArg(i, TRUE, &tempValue);   //lib func
                    args_ptr[i-2] = (void*) (int)tempValue;    
                    break;

                case ArgType_char:
                    args_ptr[i-2]=NULL;
                    AllocInCharArg(i, TRUE, &tempString);  //lib func
                    args_ptr[i-2]= tempString;
                break;
            }
        }
    }

    getInputArgs(s, formatString, (va_list) args_ptr);   //Here 
           // is the location where gcc cannot compile, 
           // Can I and how if I can create a va_list myself?
}

Answer 1

There is a way you can do this, but it is specific to gcc on Linux. It does work on Linux (tested) for both 32 and 64 bit builds.

DISCLAIMER: I am not endorsing using this code. It is not portable, it is hackish, and is quite frankly a precariously balanced elephant on a proverbial tightrope. I am merely demonstrating that it is possible to dynamically create a va_list using gcc, which is what the original question was asking.

With that said, the following article details how va_list works with the amd64 ABI: Amd64 and Va_arg.

With knowledge of the internal structure of the va_list struct, we can trick the va_arg macro into reading from a va_list that we construct ourselves:

#if (defined( __linux__) && defined(__x86_64__))
// AMD64 byte-aligns elements to 8 bytes
#define VLIST_CHUNK_SIZE 8
#else
#define VLIST_CHUNK_SIZE 4
#define _va_list_ptr _va_list
#endif

typedef struct  {
    va_list _va_list;
#if (defined( __linux__) && defined(__x86_64__))
    void* _va_list_ptr;
#endif
} my_va_list;

void my_va_start(my_va_list* args, void* arg_list)
{
#if (defined(__linux__) && defined(__x86_64__))
    /* va_args will read from the overflow area if the gp_offset
       is greater than or equal to 48 (6 gp registers * 8 bytes/register)
       and the fp_offset is greater than or equal to 304 (gp_offset +
       16 fp registers * 16 bytes/register) */
    args->_va_list[0].gp_offset = 48;
    args->_va_list[0].fp_offset = 304;
    args->_va_list[0].reg_save_area = NULL;
    args->_va_list[0].overflow_arg_area = arg_list;
#endif
    args->_va_list_ptr = arg_list;
}

void my_va_end(my_va_list* args)
{
    free(args->_va_list_ptr);
}

typedef struct {
    ArgFormatType type; // OP defined this enum for format
    union {
        int i;
        // OTHER TYPES HERE
        void* p;
    } data;
} va_data;

Now, we can generate the va_list pointer (which is the same for both 64 bit and 32 bit builds) using something like your process() method or the following:

void* create_arg_pointer(va_data* arguments, unsigned int num_args) {
    int i, arg_list_size = 0;
    void* arg_list = NULL;

    for (i=0; i < num_args; ++i)
    {
        unsigned int native_data_size, padded_size;
        void *native_data, *vdata;

        switch(arguments[i].type)
        {
            case ArgType_int:
                native_data = &(arguments[i].data.i);
                native_data_size = sizeof(arguments[i]->data.i);
                break;
            // OTHER TYPES HERE
            case ArgType_string:
                native_data = &(arguments[i].data.p);
                native_data_size = sizeof(arguments[i]->data.p);
                break;
            default:
                // error handling
                continue;
        }

        // if needed, pad the size we will use for the argument in the va_list
        for (padded_size = native_data_size; 0 != padded_size % VLIST_CHUNK_SIZE; padded_size++);

        // reallocate more memory for the additional argument
        arg_list = (char*)realloc(arg_list, arg_list_size + padded_size);

        // save a pointer to the beginning of the free space for this argument
        vdata = &(((char *)(arg_list))[arg_list_size]);

        // increment the amount of allocated space (to provide the correct offset and size for next time)
        arg_list_size += padded_size;

        // set full padded length to 0 and copy the actual data into the location
        memset(vdata, 0, padded_size);
        memcpy(vdata, native_data, native_data_size);
    }

    return arg_list;
}

And finally, we can use it:

va_data data_args[2];
data_args[0].type = ArgType_int;
data_args[0].data.i = 42;

data_args[1].type = ArgType_string;
data_args[1].data.p = "hello world";

my_va_list args;
my_va_start(&args, create_arg_pointer(data_args, 2));

vprintf("format string %d %s", args._va_list);

my_va_end(&args);

And there you have it. It works mostly the same as the normal va_start and va_end macros, but lets you pass your own dynamically generated, byte-aligned pointer to be used instead of relying on the calling convention to set up your stack frame.

Answer 2

I have tried using libffi as mentioned somewhere else and it works. Here below is the link , hope it can help others with similar issues. Thanks again for all help I got here!

Link: http://www.atmark-techno.com/~yashi/libffi.html -- simple example given http://www.swig.org/Doc1.3/Varargs.html -- printf() and other examples given

Answer 3

The type of va_list is not void ** or anything similar with 64-bit gcc (on Intel x86/64 machines). On both Mac OS X 10.7.4 and on RHEL 5, there is no header stdarg.h in /usr/include. Consider the following code:

#include <stdarg.h>
#include <stdio.h>
int main(void)
{
    printf("sizeof(va_list) = %zu\n", sizeof(va_list));
    return 0;
}

The output on both RHEL 5 and Mac OS X 10.7 with a 64-bit compilation is:

sizeof(va_list) = 24

With a 32-bit compilation, the output on each platform is:

sizeof(va_list) = 4

(You may take it that I was surprised to find this much discrepancy between the 32-bit and 64-bit versions. I was expecting a value between 12 and 24 for the 32-bit version.)

So, the type is opaque; you can't even find a header that tells you anything about; and it is much bigger than a single pointer on 64-bit machines.

Even if your code works on some machines, it is very, very far from guaranteed to work everywhere.

The GCC 4.7.1 manual does not mention any functions that allow you to build a va_list at runtime.

Answer 4

Following class works for me:

class VaList
{
    va_list     _arguments;

public:

    explicit inline VaList(const void * pDummy, ...)
    {
        va_start(_arguments, pDummy);
    }

    inline operator va_list &()
    {
        return _arguments;
    }

    inline operator const va_list &() const
    {
        return _arguments;
    }

    inline ~VaList()
    {
        va_end(_arguments);
    }
};

and it can be used like this:

void v(const char * format, const va_list & arguments)
{
    vprintf(format, const_cast<va_list &>(arguments));
}

...

    v("%d\n", VaList("", 1)); // Uses VaList::operator va_list &()
    v("%d %d\n", VaList(nullptr, 2, 3)); // Uses VaList::operator va_list &()
    vprintf("%s %s %s\n", VaList("", "Works", "here", "too!"));

    const VaList args(NULL, 4, 5, "howdy", "there");
    v("%d %d %s %s\n", args); // Uses VaList::operator const va_list &() const

The first dummy parameter can be any kind of pointer, it is only used to compute the address of the following arguments.

The same can of course be done in C too but not so niftily (use pointer instead of reference)!

Simple example of using VaList to construct a dynamic va_list:

static void VectorToVaList(const std::vector<int> & v, va_list & t)
{
    switch (v.size())
    {
        case 1: va_copy(t, VaList("", v[0])); return;
        case 2: va_copy(t, VaList("", v[0], v[1])); return;
        case 3: va_copy(t, VaList("", v[0], v[1], v[2])); return;
        case 4: va_copy(t, VaList("", v[0], v[1], v[2], v[3])); return;
        // etc
    }

    throw std::out_of_range("Out of range vector size!");
}

and usage:

    va_list t;
    VectorToVaList(std::vector<int>{ 1, 2, 3, 4 }, t);
    vprintf("%d %d %d %d\n", t);

Answer 5

If the problem you're trying to solve is inserting passing arbitrary types to a function in va_list style, then, consider using union:

#include <iostream>
#include <cstdarg>

union ARG
{
    int d;
    char* s;
    double f;
};

int main()
{
    printf("%d %s %f \n", 1, "two", 3.1415 );
    // Output: 1 two 3.141500

    char format[ 1024 ] = "%d %s %f\n";
    ARG args[ 5 ] = { };
    args[ 0 ].d = 1;
    args[ 1 ].s = "two";
    args[ 2 ].f = 3.1415;
    printf( format, args[ 0 ], args[ 1 ], args[ 2 ], args[ 3 ], args[ 4 ] );
    // Output: 1 two 3.141500

    return 0;
}

Some things you'll note about my solution:

• No attempt is made to produce the correct number of arguments. i.e. I oversupply the arguments, but, most functions will look at the first parameter to determine how to handle the rest (i.e. format)
• I didn't bother dynamically create the format, but, it is a trivial exercise to build a routine that dynamically populates format and args.

Tested this on: - Ubuntu, g++ - Android NDK

I did some more testing, and, confirmed @PeterCoordes comments about this answer not working for double precision.