processing va_list argument in JNA callback function

Question

How can I process a va_list argument with Java, having received it from the native library?

I am using a C library that facilitates logging through a callback function. The library is libghoto2 and I am using a JNA wrapper libgphoto2-java to access its features. See the errordumper method in this C file for an example of how the logging should be done.

I've managed to add a Java-written callback function using the library's gp_log_add_func. The only problem is, the callback function's signature contains a va_list argument that I don't know how to process.

As the C example from earlier shows, va_list args is passed directly into vfprintf. Reading the vfprintf manual it becomes clear that it is some sort of iterable data structure that has "been initialized with the va_start macro" and after iterating using va_arg a cleanup with va_end is required. But the only way I found for keeping the JVM from crashing, is to make the args parameter of type com.sun.jna.Pointer while a String[] or Object[] would be more suitable.

How do I get the data out of this va_list?

N.B. In order to get access to gp_log_add_func, I added some Java code:

Additions to org.gphoto2.jna.GPhoto2Native:

int gp_log_add_func(int logLevel, LogFunc func, Pointer data);

Created org.gphoto2.jna.LogFunc:

public interface LogFunc extends Callback {
    public static final int GP_LOG_ERROR = 0;
    public static final int GP_LOG_VERBOSE = 1;
    public static final int GP_LOG_DEBUG = 2;
    public static final int GP_LOG_DATA = 3;
    public static final int GP_LOG_ALL = GP_LOG_DATA;

    //the args argument is a va_list 
    public void log(int logLevel, String domain, String format, Pointer args, Pointer data);
}

The implementation and usage of org.gphoto2.jna.LogFunc:

LogFunc callback = new LogFunc() {
        public void log(int logLevel, String domain, String format, Pointer args, Pointer data) {
            System.out.println("[" + domain + "] " + format);
            System.out.println(args.toString());
        }
};
GPhoto2Native.INSTANCE.gp_log_add_func(LogFunc.GP_LOG_ALL, callback, null);

You'll have to look at the actual implementation of va_list on the platform in question (start with the header file (vararg.h), and possibly delving into compiler specifics). va_list is *usually* a pointer onto the stack, with the various extraction args simply offset operations that move the pointer along by the appropriate size offset. You'll have to extract the stuff manually using the Pointer, using the various data extraction methods like Pointer.getInt(int offset), depending on the type you're attempting to access. — technomage, Jun 12 '12 at 10:36
This is what I was afraid of. From a purist's point of view, the JNA/Java business should be separate from any native business. This is especially worrying, since I am currently developing on openSuse and may later want to deploy on OS X. Unfortunately I don't know much about C/C++, but is there maybe a way of employing a second native library that converts the va_list into something more JNA-manageable? — derabbink, Jun 12 '12 at 15:22
It's certainly an arguable case that JNA should provide a mapping for va_list handling. Feel free to open an issue on it. — technomage, Jun 13 '12 at 11:30
Today I had some spare time again to work on this; I think I may have found a way to extract the data and turn the whole thing into a Java List — derabbink, Jul 11 '12 at 22:46
To be more specific, what native library is va_arg in? it is not in `Native.loadLibrary("c")` — derabbink, Jul 11 '12 at 22:55
It's usually in varargs.h and implemented as macros. One common approach is to calculate the start of the va args as an offset from the address of the last-named argument, which basically gives you the starting address of the varargs on the stack. From there, you walk the stack, extracting a pointer, FP, or integer value from each stack address based on the expected argument type. — technomage, Jul 12 '12 at 13:29
The tricky part is going to be determining the address of your stack arguments. I'm not sure that's possible. — technomage, Jul 12 '12 at 13:32
So if i were to limit myself to va_arg and va_end, thus only being able to process va_lists that have already been initialized with va_start it should be possible? JNA does not allow loading a header file like this `Native.loadLibrary("varargs.h")` as it requires the name of a binary. Does this mean I will need a tiny C library that exposes these macros as methods? Or does such a library exist already? — derabbink, Jul 12 '12 at 15:37
If you have the results of va_start (the va_list address) you could use Pointer functions to peform the equivalent of va_arg (va_end doesn't likely do anything). Getting the address of the variadic arguments is the hard part. — technomage, Jul 12 '12 at 22:46
You *could* make a small callback function intended to call your java callback function with the format string and the argp. Things could work if you register your Java callback with *that* function, then register your new function with your logging function. But that still seems like a lot of work... — technomage, Jul 13 '12 at 12:11

score 2 · Accepted Answer · answered Jul 12 '12 at 22:41

Here is an example varargs implementation, with some hints about what the varargs macros are doing:

int printf(const char* fmt, ...) {
  va_list argp;

  va_start(argp, fmt); // usually something like: argp = (char *)&fmt - sizeof(void *);

  int arg1 = va_arg(argp, int); // *(int *)argp; argp += sizeof(int);
  void *arg2 = va_arg(argp, void*); // *(void **)argp; argp += sizeof(void *);
  float arg3 = va_arg(argp, float); // *(float *)argp; argp += sizeof(float);

  va_end(argp); // no-op

}

So it's basically a bunch of pointer arithmetic working with the stack pointer. The problematic piece w/r/t JNA is that you don't have access to the stack pointer, and you'd probably need to extend JNA's callback mechanism at the native level to specially handle variadic callbacks to provide the stack pointer.

Even that is potentially problematic. As you can see from the above example, you actually need the address of the last named argument of the variadic function signature in order to access the variadic arguments. That would be very tricky to do generically.

I guess I'll be reverting to just invoking the gphoto2 cli combined with "clever regexes" that match stdout and stderr. It may not be the slickest of solutions but at least it's more likely to work. — derabbink, Aug 25 '12 at 14:39
@derabbink FYI, it's easier to get around this with JavaCPP: It already has the infrastructure to build, bundle, and load JNI libraries automatically. I've had to do that for FFmpeg recently: https://github.com/bytedeco/javacpp-presets/commit/4ed81465898e0ab27b48f3eb232c26d92ffb69d7 — Samuel Audet, Nov 01 '15 at 01:19

score 0 · Answer 2 · answered Mar 21 '20 at 22:39

Maybe it might not be relevant for the author here, but something that might be useful for anyone new who comes across this problem as I did. Most of these issues are related with callbacks for logging and in such cases, it might be easy to just use C and another JNA call to our advantage.

For instance, if there is a callback function that needs to be provided that has a C signature as log_write(const char* format, va_list args); then one can have a JNA callback that will make another call to C's vsprintf to construct the final string.

public interface CLib extends Library {
  CLib INSTANCE = (CLib) Native.loadLibrary((Platform.isWindows() ? "msvcrt" : "c"), CLib.class);
  int vsprintf(byte[] buffer, String format, Pointer va_list);
} // interface CLib

public static String format(final String format, final Pointer va_args) {
  CLib jnaLib = CLib.INSTANCE;
  byte[] buffer = new byte[2048];
  jnaLib.vsprintf(buffer, format, va_args);
  return new String(buffer);
}

processing va_list argument in JNA callback function

2 Answers2