1

I'm trying to write a MySQL UDF in Go with cgo, in which I have a basic one functioning, but there's little bits and pieces that I can't figure out because I have no idea what some of the C variables are in terms of Go.

This is an example that I have written in C that forces the type of one of the MySQL parameters to an int

my_bool unhex_sha3_init(UDF_INIT *initid, UDF_ARGS *args, char *message) {
    if (args->arg_count != 2) {
        strcpy(message, "`unhex_sha3`() requires 2 parameters: the message part, and the bits");
        return 1;
    }

    args->arg_type[1] = INT_RESULT;

    initid->maybe_null = 1; //can return null

    return 0;
}

And that works fine, but then I try to do the same/similar thing with this other function in Go like this

//export get_url_param_init
func get_url_param_init(initid *C.UDF_INIT, args *C.UDF_ARGS, message *C.char) C.my_bool {
    if args.arg_count != 2 {
        message = C.CString("`get_url_param` require 2 parameters: the URL string and the param name")
        return 1
    }

    (*args.arg_type)[0] = C.STRING_RESULT
    (*args.arg_type)[1] = C.STRING_RESULT

    initid.maybe_null = 1

    return 0
}

With this build error

./main.go:24: invalid operation: (*args.arg_type)[0] (type uint32 does not support indexing)

And I'm not totally sure what that means. Shouldn't this be a slice of some sort, not a uint32?

And this is where it'd be super helpful have some way of dumping the args struct somewhere somehow (maybe even in Go syntax as a super plus) so that I can tell what I'm working with.

Well I used spew to dump the variable contents to a tmp file inside the init function (commenting out the lines that made it not compile) and I got this

(string) (len=3) "%#v"
(*main._Ctype_struct_st_udf_args)(0x7ff318006af8)({
 arg_count: (main._Ctype_uint) 2,
 _: ([4]uint8) (len=4 cap=4) {
  00000000  00 00 00 00                                       |....|
 },
 arg_type: (*uint32)(0x7ff318006d18)(0),
 args: (**main._Ctype_char)(0x7ff318006d20->0x7ff3180251b0)(0),
 lengths: (*main._Ctype_ulong)(0x7ff318006d30)(0),
 maybe_null: (*main._Ctype_char)(0x7ff318006d40)(0),
 attributes: (**main._Ctype_char)(0x7ff318006d58->0x7ff318006b88)(39),
 attribute_lengths: (*main._Ctype_ulong)(0x7ff318006d68)(2),
 extension: (unsafe.Pointer) <nil>
})
Brian Leishman
  • 8,155
  • 11
  • 57
  • 93
  • I don't think you can export an unexported function for starters. There's nothing to dump here, because you haven't compiled anything, the compiler is telling you have a type error: `*args.arg_type` is a `uint32`. – JimB Jul 20 '18 at 15:46
  • @JimB Yes actually I have a skeleton Go UDF that works https://gist.github.com/BrianLeishman/1720c3c3d47b799d5dd913cedea654c5, so the export function thing is fine in this case – Brian Leishman Jul 20 '18 at 15:47
  • @JimB And I get that, it makes sense that I can't dump since it's not compiled but is there *any way* at all to look at the entire structure of that variable? – Brian Leishman Jul 20 '18 at 15:49
  • 1
    I'm not sure what you mean -- args is of type `C.UDF_ARGS`, which means there's a C file somewhere that declares the `UDF_ARGS` type which you can view. (and IMO I would actually export those exported functions, since it would probably make most Go programmers question what's really happening there) – JimB Jul 20 '18 at 15:57
  • @JimB that makes sense, I could update that. And I actually got the Go syntax I was looking for by calling `fmt.Fprintf(tmpfile, "%#v", args)` in the init function and removing my type forcing to make it compile, so we'll see what I find – Brian Leishman Jul 20 '18 at 15:59
  • @JimB Actually it wont let you, the export comment has to match the name of the function – Brian Leishman Jul 20 '18 at 16:00
  • Yes, you have to change the export comment to match of course (I guess if you need to work with existing C code you might need it to stay the way it is). – JimB Jul 20 '18 at 16:02
  • @JimB Well I got the variable (Added it to the question), is the `arg_type` property supposed to be a pointer? – Brian Leishman Jul 20 '18 at 16:22
  • Again I'm confused, `arg_type` is a pointer if that's how it's declared in C, I can't say what it's _supposed_ to be. What is it you're expecting? – JimB Jul 20 '18 at 16:25
  • @JimB I'm honestly not totally sure how it's declared in C, what I'm expecting though is a way to set it to a value like I set the value in C, with `args->arg_type[1] = something`, so I would *expect* it in Go to be an array slice or an array, not just a uint32 – Brian Leishman Jul 20 '18 at 16:27
  • You can just look at the C header to see how it's declared. C arrays are pointers, and that's what I would expect if you have an array of `uint32`. If you want to convert that to a slice see https://stackoverflow.com/questions/27532523/how-to-convert-1024c-char-to-1024byte and documentation here: https://github.com/golang/go/wiki/cgo#turning-c-arrays-into-go-slices – JimB Jul 20 '18 at 16:32
  • @JimB thanks a ton for your help, I've added an answer that explains everything I needed to know before posting this question – Brian Leishman Jul 20 '18 at 19:45

1 Answers1

0

Alright so huge help with @JimB who stuck with me even though I'm clearly less adept with Go (and especially CGO) but I've got a working version of my UDF, which is an easy and straight forward (and fast) function that pulls a single parameter out of a URL string and decodes it correctly and what not (e.g. %20 gets returned as a space, basically how you would expect it to work).

This seemed incredibly tricky with a pure C UDF because I don't really know C (as well as I know other languages), and there's a lot that can go wrong with URL parsing and URL parameter decoding, and native MySQL functions are slow (and there's not really a good, clean way to do the decoding either), so Go seemed like the better-than-perfect candidate for this kind of problem, for strong performance, ease of writing, and wide variety of easy to use built ins & third party libraries.

The full UDF and it's installation/usage instructions are here https://github.com/StirlingMarketingGroup/mysql-get-url-param/blob/master/main.go

First problem was debugging output. And I did that by Fprintfing to a tmp file instead of the standard output, so that I could check the file to see variable dumps.

t, err := ioutil.TempFile(os.TempDir(), "get-url-param")

fmt.Fprintf(t, "%#v\n", args.arg_type)

And then after I got my output (I was expecting args.arg_type to be an array like it is in C, but instead was a number) I needed to convert the data referenced by that number (the pointer to the start of the C array) to a Go array so I could set it's values.

argsTypes := *(*[2]uint32)(unsafe.Pointer(args.arg_type))

argsTypes[0] = C.STRING_RESULT
argsTypes[1] = C.STRING_RESULT
Brian Leishman
  • 8,155
  • 11
  • 57
  • 93