Why passing Go bytes to C, the bytes length doesn't match in cgo in this code below?

Question

When passing golang bytes to C, the bytes length doesn't match.

The resulting strlen(key) and keylen doesnt match up.

Build with "go build file.go"

You can download the go file below here: https://pastebin.com/raw/hnMfXJKq <- does anyone know why my cgo call the []bytes to c has error? why the strlen doesnt match?

Expected output is supposed to be the same key length. Sometimes work, sometimes doesn't.

package main
/*
#include <stddef.h>
#include <stdlib.h>
#include <stdio.h>
#include <stdint.h>
#include <string.h>
#include <assert.h>

void replaceK(void *key, size_t keylen, void *value, size_t valuelen);
void replaceK(void *key, size_t keylen, void *value, size_t valuelen)
{
        printf("replaceK : key = %s = %lu = %lu = %s = %lu\n",(char*) key,keylen,strlen(key),(char*) value,valuelen);
        if (keylen != strlen(key)){
                printf("ERROR!!! keylen : %lu != strlen(key) : %lu!!!\n",keylen,strlen(key));
                exit(1);
        }
}
*/
import "C"
import (
        "fmt"
        "unsafe"
        "math/rand"
        "time"
)
func Set(key,value []byte) {
        cKey := cByteSlice(key)
        cValue := cByteSlice(value)
        C.replaceK(unsafe.Pointer(cKey),C.ulong(len(key)),unsafe.Pointer(cValue),C.ulong(len(value)))
        C.free(unsafe.Pointer(cKey))
        C.free(unsafe.Pointer(cValue))
}
func byteToChar(b []byte) *C.char {
        var c *C.char
        if len(b) > 0 {
                c = (*C.char)(unsafe.Pointer(&b[0]))
        }
        return c
}
var letterRunes = []byte("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ")
func RandStringRunes(n int) []byte {
        randNum := rand.Intn(n)+1
        b := make([]byte, randNum)
        for i := range b {
                b[i] = letterRunes[rand.Intn(len(letterRunes))]
        }
        return b
}
func cByteSlice(b []byte) *C.char {
        var c *C.char
        if len(b) > 0 {
                c = (*C.char)(C.CBytes(b))
        }
        return c
}
func main() {
        rand.Seed(time.Now().UnixNano())
        var key []byte
        var value []byte
        for i:=0;i<10000000;i++ {
                key = RandStringRunes(10)
                value = RandStringRunes(20)
                randnum := 1
                if randnum == 1 {
                        fmt.Printf(">>> = %s = %s\n",key, value)
                        Set(key,value)
                }
        }
}

Do you remember to include the C string *null-terminator* in your strings? Otherwise you can't use `strlen` in C. — Some programmer dude, Sep 12 '21 at 13:49
Please create a [mre]. You have a bunch of broken and or commented out code, and we cannot tell what you intend for this to do. — JimB, Sep 12 '21 at 14:01
@Someprogrammerdude yes u are right. i know but i'm more interested in the []byte equivalent from golang but regardless, the output of length shld be same. — superdolt, Sep 12 '21 at 14:07
Actually it doesn't have to be. Remember that the `strlen` function in C looks for the null-terminator. If it doesn't exist in the string then `strlen` will go out of bounds and continue counting until it finds something corresponding to the null-terminator somewhere in memory. If the string doesn't have a null-terminator then `strlen` will almost always report a length longer than expected (unless you're **un**lucky that the string just happens to have a null-terminator after it). Passing a string which isn't explicitly null-terminated leads to *undefined behavior* — Some programmer dude, Sep 12 '21 at 14:12
@Someprogrammerdude yes i understand the concept. so what's the issue with the code? — superdolt, Sep 12 '21 at 14:14
@superdolt: you are using CBytes which copies the exact bytes into a C byte array. You need to use CString, or append your own null byte. — JimB, Sep 12 '21 at 14:58
@JimB it's not supposed to be CString with a null terminator. it's supposed to be a C bytes. how do i append my own null byte too? — superdolt, Sep 14 '21 at 09:48
What do you mean "it's not supposed to be CString"? You are expecting strings, so why not use a string function? You append your own null byte by appending that to the end of the slice passed to `CBytes` so that you can end up with a _string_. — JimB, Sep 14 '21 at 12:30

score 1 · Answer 1 · answered Sep 16 '21 at 17:34

1

The C strlen function is for use with null-terminated strings, not pointer+length strings.

You can print a pointer+length string with the C printf function using %.*s instead of %s. Since Go string and []byte variables both use the pointer+length encoding, that is probably the right way to print a string obtained from Go.

answered Sep 16 '21 at 17:34

bcmills

4,391
24
34

See also https://stackoverflow.com/questions/3767284/using-printf-with-a-non-null-terminated-string. – bcmills Sep 16 '21 at 18:07

Why passing Go bytes to C, the bytes length doesn't match in cgo in this code below?

1 Answers1