How to use syscalls correctly in go (different results from Go unsafe.Sizeof vs C sizeof)

Question

Go's unsafe.Sizeof is returning a different result than C's sizeof.

main.go:

package main

import (
    "unsafe"
)

type gpioeventdata struct {
    Timestamp uint64
    ID        uint32
}

func main() {
    eventdata := gpioeventdata{}
    println("Size", unsafe.Sizeof(eventdata))
}

Prints 12 when compiled with env GOOS=linux GOARCH=arm GOARM=6 go build on macOS and run on Raspberry Pi Zero.

gpio.c:

#include <stdio.h>
#include <linux/gpio.h>

int main() {    
    printf("sizeof gpioevent_data %zu\n", sizeof(struct gpioevent_data));
}

Prints 16 when compiled and run on Raspberry (with gcc).

struct definition in gpio.h:

struct gpioevent_data {
    __u64 timestamp;
    __u32 id;
};

Edit

I already thought that this is due to alignment, but a lot of people are passing Go structs to syscall.Syscall (e.g. https://github.com/stapelberg/hmgo/blob/master/internal/gpio/reset.go#L49). So that's basically wrong and you should never do that?

If that's wrong, what would be the correct approach calling syscalls with go so that works correctly with different architectures. For example GPIO ioctl calls:

ret = ioctl(fd, GPIO_GET_LINEEVENT_IOCTL, &req);
...
struct gpioevent_data event;
ret = read(req.fd, &event, sizeof(event));

Conclusion: The second setup (gcc for Raspberry) pads to alignment of 64 bits. The first setup doesn't pad to alignment of 64 bits (though it probably pads to alignment of 32 bits). — goodvibration, Apr 13 '19 at 10:31

6502 · Answer 1 · 2019-04-16T13:05:23.283

2

The go compiler and the C compiler are handling alignment differently.

In C the structure has been aligned to 16 bytes (adding a 4 bytes slack space after id or before it). The go compiler instead packed the fields without adding any slack space.

Your mistake is thinking that two "structures" in different languages with different compilers should have the same memory layout.

Note that there is no way to "compute" what will be the padding in a C or C++ structure because padding is a choice of the implementer. It's well possible that two different standard-conforming C compilers for the same architecture will generate different paddings (or even the same compiler with different compiling options).

The only way to get the padding correct is to check the specific case, either manually or by writing a script that calls the compiler passing the same compiling options and checks the result (e.g. by output the results of offsetof on all the members) and then generates the needed go source code after parsing that output.

edited Apr 16 '19 at 13:05

answered Apr 13 '19 at 10:31

6502

112,025
15
165
265

@schlamar: you can do it, but in this case you need to add the padding in the `go` structure. First check where those padding bytes are (most probably at the end) and then add another `reserved` 32-bit integer to the `go` structure in the proper position. – 6502 Apr 13 '19 at 13:39
And how would you do that across multiple architectures in practice? I know you can calculate this manually (https://fresh2refresh.com/c-programming/c-structure-padding/) but this is tedious. – schlamar Apr 15 '19 at 11:32
@schlamar: there is no way to "compute" the result for different architectures (or even different compilers for the same architecture). Padding in C and C++ is a **choice** of the compiler implementer. See edited answer. – 6502 Apr 15 '19 at 13:03
That's not quite correct, usually the C platform ABI guarantees a deterministic struct alignment (if you don't use any compiler options / pragmas / etc. of course). C++ is more complicated, but that's not relevant here. See also my answer to this question. – schlamar Apr 16 '19 at 07:12

score 0 · Answer 2 · answered Apr 16 '19 at 07:03

0

According to https://go101.org/article/memory-layout.html, go generally follows the C rules for structure padding (see https://stackoverflow.com/a/38144117/851737 for details of the C memory alignment rules and here for an algorithm in pseudocode).

However, there is a known bug that go doesn't align 64bit values on 32bit architectures correctly.

answered Apr 16 '19 at 07:03

schlamar

9,238
3
38
76

You apparently ignore that the "C rules for structure padding" are extremely liberal... (basically they only forbid padding at the begin of structures, that's all). It's impossible (theoretically impossible) for `go` to be compatible at structure level with "C" programs because there can be (and actually there are) standard-conforming C compilers that are not compatible each other at structure layout level, even on the same platform. – 6502 Apr 16 '19 at 10:02
That's wrong. The aligment of a struct in C is defined by the platform's ABI if no compiler options regarding padding are set. Otherwise, FFI with structs from for example Rust (https://doc.rust-lang.org/nomicon/ffi.html#interoperability-with-foreign-code) or Python (https://gist.github.com/Overdrivr/cdd58cea15d7e28c50ea) to C wouldn't work. Actually, even the C example from the question wouldn't be portable across different compilers. – schlamar Apr 16 '19 at 12:29
You are mixing up levels. The "C" level defines no ABI. The "C" language padding is not governed by "rules" as you think them. The rules are instead "there may be padding between members and after the end" and "there must be no padding at beginning"; how much and if there is padding at all or not is not guaranteed by the C standard. Feel free to check C standard documents if you don't trust me. I passed long ago the age in which trying to correct every ignorant on the internet was my top priority; you are free to self-accept your own wrong answer if it makes you feel better. No offense taken. – 6502 Apr 16 '19 at 13:04
Yeah, I never said this is defined in the C standard. The ABI comes from the compiler. In theory a compiler can do want he wants (regarding to padding) but in practice they follow the platform's default ABI (at least that's the case for clang and gcc on Linux, they are ABI compatible: http://lists.llvm.org/pipermail/cfe-dev/2015-April/042357.html). – schlamar Apr 16 '19 at 15:18
MSVC guarantees a stable ABI for structs ("POD"), too: You should never write code that depends on a particular layout for an object that isn't a COM interface or a POD object. https://learn.microsoft.com/de-de/cpp/porting/binary-compat-2015-2017?view=vs-2019 – schlamar Apr 16 '19 at 15:35

How to use syscalls correctly in go (different results from Go unsafe.Sizeof vs C sizeof)

2 Answers2