4

I have created three enums that are nearly identical:

#[derive(Clone, Debug)]
pub enum Smoller {
    Int(u8),
    Four([u8; 4]),
    Eight([u8; 8]),
    Twelve([u8; 12]),
    Sixteen([u8; 16]),
}

#[derive(Clone, Debug)]
pub enum Smol {
    Float(f32),
    Four([u8; 4]),
    Eight([u8; 8]),
    Twelve([u8; 12]),
    Sixteen([u8; 16]),
}

#[derive(Clone, Debug)]
pub enum Big {
    Float(f64),
    Four([u8; 4]),
    Eight([u8; 8]),
    Twelve([u8; 12]),
    Sixteen([u8; 16]),
}

pub fn main() {
    println!("Smoller: {}", std::mem::size_of::<Smoller>()); // => Smoller: 17
    println!("Smol: {}", std::mem::size_of::<Smol>()); // => Smol: 20
    println!("Big: {}", std::mem::size_of::<Big>()); // => Big: 24
}

What I expect, given my understanding of computers and memory, is that these should be the same size. The biggest variant is the [u8; 16] with a size of 16. Therefore, while these enums do have a different size first variant, they have the same size of their biggest variants and the same number of variants total.

I know that Rust can do some optimizations to acknowledge when some types have gaps (e.g. pointers can collapse because we know that they won't be valid and 0), but this is really the opposite of that. I think if I were constructing this enum by hand, I could fit it into 17 bytes (only one byte being necessary for the discrimination), so both the 20 bytes and the 24 bytes are perplexing to me.

I suspect this might have something to do with alignment, but I don't know why and I don't know why it would be necessary.

Can someone explain this?

Thanks!

River Tam
  • 3,096
  • 4
  • 31
  • 51
  • It's also worth noting that this happens with `Arc`s as well (this is actually how I encountered this behavior), so while I know that both pointers and floating point numbers are a bit special, I don't know how they relate that would cause this issue exactly. – River Tam Jul 29 '19 at 09:43
  • Simpler example: https://play.rust-lang.org/?version=stable&mode=release&edition=2018&gist=93149c5ba46630d058706762a17f35ac – Boiethios Jul 29 '19 at 09:52
  • 1
    I also found a great tool to help with this: `rustc` comes with a `print-type-sizes` debug flag, which can be run like this: `cargo +nightly-2019-07-19 rustc --release -- -Z print-type-sizes` (the `+nightly...` is only necessary if it's not your default) – River Tam Jul 29 '19 at 09:54
  • For posterity, [here](https://gist.github.com/rivertam/0f3548b687865514aa3d1dab0c7b2b26) is the relevant part of the output of the command I posted above for the example i posted in the original question. (I prefer to use the less simple example because it's easier to see how some fields make a difference and others don't) – River Tam Jul 29 '19 at 10:07
  • [Smol :3](https://i.imgur.com/GTpHKYQ.jpg) – Lukas Kalbertodt Jul 29 '19 at 11:09

3 Answers3

9

The size must be at least 17 bytes, because its biggest variant is 16 bytes big, and it needs an extra byte for the discriminant (the compiler can be smart in some cases, and put the discriminant in unused bits of the variants, but it can't do this here).

Also, the size of Big must be a multiple of 8 bytes to align f64 properly. The smaller multiple of 8 bigger than 17 is 24. Similarly, Smol cannot be only 17 bytes, because its size must be a multiple of 4 bytes (the size of f32). Smoller only contains u8 so it can be aligned to 1 byte.

Lukas Kalbertodt
  • 79,749
  • 26
  • 255
  • 305
mcarton
  • 27,633
  • 5
  • 85
  • 95
  • 2
    I see, so the differences between them can be explained by alignment -- `f64` requires 8 byte alignment, `f32` requires 4 byte alignment, and `u8` does not require any alignment. I guess my next question has to be... why does the whole thing have to be aligned? Can't you treat the first (or second; we have 17 of them) 8 bytes like an aligned float and not align the entire enum? – River Tam Jul 29 '19 at 09:53
  • 3
    Got it -- as @rodrigo notes (and the page you linked to also covers), the whole struct/enum must be aligned because if it's not aligned, arrays of the value will have their inner members be unaligned. Even if the `Big` starts with the float, if the whole enum is 17 bytes, the next `Big` in the `[Big]` would start on byte 18, so *its* alignment would be wrong, even if its internal alignment is fine. Thanks for your and @rodrigo's help! – River Tam Jul 29 '19 at 10:01
4

I think that it is because of the alignment requirements of the inner values.

u8 has an alignment of 1, so all works as you expect, and you get a whole size of 17 bytes.

But f32 has an alignment of 4 (technically it is arch-dependent, but that is the most likely value). So even if the discriminant is just 1 byte you get this layout for Smol::Float:

[discriminant x 1] [padding x 3] [f32 x 4] = 8 bytes

And then for Smol::Sixteen:

[discriminant x 1] [u8 x 16] [padding x 3] = 20 bytes

Why is this padding really necessary? Because it is a requirement that the size of a type must be a multiple of the alignment, or else arrays of this type will misalign.

Similarly, the alignment for f64 is 8, so you get a full size of 24, that is the smallest multiple of 8 that fits all the enums.

rodrigo
  • 94,151
  • 12
  • 143
  • 190
4

As mcarton mentions, this is an effect of alignment of internal fields and alignment/size rules.


Alignment

Specifically, common alignments for built-in types are:

  • 1: i8, u8.
  • 2: i16, u16.
  • 4: i32, u32, f32.
  • 8: i64, u64, f64.

Do note that I say common, in practice alignment is dictated by hardware, and on 32-bits architectures you could reasonably expect f64 to be 4-bytes aligned. Further, the alignment of isize, usize and pointers will vary based on 32-bits vs 64-bits architecture.

In general, for ease of use, the alignment of a compound type is the largest alignment of any of its fields, recursively.

Access to unaligned values is generally architecture specific; on some architecture it will crash (SIGBUS) or return erroneous data, on some it will be slower (x86/x64 not so long ago) and on others it may be just fine (newer x64, on some instructions).


Size and Alignment

In C, the size must always be a multiple of the alignment, because of the way arrays are laid out and iterated over:

  • Each element in the array must be at its correct alignment.
  • Iterating is done by incrementing the pointer by sizeof(T) bytes.
  • Thus the size must be a multiple of the alignment.

Rust has inherited this behavior^1 .

It's interesting to note that Swift decided to define a separate intrinsic, strideof, to represent the stride in an array, which allowed them to remove any tail-padding from the result of sizeof. It did cause some confusions, as people expected sizeof to behave like C, but allows compacting memory more efficiently.

Thus, in Swift, your enums could be represented as:

  • Smoller: [u8 x 16][discriminant] => sizeof 17 bytes, strideof 17 bytes, alignof 1 byte.
  • Smol: [u8 x 16][discriminant] => sizeof 17 bytes, strideof 20 bytes, alignof 4 bytes.
  • Big: [u8 x 16][discriminant] => sizeof 17 bytes, strideof 24 bytes, alignof 8 bytes.

Which clearly shows the difference between the size and the stride, which are conflated in C and Rust.

^1 I seem to remember some discussions over the possible switch to strideof, which did not come to fruition as we can see, but could not find a link to them.

Community
  • 1
  • 1
Matthieu M.
  • 287,565
  • 48
  • 449
  • 722
  • >It's interesting to note that Swift decided to define a separate intrinsic, strideof, to represent the stride in an array I'm glad you mentioned this, as this is really interesting to me. Of course, this morning, I've been thinking about ways one could get around it. In practice, I won't bother (not worth it; this size is fine, in reality) but I'm glad to see some languages have this as well-supported behavior. – River Tam Jul 29 '19 at 11:57
  • C doesn't have such requirement. You mix C requirement and machine requirement to produce optimized code. If you ask to the compile to pack the structure you can get all 3 structures to have 17 bytes. https://rextester.com/XNVHF93560, and it's still correct C code. – Stargateur Jul 29 '19 at 13:00
  • @Stargateur: Note that `pack` also changes the alignment, so you are not invalidating my point. There is no explicit requirement that `sizeof` return a number that is a multiple of `alignof` AFAIK. In C++ it can be deduced from 5.3.3/2 (https://stackoverflow.com/a/4638295/147192). In C, I expect 6.5.3.4/3 to enforce the same semantics: "When applied to an operand that has structure or union type, the result is the total number of bytes in such an object, including internal **and trailing padding**" combined with alignment and arrays. – Matthieu M. Jul 29 '19 at 13:19