How to convert 'struct' to '&[u8]'?

Question

I want to send my struct via a TcpStream. I could send String or u8, but I can not send an arbitrary struct. For example:

struct MyStruct {
    id: u8,
    data: [u8; 1024],
}

let my_struct = MyStruct { id: 0, data: [1; 1024] };
let bytes: &[u8] = convert_struct(my_struct); // how??
tcp_stream.write(bytes);

After receiving the data, I want to convert &[u8] back to MyStruct. How can I convert between these two representations?

I know Rust has a JSON module for serializing data, but I don't want to use JSON because I want to send data as fast and small as possible, so I want to no or very small overhead.

Answer 1

A correctly sized struct as zero-copied bytes can be done using stdlib and a generic function.

In the example below there there is a reusable function called any_as_u8_slice instead of convert_struct, since this is a utility to wrap cast and slice creation.

Note that the question asks about converting, this example creates a read-only slice, so has the advantage of not needing to copy the memory.

Heres a working example based on the question:

unsafe fn any_as_u8_slice<T: Sized>(p: &T) -> &[u8] {
    ::core::slice::from_raw_parts(
        (p as *const T) as *const u8,
        ::core::mem::size_of::<T>(),
    )
}

fn main() {
    struct MyStruct {
        id: u8,
        data: [u8; 1024],
    }
    let my_struct = MyStruct { id: 0, data: [1; 1024] };
    let bytes: &[u8] = unsafe { any_as_u8_slice(&my_struct) };
    // tcp_stream.write(bytes);
    println!("{:?}", bytes);
}

---

Note 1) even though 3rd party crates might be better in some cases, this is such a primitive operation that its useful to know how to do in Rust.

Note 2) at time of writing (Rust 1.15), there is no support for const functions. Once there is, it will be possible to cast into a fixed sized array instead of a slice.

Note 3) the any_as_u8_slice function is marked unsafe because any padding bytes in the struct may be uninitialized memory (giving undefined behavior). If there were a way to ensure input arguments used only structs which were #[repr(packed)], then it could be safe.

Otherwise the function is fairly safe since it prevents buffer over-run since the output is read-only, fixed number of bytes, and its lifetime is bound to the input.
If you wanted a version that returned a &mut [u8], that would be quite dangerous since modifying could easily create inconsistent/corrupt data.

Answer 2

(Shamelessly stolen and adapted from Renato Zannon's comment on a similar question)

Perhaps a solution like bincode would suit your case? Here's a working excerpt:

Cargo.toml

[package]
name = "foo"
version = "0.1.0"
authors = ["An Devloper <an.devloper@example.com>"]
edition = "2018"

[dependencies]
bincode = "1.0"
serde = { version = "1.0", features = ["derive"] }

main.rs

use serde::{Deserialize, Serialize};
use std::fs::File;

#[derive(Serialize, Deserialize)]
struct A {
    id: i8,
    key: i16,
    name: String,
    values: Vec<String>,
}

fn main() {
    let a = A {
        id: 42,
        key: 1337,
        name: "Hello world".to_string(),
        values: vec!["alpha".to_string(), "beta".to_string()],
    };

    // Encode to something implementing `Write`
    let mut f = File::create("/tmp/output.bin").unwrap();
    bincode::serialize_into(&mut f, &a).unwrap();

    // Or just to a buffer
    let bytes = bincode::serialize(&a).unwrap();
    println!("{:?}", bytes);
}

You would then be able to send the bytes wherever you want. I assume you are already aware of the issues with naively sending bytes around (like potential endianness issues or versioning), but I'll mention them just in case ^_^.

Answer 3

You can use the bytemuck crate to do it safely:

#[derive(bytemuck::NoUninit, Clone, Copy)]
#[repr(C)]
struct MyStruct {
    id: u8,
    data: [u8; 1024],
}

let my_struct = MyStruct { id: 0, data: [1; 1024] };
let bytes: &[u8] = bytemuck::bytes_of(&my_struct);
tcp_stream.write(bytes);

Note this requires the struct to be Copy and #[repr(C)] or #[repr(transparent)].