0

I am trying to implement a self-referential struct in rust.

Background

I am writing a crate which is based on another crate. The underlying crate exposes a type Transaction<'a> which requires a mutable reference to a Client object:

struct Client;

struct Transaction<'a> {
    client: &'a mut Client
}

The only way to construct a Transaction is by calling a method on a client object. The function looks something like this:

impl Client {
    async fn transaction<'a>(&'a mut self) -> Transaction<'a> {
        // ...
    }
}

Now I want to expose an api that allows something like this:

let transaction = Transaction::new().await;

Of course, this doesn't work without passing a &mut Client.

Now my idea was to create a wrapper type which owns a Client as well as a Transaction with a reference to said client.

I would use unsafe to create a mutable reference which can be used to create a Transaction while still being able to pass the Client object to my struct. I would pin the Client on the heap to make sure the reference doesn't become invalid:

struct MyTransaction<'a> {
    client: Pin<Box<Client>>,
    inner: Transaction<'a>
}

impl<'a> MyTransaction<'a> {
    async fn new() -> MyTransaction<'a> {
        // ... fetch a new client object
        let pin = Box::pin(client);
        let pointer = &*pin as *const Client as *mut Client;
        
        // convert raw pointer to mutable reference 
        // to create new Transaction
        let inner = unsafe {
            &mut *pointer
        }.transaction().await;

        Self {
            inner,
            client
        }
    }
}

When I run this code however, my test fails with the following error message:

error: test failed

Caused by:
  process didn't exit successfully: `/path/to/test` (signal: 11, SIGSEGV: invalid memory reference)

This somewhat surprised me because I thought that by pinning the client object I'd ensure it won't be moved. Thus, I reasoned, a pointer/reference to it shouldn't become invalid as long as it doesn't outlive the client. I though that this could not be the case since the client is only dropped when MyTransaction is dropped, which will also drop the Transaction which holds the reference. I also though that moving MyTransaction should be possible.

What did I do wrong? Thanks in advance!

Einliterflasche
  • 473
  • 6
  • 18
  • 2
    In Rust fields are dropped in order, so `client` is dropped before `inner` which leads to a sigsegv I'm not quite sure just reordering the fields is the fix though. – cafce25 Aug 06 '23 at 13:41
  • @cafce25 it worked. Wow, that's something I wouldn't have guessed in a million years. Would you mind writing an answer so that I might accept it? – Einliterflasche Aug 06 '23 at 14:57
  • Note that creating a self-referential structure correctly is _incredibly hard_. Harder than almost everything using unsafe code. Perhaps the only thing harder than it is creating a _library_ to create self-referential structs... You should avoid it if you don't know 100% what you're doing (you can use a library though, [`ouroboros`](https://docs.rs/ouroboros) being the only sound). This code, for example, may be unsound, because moving the `Box` may invalidate all pointers to it (it is undecided for now). Also, the `Pin` is not needed. – Chayim Friedman Aug 06 '23 at 17:32
  • @ChayimFriedman I thought the whole point of `Pin`ning is to make sure the content isn't moved? And why is it undecided for now what happens when the box is moved (which shouldn't happen)? – Einliterflasche Aug 06 '23 at 21:08
  • @Einliterflasche `Pin` does not change anything. It is only usable for public APIs where the data needs to not move. When you control it, you can just not move it. And about the `Box` moving (which does happen, every time you move the containing struct), see https://github.com/rust-lang/unsafe-code-guidelines/issues/326. – Chayim Friedman Aug 07 '23 at 11:18
  • @ChayimFriedman Hey, it's me again xD I won't pretend to understand the discussion you linked. The [docs](https://doc.rust-lang.org/std/boxed/struct.Box.html#method.pin) however state that _If `T` does not implement `Unpin`, then `x` will be pinned in memory and unable to be moved_ - doesn't that mean that my approach should work (if Client is `!Unpin`)? – Einliterflasche Aug 08 '23 at 20:26
  • @Einliterflasche I'll repeat again that _`Pin` is a library guarantee, it does not change anything in the language_. And yes, `Pin` does not move, but there are also the aliasing rules - and those state that (maybe) every time the `Box` is moved its uniqueness is asserted and any pointers (or references) to it are invalidated. – Chayim Friedman Aug 09 '23 at 09:38
  • @ChayimFriedman that's unfortunate... I tried `ouroboros` but it doesn't work for me, because of the async stuff and lifetime issues... is there no way/library to actually pin something to a fixed address? – Einliterflasche Aug 09 '23 at 13:54
  • Instead of `Box`, you can use a raw pointer. – Chayim Friedman Aug 09 '23 at 13:57
  • @ChayimFriedman that's true. thank you so much for your answers. You may hear from me again when I failed to pin to the heap by allocating\deallocating myself xD – Einliterflasche Aug 09 '23 at 19:20

1 Answers1

2

The problem here is the implicit drop order placed on the fields, in Rust that order is the declaration order meaning your client gets dropped before inner so when inner is dropped and uses it's reference to client that reference is dangling and causes the SIGSEGV you see. A simple fix is to just reorder them:

struct MyTransaction<'a> {
    inner: Transaction<'a>
    client: Pin<Box<Client>>,
}

For more complex scenarios see Forcing the order in which struct fields are dropped

cafce25
  • 15,907
  • 4
  • 25
  • 31