13

I'm following this tutorial and I'm having trouble understanding the following code:

mail_account.serialize(&mut &mut account.data.borrow_mut()[..])?;

where mail_account is a struct and .serialize() is a method derived from Borsh Serialize.

One other thing to note is that account is an AccountInfo struct from the solana_program crate and data has type: Rc<RefCell<&'a mut [u8]>>

The author explains the code as follows:

You might notice the tricky &mut &mut account.data.borrow_mut()[..] expression. The serialize() method takes a reference to a mutable slice of u8 as an argument, the borrow_mut() method returns a RefMut. We can't pass RefMut to a method that expects a slice, so we take a mutable slice of RefMut which returns a mutable slice of u8

What I do understand is that we want to write the current mail_account struct into account.data, which is why we are borrowing a mutable reference to account.data. What I don't understand is why is it important to add the [..]? I assume it has something to do with the serialize() expecting a slice.

I'm also having trouble understanding how adding &mut &mut to account.data.borrow_mut()[..] creates a slice of u8. How is this being converted to u8? To me it seems like it would just be adding additional mutable references to the original one. (How is the type changing from RefMut to &[u32]?)

John Kugelman
  • 349,597
  • 67
  • 533
  • 578
Craig
  • 395
  • 3
  • 14

2 Answers2

7

So, I guess, you have multiple questions:

  1. What I don't understand is why is it important to add the [..]?

  2. I'm also having trouble understanding how adding &mut &mut to account.data.borrow_mut()[..] creates a slice of u8. How is this being converted to u8?

Those questions are kinda intertwined, as you can see inthe anwser to question 1.

Question 1

When we look at the documentation about certain cases of indexing, we see, that

account.data.borrow_mut()[..]

is sugar for

*(account.data.borrow_mut().index_mut(..))

Why is that a valid expression?

.. is a shorthand for RangeFull.

RangeFull has an implementation for SliceIndex<[u8]>.

With this blanket implementation we get a IndexMut<RangeFull> for [u8], which provides

fn index_mut(&mut [u8], index: RangeFull) -> &mut [u8]

Now the deref coercion and or the auto dereferencing mentioned in other answers and comments kicks in.

account.data.borrow_mut().index_mut(..)

And RefMut<&mut [u8]> implements DerefMut which has Deref<Target = &mut [u8]> as a super trait.

And &mut [u8] implements DerefMut with Deref<Target = [u8]> as super trait.

As mentioned in the reference, the compiler will now take the receiver expression and dereference it repeatedly, so it gets a list of candidate types. It also adds for each type resulting from a dereference the reference type and the mutable reference type to the list of candidate types. From this candidate types it selects one providing the method to call.

  1. RefMut<&mut [u8]> using account.data.borrow_mut()
  2. &RefMut<&mut [u8]>
  3. &mut RefMut<&mut [u8]>
  4. &mut [u8] using *account.data.borrow_mut().deref_mut()
  5. &&mut [u8]
  6. &mut &mut [u8]
  7. [u8] using *(*account.data.borrow_mut().deref_mut())
  8. &[u8]
  9. &mut [u8]

(In 7. we are dereferencing a pointer type &mut [u8] so no DerefMut Trait is used.)

The first (and only) type in this list providing an index_mut() method is &mut [u8], via the IndexMut<FullRange> implementation for [u8], so &mut [u8] is selected as receiver type. The return type of index_mut() is &mut [u8] as well.

So now, we hopefully understand, the type of

*(account.data.borrow_mut().index_mut(..)) is [u8].

Hence:

Question 2

&mut &mut account.data.borrow_mut()[..]

has type &mut &mut [u8].

Addendum

&mut &mut [u8] is needed as &mut [u8] has a Write implementation.

And serialize

pub fn serialize<W: Write>(&self, writer: &mut W) -> Result<()>

needs an argument of type &mut W, where W implements Write.

The reference to a value of type W implementing Write needs to mutable, as we want to keep track of the actual writing position in the value of type W. In the case of &mut [u8] we simply alter the reference to start at a different position in the underlying slice, so we need a mutable reference to the mutable reference, as we want to alter the mutable reference itself and not only the underlying data.

Addendum 2

Just use

mail_account.serialize(&mut *account.data.borrow_mut())?;
typetetris
  • 4,586
  • 16
  • 31
  • In the auto dereferencing section for 4. isn't `*account...` dereferencing and then we're also dereferencing again with `...().deref_mut()`? Why isn't it just one dereference to go from `RefMut<&mut [u8]>` to `&mut [u8]`? I tried printing the types of `account.data.borrow_mut().deref_mut()` and `*account.data.borrow_mut().deref_mut()` to understand but encounter `deref_mut() not found in RefMut<'_, &mut [u8]>` for both expressions. – Craig Oct 22 '21 at 20:14
  • Oops I wasn't using `std::ops::DerefMut` and also learnt that `deref_mut()` is actually just returning another reference. Now I get as `&mut &mut [u8]` for `account.data.borrow_mut().deref_mut()` which I understand as a mutable reference to the mutable reference for the slice. I'm running into an ownership error when I try `*account.data.borrow_mut().deref_mut()`: `Cannot move out of a mutable reference. move occurs because value has type &mut [u8]`. – Craig Oct 22 '21 at 22:30
  • 1
    As you can see in the [reference](https://doc.rust-lang.org/reference/expressions/operator-expr.html#the-dereference-operator) for non pointer types `*v` is simply `*std::ops::DerefMut::deref_mut(&mut v)`. So dereferencing implies using `deref_mut` but still apply the dereferencing operator `*` to the result. And `RefMut<_>` is a non pointer type to start with. – typetetris Oct 23 '21 at 06:36
  • 1
    `let t = *something;` for a something of type `&mut T` would mean we try to get ownership (not just a borrow) for the value referenced by `something`. (Unless `T` implements `Copy`.) You can use `*something` to assign values to and call methods or maybe also other stuff, but you can't move the value out of it. – typetetris Oct 23 '21 at 06:39
1

This magic is called type coercion. You can read about it here or here. Basically what happens is an implicit conversion of Rc<RefCell<&'a mut [u8]>> into &mut [u8]. The way Rust does it is as following:

  1. Auto-dereference when use . operator - Rc<RefCell<...> is dereferenced into RefCell<> when you call .borrow_mut().
  2. borrow_mut returns a value of type RefMut<T>, which implements trait DerefMut<T>.
  3. The method .serialize takes &mut [u8] as argument, so you cannot call it with just &mut data.borrow_mut() as it has the type of &mut RefMut<&mut [u8]>.
  4. Adding additional &mut before RefMut allows Rust compiler to use the type coercion. Seeing the type &mut &mut RefMut<&mut [u8]> it can automatically convert inner &mut RefMut<T> into just &mut T, so you get &mut &mut &mut [u8]. Then it can repeat the process twice, each time turning &mut &mut into &mut. So in the end you get &mut [u8].

The more explicit way to do the same thing is to derefrence RefMut:

&mut *account.data.borrow_mut()

It does the same thing but directly.

John Kugelman
  • 349,597
  • 67
  • 533
  • 578
Maxim Gritsenko
  • 2,396
  • 11
  • 25
  • 1
    No, there is not deref coercion happening here. It's rather auto-dereferencing as part of method lookup, [which is a distinct mechanism](https://stackoverflow.com/a/53344847/279627). – Sven Marnach Oct 22 '21 at 09:07
  • 3
    The method `.serialize()` does not accept an `&mut [u8]`. It rather accepts an `&mut W` with `W` implementing the `Write` trait. The trait is implemented for `&mut [u8]`, so to pass in an `&mut W`, you actually need `&mut &mut [u8]`. (The `serialize()` method _should_ simply accept a `W` instead.) – Sven Marnach Oct 22 '21 at 09:14
  • The additional `&mut` is completely unrelated to any type coercion, not least because there isn't any type coercion happening here. – Sven Marnach Oct 22 '21 at 09:15
  • @SvenMarnach Thank for the pointer to auto-dereferencing, so I could rework my bit of handwaving. – typetetris Oct 22 '21 at 09:32