Update a mutable reference in a loop

Question

This is an example that doesn't work:

fn main() {
    struct Node {
        children: Vec<Node>,
    }
    impl Node {
        fn new() -> Node {
            Node {
                children: Vec::new(),
            }
        }
    }

    let mut node = Node::new();
    node.children.push(Node::new());

    let mut x = &mut node;
    loop {
        x = &mut x.children[0];
    }
}

Running this, I get an error of:

error[E0506]: cannot assign to `x` because it is borrowed
  --> src/main.rs:18:9
   |
18 |         x = &mut x.children[0];
   |         ^^^^^^^^^----------^^^
   |         |        |
   |         |        borrow of `x` occurs here
   |         assignment to borrowed `x` occurs here

error[E0499]: cannot borrow `x.children` as mutable more than once at a time
  --> src/main.rs:18:18
   |
18 |         x = &mut x.children[0];
   |                  ^^^^^^^^^^ mutable borrow starts here in previous iteration of loop
19 |     }
20 | }
   | - mutable borrow ends here

Is there any way to update x to a mutable reference of one of x's child nodes without re-borrowing x?