Why can't I send multiple requests in parallel using rust tonic?

Question

I implemented the tonic helloworld tutorial. I then tried to change the client code so that I could send multiple requests before awaiting any.

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let num_requests = 10;
    let mut client = GreeterClient::connect("http://[::1]:50051").await?;

    let mut responses = Vec::with_capacity(num_requests);
    for _ in 0..num_requests {
        let request = tonic::Request::new(HelloRequest {
            name: "Tonic".into(),
        });

        responses.push(client.say_hello(request));
    }
    for resp in responses {
        assert!(resp.await.is_ok());
    }

    Ok(())
}

This results in a compilation error:

error[E0499]: cannot borrow `client` as mutable more than once at a time
  --> src/client.rs:19:24
   |
19 |         responses.push(client.say_hello(request));
   |                        ^^^^^^ mutable borrow starts here in previous iteration of loop

Does that mean 'client.say_hello()' returns a type which still references client, and therefore I can't make another call to 'say_hello', which itself requires '&mut self'? Is there a way to continue to make requests before calling to 'await'?

score 7 · Accepted Answer · answered Jan 29 '21 at 17:53

From the Tonic documentation:

Sending a request on a channel requires a &mut self and thus can only send one request in flight. This is intentional and is required to follow the Service contract from the tower library which this channel implementation is built on top of.
...
To work around this and to ease the use of the channel, Channel provides a Clone implementation that is cheap. This is because at the very top level the channel is backed by a tower_buffer::Buffer which runs the connection in a background task and provides a mpsc channel interface. Due to this cloning the Channel type is cheap and encouraged.

Therefore, you can clone the client for each concurrent request you make. This eliminates the possibility of a single client being mutably borrowed more than once at any given time, so the borrow checker is appeased.

let num_requests = 10;
let client = GreeterClient::connect("http://[::1]:50051").await?;

let mut responses = Vec::with_capacity(num_requests);

for _ in 0..num_requests {
    let mut client = client.clone();

    let request = tonic::Request::new(HelloRequest {
        name: "Tonic".into(),
    });

    responses.push(tokio::spawn(async move {
        client.say_hello(request).await
    }));
}

for resp in responses {
    let resp = resp.await;
    assert!(resp.is_ok());
    assert!(resp.unwrap().is_ok());
}

Why can't I send multiple requests in parallel using rust tonic?

1 Answers1