How can I have a grace period between starting an asynchronous operation and waiting for its result in a loop?

Question

I have a loop:

let grace = 2usize;
for i in 0..100 {
    if i % 10 == 0 {
        expensive_function()
    } else {
        cheap_function()
    }
}

The goal is that when it hits expensive_function(), it runs asynchronously and allows grace number of further iterations until waiting on expensive_function().

If expensive_function() triggers at iteration 10, it could then run iterations 11 and 12 before needing to wait for the expensive_function() run on iteration 10 to finish to continue.

How could I do this?

In my case expensive_function() is effectively:

fn expensive_function(&b) -> Vec<_> {
    return b.iter().map(|a| a.inner_expensive_function()).collect();
}

As such I plan to use multi-threading within this function.

So you need the result only until you run into the if again ? So your two branch are totally independent ? — Stargateur, Jun 22 '20 at 17:10
@Stargateur At the extreme, yes. But not ideally. This is code for calculating a bunch of productions in a game. On the last day of the month production is calculated for that month, now players might not notice if it is not done until the 2nd day of the next month (if time is passing quickly), this helps deal with the sort of stutter and time halting you might get otherwise as it runs this calculation. Hope that gives some context. — Jonathan Woollett-light, Jun 22 '20 at 17:15

Shepmaster · Accepted Answer · 2020-06-22T19:17:58.603

When you start the expensive computation, store the resulting future in a variable, along with the deadline time to wait for the result. Here, I use an Option of a tuple:

use std::{thread, time::Duration};
use tokio::task; // 0.2.21, features = ["full"]

#[tokio::main]
async fn main() {
    let grace_period = 2usize;
    let mut pending = None;

    for i in 0..50 {
        if i % 10 == 0 {
            assert!(pending.is_none(), "Already had pending work");

            let future = expensive_function(i);
            let deadline = i + grace_period;
            pending = Some((deadline, future));
        } else {
            cheap_function(i);
        }

        if let Some((deadline, future)) = pending.take() {
            if i == deadline {
                future.await.unwrap();
            } else {
                pending = Some((deadline, future));
            }
        }
    }
}

fn expensive_function(n: usize) -> task::JoinHandle<()> {
    task::spawn_blocking(move || {
        println!("expensive_function {} start", n);

        thread::sleep(Duration::from_millis(500));

        println!("expensive_function {} done", n);
    })
}

fn cheap_function(n: usize) {
    println!("cheap_function {}", n);
    thread::sleep(Duration::from_millis(1));
}

This generates the output of

cheap_function 1
expensive_function 0 start
cheap_function 2
expensive_function 0 done
cheap_function 3
cheap_function 4
cheap_function 5

Since you did not provide definitions of expensive_function and cheap_function, I have provided appropriate ones.

One tricky thing here is that I needed to add the sleep call in the cheap_function. Without it, my OS never schedules the expensive thread until it's time to poll it, effectively removing any parallel work. In a larger program, the OS is likely to schedule the thread simply because more work will be done by cheap_function. You might also be able to use thread::yield_now to the same effect.

How can I have a grace period between starting an asynchronous operation and waiting for its result in a loop?

1 Answers1