1

I want to use library with synchronous file IO in asynchronous application. I also want all file operations work asynchronously. Is that possible? Something like this:

// function in other crate with synchronous API
fn some_api_fun_with_sync_io(r: &impl std::io::Read) -> Result<(), std::io::Error> {
    // ...
}

async fn my_fun() -> anyhow::Result<()> {
    let mut async_file = async_std::fs::File::open("test.txt").await?;

    // I want some magic here ))
    let mut sync_file = magic_async_to_sync_converter(async_file);

    some_api_fun_with_sync_io(&mut sync_file)?;

    Ok(())
}
John Kugelman
  • 349,597
  • 67
  • 533
  • 578
ArtDen
  • 253
  • 2
  • 7

2 Answers2

2

I don't think this magic exists yet, but you can conjure it up yourself with async_std::task::block_on:

fn magic_async_to_sync_converter(async_file: AsyncFile) -> Magic {
    Magic(async_file)
}

struct Magic(AsyncFile);

impl SyncRead for Magic {
    fn read(&mut self, buf: &mut [u8]) -> std::io::Result<usize> {
        block_on(self.0.read(buf))
    }
}

use std::io::Read as SyncRead;

use async_std::{
    fs::File as AsyncFile,
    io::ReadExt,
    task::{block_on, spawn_blocking},
};

But since some_api_fun_with_sync_io is now doing blocking io, you'll have to shove it into a blocking io thread with spawn_blocking:

spawn_blocking(move || some_api_fun_with_sync_io(sync_file)).await?;

You might want to revise your design and see whether you can do without this though. spawn_blocking is still marked as unstable by async_std.

Caesar
  • 6,733
  • 4
  • 38
  • 44
  • Thanks for idea about `async_std::task::block_on`. I already tried to write same but didn't know how to use .await in sync code. `async_std::task::block_on` solves it. But it looks like you meant `impl Read for Magic` instead of `impl SyncRead for Magic`? – ArtDen Feb 05 '22 at 10:15
  • Ah, sorry for the confusion, I have the habit or renaming things when using both sync and async facilities in one piece of code. See that `use std::io::Read as SyncRead`? – Caesar Feb 05 '22 at 10:25
  • Opps I did't notice `use std::io::Read as SyncRead`. I post some benchmarks of your idea below – ArtDen Feb 05 '22 at 10:40
0

Benchmarking idea of @Caesar :

use async_std::prelude::*;
use std::time::*;

struct AsyncToSyncWriteCvt<T: async_std::io::Write + Unpin> (T);

impl<T: async_std::io::Write + Unpin> std::io::Write for AsyncToSyncWriteCvt<T> {
    fn write(&mut self, buf: &[u8]) -> std::io::Result<usize> {
        async_std::task::block_on(self.0.write(buf))
    }
    fn flush(&mut self) -> std::io::Result<()> {
        async_std::task::block_on(self.0.flush())
    }
}

fn test_sync<W: std::io::Write>(mut w: W) -> Result<(), std::io::Error> {
    for _ in 0..1000000 { w.write("test test test test ".as_bytes())?; }
    Ok(())
}

async fn test_async<T: async_std::io::Write + Unpin>(mut w: T) -> Result<(), std::io::Error> {
    for _ in 0..1000000 { w.write("test test test test ".as_bytes()).await?; }
    Ok(())
}

fn main() -> anyhow::Result<()> {
    async_std::task::block_on(async {
        // bench async -> sync IO
        let now = Instant::now();
        let async_file = async_std::fs::File::create("test1.txt").await?;
        let sync_file = AsyncToSyncWriteCvt(async_file);
        test_sync(sync_file)?;
        println!("Async -> sync: {:.2}s", now.elapsed().as_secs_f32());

        // bench sync IO
        let now = Instant::now();
        let sync_file = std::fs::File::create("test2.txt")?;
        test_sync(sync_file)?;
        println!("Sync: {:.2}s", now.elapsed().as_secs_f32());

        // bench async IO
        let now = Instant::now();
        let async_file = async_std::fs::File::create("test3.txt").await?;
        test_async(async_file).await?;
        println!("Async: {:.2}s", now.elapsed().as_secs_f32());

        Ok(())
    })
}

This code shows "sync -> async" file writing as fast as "async" file writing but less fast then direct sync writing. BufWriter allow to speed up and to close the speed gap between sync and async

ArtDen
  • 253
  • 2
  • 7