1

Tokio (I'm using 0.2, with futures 0.3) has process::Command.output(), which returns the combined output (both stdout and stderr) from a child process after it's complete. However, I would like to stream the output for a few reasons:

  1. It might be larger than available memory.

  2. The stdout and stderr output may be interleaved (I realise that in general the relationship between the two will not be something that can be relied upon, but I'd rather make as much of an effort as possible to keep the results as similar as running the process normally).

  3. I don't want to wait for the process to terminate before output starts being returned.

I tried using select! with a pair of futures and a loop:

let outbuf = [0u8; 512];
let errbuf = [0u8; 512];

let outfut = child_stdout.read(&mut outbuf).fuse();
let errfut = child_stderr.read(&mut errbuf).fuse();

pin_mut!(outfut, errfut);

loop {
    select! {
        result = outfut => {
            let length = result?;
            if length != 0 {
                 output.write_all(&outbuf[..length]).await?
                 outfut.set(child_stdout.read(&mut outbuf).fuse())
            }
        }
        result = errfut => {
            let length = result?;
            if length != 0 {
                 output.write_all(&errbuf[..length]).await?
                 errfut.set(child_stderr.read(&mut errbuf).fuse())
            }
        }
        complete => break
    }
}

let status = child.await?

Of course, this won't work because I've borrowed the buffers twice; I've tried rearranging the code in various ways, but when using select! like this I always seem to end up having to borrow something twice. (Note also that in practice there's a little more output in the if statements than I'm showing above, and the stdout and stderr branches differ from one another, so you can't just redirect both stdout and stderr to the same place and call it a day.)

Clearly this is not the right approach. Is there an idiomatic Rust way to achieve what I'm after here?

al45tair
  • 4,405
  • 23
  • 30
  • It looks like your question might be answered by the answers of [How do I read the output of a child process without blocking in Rust?](https://stackoverflow.com/a/55565595/155423); [How do I use async/await syntax with Tokio?](https://stackoverflow.com/a/54854883/155423). If not, please **[edit]** your question to explain the differences. Otherwise, we can mark this question as already answered. – Shepmaster Dec 17 '19 at 14:39
  • It's hard to answer your question because it doesn't include a [MRE]. We can't tell what crates (and their versions), types, traits, fields, etc. are present in the code. It would make it easier for us to help you if you try to reproduce your error on the [Rust Playground](https://play.rust-lang.org) if possible, otherwise in a brand new Cargo project, then [edit] your question to include the additional info. There are [Rust-specific MRE tips](//stackoverflow.com/tags/rust/info) you can use to reduce your original code for posting here. Thanks! – Shepmaster Dec 17 '19 at 14:39
  • @Shepmaster It might _look_ as if it might be answered by those, but it isn't. I saw those (and others). All the answers on here - and that I found in Google - either (a) only deal with one of the two streams (which is much easier), or (b) don't actually stream the output, but instead buffer it until the process ends. – al45tair Dec 17 '19 at 14:52
  • As for a "minimal reproducible example", I'm not sure there's much point; it's obvious why the code above won't build (multiple borrows of outbuf/errbuf). If there's some structural thing that I'm missing, I'm sure someone more experienced with Rust than myself will be able to point it out. – al45tair Dec 17 '19 at 14:54
  • So you are using futures 0.1 and tokio 0.1? Please read the *entirety* of my comment, where i discuss all of the parts that make up a [MRE], – Shepmaster Dec 17 '19 at 14:58
  • *I saw those (and others).* — please **link** to questions and answers that you've found. This serves two purposes: (1) it helps show that you did a minimal amount of work before asking the question (2) you can describe what you don't understand about those answers or how this question differs. Please remember that Stack Overflow is primarily about building a knowledge base of high-quality questions. – Shepmaster Dec 17 '19 at 15:01
  • You could also provide a small shell script that generates output to stdout/stderr, or anything else about the boilerplate of the solution, reducing the time it would take for someone to answer you. – Shepmaster Dec 17 '19 at 15:02
  • @Shepmaster Honestly, if it's that much trouble, I'll just delete the question, use Go instead (where it seems easy to do this) and be done with it. – al45tair Dec 17 '19 at 15:09
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/204418/discussion-between-shepmaster-and-alastair). – Shepmaster Dec 17 '19 at 15:12

0 Answers0