How can I write to a BufWriter and get the content that is written?

Question

Is there a way to use a Writer and get the content written to this writer back? Something like a BufWriterReader? I've tried this, but doesn't compile:

#[test]
fn write_to_json_test0() {
    let json = get_schema_without_optionals0(); // this is a thing serde_json can (de)serialize
    let buf = Vec::new();
    let writer = BufWriter::new(buf); // buf moves here

    serde_json::to_writer(writer, &json).unwrap(); // writer moves here

    let s = String::from_utf8(writer.into_inner().unwrap()).unwrap(); // compile error, because writer used after move

    assert_eq!(s, json.to_string());
}

There is a pull-request for a BufWriter::buffer(), but that's not available now.

What is the fastest way to test if to_string() and to_writer() returns the same? Do I really need to write to file and read the file to a string?

Answer 1

the fastest way to test if to_string() and to_writer() returns the same

Look at the source code for to_string:

let vec = try!(to_vec(value));
let string = unsafe {
    // We do not emit invalid UTF-8.
    String::from_utf8_unchecked(vec)
};
Ok(string)

Which calls to_vec:

let mut writer = Vec::with_capacity(128);
try!(to_writer(&mut writer, value));
Ok(writer)

Stuck together, it's almost the same as your code:

let mut writer = Vec::with_capacity(128);
to_writer(&mut writer, value)?;
let string = unsafe {
    // We do not emit invalid UTF-8.
    String::from_utf8_unchecked(writer)
};
Ok(string)

It is more more efficient, however:

• it pre-allocates some space in the Vec
• it avoids the BufWriter (as Sven Marnach points out)
• it skips the UTF-8 validity checks

a pull-request for a BufWriter::buffer()

No, you cannot access the buffer of the BufWriter until that pull request is merged and released — that's part of the reason that it's being added!

written to this writer

It does not seem like you fully understand the purpose of BufWriter. It buffers data that you write to it, so that the underlying writer doesn't get it. The data in the buffer is data that the underlying writer hasn't seen. The pull request you cite will not help you.

In fact, you need to be sure to flush the BufWriter to ensure that no data remains in the buffer before you try to parse the underlying Vec as a string. Thankfully, into_inner is already doing that for you.

writer moves here

This is because Serde is following the "C-RW-VALUE" guideline: Generic reader/writer functions take R: Read and W: Write by value. You can avoid giving away ownership of the writer by using Write::by_ref:

#[test]
fn write_to_json_test0() -> Result<(), io::Error> {
    let json = Value::Number(42.into());

    let buf = Vec::new();
    let mut writer = BufWriter::new(buf);
    serde_json::to_writer(writer.by_ref(), &json)?;
    let s = String::from_utf8(writer.into_inner()?).unwrap();

    assert_eq!(s, json.to_string());
    Ok(())
}

This works because by_ref returns a mutable reference to the writer, and there's a blanket implementation of Write for any mutable reference to a type that itself implements Write:

impl<'a, W: Write + ?Sized> Write for &'a mut W

by_ref is just a convenience function to avoid the relatively strange syntax of taking a mutable reference explicitly. This pattern of by_ref is repeated for Read as well as Iterator.

See also:

• How to use a file with a BufReader and still be able to write to it?
• Why does Iterator::take_while take ownership of the iterator?
• Why does the usage of by_ref().take() differ between the Iterator and Read traits?
• Cannot use moved BufReader after for loop with bufreader.lines()

Answer 2

There is a forwarding implementation of Write for mutable references to Write:

impl<'a, W: Write + ?Sized> Write for &'a mut W

Using this implementation, you can avoid either moving buf or moving writer, by instead just moving a mutable reference.

Note that wrapping a Vec<u8> in a BufWriter isn't really useful. Buffering is meant to reduce the number of writes to a slow backend, but if your backend is memory anyway, you don't gain anything by adding a layer of buffering.

If we remove the BufWriter, your code could look like this:

fn write_to_json_test0() {
    let json = get_schema_without_optionals0();
    let mut buf = Vec::new();
    serde_json::to_writer(&mut buf, &json).unwrap();
    let s = String::from_utf8(buf).unwrap();
    assert_eq!(s, json.to_string());
}