Extract chain of iterator calls to a helper function

Question

I'm trying to write a function that will encapsulate a series of chained iterator method calls (.lines().map(...).filter(...)) which I currently have duplicated. I can't figure out the type signatures to get this to compile. If this is impossible or highly unidiomatic for Rust, I'm open to suggestions for an idiomatic approach.

use std::fs;
use std::io;
use std::io::prelude::*;
use std::iter;

const WORDS_PATH: &str = "/usr/share/dict/words";

fn is_short(word: &String) -> bool {
    word.len() < 7
}

fn unwrap(result: Result<String, io::Error>) -> String {
    result.unwrap()
}

fn main_works_but_code_dupe() {
    let file = fs::File::open(WORDS_PATH).unwrap();
    let reader = io::BufReader::new(&file);
    let count = reader.lines().map(unwrap).filter(is_short).count();
    println!("{:?}", count);

    let mut reader = io::BufReader::new(&file);
    reader.seek(io::SeekFrom::Start(0));
    let sample_size = (0.05 * count as f32) as usize; // 5% sample

    // This chain of iterator logic is duplicated
    for line in reader.lines().map(unwrap).filter(is_short).take(sample_size) {
        println!("{}", line);
    }
}

fn short_lines<'a, T>
    (reader: &'a T)
     -> iter::Filter<std::iter::Map<std::io::Lines<T>, &FnMut(&str, bool)>, &FnMut(&str, bool)>
    where T: io::BufRead
{
    reader.lines().map(unwrap).filter(is_short)
}

fn main_dry() {
    let file = fs::File::open(WORDS_PATH).unwrap();
    let reader = io::BufReader::new(&file);
    let count = short_lines(reader).count();
    println!("{:?}", count);

    // Would like to do this instead:
    let mut reader = io::BufReader::new(&file);
    reader.seek(io::SeekFrom::Start(0));
    let sample_size = (0.05 * count as f32) as usize; // 5% sample
    for line in short_lines(reader).take(sample_size) {
        println!("{}", line);
    }
}


fn main() {
    main_works_but_code_dupe();
}

Answer 1

I can't figure out the type signatures to get this to compile.

The compiler told you what it was.

error[E0308]: mismatched types
  --> src/main.rs:35:5
   |
35 |     reader.lines().map(unwrap).filter(is_short)
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected reference, found fn item
   |
   = note: expected type `std::iter::Filter<std::iter::Map<_, &'a for<'r> std::ops::FnMut(&'r str, bool) + 'a>, &'a for<'r> std::ops::FnMut(&'r str, bool) + 'a>`
              found type `std::iter::Filter<std::iter::Map<_, fn(std::result::Result<std::string::String, std::io::Error>) -> std::string::String {unwrap}>, for<'r> fn(&'r std::string::String) -> bool {is_short}>`

Now, granted, you can't just copy+paste this directly. You have to replace the _ type with the actual one you already had (it left it out because it was already correct). Secondly, you need to delete the {unwrap} and {is_short} bits; those are because function items have unique types, and that's how the compiler annotates them. Sadly, you can't actually write these types out.

Recompile and...

error[E0308]: mismatched types
  --> src/main.rs:35:5
   |
32 |      -> std::iter::Filter<std::iter::Map<std::io::Lines<T>, fn(std::result::Result<std::string::String, std::io::Error>) -> std::string::String>, for<'r> fn(&'r std::string::String) -> bool>
   |         -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- expected `std::iter::Filter<std::iter::Map<std::io::Lines<T>, fn(std::result::Result<std::string::String, std::io::Error>) -> std::string::String>, for<'r> fn(&'r std::string::String) -> bool>` because of return type
...
35 |     reader.lines().map(unwrap).filter(is_short)
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected fn pointer, found fn item
   |
   = note: expected type `std::iter::Filter<std::iter::Map<_, fn(std::result::Result<std::string::String, std::io::Error>) -> std::string::String>, for<'r> fn(&'r std::string::String) -> bool>`
              found type `std::iter::Filter<std::iter::Map<_, fn(std::result::Result<std::string::String, std::io::Error>) -> std::string::String {unwrap}>, for<'r> fn(&'r std::string::String) -> bool {is_short}>`

Remember what I said about function items have unique types? Yeah, that. To fix that, we cast from a function item to a function pointer. We don't even need to specify what we're casting to, we just have to let the compiler know we want it to do a cast.

fn short_lines<'a, T>
    (reader: &'a T)
     -> std::iter::Filter<std::iter::Map<std::io::Lines<T>, fn(std::result::Result<std::string::String, std::io::Error>) -> std::string::String>, for<'r> fn(&'r std::string::String) -> bool>
    where T: io::BufRead
{
    reader.lines().map(unwrap as _).filter(is_short as _)
}

error[E0308]: mismatched types
  --> src/main.rs:41:29
   |
41 |     let count = short_lines(reader).count();
   |                             ^^^^^^ expected reference, found struct `std::io::BufReader`
   |
   = note: expected type `&_`
              found type `std::io::BufReader<&std::fs::File>`
   = help: try with `&reader`

Again, the compiler tells you exactly what to do. Make the change and...

error[E0507]: cannot move out of borrowed content
  --> src/main.rs:35:5
   |
35 |     reader.lines().map(unwrap as _).filter(is_short as _)
   |     ^^^^^^ cannot move out of borrowed content

Right, that's because you got the input of short_lines wrong. One more change:

fn short_lines<T>
    (reader: T)
     -> std::iter::Filter<std::iter::Map<std::io::Lines<T>, fn(std::result::Result<std::string::String, std::io::Error>) -> std::string::String>, for<'r> fn(&'r std::string::String) -> bool>
    where T: io::BufRead
{
    reader.lines().map(unwrap as _).filter(is_short as _)
}

And now all you have to deal with are warnings.

In short: read the compiler messages. They're useful.

Answer 2

I'd suggest doing it the simple way - collecting the iterator into a vector:

use std::fs;
use std::io;
use std::io::prelude::*;

const WORDS_PATH: &str = "/usr/share/dict/words";

fn main() {
    let file = fs::File::open(WORDS_PATH).unwrap();
    let reader = io::BufReader::new(&file);
    let short_lines = reader.lines()
                            .map(|l| l.unwrap())
                            .filter(|l| l.len() < 7)
                            .collect::<Vec<_>>(); // the element type can just be inferred
    let count = short_lines.len();
    println!("{:?}", count);

    let sample_size = (0.05 * count as f32) as usize; // 5% sample

    for line in &short_lines[0..sample_size] {
        println!("{}", line);
    }
}