What is an idiomatic way to define a function taking an iterator without consuming in Rust?

Question

I've been working through the 2020 Advent of Code in Rust, trying to produce solutions involving fairly generic functions, which could (in theory) be reused elsewhere without having to worry too much about types. For day 2, the problem involves checking each line of a file to make sure that it is a string conforming to some given condition. My current approach is to define a function similar to the following:

fn filter_lines<'a>(lines: impl IntoIterator<Item = &'a str>) -> Vec<&'a str> {
    lines
        .into_iter()
        .filter(|line| verify_line(line))
        .collect()
}

Here, verify_line is just some function which takes an &str and produces a boolean depending on whether the line conforms to a certain specification.

The issue is that this function appears to necessarily consume the iterator. I've tried rewriting it as follows to use a reference to lines, and clone lines before operating on this, but this code won't compile:

fn filter_lines<'a>(lines: &impl IntoIterator<Item = &'a str>) -> Vec<&'a str> {
    let lines = lines.clone();
    lines
        .into_iter()
        .filter(|line| verify_line(line))
        .collect()
}

The compiler produces the following error:

error[E0507]: cannot move out of `*lines` which is behind a shared reference
  --> src/main.rs:40:5
   |
40 |     lines
   |     ^^^^^ move occurs because `*lines` has type `impl IntoIterator<Item = &'a str>`, which does not implement the `Copy` trait

I think I understand why doing this isn't allowed, but I'm not sure how to idiomatically define a function such as filter_lines which won't consume the iterator. So, what would be a better way to implement this filter_lines function such that it can be called as follows?

let some_str: String = get_file_contents();
let lines = some_str.lines();
println!("Filtered lines: {:?}", filter_lines(&lines));
println!("All lines: {:?}", lines.collect::<Vec<&str>>());

Answer 1

The issue for me is that this function appears to necessarily consume the iterator.

That's going to happen in every case, because that's what IntoIterator does. So if you take an IntoIterator input, it's up to the caller to set it up correctly e.g. Vec has impl IntoIterator for Vec<T> and impl IntoIterator for &[T], so the caller can pass in &v — though that requires some adaptation anyway as the IntoIterator for &[T] yields Iterator<Item=&T>, so here it's an Iterator<Item=&&str> which wouldn't work.

But since every Iterator also implements IntoIterator as well that's not really an issue: the caller can just do some adaptation. Anyway my point is the function works as-is, it's the caller which needs changes.

Now as to the specific snippet:

let some_str: String = get_file_contents();
let lines = some_str.lines();
println!("Filtered lines: {:?}", filter_lines(&lines));
println!("All lines: {:?}", lines.collect::<Vec<&str>>());

that can not work, because Lines is an Iterator it will be consumed by any iteration: Rust iterators are not "replayable", it's only the "iterables" which may be. But it also doesn't really make much sense to try and reuse lines(), what would that even do?

• it could redo the entire thing, but then just call some_str.lines() twice and create two iterators in parallel

• or it could memoise the entire thing, but then just collect() the Lines upfront and iterate on the subsequent Vec however you need, that's basically what you're expecting, and since you're collecting to a vec anyway to print all the lines you might as well do that first:

  let some_str: String = String::from("foo\nbar\nbaz");
  let lines = some_str.lines().collect::<Vec<_>>();
  // Vec<&str> -> Iterator<Item=&&str> -> Iterator<Item=&str>
  println!("Filtered lines: {:?}", filter_lines(lines.iter().map(|s| *s)));
  println!("All lines: {:?}", lines);
  

Answer 2

Not saying this is the idiomatic way to do this, but in your snippet, you are actually cloning the reference because of which it doesn't work.

fn filter_lines<'a>(lines: &impl IntoIterator<Item = &'a str>) -> Vec<&'a str> {
    // Here you are creating a clone of the reference, not iterator
    let lines = lines.clone();
    lines
        .into_iter()
        .filter(|line| verify_line(line))
        .collect()
}

To clone the iterator, you need to impose a trait bound for Clone as well:

fn filter_lines<'a>(lines: &(impl IntoIterator<Item = &'a str> + Clone)) -> Vec<&'a str> {
    let lines = lines.clone();
    lines
        .into_iter()
        .filter(|line| verify_line(line))
        .collect()
}

Because Lines implements Clone as well, this will work.

Playground