Can I use itertools::PutBack::put_back in a loop?

Question

My use case:

• Loop over string
• React to each character in a lexical analyzer state machine
• On seeing some characters, realize that the previous character was the end of its token
• finish up the token
• transition back to the Empty state (meaning no token is partially constructed)
• push the "extra" character back onto the iterator with put_back
• continue the state machine processing, where the put back character will be available for the first character in the next token.

Here is an example of an attempt:

use itertools::put_back; // 0.8.0

fn main() {
    let hello = "Hello world".to_owned();
    let hello_iter = hello.chars();
    let mut putback_iterator = put_back(hello_iter);
    let mut already_putback = false;
    for c in putback_iterator {
        if c == 'd' && !already_putback {
            putback_iterator.put_back('!');
            already_putback = true;
        }
        println!("Char is {}", c.to_string());
    }
}

The error message:

error[E0382]: borrow of moved value: `putback_iterator`
  --> src/main.rs:10:13
   |
6  |     let mut putback_iterator = put_back(hello_iter);
   |         -------------------- move occurs because `putback_iterator` has type `itertools::adaptors::PutBack<std::str::Chars<'_>>`, which does not implement the `Copy` trait
7  |     let mut already_putback = false;
8  |     for c in putback_iterator {
   |              ---------------- value moved here
9  |         if c == 'd' && !already_putback {
10 |             putback_iterator.put_back('!');
   |             ^^^^^^^^^^^^^^^^ value borrowed here after move

How would I loop over the characters and perform a put_back? I can't find any worked examples using put_back.

Answer 1

for loops allow you to iterate over anything which implements IntoIterator. This trait defines into_iter(self) which consumes the object it is being called on and returns an iterator. This is true even when self is already an iterator (as it is in your code).

Thus, the for loop consumes the iterator, making it inaccessible within the loop.

An alternative is to use a while let loop instead:

use itertools::put_back; // 0.8.0

fn main() {
    let hello = "Hello world".to_owned();
    let hello_iter = hello.chars();
    let mut putback_iterator = put_back(hello_iter);
    let mut already_putback = false;
    while let Some(c) = putback_iterator.next() {
        if c == 'd' && !already_putback {
            putback_iterator.put_back('!');
            already_putback = true;
        }
        println!("Char is {}", c.to_string());
    }
}

Playground Link