Desire to iterate over two word list files in Rust

Question

Currently, regardless of the different attempts that I have tried. I cannot pass the wordlist words from both iterations to a new thread.

I am attempting to build a web application fuzzing tool, and I want the ability to use at least two different wordlists at the same time, like other popular fuzzing tools can do. However, I run into errors attempting to do so in Rust.

My guess is its because by the second-pass on the second list the items have already been borrowed once and I cannot borrow them a second time?

Here is the snippet of code and its error:

fn read_lines<P>(filename: P) -> io::Result<io::Lines<io::BufReader<File>>>
where P: AsRef<Path> {
    let file = File::open(filename)
        .expect("Could not open word list.");
    Ok(io::BufReader::new(file)
        .lines())
}

fn compound_multiple(word_list: &Vec<String>) {
    let pool = ThreadPool::new(10);
    if let Ok(lines) = read_lines(word_list[0].as_str()) {
        for line in lines {
            if let Ok(word) = line {
                if let Ok(lines2) = read_lines(word_list[1].as_str()) {
                    for (counter, line2) in lines2.enumerate() {
                        if let Ok(word2) = line2 {
                            pool.execute(move || {
                                println!("{:?}{:?}{:?}", word, word2, counter + 1);
                            })
                        }
                    }
                }
            }
        }
    }

The error:

error[E0382]: use of moved value: `word`
   --> src/utils/fuzzer.rs:122:42
    |
118 |             if let Ok(word) = line {
    |                       ----
    |                       |
    |                       this reinitialization might get skipped
    |                       move occurs because `word` has type `std::string::String`, which does not implement the `Copy` trait
...
122 |                             pool.execute(move || {
    |                                          ^^^^^^^ value moved into closure here, in previous iteration of loop
123 |                                 println!("{:?}{:?}{:?}", word, word2, counter + 1);
    |                                                          ---- use occurs due to use in closure

For more information about this error, try `rustc --explain E0382`.

I have attempted to clone() the words on each iteration, but it produces the same error message.

I have tried:

    let word_list_one = File::open(word_list[0]).unwrap();
    let mut reader_one = BufReader::new(word_list_one); 
    let word_list_two = File::open(word_list[1]).unwrap();
    let mut reader_two = BufReader::new(word_list_two);
    for line_one in reader_one.by_ref().lines() {
        let line_one = line_one.unwrap().clone();
        for (counter, line_two) in reader_two.by_ref().lines().enumerate() {
            let line_two = line_two.unwrap().clone();
            pool.execute(move || {
               println!("{:?}{:?}{:?}", line_one, line_two, counter + 1);
            })
        }
    }
}

I have also tried the suggestion from this Stack question --> Trying to iterate 2 files in rust

Appreciate any help!

Answer 1

First, I made your code a little more readable. You may want to go further and use and_then a few times, or change the return type of the function and use ?.

pub fn compound_multiple(word_list: &Vec<String>) {
    let pool = ThreadPool::new(10);
    let Ok(lines) = read_lines(word_list[0].as_str()) else {
        return;
    };

    for line in lines {
        let Ok(word) = line else { return };
        let Ok(lines2) = read_lines(word_list[1].as_str()) else {
            return;
        };

        for (counter, line2) in lines2.enumerate() {
            let Ok(word2) = line2 else { return };

            pool.execute(move || {
                println!("{:?}{:?}{:?}", word, word2, counter + 1);
            });
        }
    }
}

Now, the solution. You need to make another clone for every closure you move. That means cloning in the inner loop.

pub fn compound_multiple(word_list: &Vec<String>) {
    let pool = ThreadPool::new(10);
    let Ok(lines) = read_lines(word_list[0].as_str()) else {
        return;
    };

    for line in lines {
        let Ok(word) = line else { return };
        let Ok(lines2) = read_lines(word_list[1].as_str()) else {
            return;
        };

        for (counter, line2) in lines2.enumerate() {
            let Ok(word2) = line2 else { return };
            let word = word.clone();

            pool.execute(move || {
                println!("{:?}{:?}{:?}", word, word2, counter + 1);
            });
        }
    }
}

If all you need is read-only access to the strings, then you can use Arc<str>, which allows you to keep only one copy of each word from the first word list in memory, instead of making a separate copy for every word in the second word list. You can use this in the same way you use &str, and it will free the memory when the last clone is dropped. If you need to explicitly coerce Arc<str> to &str, you can dereference it, like &*word. (playground)

pub fn compound_multiple(word_list: &Vec<String>) {
    let pool = ThreadPool::new(10);
    let Ok(lines) = read_lines(word_list[0].as_str()) else {
        return;
    };

    for line in lines {
        let Ok(word) = line else { return };
        let Ok(lines2) = read_lines(word_list[1].as_str()) else {
            return;
        };

        let word: Arc<str> = word.into();

        for (counter, line2) in lines2.enumerate() {
            let Ok(word2) = line2 else { return };
            let word = word.clone();

            pool.execute(move || {
                println!("{:?}{:?}{:?}", word, word2, counter + 1);
            });
        }
    }
}