1

I want to create an iterator for iterating through records in a huge file.

for record in huge_file.iter() {
  println!("{}", record.read_field("name"));
}

Each record individually is too big to fit in memory, so the iterator can't return owned Record objects with everything loaded into memory. The underlying storage does not support random access, so returning "slices"/"views" of the storage is also not an option.

Instead, I want the iterator to return elements that allow the consumer to read from the current "cursor position" in the file. Superficially, the API would look like this:

impl Iterator for HugeFileIterator {
  fn next<'a>(&'a mut self) -> Option<&'a mut Record> {
    ...
  }
}

impl Record {
  fn read_field(name: &str) -> &str {
    ...
  }
}

With that signature of next, the mut borrow ensures that the consumer can't keep a reference to the "previous Record" when calling next() to get the "next Record."

However, I can't get that type signature for next to work the way I want it to. The Iterator trait requires me to define an associated Item type, and I don't know how it should look:

impl Iterator for HugeFileIterator {
  type Item<'a> = &'a mut Record; // ← Obviously doesn't work

  fn next<'a>(&'a mut self) -> Option<Self::Item<'a>> {
    ...
  }
}

Is it possible to express this constraint in the Rust type system? Or is there perhaps another completely different approach that I haven't thought about?

dflemstr
  • 25,947
  • 5
  • 70
  • 105
  • std::slice::IterMut is an example of an iterator that iterates over "mutable references to the values". Perhaps it can serve as an inspiration. https://doc.rust-lang.org/std/slice/struct.IterMut.html – nielsle Nov 15 '15 at 06:56
  • @nielsle That looks interesting! Sadly, the `IterMut` holds a `&mut` for the *iterator*, while I want to hold a `&mut` for every *element* in the iterator. – dflemstr Nov 15 '15 at 11:21

0 Answers0