Can i iterate by chunks using zip in rust

Question

In python I can write the following to iterate by tuples.

it = iter(range(10))
zip(it, it) # [(0, 1), (2, 3), (4, 5), (6, 7), (8, 9)]

Rust wouldn't let me borrow iterator twice (or use same iterator twice because of the move).

let mut i1 = a.iter();
let i2 = &mut i1;
i1.zip(i2).for_each(|(a, b)| println!("a: {}, b: {}", a, b));

I know about itertools crate, I just wonder if there's some hack that would allow me to get by without it if I only need this functionality.

Obviously you can do something like that.

struct Chunks<I: Iterator<Item = T, T> {
    seq: I,
}

impl<I: Iterator<Item = T>, T> Iterator for Chunks<I, T> {
    type Item = (T, T);
    fn next(&mut self) -> Option<Self::Item> {
        self.seq.next().zip(self.seq.next())
    }
}

But that works only for tuples and not for triples. Triples would require some kind of macroses.

With std::iter::from_fn you can create one-liner (thanks to @user4815162342).

let mut seq = a.iter();
let chunks = std::iter::from_fn(move || seq.next().zip(seq.next()));
chunks.for_each(|(a, b)| println!("a: {}, b: {}", a, b));

*"I know about itertools crate, I just wonder if there's some hack that would allow me to get by without it "* Obviously there is as the itertools crate does it (it's open-source). — Denys Séguret, Apr 25 '21 at 13:47
Does this answer your question? [Are there equivalents to slice::chunks/windows for iterators to loop over pairs, triplets etc?](https://stackoverflow.com/questions/42134874/are-there-equivalents-to-slicechunks-windows-for-iterators-to-loop-over-pairs) — Denys Séguret, Apr 25 '21 at 13:48
@DenysSéguret not really as I disagree that the best way is to just collect into vector. — user1685095, Apr 25 '21 at 16:46
It took me some time to realize that `zip()` in the snippet added by the edit refers to [`Option::zip`](https://doc.rust-lang.org/stable/std/option/enum.Option.html#method.zip) (a method I just learned of), not the more usual `Iterator::zip()`, which is referred by the actual question. — user4815162342, Apr 25 '21 at 22:05
@user4815162342 Yeah, by the way it's just a workaround for not having applicative functors. — user1685095, Apr 26 '21 at 00:07

score 3 · Answer 1 · answered Apr 25 '21 at 13:45

3

This could be achieved with interior mutability. Note that it is only the next method call that requires a mutable borrow.

use std::cell::RefCell;

struct CellIter<I: Iterator>(RefCell<I>);

impl<I: Iterator> CellIter<I> {
    pub fn new(iter: I) -> Self {
        Self(RefCell::new(iter))
    }
}

impl<I: Iterator> Iterator for &CellIter<I> {
    type Item = <I as Iterator>::Item;

    fn next(&mut self) -> Option<Self::Item> {
        self.0.borrow_mut().next()
    }
}

fn main() {
    let a = CellIter::new(0..10);
    let i1 = &a;
    let i2 = &a;
    i1.zip(i2).for_each(|(a, b)| println!("a: {}, b: {}", a, b));
}

Output:

a: 0, b: 1
a: 2, b: 3
a: 4, b: 5
a: 6, b: 7
a: 8, b: 9

answered Apr 25 '21 at 13:45

Alsein

4,268
1
15
35

Interesting solution. Although impractical, as just creating iterator with one line in `next` is less code. `self.seq.next().zip(self.seq.next())`. Thanks for you help, I forgot about `RefCell`. Can I achieve the same with unsafe though? In this case it's kind a guaranteed that there wouldn't be two mutable borrows at the same time, right? – user1685095 Apr 25 '21 at 16:39
@user1685095 You can use unsafe, just change `RefCell` to `UnsafeCell` and replace `self.0.borrow_mut().next()` with `(*self.0.get()).next()`. **However**, I wouldn't recommend it. First, it's unsound because you *can't* guarantee that there wouldn't be two mutable borrows at the same time. After all, `CellIter` wraps any kind of iterator, and an arbitrary iterator's `next()` might end up calling `CellIter::next()`. (That's not hard to imagine - after all, the question is about sharing an iterator between two consumers.) That mistake would panic with `RefCell` and silent UB with `UnsafeCell`. – user4815162342 Apr 25 '21 at 19:46
@user1685095 Second, `RefCell`, despite the scary name, is not particularly expensive. While not technically a "zero-cost" abstraction, its cost amounts to a boolean flag, and the cost of `borrow_mut()` to checking/setting that flag. Unsafe is notoriously hard to get right, and `RefCell` is not expensive enough to risk UB, unless benchmarks prove that it makes a difference, which is very unlikely. – user4815162342 Apr 25 '21 at 19:47

user4815162342 · Accepted Answer · 2021-04-25T15:02:08.230

2

Here is a solution based on the same idea as Alsein's solution (using a RefCell), but somewhat shorter using std::iter::from_fn as helper:

use std::cell::RefCell;

fn share<T>(iter: impl Iterator<Item = T>) -> impl Fn() -> Option<T> {
    let wrapped_iter = RefCell::new(iter);
    move || wrapped_iter.borrow_mut().next()
}

fn main() {
    let a = share(0..10);
    let i1 = std::iter::from_fn(&a);
    let i2 = std::iter::from_fn(&a);
    i1.zip(i2).for_each(|(a, b)| println!("a: {}, b: {}", a, b));
}

edited Apr 25 '21 at 15:02

answered Apr 25 '21 at 14:46

user4815162342

141,790
18
296
355

Wait a second? Does `Fn() -> Option` implements an Iterator? That would be cool! – user1685095 Apr 25 '21 at 16:47
1

@user1685095 It doesn't, but iterator is one `std::iter::from_fn` away. :) – user4815162342 Apr 25 '21 at 17:06

score -1 · Answer 3 · answered Apr 25 '21 at 13:24

-1

You can use slice windows rather than zipping the iterators: https://doc.rust-lang.org/std/slice/struct.Windows.html

answered Apr 25 '21 at 13:24

moy2010

854
12
18

Firstly, the operation I'm doing here is `chunks`. Secondly, it's about working with any iterator not with slices specifically. – user1685095 Apr 25 '21 at 13:31

Can i iterate by chunks using zip in rust

3 Answers3