How to zip two sequences (one shifted by N elements)?

Question

I have two lists A and B and I want to traverse pairs of elements (A[0], B[2]), (A[1], B[3]), etc. Without the shift I could just use zip(A, B) to do it efficently.

In this example the shift is 2 but may happen that I will need a shift of N. How to do it efficiently (not indexing into two lists)?

[`more_itertools.zip_offset`](https://more-itertools.readthedocs.io/en/latest/api.html#more_itertools.zip_offset)? — hoefling, Feb 29 '20 at 12:45

score 2 · Accepted Answer · answered Feb 29 '20 at 12:40

2

I think a concise way to do this would be

zipped = zip(list1, list2[n:])

Where n is an integer representing the offset.

answered Feb 29 '20 at 12:40

Haleemur Ali

26,718
5
61
85

The issue is that it seems to require at least `len(list2) - n` additional memory for almost a full copy of `list2`, doesn't it? – sophros Feb 29 '20 at 19:33
The data in the list is not copied but a new `slice` object is created that may require a significant amount of memory. On the other hand the slice creation is FAST. – gboffi Feb 29 '20 at 20:24
@sophros, from my understanding, when a list is sliced, the list elements, i.e. _data_ is not copied. However a slice object is created, and the pointers to the _data_ are copied. I think each pointer is 8b, and the object itself has some memory. At any rate, slicing is unlikely to cause performance issues. Also, I'm not certain that the solution you present is more efficient than this shorter alternative. – Haleemur Ali Mar 03 '20 at 17:46

sophros · Answer 2 · 2020-02-29T19:34:41.410

Eventually, I arrived at the following generic function:

def zip_shift(s1, s2, shift, placeholder):
    i1 = iter(s1)
    i2 = iter(s2)

    while shift > 0:
        yield placeholder, next(i2)  # leave only next(i2) if no placeholder pairs are needed
        shift -= 1

    yield from zip(i1, i2)  # or return zip

The above function works like a generator that first fills in the placeholder values shift times instead of taking values from the second sequence s2. Afterward, it works just like zip. The yield from is a relatively new construct (since Python 3.3). There are subtle differences between yield from and return above but can be neglected in this situation.

Of course if the shift value is 1 almost (with the exception of the pairs with the placeholder) the same can be achieved with:

zip(s1, next(s2))

As with zip sequences do not have to be of the same length. The generator works as long as the shorter sequence is not exhausted.

How to zip two sequences (one shifted by N elements)?

2 Answers2