for..of and the iterator state

Question

Consider this python code

it = iter([1, 2, 3, 4, 5])

for x in it:
    print x
    if x == 3:
        break

print '---'

for x in it:
    print x

it prints 1 2 3 --- 4 5, because the iterator it remembers its state across the loops. When I do seemingly the same thing in JS, all I get is 1 2 3 ---.

function* iter(a) {
    yield* a;
}

it = iter([1, 2, 3, 4, 5])

for (let x of it) {
    console.log(x)
    if (x === 3)
        break
}

console.log('---')

for (let x of it) {
    console.log(x)
}

What am I missing?

You have a generator and it is once and done. https://stackoverflow.com/questions/23848113/is-it-possible-to-reset-an-ecmascript-6-generator-to-its-initial-state — epascarello, Mar 26 '18 at 13:43

score 6 · Answer 1 · answered Mar 26 '18 at 13:40

6

Generator objects in JS are not reusable unfortunately. Clearly stated on MDN

Generators should not be re-used, even if the for...of loop is terminated early, for example via the break keyword. Upon exiting a loop, the generator is closed and trying to iterate over it again does not yield any further results.

answered Mar 26 '18 at 13:40

Andrey

4,020
21
35

Yes, this (sadly) seems to be the answer. ECMA standard link http://www.ecma-international.org/ecma-262/7.0/#sec-runtime-semantics-forin-div-ofbodyevaluation-lhs-stmt-iterator-lhskind-labelset , item k. – georg Mar 26 '18 at 15:17

Keith · Answer 2 · 2018-03-26T15:55:07.950

3

As mentioned generators are a one off.

But it's easy to to simulate a re-usable iterator by wrapping the array inside a closure, and returning a new generator..

eg.

function resume_iter(src) {
  const it = src[Symbol.iterator]();
  return {
    iter: function* iter() {
      while(true) {
        const next = it.next();
        if (next.done) break;
        yield next.value;
      }
    }
  }
}

const it = resume_iter([1,2,3,4,5]);

for (let x of it.iter()) {
    console.log(x)
    if (x === 3)
        break
}

console.log('---')

for (let x of it.iter()) {
    console.log(x)
}



console.log("");
console.log("How about travesing the DOM");

const it2 = resume_iter(document.querySelectorAll("*"));

for (const x of it2.iter()) {
  console.log(x.tagName);
  //stop at first Script tag.
  if (x.tagName === "SCRIPT") break;
}

console.log("===");

for (const x of it2.iter()) {
  console.log(x.tagName);
}

edited Mar 26 '18 at 15:55

answered Mar 26 '18 at 13:50

Keith

22,005
2
27
44

Nice, but I'd like an iterator to be unaware of the underlying type, which is not necessary an array. – georg Mar 26 '18 at 15:23
@georg Oh, in that case how about wrapping an iterable inside a closure,.. Updated snippet to handle any iterable.. – Keith Mar 26 '18 at 15:48
@georg Updated snippet to traverse DOM nodes, as that's not an array, basically stop at first SCRIPT tag, and then it resumes again.. – Keith Mar 26 '18 at 15:55

score 3 · Answer 3 · edited May 17 '21 at 16:36

This behavior is expected according to the spec, but there is a simple solution. The for..of loop calls the return method after the loop ends:

Invoking this method notifies the Iterator object that the caller does not intend to make any more next method calls to the Iterator.

Solution

You can of course just replace that function with a custom one which does not close the actual iterator, right before utilizing it in a loop:

iter.return = value => ({ value, done: true });

Example:

function* iter(a) {
    yield* a;
}

it = iter([1, 2, 3, 4, 5])
it.return = () => ({})

for (let x of it) {
    console.log(x)
    if (x === 3)
        break
}

console.log('---')

for (let x of it) {
    console.log(x)
}

this is great! I didn't know about `return` – georg May 17 '21 at 16:36 — georg, May 17 '21 at 16:36

vol7ron · Answer 4 · 2018-03-26T14:59:56.150

This has more to do with how the for..of operates than the reusability of the iterator. If you were to manually pull the next value of the iterator, you call it as many times as necessary and it would resume from the previous state.

Which makes something like this possible:

function* iter(a) {
  yield* a;
}

let values = [1, 2, 3, 4, 5];
let it = iter(values)

for (let i = 0, n = values.length; i < n; i++) {
  let x = it.next().value
  console.log(x)
  if (x === 3)
    break
}

console.log('---')

for (let x of it) {
  console.log(x)
}

And the same could be done for a while loop that isn't dependent on a values array:

function* iter(a) {
  yield* a;
}

let it = iter([1, 2, 3, 4, 5]),
  contin = true

while (contin && (x = it.next().value)) {
  console.log(x)
  if (x === 3)
    contin = false
}

console.log('---')

for (let x of it) {
  console.log(x)
}

The second example (while loop) deviates slightly as x is assigned during the condition evaluation. It assumes that all values of x are truthy so undefined can be used as a terminating condition. If that is not the case, it would need to be assigned in the loop block and a terminating condition would have to be set. Something like if(x===undefined)contin=false or checking if the iterator has reached the end of its inputs.

Good idea, I've posted a wrapper that does the same (manually pull values from an iterator thus preserving its state). — georg, Mar 26 '18 at 15:21

Nathangrad · Answer 5 · 2018-03-26T14:02:18.803

In addition to Andrey's answer, if you want to have the same functionality as in the Python script, since generators are not able to be re-used when the loop is exited, you can re-create the iterator before looping through each time and keep track of where the loop ends up being broken to exclude processing of already-processed results like so:

function* iter(a) {
  yield* a;
}

var broken = 0;

iterate();
console.log('---');
iterate();

function iterate() {
  var it = iter([1, 2, 3, 4, 5]);
  for (let x of it) {
    if (x <= broken)
      continue;
    console.log(x);
    if (x === 3) {
      broken = x;
      break;
    }
  }
}

you are still looping twice. – Nina Scholz Mar 26 '18 at 14:01 — Nina Scholz, Mar 26 '18 at 14:01

georg · Accepted Answer · 2021-04-21T10:45:38.527

As pointed out in other answers, for..of closes the iterator in any case, so there's another wrapper necessary to preserve the state, e.g.

function iter(a) {
    let gen = function* () {
        yield* a;
    }();

    return {
        next() {
            return gen.next()
        },
        [Symbol.iterator]() {
            return this
        }
    }
}


it = iter([1, 2, 3, 4, 5]);

for (let x of it) {
    console.log(x);
    if (x === 3)
        break;
}

console.log('---');

for (let x of it) {
    console.log(x);
}

for..of and the iterator state

6 Answers6

Solution

Linked