Why does my generator become empty after being iterated through?

Question

I have a generator being returned to me by a function call from a library I'm using. I then pass this generator to a function which iterates through it and does a bunch of logic on each of the items. I then want to refer to this same generator after that function has been called. However, it seems the generator no longer has/generates any items. The code is along these lines:

let myGenerator = this.generatorFunc();
console.log(Array.from(myGenerator).length); //prints N which is specified elsewhere
this.iterateThroughGenerator(myGenerator);
console.log(Array.from(myGenerator).length); //now prints 0 when I need it to be N still

iterateThroughGenerator(generator) {
    for(let element of generator) {
        // do a bunch of stuff with element
    }
}

@PatrickRoberts here's the definition in the library I'm using. Basically just used to generate a set of polygons. https://github.com/d3/d3-delaunay/blob/1807a51dc5ad450cac9bfcb0ec9e5946d1970256/src/voronoi.js#L98 — TomLisankie, Jun 28 '19 at 18:29
In that case `getGeneratorFunc()` is poorly named. That function _is_ the generator function. It returns a generator iterator, which can only be consumed once. — Patrick Roberts, Jun 28 '19 at 18:30
@PatrickRoberts Right, definitely a misnomer. Will update. Can you explain why an iterator can only be consumed once? Surely there are situations where you'll want to reuse an iterator. — TomLisankie, Jun 28 '19 at 18:38
Because a generator iterator has internal state to keep track of what point in the control flow of its generator function to resume from every time `next()` is called. Once its internal state reaches the end of the function, `next()` simply returns `{ value: undefined, done: true }` as Bergi explains in his answer. — Patrick Roberts, Jun 28 '19 at 18:42
@PatrickRoberts Okay, I see. But why would someone want that? Why have iterators when you could just have iterables? Is it just for memory preservation when you know you're only going to need to iterate through a collection once? — TomLisankie, Jun 28 '19 at 18:48
The purpose of iterators isn't that they iterate a collection once, it's that they can be incrementally consumed because they're stateful. The fact that they iterate the underlying collection only once is just a limitation of being stateful. — Patrick Roberts, Jun 28 '19 at 19:01

score 1 · Answer 1 · answered Jun 28 '19 at 18:28

1

That's how iterators work. Invoking a generator returns an iterator, which can be iterated once. This is the same in most other languages.

let generator = function* () {
  for (let i = 0; i < 3; i++) 
    yield i;
};

let iterator = generator();

console.log(Array.from(iterator)); // [1...3]
console.log(Array.from(iterator)); // []

console.log(Array.from(generator())); // [1..3]
console.log(Array.from(generator())); // [1..3]

answered Jun 28 '19 at 18:28

junvar

11,151
2
30
46

I see. Why can iterators only be iterated once? There must be circumstances where you want an iterator that you'll be able to use multiple times. Is there some sort of property for iterators that would allow this to happen? – TomLisankie Jun 28 '19 at 18:37
My hunch is for simplicity; most implementations of iterators include a `bool hasNext()` and `T next()` methods, but no `void restart()` method. If you haven't already, as a fun practice problem, try implementing an iterator. Also, see https://stackoverflow.com/questions/7689261/why-iterator-doesnt-have-any-reset-method – junvar Jun 28 '19 at 18:50
@TomLisankie An iterator is a stateful object, you iterate it once until it ends and then you throw it away. If you want to iterate your iterable object (array, whatever, ...) again, you create a new iterator. (Of course it would be possible to write an iterator that resets when it has ended, but that's not how they usually work and and get consumed). – Bergi Jun 28 '19 at 19:03

Patrick Roberts · Answer 2 · 2019-06-28T19:03:03.290

Others have already explained why this is happening, but I'll recap anyway.

Generator functions return a generator object, which implements both the iterable protocol (a Symbol.iterator property) using a simple circular reference to itself, and the iterator protocol (a next() method) in order to statefully iterate through the control flow of its generator function.

To fix your problem, encapsulate the generator function call with an object that implements the iterable protocol by returning separate instances of your generator object, and you can use it the same way:

const iterable = { [Symbol.iterator]: () => this.generatorFunc() };

console.log(Array.from(iterable).length); //prints N which is specified elsewhere
this.iterateThroughGenerator(iterable);
console.log(Array.from(iterable).length); //still prints N

iterateThroughGenerator(iterable) {
    for(let element of iterable) {
        // do a bunch of stuff with element
    }
}

score 0 · Answer 3 · answered Jun 28 '19 at 18:27

Once the generator function is completed, then you have to call this.getGeneratorFunc() to recreate the generator again. Also, when you do Array.from(myGenerator), it will complete that generator as well, so when you call this.iterateThroughGenerator(myGenerator) then nothing will occur because there are no elements being returned from the generator anymore. So you can either save the result of the generator into an array and reuse that array, or call this.getGeneratorFunc() three times for each time you want to get elements from it. In this specific example, I would do

 const generated = Array.from(this.getGeneratorFunc());
 console.log(generated.length);
 this.iteratedItems(generated);
 console.log(generated.length);

Check out this answer as well. Previous Answer
Id also read this.

score 0 · Answer 4 · answered Jun 28 '19 at 18:28

Array.from() exhausts the generator (iterating it until the end), and iterating it further won't yield any more elements. next() calls will always return {value: undefined, done: true}.

To create a new generator that starts from the beginning again, you need to call the generatorFunc() again.

Why does my generator become empty after being iterated through?

4 Answers4

Linked