9

I found that some iterable can be repeatedly iterable:

const iterable = {
  [Symbol.iterator]: function* () {
    yield 1;
    yield 3;
    yield 5;
  }
}

console.log([...iterable]);
console.log([...iterable]);
console.log([...iterable]);

While some cannot:

function* generatorFn() {
  yield 1;
  yield 3;
  yield 5;
}

const iterable = generatorFn();

console.log([...iterable]);
console.log([...iterable]);
console.log([...iterable]);

Is there a rule whether an iterable should or should not be repeatedly iterable?

I understand why they behave differently (it is because the second case, when the iterable[Symbol.iterator] function is invoked, the same iterator is returned (which is iterable itself. Can try iterable[Symbol.iterator]() === iterable and it would return true. iterable.next is a function too. So in this case, iterable is a generator object, an iterable, and an iterator, all three). But I wonder iterable being an object type, is there a well-defined behavior as to whether it should or should not be repeatedly iterable.)

nonopolarity
  • 146,324
  • 131
  • 460
  • 740
  • 1
    Not really but the two are not really equivalent, either. The first one is an object that you force to iterate over the second you take an *iterator* and try to iterate after it's exhausted. Basically, an iterator will signify if it holds no more values and you cannot go on afterwards. So, in the first case, if you had done `it = iterable[Symbol.iterator]` you'd not be able to do `[...it]` more than once since the iterator is finished. In the second case, repeat calls `[...generatorFn()]` are fine, since you take a new iterator every time (same as the first version). – VLAZ Jan 04 '20 at 22:51
  • 2
    An iterator should iterate until it's done and then be done. Once it reports `{done: true}`, it is done. An iterable should be able to supply a new iterator upon demand to start a new iteration. It's important to understand the distinction between an `iterable` and an `iterator`. An iterable is something you can get an iterator for and can use that iterator to go through all the items in the iterable. – jfriend00 Jan 05 '20 at 00:19
  • @jfriend00 so you mean an iterable should return a new iterator every time (ideally?). In the second case, `iterable` is an iterable but obviously it is not doing that – nonopolarity Jan 05 '20 at 00:47
  • @jfriend00 so just because in the second case, the iterable comes from a generator function, then this iterable is not quite an iterable but a different type of iterable? I only know if an object complies to the iterable protocol, then it is formally an iterable. There is no "yeah it is an iterable but it is sort of a different type of iterable" – nonopolarity Jan 05 '20 at 01:30
  • @jfriend00 you can't use `[...iter]` on an iterator – nonopolarity Jan 05 '20 at 04:27
  • Uhhh, that doesn't seem right. I can do this: `let s = new Set([1,2,3,4,5,6]); let iter = s[Symbol.iterator](); let x = [...iter]; console.log(x);` or `let s = new Set([1,2,3,4,5,6]); let iter = s.values(); let x = [...iter]; console.log(x);` In both cases, `iter` is an iterator. – jfriend00 Jan 05 '20 at 04:50
  • your `iter[Symbol.iterator]` is defined, and in fact `iter[Symbol.iterator]() === iter` returns `true`, so what you have is both an iterable and iterator. Iterator only has to have `next()`, so try this: `obj = { i: 1, next: function() { if (this.i <= 3) return { value: this.i++, done: false }; else return { value: undefined, done: true }; } };` you can do `obj.next()` 4 times and see, but set `obj` to that same thing again and `[...obj]` won't work – nonopolarity Jan 05 '20 at 05:08
  • Yep, your `iterable` is both an iterable and an iterator and it does have that weird property that every time you ask the iterable for an iterator, it gives you the same one. I guess that's what a generator function returns. I'm more familiar with collection-style iterators and iterables than I am with generators. – jfriend00 Jan 05 '20 at 05:09
  • I'm more of a practical problem solver and how to use language tools to solve a particular problem than one curious about language details just for the sake of language curiosity so I think I'm going to bow out of this unless it turns into a practical problem solving question. – jfriend00 Jan 05 '20 at 05:15
  • @jfriend00 sometimes it is like, something works or doesn't work and a group of people may stand there wondering why that is, and this kind of details will tell why – nonopolarity Jan 05 '20 at 05:19
  • Just discovered this in the spec. All standard, built-in iterators [derive from the same IteratorPrototype](https://www.ecma-international.org/ecma-262/10.0/index.html#sec-%iteratorprototype%-object) and that parent class does [by specification](https://www.ecma-international.org/ecma-262/10.0/index.html#sec-%iteratorprototype%-@@iterator) just `return this` when you ask it for a new iterator. So, that is a standard behavior of built-in iterators. It is not required for an iterator to do that (your simple iterator a couple comments above does not do that, but is still a legal iterator). – jfriend00 Jan 05 '20 at 05:24
  • 1
    So, `[...iterator]` apparently only works for an iterator that pretends to be an iterable too. You can see that the [`iterator` interface spec](https://www.ecma-international.org/ecma-262/10.0/index.html#sec-iterator-interface) does not require (or even mention) an iterator pretending to be an iterable by just returning itself. That is something the built-in iterators decided to do themselves. I'm sure it may be convenient sometimes, but it certainly does confuse the line between an iterable and an iterator (as we discovered). – jfriend00 Jan 05 '20 at 05:58
  • Yes, just confirmed that `[...iterator]` only works if there's a `Symbol.iterator` property on the object that is a function that returns an iterator when called. That syntax is only for a something that responds when you ask it for an iterator (thus an iterable). Which is true for the built-in iterators, not required to be true for custom iterators. – jfriend00 Jan 05 '20 at 06:05
  • @nopole - OK, you got me hooked and I wrote an answer that I think explains it all including the specific questions you pose in your question and everything we covered in the comments. – jfriend00 Jan 05 '20 at 07:31

3 Answers3

4

OK, I thought I'd summarize some of the things we've learned in the comments and add a few more and then finish off by writing answers to your specific questions.

[...x] syntax

The [...x] syntax works for things that support the iterables interface. And, all you have to do to support the iterable interface is support the Symbol.iterator property to supply a function that (when called) returns an iterator.

Built-In Iterators Are Also an Iterable

All iterators built into Javascript derive from the same IteratorPrototype. It is not required that an iterator do this, this is a choice the built-in iterators make.

This built-in IteratorPrototype is also an Iterable. It supports the Symbol.iterator property which is a function that just does return this. This is by specification.

This means that all built-in iterators such as someSet.values() will work with the [...x] syntax. I'm not sure why that's super useful, but it certainly can lead to confusion about what an Iterable can do and what an Iterator can do because these built-in iterators can behave as either.

It leads to some funky behavior because if you do this:

let s = new Set([1,2,3]);
let iter = s.values();    // gets an iterator
let x = [...iter];
let y = [...iter];
console.log(x);
console.log(y);

The second [...iter] is an empty array because there's only one iterator here. In fact, x === y. Thus the first let x = [...iter]; exhausts the iterator. It's sitting on done and can't iterate the collection again. That's because of this funky behavior of the built-in iterators where they behave as an iterable, but just return this. They do NOT create a new iterator that can iterate the collection again like you can when you use the actual collection iterable. This collection iterable returns a brand new iterator each time you access s[Symbol.iterator]() as shown below:

let s = new Set([1,2,3]);
let x = [...s];
let y = [...s];
console.log(x);
console.log(y);

Plain Iterators Do Not Work with [...x]

All you need to implement to be an Iterator is to support the .next() method and respond with the appropriate object. In fact, here's a super simple iterator that meets the specification:

const iter = { 
    i: 1, 
    next: function() { 
        if (this.i <= 3) {
            return { value: this.i++, done: false }; 
        } else {
            return { value: undefined, done: true }; 
        } 
    }
}

If you try to do let x = [...iter];, it will throw this error:

TypeError: object is not iterable (cannot read property Symbol(Symbol.iterator))

But, if you make it an Iterable by adding the appropriate [Symbol.iterator] property to it, it will work as [...iter];

const iter = { 
    i: 1, 
    next: function() { 
        if (this.i <= 3) {
            return { value: this.i++, done: false }; 
        } else {
            return { value: undefined, done: true }; 
        } 
    },
    [Symbol.iterator]: function() { return this; }
}

let x = [...iter];
console.log(x);

Then, it can work as [...iter] because it's now also an iterable.

Generators

A Generator function returns a Generator object when it is called. Per spec, that Generator object behaves as both an Iterator and an Iterable. There is purposely no way to tell if this Iterator/Iterable came from a generator or not and this is apparently done on purpose. The calling code just knows it's an Iterator/Iterable and the generator function is just one means of creating the sequence which is transparent to the calling code. It is iterated just like any other iterator.


The Tale of Your Two Iterators

In your original question, you show two iterators, one that works repeatedly and one that does not. There are two things at work here.

First, some iterators "consume" their sequence and there is no way to just repeatedly iterate the same sequence. These would be manufactured sequences, not static collections.

Second, in your first code example:

const iterable = {
  [Symbol.iterator]: function* () {
    yield 1;
    yield 3;
    yield 5;
  }
}

console.log([...iterable]);
console.log([...iterable]);
console.log([...iterable]);

Separate Iterators

That iterable is an iterable. It isn't an iterator. You can ask it for an iterator by calling iterable[Symbol.iterator]() which is what [...iterable] does. But, when you do that, it returns a brand new Generator object that is a brand new iterator. Each time you call iterable[Symbol.iterator]() or cause that to be called with [...iterable], you get a new and different iterator.

You can see that here:

    const iterable = {
      [Symbol.iterator]: function* () {
        yield 1;
        yield 3;
        yield 5;
      }
    }

    let iterA = iterable[Symbol.iterator]();
    let iterB = iterable[Symbol.iterator]();
    
    // shows false, separate iterators on separate generator objects
    console.log(iterA === iterB);      

So, you're creating an entirely new sequence with each iterator. It freshly calls the generator function to get a new generator object.

Same Iterator

But, with your second example:

function* generatorFn() {
  yield 1;
  yield 3;
  yield 5;
}

const iterable = generatorFn();

console.log([...iterable]);
console.log([...iterable]);
console.log([...iterable]);

It's different. What you call iterable here is what I like to think of as a pseudo-iterable. It implements both the Iterable and the Iterator interfaces, but when you ask it for an Iterator like [...iterable] does, it just returns the same object every time (itself). So, each time you do [...iterable], it's operating on the same iterator. But that iterator was exhausted and is sitting in the done state after the first time you executed [...iterable]. So, the second two [...iterable] are empty arrays. The iterator has nothing more to give.

Your Questions

Is there a rule whether an iterable should or should not be repeatedly iterable?

Not really. First, a given iterator that eventually gets to the done state (a non-infinite iterator) is done giving any results once it gets to the done state. That per the definition of iterators.

So, whether or not an Iterable that represents some sort of static sequence can be repeatedly iterated depends upon whether the Iterator that it provides when asked for an iterator is new and unique each time it is asked and we've seen in the above two examples, that an Iterable can go either way.

It can produce a new, unique iterator each time that presents a fresh iteration through the sequence each time.

Or, an Iterable can produce the exact same Iterator each time. If it does that, once that iterator gets to the done state, it is stuck there.

Keep in mind also that some Iterables represent a dynamic collection/sequence that may not be repeatable. This isn't true for things like a Set or a Map, but more custom types of Iterables might essentially "consume" their collection when it is iterated and when it's done, there is no more, even if you get a new fresh Iterator.

Imagine an iterator that handed you a code worth some random amount between $1 and $10 and subtracted that from your bank balance each time you ask the iterator for the next value. At some point, your bank balance hits $0 and that iterator is done and even getting a new iterator will still have to deal with the same $0 bank balance (no more values). That would be an example of an iterator that "consumes" values or some resource and just isn't repeatable.

But I wonder iterable being an object type, is there a well-defined behavior as to whether it should or should not be repeatedly iterable.

No. It is implementation specific and depends entirely upon what you're iterating. With a static collection like a Set or a Map or an Array, you can fetch a new iterator and generate a fresh iteration each time. But, what I called a psuedo-iterable (an iterable that returns the same iterator each time it is requested) or an iterable where the sequence is "consumed" when it's iterated may not be able to be repeatedly iterated. So, it can purposely be either way. There is no standard way. It depends upon what is being iterated.

Testing What You Have

Here a few useful tests that help one understand things a bit:

// could do a more comprehensive test by calling `obj.next()` to see if
// it returns an appropriate object with appropriate properties, but
// that is destructive to the iterator (consumes that value) 
// so we keep this one non-destructive
function isLikeAnIterator(obj) {
    return typeof obj === "object" && typeof obj.next === "function)";
}

function isIterable(obj) {
    if (typeof obj === "object" && typeof obj[Symbol.iterator] === "function") {
        let iter = obj[Symbol.iterator]();
        return isLikeAnIterator(iter);
    }
    return false;
}

// A pseudo-iterable returns the same iterator each time
// Sometimes, the pseudo-iterable returns itself as the iterator too
function isPseudoIterable(obj) {
   if (isIterable(obj) {
       let iterA = obj[Symbol.iterator]();
       if (iterA === this) {
          return true;
       }
       let iterB = obj[Symbol.iterator]();
       return iterA === iterB;
   }
   return false;
}

function isGeneratorObject(obj) {
    if (!isIterable(obj) !! !isLikeAnIterator(obj) {
        // does not meet the requirements of a generator object
        // which must be both an iterable and an iterator
        return false;
    }
    throw new Error("Can't tell if it's a generator object or not by design");
}
jfriend00
  • 683,504
  • 96
  • 985
  • 979
1

iterable of const iterable = generatorFn(); is an iterable and a Generator object, too.

The Generator object is returned by a generator function and it conforms to both the iterable protocol and the iterator protocol.

This generator follows the protocol and runs with the iterable only once.

Nina Scholz
  • 376,160
  • 25
  • 347
  • 392
  • 2
    Mozilla distinguishes between a generator function and a generator object (or just generator). A generator is an iterable too, I think, or else `[...iterable]` would not work. If it quacks like a duck then it is a duck. (or at least it is quackable) – nonopolarity Jan 04 '20 at 22:57
  • 1
    So, this is not really correct. `generatorFn()` does indeed return a Generator object. But a Generator object IS both an Iterable and an Iterator. It fulfills both of those interfaces. You see that expressed [here in the specification](https://www.ecma-international.org/ecma-262/10.0/index.html#sec-generator-objects) where it says ***A Generator object is an instance of a generator function and conforms to both the Iterator and Iterable interfaces.***. It is purposely done that you can't even tell that it's a Generator object. You can just detect that it is both an Iterable and an Iterator. – jfriend00 Jan 05 '20 at 07:28
0

2021 Update

the MDN doc has been changed to reflect that an exhausted iterator should not make itself iterable again. Their example now reaches done: true and does NOT reset this.index = 0. This is in line with the conclusions reached by jfriend00's answer.

This correction predates MDN's migration to github so I don't have a history of the change. I'm leaving the previous answer below.


This answer is probably wrong (see update above)

The MDN doc has a good implementation that seems to suggest we'd prefer and iterable to be re-iterable:

[Symbol.iterator]() {
  return {
    next: () => {
      if (this.index < this.data.length) {
        return {value: this.data[this.index++], done: false};
      } else {
        this.index = 0; //If we would like to iterate over this again without forcing manual update of the index
        return {done: true};
      }
    }
  };
}
Sheraff
  • 5,730
  • 3
  • 28
  • 53