Is there a way to subclass a generator in Python 3?

Question

Aside from the obvious, I thought I'd try this, just in case:

def somegen(input=None):
    ...
    yield
    ...

gentype = type(somegen())
class subgen(gentype):
    def best_function_ever():
        ...

Alas, Python's response was quite hostile:

"TypeError: Type generator is not an acceptable base type"

As luck would have it, that's a problem for me. See, I was thinking that maybe it would be a fun base type to play with, if I gave it a chance. Imagine my surprise! ..and dismay. Is there no way to get the almighty Python to see things my way on this one?

This is most certainly an outside-the-box kinda question, so please don't just say that it's not possible if you can't think of a way immediately. Python (especially Py3) is very flexible.

Of course, if you have evidence of why it cannot (not "should not") be a base type (Py3), then I do want to see and understand that.

A `generator` is a type of `method`. A `class` can only inherit from other `class`'s. It would be analogous to building a car out of driving. I can understand the frustration because I am of the opinion that python should be able to do anything and everything, but I do see why they wouldn't take the time to implement it. — Farmer Joe, Nov 04 '14 at 15:52
@FarmerJoe I'm not sure that I get the analogy, but a `method` is essentially (maybe even actually) a subtype of `function`. Also, everything is a type, even type itself ;) — Inversus, Nov 06 '14 at 22:45
@FarmerJoe What I mean by that is that, under the covers, everything is a `class`, even `generator`. Somewhere deep in the internals, one would surely find some C equivalent of `class generator(function)` and it's that class that I'm looking to "subclass" (ie: I am perfectly happy doing something else that allows me similar flexibility). — Inversus, Nov 06 '14 at 22:58

score 2 · Answer 1 · edited May 23 '17 at 12:29

2

A relevant other question is Which classes cannot be subclassed?.

It's reason 2 in the accepted answer there -- subclassing of a type needs to be implemented in C, and it wasn't implemented for generators, probably because nobody saw a use case.

The source code for generator objects is genobject.c, and you can see on line 349 that the Py_TPFLAGS_BASETYPE flag is not set.

edited May 23 '17 at 12:29

Community

1
1

answered Nov 04 '14 at 15:09

RemcoGerlich

30,470
6
61
79

1

Ooo. That's nice. Is that the only thing preventing my from subclassing a gen? Are there other parts of the internals that would need to be convinced? Can this be done at runtime (monkeypatch?) or am I could to need to make a custom build? (a long road I'm trying to avoid for the moment lol) – Inversus Nov 06 '14 at 22:49
Also.. how did you find that flag? Like, you just knew about it, or you followed the code there or something? – Inversus Nov 06 '14 at 22:50
I Googled and found the previous Stack Overflow thread that I linked to, and read it... And the flag isn't enough, you also have to _implement_ how it would work (see, say, the code for list), and recompile Python. – RemcoGerlich Nov 07 '14 at 17:51
Or actually it was another question like it I guess, I found a few and saw the flag in one of them. – RemcoGerlich Nov 07 '14 at 17:57
Even though I'm accepting the other answer, I want you to know that I found this answer useful. Thank you. – Inversus Nov 12 '14 at 18:26

score 1 · Accepted Answer · edited Nov 05 '14 at 00:17

1

You cannot subclass a generator that is defined as a function using yield, but you can use it in another generator.

Just take this simple one :

def alphagen(n=27):
    if n<0 or n > 27: n = 27
    for i in range(n):
        yield chr(ord('A') + i)

You get :

>>>> [ a for a in alphagen(10)]
['A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J']

You can use it in:

def alphastartgen(n=27):
    resul = ""
    for i in alphagen(n):
        resul += i
        yield resul

And now you get :

>>> [ a for a in alphastartgen(8) ]
['A', 'AB', 'ABC', 'ABCD', 'ABCDE', 'ABCDEF', 'ABCDEFG', 'ABCDEFGH']

edited Nov 05 '14 at 00:17

JasonMArcher

14,195
22
56
52

answered Nov 04 '14 at 15:39

Serge Ballesta

143,923
11
122
252

That's a fascinating angle.. I'm gonna go play with it for a while. I'll be back. Thanks – Inversus Nov 06 '14 at 22:54
I'm back from playing with that a bit, and I've gotta say: "Oh ya. That's the good stuff". Treating the outer generator (function) definition like a class definition is exactly the kind of crazy hack I was hoping for. Thank you very much. – Inversus Nov 12 '14 at 18:32

Is there a way to subclass a generator in Python 3?

2 Answers2

Linked