Class function returns generator

Question

I'm sure this is a stupid error on my part but I'm sick of debugging so hoping someone can spot the problem. I've included the larger script for reference sake but the part that I'm struggling with is the method _get_subcat within Categories. When I attempt to print the results I get [<generator object Category._get_subcat at 0x000002873CFE76F0>]. Can someone explain this to me?

class GamesDict:
    def __init__(self):
        
        self._dic = json.load(open('full.json'))['eventGroup']
        self.categories = self._dic['offerCategories']

        self.export_runline()
        self.export_moneyline()
        self.export_total()
        self.export_propshomeruns()
        self.export_propshits()
        self.export_propstotalbases()

    def export_runline(self):
        self.runline = Runline().to_dict(self.categories)

    def export_moneyline(self):
        self.moneyline = Moneyline().to_dict(self.categories)
        
    def export_total(self):
        self.total = Total().to_dict(self.categories)

    def export_propshomeruns(self):
        self.propshomeruns = BatterPropsHomeruns().to_dict(self.categories)
        print(self.propshomeruns)

    def export_propshits(self):
        self.propshomeruns = BatterPropsHits().to_dict(self.categories)

    def export_propstotalbases(self):
        self.propshomeruns = BatterPropsTotalBases().to_dict(self.categories)
 
  
class Parser:
    CATEGORY = int
    SUBCATEGORY = int
    DISPLAY_NAME: str

    def to_dict(self, markets: Iterable[dict[str, Any]]) -> list:
        return [market for market in self._get_records(markets)]

    @classmethod
    def _get_records(cls, data: Iterable[dict[str, Any]]) -> Iterator[tuple]:
        raise NotImplementedError()


class Category(Parser):
    @classmethod
    def _get_records(cls, data: Iterable[dict[str, Any]]) -> tuple[tuple, ...]:
        for item in data:
            if item['offerCategoryId'] == cls.CATEGORY:
                yield from cls._get_subcatdes(item['offerSubcategoryDescriptors'])

    @classmethod
    def _get_subcatdes(cls, data: Iterable[dict[str, Any]]) -> tuple[tuple, ...]:
        for item in data:
            if item['subcategoryId'] == cls.SUBCATEGORY:
                yield cls._get_subcat(item['offerSubcategory'].get('offers'))

    @classmethod
    def _get_subcat(cls, data: Iterable[dict[str, Any]]) -> tuple[tuple, ...]:
        for item in data:
            for x in item:
                if x.get('isOpen',()):
                    yield from cls._get_data(x['outcomes']) # why does this not work? 
                        
    
class Runline(Category):
    CATEGORY = 493
    SUBCATEGORY = 4519
    DISPLAY_NAME = 'Run Line'
            
    @classmethod
    def _get_data(cls, market: dict[str, Any]) -> tuple[tuple, ...]:
        event_id = market['eventId']
        away_team, home_team = market['outcomes'][:2]
        away_abbr = TEAMS.get(away_team['label'])
        home_abbr = TEAMS.get(home_team['label'])

        away_odds = away_team['oddsDecimal']
        home_odds = home_team['oddsDecimal']
        away_line = away_team['line']
        home_line = home_team['line']
        return (
            (away_abbr, event_id, away_odds, away_line, BOOK),
            (home_abbr, event_id, home_odds, home_line, BOOK),
        )


class Moneyline(Category):
    CATEGORY = 493
    SUBCATEGORY = 4519
    DISPLAY_NAME = 'Moneyline'
                
    @classmethod
    def _get_data(cls, market: dict[str, Any]) -> tuple[tuple, ...]:
        event_id = market['eventId']
        away_team, home_team = market['outcomes'][:2]
        away_abbr = TEAMS.get(away_team['label'])
        home_abbr = TEAMS.get(home_team['label'])

        away_odds = away_team['oddsDecimal']
        home_odds = home_team['oddsDecimal']
        return (
            (away_abbr, event_id, away_odds, BOOK),
            (home_abbr, event_id, home_odds, BOOK),
        )


class Total(Category):
    CATEGORY = 493
    SUBCATEGORY = 4519
    DISPLAY_NAME = 'Total'
                
    @classmethod
    def _get_data(cls, market: dict[str, Any]) -> tuple[tuple, ...]:
        event_id = market['eventId']
        
        away_team, home_team = market['outcomes'][:2]
        line = away_team['line']
        over = away_team['oddsDecimal']
        under = home_team['oddsDecimal']
        return (
            (event_id, line, over, under, BOOK),
        )


class Props(Category):
    @classmethod
    def _get_data(cls, market: dict[str, Any]) -> tuple[tuple, ...]:
        event_id = market['eventId']

        cont1, cont2 = market['outcomes'][:2]
        player = cont1.get('participant','')
        line = cont1.get('line','')

        over = cont1.get('oddsAmerican','')
        under = cont2.get('oddsAmerican','')
        return (
            (player, event_id, line, over, under),
        )
    

class BatterPropsHomeruns(Props):
    CATEGORY = 743
    SUBCATEGORY = 6606
    DISPLAY_NAME = 'Home Runs'


class BatterPropsHits(Props):
    CATEGORY = 743
    SUBCATEGORY = 6606
    DISPLAY_NAME = 'Hits'

    
class BatterPropsTotalBases(Props):
    CATEGORY = 743
    SUBCATEGORY = 6606
    DISPLAY_NAME = 'Total Bases'



GamesDict()

Answer 1

I think you meant for _get_subcatdes to yield from cls._get_subcat(...) instead of yield cls._get_subcat(...).

cls._get_subcat is a generator, so cls._get_subcat(...) returns an iterator (over the values yielded by _get_subcat), and that iterator is what _get_subcatdes will yield. yield from cls._get_subcat(...) would cause _get_subcatdes to successively yield the values produced by that iterator, which I think is what you're expecting.

A couple of notes:

1. One of my debugging heuristics is that if I see that a generator is yielding an iterator instead of the iterator's values, I make sure that I didn't somehow forget to use yield from when I meant yield.

2. But since OP went to the trouble of annotating the expected types, a more structured approach would be to actually hand this code over to a static type checker. I used mypy, and I had to make some adjustments to the annotations:

   • in class Parser, I replaced CATEGORY = int with CATEGORY: int (and the same with SUBCATEGORY).
   • I changed the return type annotations for the generators from -> tuple[tuple, ...] to -> Iterator[tuple] (although I suppose I could have been more precise).
   • At the suggestion of mypy, I added from typing import Iterable, Iterator, Any.

   I probably could have done some more. That done, mypy reported various problems, two of which seem relevant:

   main.py:58: error: Incompatible types in "yield"
      (actual type "Iterator[Tuple[Any, ...]]", expected type "Tuple[Any, ...]")
   

   That's because yield was used instead of yield from.

   main.py:64: error: "str" has no attribute "get"
   main.py:65: error: Invalid index type "str" for "str";
      expected type "Union[SupportsIndex, slice]"
   

   Those (which might answer the question "why does this not work?") are the result of using for x in item where item is a dict, which iterates over the dictionary's keys (which are of type str). I'm pretty sure the intent was to iterate over item's values.

   There are lots more errors and warnings which would be useful to attend to, but for me it's a clear example of the value of static type-checking. Particularly if you've already gone to the trouble to annotate.