0
class DownloadDict(dict):
    """dictionary that will fetch if missing data"""
    def __init__(self, name, *args, **kwargs):
        super(dict, self).__init__(*args, **kwargs)
        self.name = name

    def download(self):
        print(f"Fetching New Data: {self.name}")
        [self.__setitem__(key,value) for (key, value) in makeReferenceJson()[self.name].items()]


    def __getitem__(self, key):
        try:
            return getattr(self, key)
        except AttributeError:
            self.download()
            return getattr(self, key)

makeReferenceJson performs a fetch.

My goal is for this dictionary class to operate as a normal dict, except when there is a key-miss; I want it to perform a fetch and try again (I have a json dict that hardly ever gets updated, but when it does I want to handle it natively w/out polling an API).

The problem is self.name = name assigns the name value to a "name" key and shows up in all of my normal dictionary actions (like .keys())

Is there a way to easily (without custom writing ALL of the methods) create a private class attribute?

Aside: I'm taking this approach because I have dictionaries all over my code that I invoke in different ways and I want to update their construction in one place instead of hunting down all of the instance -- so the API needs to remain unchanged. (Something like myDict.get(key) or myClass.dict[key] wont work here

Right now:

d = DownloadDict("Dataset1")
d
# {"name":"Dataset1"}
d.name
# "Dataset1"

Desired Behavior:

d = DownloadDict("Dataset1")
d
# {}
d.name
# "Dataset1"

After tinkering & reading through the comments I've made some minor adjustments, and think I've narrowed down this issue to the __getitem__ override.

I'm not sure what/how to call the getitem within that function (I don't want self.getitem; there's probably some syntax involving super?) but my issue is using the getattr as others have pointed out -- since I'm purposefully trying to avoid writing to the class attributes

Schalton
  • 2,867
  • 2
  • 32
  • 44
  • 1
    The assignment `self.__dict__ = self` seems suspect. In all the times I inherited from `dict` I never thought I needed to do this. Also, the fact that the dictionary's value includes the key-value pair of the object's name is a direct result of the assignment. – Amitai Irron Jun 05 '20 at 22:52
  • My goal is change the base dictionary as minimally as possible -- I need to store an attribute on construction (dataset name) and insert the exception handler on key lookups -- those are my only requirements. My code *seems* to work -- with the exception that my data is being stored at the object attribute level; so the "name" is added to the dict keys/index/etc. and I'd like to maintain the separation between dictionary attributes from class attributes – Schalton Jun 05 '20 at 22:54
  • @Amitai -- I agree, I'm still playing with different things, and also thought the self.__dict__ = self was suspect (I don't want to commingle the dict w/ it's attributes); so I removed it. I believe the issue now is in my download method -- I can't manage to store a key value pair to the dict object instead of the class attributes – Schalton Jun 05 '20 at 22:58
  • I think all that's required is that you remove this assignment. It is the direct reason for the existence of the `"name"` key in your dictionary. The override of the `__getitem__` method is all you need. – Amitai Irron Jun 05 '20 at 22:58
  • Why do you keep using `setattr` and `getattr` when you set the key? Why `self.__dict__ = self`? That's the problem. Although note, deriving directly from `dict` is fraught with problems, mostly that overriding `__getitem__` won't affect other methods that you might assume will use that. Python built-ins use optimizations at the C level and skip this often. Generally, if you want a custom dict-like class, use `collections.abc.MutableMapping` – juanpa.arrivillaga Jun 05 '20 at 23:06
  • **Although note**, you *may* be able to use the special `__missing__` method if you derive from dict in *this particular case* – juanpa.arrivillaga Jun 05 '20 at 23:06
  • Also, `{setattr(self, key, value) for (key, value) in makeReferenceJson()[self.name].items()}` is bad. Don't use set comprehensions for side-effects... set comprehensions are for *creating set objects*. You create a set of `{None}` simply to throw it away. Use *a for loop*. – juanpa.arrivillaga Jun 05 '20 at 23:12
  • @juanpa; I tried w/ private attributes, but that didn't work for me. FWIW I haven't tried it since python 3.2, but in last time I horse raced the comprehension vs the loop it was faster to run comprehension -- even if I was throwing away the empty object -- I've stuck to it since, but will re-evaluate – Schalton Jun 05 '20 at 23:15
  • No, **don't do that**. The speed differences are marginal at best. And you are using a delcarative programming construct for imperative side effects, it's terrible style and confusing. Anyway, *python doesn't have private attributes*. I suppose you mean double-underscore name-mangling, but again, the **problem** is that you are using *attributes* in your `__getitem__` method, and setting the namespace dictionary to the dict object itself. Don't do that. In `__getitem__` use `dict.__getitem__` or `super().__getitem__` if you don't want your attribute and key-value pairs to be conflated – juanpa.arrivillaga Jun 05 '20 at 23:20
  • Juanpa -- the speed differences weren't marginal; and I was processing hundreds of gigabytes of data on a local machine (Gov. Job -- not ideal) it helped. In terms of it being "terrible style" -- sure, okay -- if it's more efficient and equally understandable I'll lose some style points. "Special cases aren't special enough to break the rules. Although practicality beats purity." I was reminded of that same piece `super().__getitem__` and implemented -- it's been a while since I overwrote a class and needed (and I've been coding in go/node more lately) Thank you for your help – Schalton Jun 05 '20 at 23:26

1 Answers1

0

This:

Add a decorator to existing builtin class method in python

Reminded me of the syntax to call the subclass method -- solved my problem

class DownloadDict(dict):
    """dictionary that will fetch if missing data"""
    def __init__(self, name, *args, **kwargs):
        super(DownloadDict, self).__init__(*args, **kwargs)
        self.name = name

    def download(self):
        [self.__setitem__(key,value) for (key, value) in makeReferenceJson()[self.name].items()]

    def __getitem__(self, key):
        try:
            return super(DownloadDict, self).__getitem__(key)
        except AttributeError:
            self.download()
            return super(DownloadDict, self).__getitem__(key)
Schalton
  • 2,867
  • 2
  • 32
  • 44