Alternatives to attrdict library for nested AttrDict

Question

My program contains a lot of configuration parameters so I was looking for a way to have them all in one place, accessible from every file in the project. I thought about a config module which would act as an interface to a yaml file that contains the configuration parameters. One requirement I want is to be able to access the config object with attribute (dot) notation. I found the AttrDict library and wrote the following code:

import yaml
from attrdict import AttrDict


def get_cfg():
    return cfg_node


def dump(out_filepath):
    with open(out_filepath, 'w') as file:
        yaml.dump(attrdict_to_dict(cfg_node), file)


def load_yaml(filepath):
    global cfg_node
    with open(filepath, 'r') as file:
        cfg_node = dict_to_attrdict(yaml.safe_load(file))


def attrdict_to_dict(myAttrdict):
    dictionary = dict(myAttrdict)
    for k in myAttrdict:
        value = dictionary[k]
        if isinstance(value, AttrDict):
            dictionary[k] = attrdict_to_dict(value)
    return dictionary


def dict_to_attrdict(dictionary):
    myAttrdict = AttrDict()
    for k in dictionary:
        value = dictionary[k]
        if isinstance(value, dict):
            myAttrdict.__setattr__(k, attrdict_to_dict(value))
    return myAttrdict


cfg_node = AttrDict()
cfg_node.b = "value_b"
cfg_node.a = AttrDict()
cfg_node.a.b = AttrDict()
cfg_node.a.c = "nested_c"
cfg_node.a.b.e= "double_nested_e"

The problem is that this library does not allow nested AttrDict().

print(cfg_node)
 >>> {'b': 'value_b', 'a': AttrDict({})}

However if I do the following, nested assignment works but I'm forced to explicitly use the nested object which is not useful for my purpose:

cfg_node = AttrDict()
cfg_node.b = "value_b"
a =  AttrDict()
a.c = "nested_value_c"
cfg_node.a = a
print(cfg_node)
 >>> {'b': 'value_b', 'a': AttrDict({'c':'nested_value_c'})}

Answer 1

I've looked into the documentation of AttrDict and I'm afraid that recursive attribute access is not possible. See here:

Recursive attribute access results in a shallow copy, so recursive assignment will fail (as you will be writing to a copy of that dictionary):

> attr = AttrDict('foo': {})
> attr.foo.bar = 'baz'
> attr.foo
{}

They suggest using an AttrMap instead. However I've tried using it and I was getting an error right from the import. I'll look into it. *update below

For now, what you can do is define instead your custom AttrDict.

class AttrDict(dict):

    def __init__(self, *args, **kwargs):
        super(AttrDict, self).__init__(*args, **kwargs)
        self.__dict__ = self


if __name__ == '__main__':
    cfg_node = AttrDict()
    cfg_node.b = "value_b"
    cfg_node.a = AttrDict()
    cfg_node.a.b = AttrDict()
    cfg_node.a.c = "nested_c"
    cfg_node.a.b.e = "double_nested_e"
    print(cfg_node)
    print(cfg_node.a.b.e)

The output here is, as you needed:

{'b': 'value_b', 'a': {'b': {'e': 'double_nested_e'}, 'c': 'nested_c'}}
double_nested_e

If you are using an IDE you will also get code completion while you access the keys of AttrDict.

See also here, there are additional methods you can add to the custom class that can enhance its functionalities.

Hope this helps!

Update on the AttrMap library. I've asked about it here, I was using python 3.8 but the library contains a list[str], which is available only on python >= 3.9.

Second update on AttrMap. The creator has fixed the issue in the new version, now it supports python 3.6

Answer 2

Representing Configuration in Code

I like pydantic and dataclasses for such use-cases. I feel like pydantic makes it incredible clean and easy.

from datetime import datetime
from typing import List, Optional
from pydantic import BaseModel

class User(BaseModel):
    id: int
    name = 'John Doe'
    signup_ts: Optional[datetime] = None
    friends: List[int] = []

You can create the objects just like normal objects, parse it from a JSON / YAML (or anything that creates a dict). You can have validation logic and data transformations.

Attribute-like access

If it's just about making quick objects: types.SimpleNamespace:

from types import SimpleNamespace

sn = SimpleNamespace(param_a='1')
print(sn.param_a)

Answer 3

Checkout the module attridict, if offers what you are looking for. It 100% allows for nested dicts. It basically works the same

> att = attridict({'foo': {}})
> att.foo.bar = 'baz'
> att.foo
{'bar':'baz'}