Construct an object using ruamel

Question

I want to create some objects based on arguments in a yaml file. Everything was working fine, but now I want to supply the class with some options from which one should be chosen randomly. I thought I could use the __post_init__ method for this, but the learned, that this is intentionally not called because the use case is to serialize and deserialize objects.

Is there a way to tell ruamel to use the constructor of the class and call __post_init__?

This is almost what I want, but I couldn't get it to work with ruamel despite it saying that it was tested using ruamel at the bottom:

Answer on SO: Is there a way to construct an object using PyYAML construct_mapping after all nodes complete loading?

My current setup is like this:

import random

from abc import ABC, abstractmethod
from dataclasses import dataclass
from ruamel.yaml import YAML, yaml_object

yaml = YAML()

@dataclass
class A(ABC):
    some_var: str
    options: list[str]

    def __post_init__(self):
        self.other_var = random.choice(self.options)

@yaml_object(yaml)
@dataclass
class B(A):
    yaml_tag = "!B"
    more: int

@yaml_object(yaml)
@dataclass
class C(A):
    yaml_tag = "!C"
    foo: list[int]

    def __post_init__(self):
        super().__post_init__()
        self.bar = random.choice(self.foo)

# …
data = yaml.load("config.yml")

yaml to load:

test_classes:
  - !B
    some_var: abc123
    options: ['X', 'Y', 'Z']
    more: 7
  - !C
    some_var: abc123
    options: ['X', 'Y', 'Z']
    foo: [7, 12, 42]

score 1 · Accepted Answer · answered Apr 21 '23 at 11:28

Your YAML document consists of a single unquoted scalar config.yml , and that loads as the Python string config.yml. You can easily check that by doing print(repr(data)) at the end of your program. And for that you don't need to define dataclasses.

If you want to load the contents of the file config.yaml you should do:

yaml.load(Path('config.yaml'))

Please note that the recommended extension for documents containing YAML files has been .yaml since at least September 2006.

Using @yaml_object(yaml) on your dataclasses doesn't give you any option to make yaml_object() call __post_init__(). For that you need to use the "normal" registration, or provide a from_yaml classmethod that does that. However if you pre-construct the dictionary, you can create the dataclass with the constructed values:

import random
from pathlib import Path

from abc import ABC, abstractmethod
from dataclasses import dataclass

from ruamel.yaml import YAML, yaml_object

yaml = YAML()

@dataclass
class A(ABC):
    some_var: str
    options: list[str]

    def __post_init__(self):
        self.other_var = random.choice(self.options)

    @classmethod
    def from_yaml(cls, constructor, node):
        # wrong, but cannot do better with dataclasses without defaults
        data = ruamel.yaml.comments.CommentedMap()  
        constructor.construct_mapping(node, data, deep=True)
        return cls(**data)

@yaml_object(yaml)
@dataclass
class B(A):
    yaml_tag = "!B"
    more: int

@yaml_object(yaml)
@dataclass
class C(A):
    yaml_tag = "!C"
    foo: list[int]

    def __post_init__(self):
        super().__post_init__()
        self.bar = random.choice(self.foo)

data = yaml.load(Path("config.yaml"))
print('other_var:', data['test_classes'][0].other_var)
print('bar:', data['test_classes'][1].bar)

which gives something like (your result can be different because of the randomly picked values):

other_var: Y
bar: 42

Note that from_yaml as presented is not the right way to handle non-scalar nodes. What it should do is something like:

@classmethod
def from_yaml(cls, constructor, node):
    res = cls()
    yield res
    data = ruamel.yaml.comments.CommentedMap()  # wrong, but cannot do any better
    constructor.construct_mapping(node, data, deep=True)
    res.update(**data)
    res.__post_init__() # now the data to choose from is there

But res = cls() fails, as you cannot make instances of B() and C() without providing the attributes (that have no defaults).

For your config file this will only become a problem when there are (indirect) self references in the input YAML document (i.e. when using achors and aliases). When you need those I recommend you change the code and add defaults so that res = cls() will work.

Construct an object using ruamel

1 Answers1