6

Recently I have started to use hydra to manage the configs in my application. I use Structured Configs to create schema for .yaml config files. Structured Configs in Hyda uses dataclasses for type checking. However, I also want to use some kind of validators for some of the parameter I specify in my Structured Configs (something like this).

Do you know if it is somehow possible to use Pydantic for this purpose? When I try to use Pydantic, OmegaConf complains about it:

omegaconf.errors.ValidationError: Input class 'SomeClass' is not a structured config. did you forget to decorate it as a dataclass?
Kıvanç Yüksel
  • 701
  • 7
  • 17
  • In addition to the answers below, it might be possible to use [`hydra_zen.hydrated_dataclass`](https://mit-ll-responsible-ai.github.io/hydra-zen/generated/hydra_zen.hydrated_dataclass.html) in combination with [`hydra_zen.validates_with_pydantic`](https://mit-ll-responsible-ai.github.io/hydra-zen/generated/hydra_zen.third_party.pydantic.validates_with_pydantic.html), though I haven't tried it myself. – Jasha Jan 15 '22 at 12:03

2 Answers2

8

For those of you wondering how this works exactly, here is an example of it:

import hydra
from hydra.core.config_store import ConfigStore
from omegaconf import OmegaConf
from pydantic.dataclasses import dataclass
from pydantic import validator


@dataclass
class MyConfigSchema:
    some_var: float

    @validator("some_var")
    def validate_some_var(cls, some_var: float) -> float:
        if some_var < 0:
            raise ValueError(f"'some_var' can't be less than 0, got: {some_var}")
        return some_var


cs = ConfigStore.instance()
cs.store(name="config_schema", node=MyConfigSchema)


@hydra.main(config_path="/path/to/configs", config_name="config")
def my_app(config: MyConfigSchema) -> None:
    # The 'validator' methods will be called when you run the line below
    OmegaConf.to_object(config)


if __name__ == "__main__":    
    my_app()

And config.yaml :

defaults:
  - config_schema

some_var: -1  # this will raise a ValueError
Kıvanç Yüksel
  • 701
  • 7
  • 17
  • Nice, looks good! – Jasha Jan 15 '22 at 11:54
  • Kıvanç, have you been able to get this to work with [custom](https://pydantic-docs.helpmanual.io/usage/types/#pydantic-types)- or [constrained](https://pydantic-docs.helpmanual.io/usage/types/#constrained-types) types? I get ValidationErrors. I would prefer to use such types instead of @validators. – Emre Feb 10 '22 at 01:51
  • This doesn't seem to work for me. When you call `OmegaConf.to_object()`, it converts the object to a `dict` instead of a `MyConfigSchema`. Because of this, `MyConfigSchema` is never constructed and the validator never runs. – Antyos Feb 11 '22 at 20:17
  • @Emre I don't think it is possible to use types other than what python [dataclasses](https://docs.python.org/3/library/dataclasses.html) support. – Kıvanç Yüksel Feb 12 '22 at 14:39
  • 1
    @Antyos are you sure you are using the Structred Config you register in your `.yaml` file (in `defaults` key)? – Kıvanç Yüksel Feb 12 '22 at 14:41
  • @KıvançYüksel That's what I was missing! I was always curious what the purpose of the `name` parameter in `cs.store()` did. Thank you! It might be helpful for other people to include a sample YAML file in the response and include the import of `omegaconf` so it works as is. – Antyos Feb 12 '22 at 19:55
  • 1
    @Antyos Glad it's working now! :) I have already edited my answer based on your suggestions. – Kıvanç Yüksel Feb 12 '22 at 20:44
0

See pydantic.dataclasses.dataclass, which are a drop-in replacement for the standard-library dataclasses with some extra type-checking.

Jasha
  • 5,507
  • 2
  • 33
  • 44