Pydantic: Create model with fixed and extended fields from a Dict[str, OtherModel], the Typescript [key: string] way

Question

From a similar question, the goal is to create a model like this Typescript interface:

interface ExpandedModel {
  fixed: number;
  [key: string]: OtherModel;
}

However the OtherModel needs to be validated, so simply using:

class ExpandedModel(BaseModel):
    fixed: int

    class Config:
        extra = "allow"

Won't be enough. I tried root (pydantic docs):

class VariableKeysModel(BaseModel):
    __root__: Dict[str, OtherModel]

But doing something like:

class ExpandedModel(VariableKeysModel):
    fixed: int

Is not possible due to:

ValueError: root cannot be mixed with other fields

Would something like @root_validator (example from another answer) be helpful in this case?

Are you sure this is not an [XY Problem](https://xyproblem.info)? Can you explain, what the purpose of such a model would be? Because it is certainly possible, it just seems like trying to hammer in a screw. The beauty of class-based models is that the type safety that comes with fixed defined attributes. If you are dealing with dynamic key-object-pairs, why not just use a dynamic data structure like a `dict` and validate each item separately? Where is your data coming from and what do you intend to do with it? — Daniil Fajnberg, Feb 18 '23 at 00:34
I agree it's not pretty. I'm using Pydantic to parse all models from a file and this is just one of those that need to be validated properly. I don't agree with the [XY problem](https://xyproblem.info/) assessment though: this kind of interface is common in TS, it should be simple to replicate in pydantic (e.g. class inheritance) and others have asked for similar solutions… — CPHPython, Feb 20 '23 at 08:56

score 1 · Accepted Answer · answered Feb 20 '23 at 09:38

Thankfully, Python is not TypeScript. As mentioned in the comments here as well, an object is generally not a dictionary and dynamic attributes are considered bad form in almost all cases.

You can of course still set attributes dynamically, but they will for example never be recognized by a static type checker like Mypy or your IDE. This means you will not get auto-suggestions for those dynamic fields. Only attributes that are statically defined within the namespace of the class are considered members of that class.

That being said, you can abuse the extra config option to allow arbitrary fields to by dynamically added to the model, while at the same time enforcing all corresponding values to be of a specific type via a root_validator.

from typing import Any

from pydantic import BaseModel, root_validator


class Foo(BaseModel):
    a: int


class Bar(BaseModel):
    b: str

    @root_validator
    def validate_foo(cls, values: dict[str, Any]) -> dict[str, Any]:
        for name, value in values.items():
            if name in cls.__fields__:
                continue  # ignore statically defined fields here
            values[name] = Foo.parse_obj(value)
        return values

    class Config:
        extra = "allow"

Demo:

if __name__ == "__main__":
    from pydantic import ValidationError

    bar = Bar.parse_obj({
        "b": "xyz",
        "foo1": {"a": 1},
        "foo2": Foo(a=2),
    })
    print(bar.json(indent=4))

    try:
        Bar.parse_obj({
            "b": "xyz",
            "foo": {"a": "string"},
        })
    except ValidationError as err:
        print(err.json(indent=4))

    try:
        Bar.parse_obj({
            "b": "xyz",
            "foo": {"not_a_foo_field": 1},
        })
    except ValidationError as err:
        print(err.json(indent=4))

Output:

{
    "b": "xyz",
    "foo2": {
        "a": 2
    },
    "foo1": {
        "a": 1
    }
}

[
    {
        "loc": [
            "__root__",
            "a"
        ],
        "msg": "value is not a valid integer",
        "type": "type_error.integer"
    }
]

[
    {
        "loc": [
            "__root__",
            "a"
        ],
        "msg": "field required",
        "type": "value_error.missing"
    }
]

A better approach IMO is to just put the dynamic name-object-pairs into a dictionary. For example, you could define a separate field foos: dict[str, Foo] on the Bar model and get automatic validation out of the box that way.

Or you ditch the outer base model altogether for that specific case and just handle the data as a native dictionary with Foo values and parse them all via the Foo model.

Appreciate all details and suggestions you provided, a minor alternative to avoid the **Config** in the _Bar_ model would be to declare `class Bar(BaseModel, extra=Extra.allow):`. — CPHPython, Mar 01 '23 at 10:18
@CPHPython Yes, you can always pass your config as `__init_subclass__` keword-arguments. What is more readable/usable, depends on the situation and personal preference of course. — Daniil Fajnberg, Mar 01 '23 at 10:26

Pydantic: Create model with fixed and extended fields from a Dict[str, OtherModel], the Typescript [key: string] way

1 Answers1

Linked