8

For all my projects, I load all env variables at the start and check that all the expected keys exist as described by an .env.example file following the dotenv-safe approach.

However, the env variables are strings, which have to be manually cast whenever they're used inside the Python code. This is annoying and error-prone. I'd like to use the information from the .env.example file to cast the env variables and get Python typing support in my IDE (VS Code). How do I do that?

env.example

PORT: int
SSL: boolean

Python Ideal Behavior

# Set the env in some way (doesn't matter)
import os
os.environment["SSL"] = "0"
os.environment["PORT"] = "99999"

env = type_env()
if not env["SSL"]: # <-- I'd like this to be cast to boolean and typed as a boolean
    print("Connecting w/o SSL!")
if 65535 < env["PORT"]:  # <-- I'd like this to be cast to int and typed as an int
    print("Invalid port!")

In this code example, what would the type_env() function look like assuming it only supported boolean, int, float, and str?

It's not too hard to do the casting as shown in e.g. https://stackoverflow.com/a/11781375/1452257, but it's unclear to me how to get it working with typing support.

pir
  • 5,513
  • 12
  • 63
  • 101
  • What you are asking is not a within the scope of dotenv files. In simplest terms, dotenv files contain what look like (Unix) shell variables assigned to known values. There is no interpolation done in a dotenv file, no variable substitution, just a file containing lines that look like `key=value`. When you begin to ask to add type information to a dotenv file, you are now creating an entirely new format and it is no longer suitable to be called dotenv, given what we know of a dotenv file. Either do it manually, or look into other means of configuration such as the builtin `configparser` – smac89 Apr 14 '20 at 20:21
  • I don't think it's possible to have static code analysis loading types from `env.example`. For static analysis, you should move your types to your code, as shown in many examples below. – Aleksandr Borisov Apr 21 '20 at 10:45

5 Answers5

16

I will suggest using pydantic.

From StackOverflow pydantic tag info

Pydantic is a library for data validation and settings management based on Python type hinting (PEP484) and variable annotations (PEP526). It allows for defining schemas in Python for complex structures.

let's assume that you have a file with your SSL and PORT envs:

with open('.env', 'w') as fp:
    fp.write('PORT=5000\nSSL=0')

then you can use:

from pydantic import BaseSettings

class Settings(BaseSettings):
    PORT : int
    SSL : bool
    class Config:
        env_file = '.env'

config = Settings()

print(type(config.SSL),  config.SSL)
print(type(config.PORT),  config.PORT)
# <class 'bool'> False
# <class 'int'> 5000

with your code:

env = Settings()

if not env.SSL:
    print("Connecting w/o SSL!")
if 65535 < env.PORT: 
    print("Invalid port!")

output:

Connecting w/o SSL!
kederrac
  • 16,819
  • 6
  • 32
  • 55
  • This is so simple. Thank you! We need to avoid 3rd party dependencies as much as possible so we'll likely go with Kevin's answer, but this is the correct answer for most people so I've selected it. – pir Apr 23 '20 at 04:57
7

The following solution offers both runtime casting to the desired types and type hinting help by the editor without the use of external dependencies.

Also check kederrac's answer for an awesome alternative using pydantic, which takes care of all of this for you.


Working directly with a non-Python dotenv file is going to be too hard, if not impossible. It's way easier to handle all the information in some Python data structure, as this lets the type checkers do their job without any modification.

I think the way to go is to use Python dataclasses. Note that although we specify types in the definition, they are only for the type checkers, not enforced at runtime. This is a problem for environment variables, as they are external string mapping information basically. To overcome this, we can force the casting in the __post_init__ method.

Implementation

First, for code organization reasons, we can create a Mixin with the type enforcing logic. Note that the bool case is special since its constructor will output True for any non-empty string, including "False". If there's some other non-builtin type you want to handle, you would need to add special handling for it, too (although I wouldn't suggest making this logic handle more than these simple types).

import dataclasses
from distutils.util import strtobool

class EnforcedDataclassMixin:

    def __post_init__(self):
        # Enforce types at runtime
        for field in dataclasses.fields(self):
            value = getattr(self, field.name)
            # Special case handling, since bool('False') is True
            if field.type == bool:
                value = strtobool(value)
            setattr(self, field.name, field.type(value))

This implementation can also be done with a decorator, see here.

Then, we can create the equivalent of a ".env.example" file like this:

import dataclasses

@dataclasses.dataclass
class EnvironmentVariables(EnforcedDataclassMixin):
    SSL: bool
    PORT: int
    DOMAIN: str

and for easy parsing from os.environ, we can create a function like

from typing import Mapping

def get_config_from_map(environment_map: Mapping) -> EnvironmentVariables:
    field_names = [field.name for field in dataclasses.fields(EnvironmentVariables)]
    # We need to ignore the extra keys in the environment,
    # otherwise the dataclass construction will fail.
    env_vars = {
        key: value for key, value in environment_map.items() if key in field_names
    }
    return EnvironmentVariables(**env_vars)


Usage

Finally, taking these things together, we can write in a settings file:

import os
from env_description import get_config_from_map


env_vars = get_config_from_map(os.environ)

if 65535 < env_vars.PORT:
    print("Invalid port!")

if not env_vars.SSL:
    print("Connecting w/o SSL!")

Static type checking works correctly in VS Code and mypy. If you assign PORT (which is an int) to a variable of type str, you will get an alert!

Type hinting working

To pretend it's a dictionary, Python provides the asdict method in the dataclasses module.

env_vars_dict = dataclasses.asdict(env_vars)
if 65535 < env_vars_dict['PORT']:
    print("Invalid port!")

But sadly (as of the time of this answer) you lose static type checking support doing this. It seems to be work in progress for mypy.

Kevin Languasco
  • 2,318
  • 1
  • 14
  • 20
3

Given my comment above, I would suggest the following format for your config file:

settings.py, config.py, etc

from dotenv import load_dotenv
import os

load_dotenv()

SSL = os.getenv("SSL").lower() == 'true'
PORT = int(os.getenv("PORT", 5555)) # <-- can also set default

# Check all your other variables and expected keys here...

No need for type_env() function, as it is all done in a python file which can be imported in any module.

Now where ever you want to those variables, just import them because they have already been converted to the right type. For example:

main.py

import config

if not config.SSL:
    print("Connecting w/o SSL!")
if 65535 < config.PORT:
    print("Invalid port!")

The above will work because all the conversion has been done when the .env file was loaded.

Doing it this way, you can see that if the value used in the .env file cannot be converted to the type you were expecting, the configuration will fail.

smac89
  • 39,374
  • 15
  • 132
  • 179
0
  • Assuming that your env.example is in yaml format (at least what you wrote is valid yaml)
  • And assuming that you have PyYaml installed (pip install pyyaml)

... Then the following code works:

# do this or anything else to make a dict from your env.example
import yaml
example=yaml.safe_load("""
PORT: int
SSL: bool
""")

# the missing implementation
def type_env():
    env={}
    for k, v in os.environ.items():
        t=example.get(k)
        if t == "bool":
            env[k] = v.lower() not in ["false", "no", "0", ""] # whatever you want to consider as False
            # or env[k] = v.lower() in ["true", "yes", "1"] # whatever you want to consider as True
        elif t == "int":
            env[k] = int(v)
        elif t == "float":
            env[k] = float(v)
        else:
            env[k] = v
    return env

# From now on your code (exactly your code, except amending os.environment to os.environ)

# Set the env in some way (doesn't matter)
import os
os.environ["SSL"] = "0"
os.environ["PORT"] = "9999"

env = type_env()
if not env["SSL"]: # <-- I'd like this to be cast to boolean and typed as a boolean
    print("Connecting w/o SSL!")
if 65535 < env["PORT"]:  # <-- I'd like this to be cast to int and typed as an int
    print("Invalid port!")

yaccob
  • 1,230
  • 13
  • 16
0

You have 2 options - either you explicitly specify variables types, or you let the type_env function to infer the types from the actual values. Other commenters have already provided examples how to go with explicit types and I would personally use PORT = int(os.getenv("PORT", 5555)) or dataclass approaches, depending on how many variables you have to use.

However, specifying types explicitly imposes a bit of overhead. Here is my contribution how to infer. It won't let mypy know the exact types, they all will be Any.

import os
from distutils.util import strtobool
from typing import Dict, Any

os.environ["SSL"] = "0"
os.environ["PORT"] = "99999"


def type_env() -> Dict[str, Any]:
    d: Dict[str, Any] = dict(os.environ)
    for key in d:
        try:
            d[key] = bool(strtobool(d[key]))
            continue
        except ValueError:
            pass
        try:
            d[key] = int(d[key])
            continue
        except ValueError:
            pass
        try:
            d[key] = float(d[key])
            continue
        except ValueError:
            pass
    return d


env = type_env()
print(type(env["SSL"]))
print(type(env["PORT"]))

if not env["SSL"]:  # <-- I'd like this to be cast to boolean and typed as a boolean
    print("Connecting w/o SSL!")
if 65535 < env["PORT"]:  # <-- I'd like this to be cast to int and typed as an int
    print("Invalid port!")
Aleksandr Borisov
  • 2,136
  • 1
  • 13
  • 14