Pytest environment fixtures not seen by imported class

Question

I'm trying to set up pytest so that whenever I run my tests (locally or in github actions), the environment variables all point to files and locations in my test directory instead of wherever they're set to based on the user.

The problem is, the fixture changes are visible if I add an ipdb trace in the test_database function and print os.getenv('DB_URL') but the assert will always fail because the DataBase object always has the original non-mocked url (set in .bash_profile).

database.py

import h5py
import os

class DataBase:

    route = os.environ.get('DB_URL')

    def __init__(self):
        self.connected = False

    def connect(self):
        if not connected:
            self.db = h5py.File(self.route, 'r')
            self.connected = True

conftest.py

import os
import pytest

@pytest.fixture(autouse=True)
def mock_test_env(monkeypatch):
    cwd = os.getcwd()
    monkeypatch.setenv('DB_URL', cwd + '/sample_db.hdf5')

test_database.py

import pytest
from repo import DataBase

def test_database():
    db = DataBase()
    import ipdb; ipdb.set_trace()
    '''
    os.getenv('DB_URL') returns cwd + '/sample_db.hdf5'
    db.route returns original database, not the sample one above
    '''
    assert db.connected = False, 'DataBase must be instantiated to connected == False'

How do I globally set environment variables so all objects see the same envs?

The `DataBase` class is hard to test in its current state. `DataBase.route` will be set on `database` module import which will happen at tests collection, fixtures are executed much later, so monkeypatching will have no effect. You can either refactor `DataBase.route` to not be a class attribute, or move `DataBase` import into the test to defer it, but you may also need a fixture that reloads the module before the test starts and before `mock_test_env` executes, depending on the rest of the test suite. Rule of thumb - try executing as little code on import as possible. — hoefling, Jul 15 '20 at 22:04
@hoefling Do you have an example of how it should be refactored? Maybe I could write another method that sets the route upon instantiation of the class object or write another class that loads env vars that can be a base class for the database class and any other classes that use env vars. — pmdaly, Jul 15 '20 at 22:20
Or you can simply say `database.DataBase.route = 'whatever'` in the fixture without needing to monkeypatch at all. — hoefling, Jul 15 '20 at 22:30
_Do you have an example of how it should be refactored?_ Class variables are usually constants; if you have to calculate the value of `route`, make it an instance variable like `connected`. Even better would be passing `route` as argument in `DataBase.__init__`. — hoefling, Jul 15 '20 at 22:36
If you need a good example, check out Django's [`BaseDatabaseWrapper`](https://github.com/django/django/blob/156a2138db20abc89933121e4ff2ee2ce56a173a/django/db/backends/base/base.py#L26): none of the class variables are calculated, all settings are passed in the constructor. — hoefling, Jul 15 '20 at 22:38
@pmdaly Do `self.route = os.environ.get('DB_URL')` in `Database.__init__`. — aaron, Jul 16 '20 at 18:12
Define route as property: https://stackoverflow.com/questions/17330160/how-does-the-property-decorator-work — Pablo Henkowski, Jul 22 '20 at 17:41

score 5 · Accepted Answer · answered Jul 18 '20 at 13:05

As others have mentioned in your comment that class variables to be avoided for this assignment because it's a constant which gets assigned the moment the import statement is scanned.

To better understand this situation, try placing the from repo import DataBase inside your method

def test_database():

Like this:

import os
import pytest

@pytest.fixture(autouse=True)
def mock_test_env(monkeypatch):
    cwd = os.getcwd()
    monkeypatch.setenv('DB_URL', cwd + '/sample_db.hdf5')

def test_database(mock_test_env):
    from repo import DataBase # <<< This here works
    db = DataBase()
    assert db.route == (os.getcwd() + '/sample_db.hdf5') # Works!

Now, when you place the from repo import Database at the start of the file, pytest would scan and start reading all the imports and starts initialising the blueprints and sets the value of router the moment its imported.

Read: When is the class variable initialised in Python?

So the ideal way would be to avoid maybe such important initialisation and assing the same in the constructor of the Database class. Thus ensuring it does calculate when needed.

I feel, I for one like to think of it like Explicit is better than implicit. from the Zen Of Python and do it like:

import h5py
import os

class DataBase:

    def __init__(self):
        self.route = os.environ.get('DB_URL')
        self.connected = False

    def connect(self):
        if not connected:
            self.db = h5py.File(self.route, 'r')
            self.connected = True

When you say `autouse=True`, you don't need to pass the `mock_test_env` fixture as a param. Hope it makes sense! — Parvathirajan Natarajan, Oct 20 '21 at 20:06

Pytest environment fixtures not seen by imported class

1 Answers1