0

It looks like garbage collection is a problem with python3 -m unittest discover.

Look at this example:

file: model.py # a basic SQLAlchemy declarative model, as well as the DB manager to insert the data

from sqlalchemy.engine import create_engine
from sqlalchemy.orm import sessionmaker
from sqlalchemy import Table, Column, Integer, String
from sqlalchemy.ext.declarative import declarative_base

Base = declarative_base()


class Model(Base):
    __tablename__ = 'model'

    id = Column('id', Integer, primary_key=True)
    name = Column('name', String, unique=True)
    val = Column('val', Integer)



class DBManager:
    
    def __init__(self, dbfile):
        self.engine = create_engine(f'sqlite:///{dbfile}')
        Base.metadata.create_all(bind=self.engine)

    def insert_into_db(self, data):
        Session = sessionmaker(bind=self.engine)
        session = Session()
        for model in data:
            session.add(model)
        session.commit()
        session.close()

file: creator.py # it creates a single model and put it into a sqlite database

from model import Model

class ModelCreator:

    data = []

    def add_data(self, dic):
        self.data.append(Model(**dic))

    def get_data(self):
        if self.data:
            return self.data
        else:
            raise RuntimeError('No data found')


Now the tests (2 separate unittest.TestCase files with one test each)

file: test_model.py

import unittest
from creator import ModelCreator

class TestModelCreator(unittest.TestCase):
    
    def setUp(self):
        self.mc = ModelCreator()


    def test_add_data(self):
        with self.assertRaises(RuntimeError):
            self.mc.get_data()
        d = {'name': 'model_test', 'val': 1}
        self.mc.add_data(d)
        self.assertEqual(len(self.mc.get_data()), 1)

    def tearDown(self):
        return super().tearDown()

file: test_creator.py

import unittest
import os
from model import DBManager
from creator import ModelCreator


class TestDB(unittest.TestCase):

    @classmethod
    def setUpClass(cls):
        cls.dbfile = './test.db'

    def setUp(self):
        self.dbm = DBManager(self.dbfile)
        self.mc = ModelCreator()

    def test_insert_into_db(self):
        d = {'name': 'model_test', 'val': 1}
        self.mc.add_data(d)
        to_insert = self.mc.get_data()
        self.dbm.insert_into_db(to_insert)

    def tearDown(self):
        return super().tearDown()

    @classmethod
    def tearDownClass(cls):
        os.remove(cls.dbfile)

Each test runs smoothly when launched separately.

$ python3 -m unittest test_model.py 
.
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK

$ python3 -m unittest test_creator.py 
.
----------------------------------------------------------------------
Ran 1 test in 0.014s

OK

BUT

$ python3 -m unittest discover .
.F
======================================================================
FAIL: test_add_data (test_model.TestModelCreator)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/marco/sw/temp/sqlalchemytest/test_model.py", line 12, in test_add_data
    self.mc.get_data()
AssertionError: RuntimeError not raised

----------------------------------------------------------------------
Ran 2 tests in 0.013s

FAILED (failures=1)

$ nosetests -vw .
test_insert_into_db (test_creator.TestDB) ... ok
test_add_data (test_model.TestModelCreator) ... FAIL

======================================================================
FAIL: test_add_data (test_model.TestModelCreator)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/marco/sw/temp/sqlalchemytest/test_model.py", line 12, in test_add_data
    self.mc.get_data()
AssertionError: RuntimeError not raised

----------------------------------------------------------------------
Ran 2 tests in 0.128s

FAILED (failures=1)

After a couple of print statements across the tests I found that maybe it is a problem of garbage collection. So, my question is: how to enforce this?

Looks like the tearDown methods do not work properly....

It looked solved in Python 3.4. Also, same problem was discusses (with no luck) here back in the days.

Now I'm using Python 3.9.2

Looking forward for any comments. Thanks

snakecharmerb
  • 47,570
  • 11
  • 100
  • 153
  • 1
    I'm not sure garbage collection is the issue here. `ModelCreator.data` is a class attribute, so all instances of `ModelCreator` share the same list. The list will only be "cleared" if the `creator` module is removed from `sys.modules` and reimported (or possibly if the module is reloaded). Or if you clear it manually. – snakecharmerb Sep 15 '21 at 15:51
  • 1
    If you want `ModelCreator.data` to be garbage collected, make it an instance variable by doing `self.data = []` in `ModelCreator.__init__`. – snakecharmerb Sep 15 '21 at 18:17
  • @snakecharmerb this was the right solution! Thank you very much. – Marco Milanesio Sep 16 '21 at 07:54
  • More or less. Quoting: "Class attributes become instance attributes if and only if a value is assigned to them after instantiation, being in the `__init__` method or not". Then I don't get why `unittest discover` does not clean instance attributes set in that way. Thanks again for your time. – Marco Milanesio Sep 16 '21 at 11:16
  • 1
    If you assign to a class var at instance level it will override the class var in the instance's `__dict__`, for example `instance.data = [1, 2, 3]`. But if you mutate the class var - `instance.data.append(1)` - the class var is mutated. Your code is mutating, not assigning, so there is no instance attribute to clear. Compare the contents of the class and instance `__dict__` to see what is happening. – snakecharmerb Sep 16 '21 at 11:33
  • I see it, now. Thank you very much! – Marco Milanesio Sep 16 '21 at 11:45

0 Answers0