4

I have a parametrized (py)test that cleans up some table in the DB before running the logic:

@pytest.mark.parametrize('limit', [1, 1000, 5000000])
def test_calc(limit):
    # clean test table in database !!!
    assert calc(limit) == None

At the end of the tests I get:

=========== 3 passed in 357.65s (0:05:57) ===========

The problem is that about 2-3 minutes of that is the table cleanup.

How can I pause the timer before the cleanup and continue the timer after the cleanup finished ?

Like in Golang's testing library - T.StopTimer() & T.StartTime()

I searched all over google, and the closest thing I stumbled upon was freezegun.
I tried it out but couldn't get rid of the cleanup time, maybe I was using it somehow wrong :/
As far I understood it manipulates the datetime object but not pytest's timer (mechanism) :(

EvgenyKolyakov
  • 3,310
  • 2
  • 21
  • 31
  • Well, that's the total test duration - it's a statistic, why would you care about shortening it? – AKX Apr 16 '21 at 17:50
  • Why on earth would "table cleanup" take so long? I'd be working on getting that down to < 1 second .. – wim Apr 16 '21 at 17:52
  • Because I need to know if my new code improved or harmed performance without counting the bootstrap steps... – EvgenyKolyakov Apr 16 '21 at 17:53
  • When every row takes couple of megabytes and there're 1.5B records with having slow HDD, a delete query can take some time... – EvgenyKolyakov Apr 16 '21 at 17:54
  • Can you add the code which is responsible for the table cleanup? – Lars Blumberg Apr 16 '21 at 20:16
  • No, because this is irrelevant to the question, imagine it’s not a db clean up but some other bootstrap process. How do I pause pytest’s timer ? – EvgenyKolyakov Apr 16 '21 at 20:19
  • Could you store the table cleanup time (gathered separately) and later subtract it from the pytest time? (I haven't used pytest, just thinking aloud) – RufusVS Apr 16 '21 at 20:21
  • I see. I personally wouldn't mess with the way how pytest measures test execution time. If it's that important to show the actual time of test execution minus the test setup, then I'd run a command before executing pytest itself which does the initial setup. Not that elegant but at least it doesn't fiddle with the expected behavior of pytest. – Lars Blumberg Apr 16 '21 at 20:23
  • @LarsBlumberg if I wanted to measure time myself, I wouldn’t have used a testing framework; and Yes, it’s the most important thing to show regarding time, how long did My code run, I trust the databases’ developers to test their own code. – EvgenyKolyakov Apr 16 '21 at 20:30

2 Answers2

3

There is no timer to stop; pytest stores a timestamp on session start and prints the difference between the current time and the timestamp on session finish (if you're curious about the exact spot in the code where the duration is calculated, it's here). If the cleanup code is not tied to the rest of the test suite, you can move it into a custom impl of the pytest_unconfigure hook. Example:

import time
import pytest


@pytest.fixture(autouse=True)
def db():
    yield
    import time
    time.sleep(3)


def test_db():
    assert True

Running the test will yield

============================== 1 passed in 3.01s ==============================

Moving the db cleanup to the hookimpl:

# conftest.py

import time


def pytest_unconfigure(config):
    import time
    time.sleep(3)

The reported duration is now

============================== 1 passed in 0.01s ==============================
hoefling
  • 59,418
  • 12
  • 147
  • 194
0

As there was no way to do so for parametrized tests, I just created a PR to pytest - https://github.com/pytest-dev/pytest/issues/8568

Let's see how it goes :)

EvgenyKolyakov
  • 3,310
  • 2
  • 21
  • 31