Pytest passing test I think it should fail on sys.exit() call

Question

I am trying to check the exit code I have for a script I'm writing in python3 on a Mac (10.14.4). When I run the test it doesn't fail which I think is wrong. But I can't see what it is that I've got wrong.

The test file looks like this:

import pytest
import os
import sys

sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))

import my_script

class TestMyScript():
    def test_exit(self):
         with pytest.raises(SystemExit) as pytest_wrapped_e:
            my_script.main()
            assert pytest_wrapped_e.type == SystemExit

    def test_exit_code(self):
         with pytest.raises(SystemExit) as pytest_wrapped_e:
            my_script.main()
            self.assertEqual(pytest_wrapped_e.exception.code, 42)

My script looks like:

#!/usr/bin/env python3
import sys

def main():
    print('Hello World!')
    sys.exit(0)

if __name__ == '__main__':
    main()

The output I get is:

$ py.test -v
============================= test session starts ==============================
platform darwin -- Python 3.7.3, pytest-3.10.1, py-1.8.0, pluggy-0.9.0 -- /usr/local/opt/python/bin/python3.7
cachedir: .pytest_cache
rootdir: /Users/robertpostill/software/gateway, inifile:
plugins: shutil-1.6.0
collected 2 items

test/test_git_refresh.py::TestGitRefresh::test_exit PASSED               [ 50%]
test/test_git_refresh.py::TestGitRefresh::test_exit_code PASSED          [100%]

=========================== 2 passed in 0.02 seconds ===========================
$

I would expect the second test(test_exit_code) to fail as the exit call is getting a code of 0, not 42. But for some reason, the assert is happy whatever value I put in the sys.exit call.

score 4 · Accepted Answer · answered Apr 24 '19 at 14:03

Good question, that's because your Asserts are never called (either of them). When exit() is called the program is done (at least within the with clause), it turns off the lights, packs up its bags, and goes home. No further functions will be called. To see this add an assert before and after you call main:

def test_exit_code(self):
     with pytest.raises(SystemExit) as pytest_wrapped_e:
        self.assertEqual(0, 1) # This will make it fail
        my_script.main()
        self.assertEqual(0, 1) # This will never be called because main `exits`
        self.assertEqual(pytest_wrapped_e.exception.code, 42)

A test passes if no asserts fail and nothing breaks, so in your case both tests passed because the assert was never hit.

To fix this pull your asserts out of the with statement:

def test_exit_code(self):
    with pytest.raises(SystemExit) as pytest_wrapped_e:
        my_script.main()
    self.assertEqual(pytest_wrapped_e.exception.code, 42)

Though now you will need to fix the pytest syntax because you are missing some other stuff.

See: Testing sys.exit() with pytest

Pytest passing test I think it should fail on sys.exit() call

1 Answers1

Linked