how to make pytest fail on shellcommand invocation failure

Question

For a python pytest file with contents of:

import os
def test_that():
    print("step 1: should work")
    os.system("echo hello")
    
    print("step 2: should fail")
    os.system("asdf_failing_non_existing_command")

and an invocation of pytest like:

 pytest -s testing.py

The output is:

1 passed

I would expect the test to fail. How can I get the exit code from os system call to influence the test failure and actually fail the test?

can you try with `subprocess` https://stackoverflow.com/a/89243 ? — Jaydeep Devda, Jul 14 '22 at 11:39

score 1 · Accepted Answer · answered Jul 14 '22 at 11:39

How can I get the exit code from os system call to influence the test failure and actually fail the test?

Don't use os.system. (You almost never need it, and the docs are pretty strong on using subprocess instead).

With subprocess:

import subprocess

subprocess.run(["echo", "hi"], check=True)
subprocess.run(["asdfasdfasdfdas"], check=True)

Or you can use os.system and check yourself, but why reinvent the wheel?

score 1 · Answer 2 · answered Jul 14 '22 at 11:41

You can assert that the result of os.system() is equal to 0:

import os
def test_that():
    print("step 1: should work")
    assert os.system("echo hello") == 0
    
    print("step 2: should fail")
    assert os.system("asdf_failing_non_existing_command") == 0

The output is:

    def test_that():
        print("step 1: should work")
        assert os.system("echo hello") == 0

        print("step 2: should fail")
>       assert os.system("asdf_failing_non_existing_command") == 0
E       AssertionError: assert 1 == 0
E        +  where 1 = <built-in function system>('asdf_failing_non_existing_command')
E        +    where <built-in function system> = os.system

test.py:7: AssertionError
============================================================================== short test summary info =============================================================================== 
FAILED test.py::test_that - AssertionError: assert 1 == 0
================================================================================= 1 failed in 0.16s ==================================================================================

great answer - I will accept the one above as it is cleaner – Georg Heiler Jul 14 '22 at 11:49 — Georg Heiler, Jul 14 '22 at 11:49
Thanks, the answer above is cleaner, you're right – Yannis Sauzeau Jul 14 '22 at 12:26 — Yannis Sauzeau, Jul 14 '22 at 12:26

score 0 · Answer 3 · answered Jul 14 '22 at 11:37

0

Use subprocess.getstatusoutput, just raise an Exception if the first item it returns (the return code) isn't 0.

answered Jul 14 '22 at 11:37

Ludwig

328
1
6

how to make pytest fail on shellcommand invocation failure

3 Answers3