4

I'm using Python with Selenium and unittest. I have four tests in one class since they're all related unit tests. How do I have it skip the next test(s) if the preceding test fails? I've read up on all the documentation for unittest's skip methods, but none of it is exactly what I need. Is there a way to tell it to exit the class?

Here is a jist of what my code currently looks like:

def test_dealer_search_id_contains(self):
    try:
        LoginPage.login(self, ULV.AA_USERNAME, ULV.AA_PASSWORD)
    except TimeoutException:
        add_test_result(3, 5, "This test failed attempting to login with the user's credentials.")
        add_test_result(4, 2, 'This test is blocked due to failure of a preceding test.')
        self.fail(msg='Dealer: Search - Test failure.')

def test_dealer_search_id_exact(self):
    code for this test here

How do I have it skip the remaining test or exit the class all together?

I appreciate any help that can be provided.

EDIT: I am using PyTest as my runner and I do have additional tests after these that will need to continue to run, just not in this class or .py file.

EDIT: So the tests in this case all depend on the prior test passing, otherwise it would fail for the same defect as the above test. However, my main dilemma is I'm not sure how to tell it to skip those tests and continue with the rest of the tests that PyTest has picked up. I looked at PyTest's skip handling and it's pretty much identical to unittest. I'm trying to avoid coding in a logic check for test failures if it's unnecessary, but it appears that may be the only solution.

Jay
  • 116
  • 1
  • 10
  • Possible duplicate of [Is there a way to prevent a SystemExit exception raised from sys.exit() from being caught?](http://stackoverflow.com/questions/173278/is-there-a-way-to-prevent-a-systemexit-exception-raised-from-sys-exit-from-bei) – Tadhg McDonald-Jensen May 24 '16 at 20:06
  • 1
    The difference between that question posted and mine is I do have tests after this that I will want to continue to run, just not in this class. I'm using PyTest as my test runner, I only use unittest to manage the tests in the code for my POM. – Jay May 24 '16 at 20:19
  • I hadn't realized that there were other tests that needed to run other then the class in question, In that case this is definitely something to do with "these other tests depend on this one succeeding" – Tadhg McDonald-Jensen May 25 '16 at 02:54
  • @JayWright please take a look at my answer http://stackoverflow.com/a/37493283/4988742 it should solve your question elegantly. Please let me know if that worked. – Dmitry Tokarev Jun 05 '16 at 06:52

4 Answers4

3

I think i found what will solve this problem. It's called incremental testing in py.test framework: https://pytest.org/latest/example/simple.html#incremental-testing-test-steps

I tested locally - worked like a charm!

Used same conftest.py as in the py.test docs. Slightly different test file:

# content of test_incremental.py

import pytest

@pytest.mark.incremental
class TestUserHandling:
    def test_login(self):
        assert 1
    def test_modification(self):
        assert 0
    def test_deletion(self):
        assert 1
    def test_foo(self):
        assert 1


class TestAnotherStuff:
    def test_normal(self):
        assert 1

mac:[incremental_testing]: py.test -v test_incremental.py 
============================================================================== test session starts ===============================================================================
platform darwin -- Python 2.7.11, pytest-2.9.1, py-1.4.31, pluggy-0.3.1 -- /Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python
cachedir: .cache
rootdir: /Users/dmitry.tokarev/Repos/playground/incremental_testing, inifile: 
plugins: hidecaptured-0.1.2, instafail-0.3.0
collected 5 items 

test_incremental.py::TestUserHandling::test_login PASSED
test_incremental.py::TestUserHandling::test_modification FAILED
test_incremental.py::TestUserHandling::test_deletion xfail
test_incremental.py::TestUserHandling::test_foo xfail
test_incremental.py::TestAnotherStuff::test_normal PASSED

==================================================================================== FAILURES ====================================================================================
_______________________________________________________________________ TestUserHandling.test_modification _______________________________________________________________________

self = <test_incremental.TestUserHandling instance at 0x103b7a5a8>

    def test_modification(self):
>       assert 0
E       assert 0

test_incremental.py:10: AssertionError
================================================================= 1 failed, 2 passed, 2 xfailed in 0.02 seconds ==================================================================
Dmitry Tokarev
  • 1,851
  • 15
  • 29
0

Why don't you create dependencies in you tests for example: 

def test_mainTest(): ...

@dependsOn(test_mainTest)
def test_dependantTest(): ...

def test_non_dependant_test(): ...
Boris Stoyanov
  • 96
  • 12
  • I will try this next. Thank you, Boris. I will let you know the outcome. Also, in my above code where I have the function add_test_result, would I just need to set those to blocked until the dependency is met, then have it overwrite it in the next test? That would make my test management tool go a little nuts with the notifications. – Jay May 24 '16 at 20:59
0

Use flags -x or --exitfirst to exit instantly on first error or failed test.

Or --maxfail=num to exit after first num failures or error.

http://pytest.org/latest/usage.html#stopping-after-the-first-or-n-failures

Dmitry Tokarev
  • 1,851
  • 15
  • 29
  • That will not work. I only need it to skip the tests in that class. That would abort my testing and the remaining tests after that class would not be tested. – Jay May 27 '16 at 16:08
  • @JayWright i see now, it wasn't clear to me that you only needed to skip subsequent tests in that class only and run tests from other classes. May be then this will help: http://stackoverflow.com/a/12579625/4988742 Also you can mark each test with the @pytest.mark.skipif(reason=your_condition) see more in https://pytest.org/latest/skipping.html#id1 need to make previous test results available as a condition for the latter tests skipif markers. – Dmitry Tokarev May 27 '16 at 19:38
0

So what I ended up having to do due to the complex situation of my tests setup is I surrounded the three tests in the Class that would fail if the first test failed since everything is identical except for the search data.

For example:

class TestDealerSearchID(BaseSetup, LoginPage, DealerPage):
    testSuiteResult = 0
    def test_dealer_search_id_contains(self):
        try:
            LoginPage.login(self, ULV.AA_USERNAME, ULV.AA_PASSWORD)
        except (Exception, TimeoutException):
            add_test_result(3, 5, "This test failed attempting to login with the user's credentials.")
            add_test_result(4, 2, 'This test is blocked due to the failure of a preceding test in the suite.')
            add_test_result(5, 2, 'This test is blocked due to the failure of a preceding test in the suite.')
            add_test_result(6, 2, 'This test is blocked due to the failure of a preceding test in the suite.')
            TestDealerSearchID.testSuiteResult = 1
            self.fail(msg='AAv4 - Dealer: Search - Test failure.')

    def test_dealer_search_id_exact(self):
        if TestDealerSearchID.testSuiteResult == 1:
            TestDealerSearchID.skipTest(self, 'This test is blocked due to the failure of a preceding test in the suite.')
        else:
            pass

        REST OF CODE HERE AND REMAINING TWO TESTS WITH SAME IF/ELSE STATEMENT.

This was the best solution I could come up with and it works exactly as needed. If anyone knows of a better way, please don't hesistate to comment or post an answer.

Thank you,
Jay Wright

Jay
  • 116
  • 1
  • 10