Python unittests in Jenkins?

Question

How do you get Jenkins to execute python unittest cases? Is it possible to JUnit style XML output from the builtin unittest package?

All of the answers presume you want to initiate the test cases from the command-line. But if you want to run the tests programmatically, try this: `import nose ; nose.runmodule() # aka nose.run(defaultTest=__name__)` — MarkHu, Sep 03 '15 at 05:48
IMHO the simple 'py.test --junitxml results.xml test.py' suggestion answers the question best. 'yum install pytest' to get py.test installed. Then you can run any unittest python script and get jUnit xml results — gaoithe, May 06 '16 at 11:48
@gaoithe that answers the jenkins part, but doesn't fulfill the requirement to use the builtin unittest module. In that project it was a given requirement. — erikbstack, May 10 '16 at 11:14
@erikb85 When I say "run any unittest python script" I mean a script which uses the unittest module. — gaoithe, May 10 '16 at 11:48

dnozay · Accepted Answer · 2014-08-14T02:55:26.780

sample tests:

tests.py:

# tests.py

import random
try:
    import unittest2 as unittest
except ImportError:
    import unittest

class SimpleTest(unittest.TestCase):
    @unittest.skip("demonstrating skipping")
    def test_skipped(self):
        self.fail("shouldn't happen")

    def test_pass(self):
        self.assertEqual(10, 7 + 3)

    def test_fail(self):
        self.assertEqual(11, 7 + 3)

JUnit with pytest

run the tests with:

py.test --junitxml results.xml tests.py

results.xml:

<?xml version="1.0" encoding="utf-8"?>
<testsuite errors="0" failures="1" name="pytest" skips="1" tests="2" time="0.097">
    <testcase classname="tests.SimpleTest" name="test_fail" time="0.000301837921143">
        <failure message="test failure">self = &lt;tests.SimpleTest testMethod=test_fail&gt;

    def test_fail(self):
&gt;       self.assertEqual(11, 7 + 3)
E       AssertionError: 11 != 10

tests.py:16: AssertionError</failure>
    </testcase>
    <testcase classname="tests.SimpleTest" name="test_pass" time="0.000109910964966"/>
    <testcase classname="tests.SimpleTest" name="test_skipped" time="0.000164031982422">
        <skipped message="demonstrating skipping" type="pytest.skip">/home/damien/test-env/lib/python2.6/site-packages/_pytest/unittest.py:119: Skipped: demonstrating skipping</skipped>
    </testcase>
</testsuite>

JUnit with nose

run the tests with:

nosetests --with-xunit

nosetests.xml:

<?xml version="1.0" encoding="UTF-8"?>
<testsuite name="nosetests" tests="3" errors="0" failures="1" skip="1">
    <testcase classname="tests.SimpleTest" name="test_fail" time="0.000">
        <failure type="exceptions.AssertionError" message="11 != 10">
            <![CDATA[Traceback (most recent call last):
File "/opt/python-2.6.1/lib/python2.6/site-packages/unittest2-0.5.1-py2.6.egg/unittest2/case.py", line 340, in run
testMethod()
File "/home/damien/tests.py", line 16, in test_fail
self.assertEqual(11, 7 + 3)
File "/opt/python-2.6.1/lib/python2.6/site-packages/unittest2-0.5.1-py2.6.egg/unittest2/case.py", line 521, in assertEqual
assertion_func(first, second, msg=msg)
File "/opt/python-2.6.1/lib/python2.6/site-packages/unittest2-0.5.1-py2.6.egg/unittest2/case.py", line 514, in _baseAssertEqual
raise self.failureException(msg)
AssertionError: 11 != 10
]]>
        </failure>
    </testcase>
    <testcase classname="tests.SimpleTest" name="test_pass" time="0.000"></testcase>
    <testcase classname="tests.SimpleTest" name="test_skipped" time="0.000">
        <skipped type="nose.plugins.skip.SkipTest" message="demonstrating skipping">
            <![CDATA[SkipTest: demonstrating skipping
]]>
        </skipped>
    </testcase>
</testsuite>

JUnit with nose2

You would need to use the nose2.plugins.junitxml plugin. You can configure nose2 with a config file like you would normally do, or with the --plugin command-line option.

run the tests with:

nose2 --plugin nose2.plugins.junitxml --junit-xml tests

nose2-junit.xml:

<testsuite errors="0" failures="1" name="nose2-junit" skips="1" tests="3" time="0.001">
  <testcase classname="tests.SimpleTest" name="test_fail" time="0.000126">
    <failure message="test failure">Traceback (most recent call last):
  File "/Users/damien/Work/test2/tests.py", line 18, in test_fail
    self.assertEqual(11, 7 + 3)
AssertionError: 11 != 10
</failure>
  </testcase>
  <testcase classname="tests.SimpleTest" name="test_pass" time="0.000095" />
  <testcase classname="tests.SimpleTest" name="test_skipped" time="0.000058">
    <skipped />
  </testcase>
</testsuite>

JUnit with unittest-xml-reporting

Append the following to tests.py

if __name__ == '__main__':
    import xmlrunner
    unittest.main(testRunner=xmlrunner.XMLTestRunner(output='test-reports'))

run the tests with:

python tests.py

test-reports/TEST-SimpleTest-20131001140629.xml:

<?xml version="1.0" ?>
<testsuite errors="1" failures="0" name="SimpleTest-20131001140629" tests="3" time="0.000">
    <testcase classname="SimpleTest" name="test_pass" time="0.000"/>
    <testcase classname="SimpleTest" name="test_fail" time="0.000">
        <error message="11 != 10" type="AssertionError">
<![CDATA[Traceback (most recent call last):
  File "tests.py", line 16, in test_fail
    self.assertEqual(11, 7 + 3)
AssertionError: 11 != 10
]]>     </error>
    </testcase>
    <testcase classname="SimpleTest" name="test_skipped" time="0.000">
        <skipped message="demonstrating skipping" type="skip"/>
    </testcase>
    <system-out>
<![CDATA[]]>    </system-out>
    <system-err>
<![CDATA[]]>    </system-err>
</testsuite>

+1 for the simple 'py.test --junitxml results.xml test.py' suggestion. 'yum install pytest' to get py.test installed. Then you can run any unittest python script and get jUnit xml results. — gaoithe, May 06 '16 at 11:47
If you want to use _unittest-xml-reporting_ and benefit from the [Test Discovery feature](https://docs.python.org/3/library/unittest.html#unittest-test-discovery), you can put `unittest.main(module=None, testRunner=xmlrunner.XMLTestRunner(output='test-reports'))`. — Rosberg Linhares, Mar 04 '17 at 02:23
@RosbergLinhares why do you need `module=None` to use Test Discovery? It works exactly as described in the answer `unittest.main(testRunner=xmlrunner.XMLTestRunner(output='test-reports'))`. — acm, Aug 09 '17 at 12:48
@RosbergLinhares, during test discovery, the modules are only imported but not executed. So, how is any of those solution supposed to work with discovery? I just tried it out, none of it works. Or am I missing something? — Konstantin, Oct 23 '18 at 06:20
@Konstantin, I also had to include module=None for it to run correctly. — Brent Fisher, Nov 29 '20 at 16:13
Even for me adding module=None is not working . It is still not able to discover tests am I missing something? — Gaurav Parek, Sep 03 '21 at 15:06

score 20 · Answer 2 · edited Jun 14 '13 at 11:09

20

I would second using nose. Basic XML reporting is now built in. Just use the --with-xunit command line option and it will produce a nosetests.xml file. For example:

nosetests --with-xunit

Then add a "Publish JUnit test result report" post build action, and fill in the "Test report XMLs" field with nosetests.xml (assuming that you ran nosetests in $WORKSPACE).

edited Jun 14 '13 at 11:09

Nakilon

34,866
14
107
142

answered Jul 13 '12 at 03:12

Joshua D. Boyd

4,808
3
29
44

score 12 · Answer 3 · answered Jun 28 '12 at 23:24

You can install the unittest-xml-reporting package to add a test runner that generates XML to the built-in unittest.

We use pytest, which has XML output built in (it's a command line option).

Either way, executing the unit tests can be done by running a shell command.

score 6 · Answer 4 · answered Aug 15 '17 at 09:23

6

python -m pytest --junit-xml=pytest_unit.xml source_directory/test/unit || true # tests may fail

Run this as shell from jenkins , you can get the report in pytest_unit.xml as artifact.

answered Aug 15 '17 at 09:23

Rajib Mitra

340
4
15

score 4 · Answer 5 · answered Jun 28 '12 at 09:45

4

I used nosetests. There are addons to output the XML for Jenkins

answered Jun 28 '12 at 09:45

John La Rooy

295,403
53
369
502

score 3 · Answer 6 · answered Jun 28 '12 at 09:45

3

When using buildout we use collective.xmltestreport to produce JUnit-style XML output, perhaps it's source code or the module itself could be of help.

answered Jun 28 '12 at 09:45

Martijn Pieters

1,048,767
296
4,058
3,343

Python unittests in Jenkins?

6 Answers6

sample tests:

JUnit with pytest

JUnit with nose

JUnit with nose2

JUnit with unittest-xml-reporting

Linked