31

I am writing a Pythonic tool which validates the correctness of a certain system. Each validation is written as a Python unittest, and the report looks like:

test_exclude_list_not_empty (__main__.TestRepoLists)
Assert the the exclude list is not empty ... ok
test_include_list_not_empty (__main__.TestRepoLists)
Assert the the include list is not empty ... ok
test_repo_list_not_empty (__main__.TestRepoLists)
Assert the the repo list is not empty ... ok

In my opinion, this format is hard to read, especially for non-Pythonists. Is there any report generator that can generate a report in a nice, tabular form, e.g.:

+----------------------------------------------------------------+-----------+
| Test                                                           |  Status   |
+----------------------------------------------------------------+-----------+
| Assert the the exclude list is not empty                       |  OK       |
| Assert the the include list is not empty                       |  OK       |
| Assert the the repo list is not empty                          |  OK       |
| All the items in the include list should be in the repo list   |  OK       |
+----------------------------------------------------------------+-----------+

Clarification The test suite runs on a remote terminal, so I prefer command line reporting tools.

alecxe
  • 462,703
  • 120
  • 1,088
  • 1,195
Adam Matan
  • 128,757
  • 147
  • 397
  • 562

3 Answers3

34

This is not exactly what you are asking, but there are several options for having a readable test output there:

  • HTMLTestRunner generates easy to use HTML test reports in a tabular form. Here's a sample report.
  • nose-html-output plugin for nose test runner
  • unittest-xml-reporting - PyUnit-based test runner with JUnit like XML reporting
  • nose with --with-xunit option will produce junit xml style reports that are easy to read and convert

Also see:

If you want to see test results in a tabular form in the console anyway, I think that a good idea would be to write your own nose plugin or test runner based on unittest.TestProgram as it was done in HTMLTestRunner.

Hope that helps.

Community
  • 1
  • 1
alecxe
  • 462,703
  • 120
  • 1,088
  • 1,195
  • 1
    Well, sure, you can install any of these xml/html producing tools on the server-side and use on a remote terminal shell. But, in this case you'll need to download reports first to see them in the browser, of course. Or, alternatively, you can use a text-based web browser like [lynx](http://lynx.browser.org/) to see reports right in the console. – alecxe Jun 18 '13 at 09:19
  • That would overcomplicate the matters. I think that a simple tabulation within the shell will do. – Adam Matan Jun 18 '13 at 12:15
  • Agreed, then consider going with custom runner or a nose plugin. I don't think there is anything written out there specifically for this particular problem. – alecxe Jun 18 '13 at 12:21
  • Thanks. I will try to create a test class of my own. Just wanted to make sure I'm not reinventing the wheel. – Adam Matan Jun 18 '13 at 12:31
  • You know that an answer is not worthy to be read nor posted if it starts with "This is not exactly what you are asking" – User9123 Apr 06 '21 at 20:30
  • @User9123 hey, it was worth a worthy comment. Achievement unlocked. – alecxe Apr 07 '21 at 12:56
11

I would like to add my information as a comment into alecxe's answer, but I do not have enough reputation for that.

In case of someone still looking for an answer, I forked HTMLTestRunner into a simple TestRunner, which has a tabular, colored, terminal-friendly output. This is a sample of its output:

Example

The source code is at https://gist.github.com/viniciusd/73e6eccd39dea5e714b1464e3c47e067

I shall rewrite it from scratch soon but keeping the output format.

Vinicius Dantas
  • 165
  • 1
  • 10
3

Take a look at Twisted's Trial.

By default, it uses the TreeReporter test runner, which looks like:

Trial's reporting

It has the following:

  • It's a command line report, just run:

    trial test_name.py

  • Colored output: red for failure, green for success

  • The report uses a tree like structure. It displays the tests under the TestCases they belong, allowing you to quickly traverse the results to find a specific test. (Although it provides a couple of more reports).

  • It also includes a test library, derived from Python's unittest.TestCase. You can use this library by subclassing twisted.trial.unittest.TestCase. This provides a few more assertion methods.

  • It includes the option to generate statement coverage for your tests.

Carlos
  • 2,222
  • 12
  • 21