3

I am looking at a way to add a new "mode" in the Pysys Baserunner.

In particular I would like to add a validate mode, that just re-run the validation portion. Useful when you are writing your testcase and trying to tune the validation condition so to fit the current output without having to re-reun the complete testcase.

What is the best way to do this without having to change the original class?

Saurav Sahu
  • 13,038
  • 6
  • 64
  • 79
vito imburgia
  • 90
  • 5

2 Answers2

1

This requires support from the framework unfortunately. The issue is that the BaseRunner class will always automatically purge the output directory, and there is no hook into the framework to allow you to avoid this. You can for instance move the output subdirectory manually you want to re-run the validation over to say 'repeat' (same directory level), and then use;

from pysys.constants import *
from pysys.basetest import BaseTest

class PySysTest(BaseTest):
    def execute(self):
        if self.mode=='repeat': pass

    def validate(self):
        if self.mode=='repeat':
            self.output=os.path.join(self.descriptor.output, 'repeat')

where I have ommitted the implementations of the execute and validate. You would need to add the mode into the descriptor for the test

  <classification>
    <groups>
      <group></group>
    </groups>
    <modes>
      <mode>repeat</mode>
    </modes>
  </classification>

and run using "pysys.py run -mrepeat". This would help with the debugging if your execute takes a long time, but probably not want you want out-of-the-box i.e. a top level option to the runner to just perform validation over a previously run test. I'll add a feature request for this.

moraygrieve
  • 466
  • 4
  • 11
  • I can see a new version of Pysys is out (1.1.0) with a related functionality:'It is now possible to register cleanup functions in the BaseTest to negate the need to override the cleanup() action where a call to BaseTest.cleanup(self) must specifically be made. See the epydoc for the addCleanupFunction() in the ProcessUser module'. Wold this make possible to create a new mode that doesn't run cleanup? – vito imburgia Feb 02 '16 at 09:51
  • Not really unfortunately. The reasoning behind this change was that should a test override the cleanup() method before, it would have to explicitly call BaseTest.cleanup(self) in the overridden method ... which is clumsy. This new approach allows adding behaviour through registering cleanup methods to be called, rather than using inheritance where PySys effectively looses control of stopping background processes. It still does the file purge. We were abit tight for time so I didn't implement the change request for this release, but will aim to do so for the next. – moraygrieve Feb 03 '16 at 11:42
1

Since the original discussion, a --validateOnly command line option was added to PySys (in v1.1.1) which does pretty much what you suggest - it skips the execute method and just runs validate.

Assumes you aren't running with --purge (which I imagine is a safe assumption for this use case), and that you don't have validation commands that try to read zero-byte files from the output dir (which always get deleted even if --purge is not specified). However assuming those conditions are met, your (non-empty) output files will still be there after you've completed the first run of the test and you can re-run just validation using the --validateOnly command.

To get this feature you can install the latest PySys version (1.4.0) - see https://pypi.org/project/PySys/

Ben Spiller
  • 477
  • 4
  • 12