3

I have being taking a look at some modules of perl, and I have seen it is kind of a standard to have a lot of tests under a folder t whith the extension .t.

Those scripts can be executed directly with perl <test>.t.

I want to do a little framework to test one application. I want to have many simple tests. And then a main script which calls this tests to validate everything at once, but still being able to run the little tests individually without having to disable manually the other tests in the main script.

My question is how should I do this calling? As suggested in How do I run a Perl script from within a Perl script? this interprocess communication should be avoided and instead use modules but that would make not possible to run the tests directly.

My guess was to do something as follows. In an example where I need to give some parameters in a selenium script This is the folder structure:

├── main_script.pl
└── t
    ├── 00-Test_1.t
    ├── 01-Test_2.t
    ├── 02-Test_3.t
    └── 03-Test_4.t

And I would call them like this:

system($^X, "./t/00-Test_1.t", "-b $browser", "-s $server", "-d $debug");
my $results = capture($^X, "./t/00-Test_1.t", "-b $browser", "-s $server", "-d $debug";
$log->info("Result is $results");

And each test would have something like this:

my ($browser, $server, $debug);
GetOptions(
    'debug|d'                   => \$debug,
    'debug|d'                   => \$trace,
    'trace|t'                   => \$browser,
    'server|s=s'                => \$server,
) or die "[ERROR] Invalid options passed to $0\n";

Is this a correct approach or is there something more appropiate for doing this using modules?

simbabque
  • 53,749
  • 8
  • 73
  • 136
nck
  • 1,673
  • 16
  • 40
  • 2
    You already have the `prove` utility to run several tests at once. If you want to control their behaviour, you might want to look at one of the existing frameworks like Test::Class or Test::Class::Moose to write your tests in. If you want to ship to CPAN that'll pull in extra dependencies, but for proprietary stuff it's great. – simbabque Sep 24 '20 at 12:24
  • @simbabque I have been taking a look at it. But I think by using you can not get the result from the main script, for example to make a report from it. – nck Sep 24 '20 at 13:21

2 Answers2

6

You already have "a main script which calls this tests to validate everything at once, but still being able to run the little tests individually" as part of the standard Perl distribution. It's called prove.

Running prove with no parameters will run all test scripts matching t/*.t (so you want the tests in a t/ subdirectory when you run prove, not in the same directory), or you can use prove t/00-Test_1.t to run a single test script.

Note that prove expects the test scripts to emit output in TAP ("Test Anything Protocol") format, which is usually accomplished by using Test::More or other Test::* modules in the test scripts.

If you follow the convention of putting your application-specific modules in a lib/ subdirectory, then running prove -l will add them to @INC to make them available when running the test scripts. i.e.,

Root directory (run `prove` from here)
├── lib
|   └── MyProject
|       └── MyModule.pm
└── t
    ├── 00-Test_1.t
    ├── 01-Test_2.t
    ├── 02-Test_3.t
    └── 03-Test_4.t
Dave Sherohman
  • 45,363
  • 14
  • 64
  • 102
  • But does this allow to do post processing on the results of the test?. For example if I want to generate a personalized report in the main script given the results. – nck Sep 24 '20 at 13:24
  • 2
    @nck how do you mean? You can use various Tap::Harness:: modules that will work with `prove`, or make your own. For example, there's https://metacpan.org/pod/TAP::Harness::JUnit. You will see that `prove` and tests written in Perl are very widely used in a lot of open source software where the main program isn't written in Perl. Good examples are Linux core utilities written in C that all have tests in Perl. – simbabque Sep 24 '20 at 15:00
  • @nck Can always run `prove` out of a script that you'd write specifically for processing its output and generating the report. (Or out of whatever other "main" program you have.) Make sure to browse the docs of related components, and of `prove` itself, for a myriad of options. – zdim Sep 28 '20 at 06:26
1

Although the answer of @Dave Sherhman is good enough, I will post how I made it work for my project together with the suggestion of @simbabque. As it took me a while to understand how it works and maybe it helps another one save some time:

My *.t script are now as the following script. Where I specify the number of tests that are inside it, and also I can give some input parameters:

use strict;
use warnings;

use Test::More tests => 2;

use Getopt::Long qw(GetOptions);
use Drivers::Driver;
use Data::Dumper;

my ($browser, $server, $debug, $trace, $user, $password);
GetOptions(
    'debug|d'                   => \$debug,
    'trace|t'                   => \$trace,
    'user|u:s'                  => \$user,
    'password|p:s'              => \$password,
    'browser|b=s'               => \$browser,
    'server|s=s'                => \$server,
) or die "[ERROR] Invalid options passed to $0\n";

ok(do_some_test($browser) == 1);
ok(do_another_test($user, $password) == 1);

done_testing;

Then my main script calls the scripts I want as follows:

use strict;
use utf8;
use open ':std', ':encoding(UTF-8)';

use TAP::Harness;
use TAP::Formatter::HTML;

my $fmt = TAP::Formatter::HTML->new();
$fmt->output_file( 'foo.html' );

my $test_args = {
  '00_test'    => ['--server', "$server", $debug?'-d':undef, $trace?'-t':undef, '--browser', "firefox"],
  '01_test'    => ['--server', "$server", $debug?'-d':undef, $trace?'-t':undef, '--browser', "chrome"],
};
my $test_output;
my $harness = TAP::Harness->new( {
        test_args => $test_args,
        formatter => $fmt,
        merge => 1
    } );
    
my $result = $harness->runtests( 
    [ 't/00-test.t',    '00_test' ],
    [ 't/01-test.t',    '01_test' ],
);


print "Passed: ". scalar $result->passed . "\n";
print "Failed: ". scalar $result->failed . "\n";

generate_report($result);

This won't work out of the box as I just deleted some lines from my project but gives an idea on how is it able to run the other files and get the results to generate a report, while still being able to run each project individually and also being able to send common parameters to the different tests from the main script.

nck
  • 1,673
  • 16
  • 40