I'm trying to get PHPUnit to stop on failure and to order tests by defect, in order to speed up my dev experience (no pun intended).
Here is what I try: I use "--cache-result --order-by=defects --stop-on-defect" on the command line, and additionally I use a phpunit.xml.dist with these flags:
<?xml version="1.0" encoding="UTF-8"?>
<!-- http://phpunit.de/manual/4.1/en/appendixes.configuration.html -->
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="http://schema.phpunit.de/4.1/phpunit.xsd"
backupGlobals="false"
colors="true"
bootstrap="app/autoload.php"
cacheResult = "true"
convertErrorsToExceptions = "true"
convertNoticesToExceptions = "true"
convertWarningsToExceptions = "true"
processIsolation = "false"
stopOnFailure = "true"
executionOrder = "defects"
beStrictAboutCoversAnnotation = "true"
beStrictAboutOutputDuringTests = "true"
enforceTimeLimit = "false"
>
[...]
But still my (second) test run looks like this:
[me@horus server]$ ./vendor/bin/phpunit --cache-result --order-by=defects --stop-on-defect
Runtime: PHP 7.3.10
Configuration: /home/.../phpunit.xml.dist
PHPUnit 7.5.16 by Sebastian Bergmann and contributors.
S..S........................................................... 63 / 619 ( 10%)
...SSS......................................................... 126 / 619 ( 20%)
............................................................EEE 189 / 619 ( 30%)
EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE.....................F
Time: 8.41 minutes, Memory: 483.04 MB
There were 43 errors:
[...]
I can tell it's really using my xml because when I mess up the format, the call to PHPUnit fails. But why are the tests not sorted to start with the first failure ? And why is there a whole line of failures, instead of just a single E ? Shouldn't it stop right after the first E ?
