How to execute tests that match a regular expression only?

Question

In sbt 0.10.1, I frequently use test-only to narrow down the number of my tests.

sbt> test-only com.example.MySpec

However, I want to narrow down such that I only run tests whose name/description matches a regular expression. Is there some syntax to achieve something like this?

sbt> test-only .*someRexExp.*

In case there is not (I doubt it, if it does not work), this should be a feature request in SBT's bugtracker, not here. — Raphael, Aug 09 '11 at 16:41
I can only suggest to edit your project file, because that accepts regex. — Enrico Susatyo, Aug 10 '11 at 11:36

score 51 · Accepted Answer · edited Jun 05 '15 at 07:04

Full regular expressions are not supported by testOnly. Wildcards are supported, however.

sbt> testOnly com.example.*Spec

Only the asterisk * is interpreted specially here and not the periods. This will select all tests beginning with com.example. and ending with Spec.

Or just all test Specs:

sbt> testOnly *Spec

testOnly and other testing information is documented here: http://www.scala-sbt.org/release/docs/Detailed-Topics/Testing

score 15 · Answer 2 · answered May 25 '15 at 13:39

You can match on test cases by their name (instead of or in addition to suite class names) by using framework-specific runner arguments. ScalaTest supports a substring match with the -z argument:

> testOnly -- -z insert
> testOnly *TreeSpec -- -z insert

This runs all tests with "insert" in their name, then only the matching cases within suites ending in TreeSpec, as you would intuit. You can also use -n TagName and -l TagName to include or exclude, respectively, tags from ScalaTest's tagging support, and -t to match an exact test name.

Specs2 supports full Java regular expressions with an -ex argument:

> testOnly -- -ex ".*someRexExp.*"

-include and -exclude support Spec2's tagging features.

See the inline links for full lists of arguments that the runners support. These appear to only work with the testOnly sbt command and not test.

It's also worth mentioning that, if you want this without entering sbt first, you have to properly quote the args after sbt: `sbt "testOnly -- -z insert"` — catalin.costache, Jan 13 '16 at 14:58
Not sure when things changed but for some reason with specs2 I now have to use just `ex` with no dash. — James Ward, Dec 04 '18 at 23:59

How to execute tests that match a regular expression only?

2 Answers2

Linked