How to use xcodebuild with -only-testing and -skip-testing flag?

Question

I've noticed that there are two options in xcodebuild's man page.

-only-testing:TEST-IDENTIFIER

constrains testing by specifying tests to include, and excluding other tests

-skip-testing:TEST-IDENTIFIER

constrains testing by specifying tests to exclude, but including other tests

What I try:

xcodebuild -workspace MyWorkSpace.xcworkspace / 
-sdk iphonesimulator / 
-destination id=7F52F302-C6AF-4215-B269-39A6F9913D5B / 
-scheme SCHEME-iOS / 
test -only-testing:???

What is TEST-IDENTIFIER mean ?

score 31 · Accepted Answer · edited May 15 '18 at 12:27

31

Like what Marcio said, it's a path like string.

For example, say you have a scheme named MyScheme, a test target MyUITests, and testing class LoginTest, then testing method testUserLogin, to run only the method, you can run

xcodebuild -workspace Envoy.xcworkspace \
    -scheme MyScheme \
    -sdk iphonesimulator \
    -destination 'platform=iOS Simulator,name=iPad Air 2,OS=10.1'
    '-only-testing:MyUITests/LoginTest/testUserLogin' test

Likewise, if you want to run all tests under LoginTest, here you run

xcodebuild -workspace Envoy.xcworkspace \
    -scheme MyScheme \
    -sdk iphonesimulator \
    -destination 'platform=iOS Simulator,name=iPad Air 2,OS=10.1'
    '-only-testing:MyUITests/LoginTest' test

edited May 15 '18 at 12:27

Kjuly

34,476
22
104
118

answered Nov 12 '16 at 00:14

Fang-Pen Lin

13,420
15
66
96

is it some way to use regular expression for this parameter. Something like -only-testing:MyUItests/LoginTest[1-2]??? – emoleumassi Dec 07 '16 at 09:26
5

I just updated this answer, it was partially wrong and led me to waste about 30 minutes. In the `-only-testing` argument you specify the target not the scheme. – Liron Yahdav Jan 07 '17 at 00:45
Updated the answer. Instead of `'-only-testing:MyUITests/LoginTest/testUserLogin()'`, it suppose to be `'-only-testing:MyUITests/LoginTest/testUserLogin` w/o `()` for the test case. – Kjuly May 15 '18 at 12:26

N.K. · Answer 2 · 2017-01-19T07:07:19.320

14

You can check the video https://developer.apple.com/videos/play/wwdc2016/409/

I used it like this:

-only-testing:UITests/TC_TextArea/test1

for my tests tree. Works fine

Full command looks as follows:

command = 'xcodebuild test 
-workspace ' + pathToProjectWorkspaceFolder + '/project.xcworkspace 
-scheme yourApp.app 
-destination "platform=iOS,name=' + deviceName + '" 
-only-testing:UITests/TC_TextArea/test1'

edited Jan 19 '17 at 07:07

answered Sep 15 '16 at 10:08

N.K.

301
1
8

3

TEST-IDENTIFIER is how Xcode maps your tests. Check your testing tree--the 5th tab on your left. In order to work you always have to start from the root, i.e, `xcodebuild test skip-testing:MyTestsBundleName/MyClassName/MyTestCaseName` – Marcio Klepacz Oct 27 '16 at 18:35

score 6 · Answer 3 · answered Mar 12 '19 at 14:27

6

In case you need to include several tests:

xcodebuild -project Some.xcodeproj \
-scheme AllTests -only-testing:PersistenceTests -only-testing:FoundationTests test

Documentation:

An xcodebuild command can combine multiple constraint options, but -only-testing: has precedence over -skip-testing:.

answered Mar 12 '19 at 14:27

Vlad

6,402
1
60
74

Having trouble finding a reference to this, can you source this? Cheers! – GrayedFox Sep 28 '21 at 09:43
1

`man xcodebuild` :) – Vlad Sep 28 '21 at 15:32

Sameer Technomark · Answer 4 · 2021-07-05T22:26:48.240

2

xcodebuild \
 -workspace MyApp.xcworkspace \
 -scheme Automation \
 -destination 'platform=ios,name=My Real iPhone' \
 -only-testing:MyTestDirectory/TestClass/testMethodName \
 test-without-building

No need for single quotation marks around only-testing
No need for the sub-directory names as they are ignored e.g. MyTestDirectory/E2E/

edited Jul 05 '21 at 22:26

answered Jul 20 '18 at 04:25

Sameer Technomark

1,399
1
10
12

score 1 · Answer 5 · answered Oct 09 '18 at 15:11

To test an application you need to go with the two steps:

build the application

    xcodebuild build-for-testing \
        -workspace "<your_xcworkspace>" \
        -scheme "<your_scheme>" \
        -destination "platform=iOS Simulator,name=<your_simulator>,OS=<simdevice_os_version>" \
        -derivedDataPath "All"

test it without building

    xcodebuild test-without-building \
        -xctestrun "All/Build/Products/<your_scheme>_iphonesimulator<simdevice_os_version>-x86_64.xctestrun" \
        -destination "platform=iOS Simulator,name=<your_simulator>,OS=<simdevice_os_version>" '-only-testing:<your_test_bundle_to_run>'  \
        -derivedDataPath 'build/reports/<your_test_bundle_to_run>'

Here, <your_test_bundle_to_run> indicates the TEST-IDENTIFIER that means
How many categories or how many test cases under a category you want to run, that should be included under a test bundle[<your_test_bundle_to_run>]

How to use xcodebuild with -only-testing and -skip-testing flag?

5 Answers5

Linked