59

Some background:

I have iOS application with a target configured to run unitTests. And I am running build automation tool jenkins on my MacBook which automatically builds this application and run all tests (using command line xcodebuild tool).

Everything worked fine with Xcode 4. This build automation tool was running under different user and was running all these tests.

I switched to Xcode 5 recently and it started to fail, because it can't launch Simulator.

The problem

I have a scheme UnitTests which is configured to run tests (logic tests). A I run these test using one of two methods:

  • Command U in Xcode

  • Or command line "/usr/bin/xcodebuild -scheme UnitTests -sdk iphonesimulator -configuration Release clean build TEST_AFTER_BUILD=YES "

In both cases, it tries to start simulator. However, per my understand it doesn't need it. Anyway it runs on top x86 and it doesn't look like any apps are installed on Simulator.

Is there a way to get rid of this pesky simulator start (because it breaks my build automation)?

Update 1

Seems to find very similar question, but can't get it working: Run logic tests in Xcode 4 without launching the simulator

Update 2

I found VERY relevant and interesting question/answer: Apple CI / Xcode Service and Jenkins

Community
  • 1
  • 1
Victor Ronin
  • 22,758
  • 18
  • 92
  • 184
  • This question might help too. The steps should be pretty much the same for an iOS app. http://stackoverflow.com/questions/19219706/xcode-5-unit-testing-starts-my-app – Craig Siemens Nov 08 '13 at 20:55
  • Why don't you want to run the simulator? – quellish Nov 11 '13 at 19:53
  • @quellish: Because starting from Xcode 5, it's a problem if you are running it under different user (which doesn't UI session) – Victor Ronin Nov 11 '13 at 22:06
  • 1
    @VictorRonin , that's true on earlier versions of Xcode as well. Xcode 5, however, does support running in a headless mode for just these kinds of scenarios. Xcode Server works in this way. On earlier versions of Xcode it was possible to run the simulator as a daemon user but difficult to set up correctly. To use iPhone specific functionality like UIKit in your tests, you DO need the simulator. – quellish Nov 15 '13 at 07:03
  • @quellish: Got it. Thanks. I setup Xcode Server about a week ago, but didn't have time to try it yet. – Victor Ronin Nov 15 '13 at 16:15
  • Xcode now supports 'test' action that you can run with xcodebuild.But Jenkins doesn't support 'test' action for the current release. Spend a day to figure it out. – TS.xy Jan 28 '14 at 08:23
  • @TS.xy. Actually, Jenkins can use whatever xcodebuid can use, because it will be running it just through a command line. However, there are some problems with UI session. – Victor Ronin Jan 28 '14 at 17:02
  • @Victor Ronin, we did a work around that use shell script to invoke the 'test' action for xcode 5, but we can't get test result. So seems we have to configure the action from UI – TS.xy Jan 28 '14 at 20:59
  • @TS.xy: What exactly do you mean you can't get test results? Do test run or not? There is a problem that Xcode spits out results in wrong format, but it's solvable. – Victor Ronin Jan 29 '14 at 00:14

6 Answers6

41

Using xCode 7 and xCtool.

xctool is capable of executing unit tests without the simulator.

To get this working,

1 . Update target settings to run without a host app.

Select your project --> then test target --> Set the host application to none.

enter image description here

2. Install xctool , if you don't have it.

brew install xctool

3. Run the tests using terminal with xctool. 

xctool -workspace yourWorkspace.xcworkspace -scheme yourScheme run-tests -sdk iphonesimulator
rustylepord
  • 5,681
  • 6
  • 36
  • 49
  • 3
    You might encounter linker errors with classes that is not unavailable in the test bundle, resolve them to get this working . – rustylepord Jun 15 '16 at 06:23
  • This is odd. xctool's documentation states [If you are running xctool in continuous integration, the user account calling xctool must have an active GUI context. If not, the simulator will fail to start with cryptic warnings](https://github.com/facebook/xctool). – André Fratelli Jan 10 '17 at 16:35
24

You can create a Mac OSX Unit Test instead of an iOS unit test. This requires that you not include any iOS specific libraries in the unit tests though. You can do this via the following:

  1. Select the project -> the target drop down -> "Add Target..."
  2. Select "Mac OSX" -> "Other" -> "Cocoa Unit Testing Bundle"
  3. Create the testing bundle as you would a normal project

You can now add sources to the unit test and run it like an iOS test without launching the simulator.

Saltymule
  • 2,907
  • 2
  • 22
  • 30
  • 3
    That's the direction in which I am moving right now. However, iOS and OS X aren't exactly same operation systems (especially taking into account all the differences between frameworks) – Victor Ronin Nov 08 '13 at 17:31
14

I've asked the same question to apple engineers. Unfortunately it doesn't seem you can accomplish this and stay with iOS at the same time. There are some tricks you can do to check if testing. You could put this code snippet in your AppDelegate.h or some other global class to say not load a root viewcontroller and prevent any wierdo ui stuff from corrupting your unit tests:

static BOOL isTesting() {
    BOOL isTesting = !isEmpty([[[NSProcessInfo processInfo] environment] objectForKey:@"XCInjectBundle"]);
    return isTesting;
}

I've also had an apple engineer verify this is a legitimate check. And to give credit where credit is due, this is from: Programmatically determine current target (run or test) in iOS project

EDIT: I've also had success with this and it's a little more straight forward:

static BOOL isTesting() {
    return [[[NSProcessInfo processInfo] processName] isEqualToString:@"xctest"];
}
Community
  • 1
  • 1
rfrittelli
  • 1,211
  • 13
  • 15
7

A osx test target can become a huge hassle because you have to manage yourself which source file to include. Putting @testable import YourAppName on top of your XCTest files is way more convenient. So just prevent your app from launching in case of a XCTest run.

In your AppDelegate put: (Swift 3 solution) 

func application(_ application: UIApplication, didFinishLaunchingWithOptions launchOptions: [UIApplicationLaunchOptionsKey: Any]?) -> Bool
{
    if ProcessInfo.processInfo.environment["XCInjectBundleInto"] != nil {
        return false
    } 

...

This wont prevent the simulator from launching, but will save you a lot of time.

LimeRed
  • 1,278
  • 13
  • 18
  • I don't get it, if it doesn't stop the simulator from launching how is this an answer to the question? – SmileBot Jul 13 '17 at 15:05
2

Workaround:

App will still launch but you can #if to define what you don't want to run.

Approach:

  1. Create a custom build configuration called Test by duplicating Debug (Project > Info > create new configuration)
  2. In Build Settings > Active Compilation Conditions for Test add TESTING
  3. Edit Scheme > Info > Build Configuration, set build configuration as Test
  4. Use #if !TESTING #endif around the code you don't want to execute when testing.

Frameworks:

If you have embedded frameworks, create the same build configuration in the framework, so that the framework binary is properly linked.

Community
  • 1
  • 1
user1046037
  • 16,755
  • 12
  • 92
  • 138
2

You can run unit and ui tests in headless mode with Xcode 9 and a command from your terminal. For reference sample commands:

Building and testing a workspace

xcrun xcodebuild -workspace "YOUR_WORKSPACE_NAME.xcworkspace" -scheme "YOUR_SCHEME" -sdk "iphonesimulator12.0" -destination "OS=12.0,name=iPhone X" -configuration Debug -enableCodeCoverage YES clean build test

For project

xcrun xcodebuild -project "YOUR_PROJECT_NAME.xcodeproj" -scheme "YOUR_SCHEME" -sdk "iphonesimulator12.0" -destination "OS=12.0,name=iPhone X" -configuration Debug -enableCodeCoverage YES clean build test
abdullahselek
  • 7,893
  • 3
  • 50
  • 40