Xcode 4.5 command line unit testing

Question

Having an issue since updating to Xcode 4.5 when running my unit tests via command line. The following is the output i'm seeing when i try to run my tests

Unknown Device Type. Using UIUserInterfaceIdiomPad based on screen size
Terminating since there is no workspace.
/Applications/Xcode.app/Contents/Developer/Tools/RunPlatformUnitTests.include:334: note: Passed tests for architecture 'i386' (GC OFF)

/Applications/Xcode.app/Contents/Developer/Tools/RunPlatformUnitTests.include:345: note: Completed tests for architectures 'i386'

Even though it does say the tests have passed and completed, I don't think they have actually have been run.

I'm using the following command to run the tests:

xcodebuild -workspace MyApp.xcworkspace -scheme MyAppTests -sdk iphonesimulator -configuration Debug clean build TEST_AFTER_BUILD=YES

Has anyone run into the same problem and can offer a solution?

I have the same issue, and other people do as well: https://github.com/gabriel/gh-unit/issues/96. No solution for now. — Jean Regisser, Sep 24 '12 at 19:02
I can confirm the same issue. It's been there since the first beta, but since it's "unsupported functionality" Apple hasn't helped with it. My solution is to write a separate OCUnit/SenTest test runner launched using WaxSim. It's partially written, but is working so far, will share once it's finished — Stew, Sep 26 '12 at 07:57
same problem. best i found is this (yet I have not got it working) - http://www.gerardcondon.com/blog/2012/09/20/further-jenkins-setup-code-signing/ — mr.andy.peters, Sep 26 '12 at 20:52
Any news with Xcode 4.5.1? Note that I've got a bounty on a duplicate question: http://stackoverflow.com/questions/12604628/running-ios-unit-tests-from-command-line-with-xcode-4-5 — fabb, Oct 05 '12 at 06:11
Have you tried any of the solutions below? I have been successful with my approach. — Edward Huynh, Oct 05 '12 at 06:20
Any solution that doesn't involve installing additional software? Really looking for a solution that allows me to fix my current installation so that I don't have to alter my build server. — groomsy, Oct 23 '12 at 23:12
Xcode 5 supports its own continuous integration solution. Maybe this can also be accessed via other CI tools like Jenkins? https://developer.apple.com/library/prerelease/ios/documentation/IDEs/Conceptual/xcode_guide-continuous_integration/000-About_Continuous_Integration/about_continuous_integration.html — fabb, Jun 15 '13 at 16:14

score 29 · Accepted Answer · edited May 23 '17 at 12:22

29

Just thought I should also share what I did for a solution to this issue. I followed the solution outlined in https://stackoverflow.com/a/10823483/666943 but converted the ruby script to shell. At the end I basically installed ios-sim via homebrew and replace the Run Script in the Build Phases of my Test target with the following:

if [ "$RUN_UNIT_TEST_WITH_IOS_SIM" = "YES" ]; then
    test_bundle_path="$BUILT_PRODUCTS_DIR/$PRODUCT_NAME.$WRAPPER_EXTENSION"
    ios-sim launch "$(dirname "$TEST_HOST")" --setenv DYLD_INSERT_LIBRARIES=/../../Library/PrivateFrameworks/IDEBundleInjection.framework/IDEBundleInjection --setenv XCInjectBundle="$test_bundle_path" --setenv XCInjectBundleInto="$TEST_HOST" --args -SenTest All "$test_bundle_path"
    echo "Finished running tests with ios-sim"
else
    "${SYSTEM_DEVELOPER_DIR}/Tools/RunUnitTests"
fi

To start the test now I pass in the argument RUN_UNIT_TEST_WITH_IOS_SIM=YES e.g.

xcodebuild -workspace MyApp.xcworkspace -scheme MyAppTests -sdk iphonesimulator -configuration Debug clean build RUN_UNIT_TEST_WITH_IOS_SIM=YES

edited May 23 '17 at 12:22

Community

1
1

answered Oct 01 '12 at 23:40

Edward Huynh

2,907
1
27
26

3

I always get the following error: `[DEBUG] Session could not be started: Error Domain=DTiPhoneSimulatorErrorDomain Code=2 "Simulator session timed out." UserInfo=0x7fd323c2a4a0 {NSLocalizedDescription=Simulator session timed out.}` – fabb Oct 16 '12 at 15:10
If this is for jenkins as I see your other question was related to then you will require another step to get it working. When installing Jenkins on our build server, it created the launchd plist in /Library/LaunchDaemons. When trying to launch ios-sim, I encountered the same error as you described, the issue seems to be that jenkins is no run as the logged in user. By moving the launchd plist to /Library/LaunchAgents instead, so that its run as the logged in user, this solved the problem. – Edward Huynh Oct 17 '12 at 00:41
1

Nono, I tried this one without Jenkins for now. I ran it from Xcode by running the Test Scheme (I'll also try from the command line, maybe that's where the error comes from). And: I have a separate Jenkins Server and Build Node. In most cases, the `jenkins` user on the build node is *not* logged in. The server just connects to it via SSH. It used to work fine in Xcode 4.4.1 with this hack: http://www.raingrove.com/2012/03/28/running-ocunit-and-specta-tests-from-command-line.html – fabb Oct 17 '12 at 06:48
Ok, sorry, it works when run from the command line with the command you provide (appending `-arch i386`). Now I need to try it out with Jenkins. – fabb Oct 17 '12 at 08:23
The jenkins user has to be logged in for my approach. – Edward Huynh Oct 17 '12 at 12:17
Too bad, it's the same with the other approach using WaxSim. – fabb Oct 17 '12 at 12:36
Using iOS-sim from home-brew is the key to success. I can confirm, this works fine using Xcode 4.5 - though the referenced Xcode script patch needs to look a little different than quoted. – Till Nov 15 '12 at 21:10
This works for me in our TeamCity environment. I installed ios-sim using Homebrew and changed the run script so it includes the absolute path for starting ios-sim. – plaetzchen Dec 17 '12 at 10:52
OK I need to correct my comment: It worked fine since I rebooted the system, now I get the same error that @fabb gets. – plaetzchen Dec 17 '12 at 15:24
@fabb did you run xcodebuild while the iOS simulator is in use? I also got the session timeout error in that case. – Hlung Feb 04 '13 at 07:27
Nice one! I was stuck at the Ruby code, didn't know where to use it. So your code helped a lot! But I also have to set `ONLY_ACTIVE_ARCH=NO` to pass over this error: `Check dependencies [BEROR]No architectures to compile for (ONLY_ACTIVE_ARCH=YES, active arch=x86_64, VALID_ARCHS=i386).` – Hlung Feb 04 '13 at 07:31
@Hlung. Yes that is a build setting that you can set in your project so that you don't have to specify it as part of the xcodebuild command – Edward Huynh Feb 04 '13 at 11:31
I have a target with a space in the name and had to fix the script to escape paths: https://gist.github.com/JohannesRudolph/5353093. Feel free to incorporate this (or improve it, my bash skills are < epsilon) – Johannes Rudolph Apr 10 '13 at 09:13

score 9 · Answer 2 · answered Sep 28 '12 at 10:15

9

I noticed this issue in the beta versions of Xcode 4.5 / iOS 6. I've been working on a standalone unit tests runner to work around this problem. It works by compiling your unit test bundle, then compiling a version of your app that automatically runs the unit tests in a simulator environment.

The tool is by no means complete, but enough people seem to be having this issue that I'm releasing the tool as is for now. Please fork or comment so I can improve the tool.

xcodetest: https://github.com/sgleadow/xcodetest

Also keep an eye on this radar on the issue http://openradar.appspot.com/12306879

answered Sep 28 '12 at 10:15

Stew

1,901
10
9

I can confirm that Stew's answer was what helped me get unit tests running. – Jessedc Oct 01 '12 at 09:34
It works! But I had to append `-arch i386` to the two `xcodebuild` occurrences in the script. – fabb Oct 16 '12 at 13:37
What not works is executing other tests from a library subproject within the same scheme. – fabb Oct 16 '12 at 13:37
Problem: Sometimes WaxSim has a timeout before finishing its unit tests. The script still success and exit code 0. – fabb Oct 16 '12 at 14:05
Small problem: when several users use the script, it only works for the first one, as the others have no access to the `/tmp/xcodetest/${MAIN_APP_TARGET}` directory. This should be made unique. – fabb Oct 16 '12 at 14:32
I use `OUTPUT_DIR=\`mktemp -d -t "xcodetest"\`/${MAIN_APP_TARGET}` instead. – fabb Oct 16 '12 at 14:39
Hmmmm, running the testcases via SSH (needed for Jenkins) does *not* work. It just hangs after starting WaxSim. Running it locally works though. Some issue with WaxSim and not having a UI? – fabb Oct 16 '12 at 14:44
sometimes, I get this as well. Session ended with error. The simulated application quit. But the test still passed. – kkurni Nov 27 '12 at 07:41

Victor Lima · Answer 3 · 2012-09-29T20:36:49.617

6

xcodebuild -project ${PROJECT_PATH}/${PROJECT_NAME}.xcodeproj \ -scheme ${TEST_SCHEME} \ -configuration Debug \ -sdk iphonesimulator5.1 \ clean build \ TEST_AFTER_BUILD=YES

Setting the iphonesimulator to version 5.1 seems to solve the problem. There are radar bugs filled upon this issue.

This article also mention a good solution to follow:

http://baolei.tumblr.com/post/32428168156/ios-unit-test-from-command-line-ios6-xcode4-5

edited Sep 29 '12 at 20:36

answered Sep 29 '12 at 19:37

Victor Lima

136
1
4

1

I ended going for a solution similar to the link you provided based on the answer http://stackoverflow.com/a/10823483/666943. – Edward Huynh Oct 01 '12 at 05:24
I guess you still needed to patch `RunPlatformUnitTests` like described here? http://www.raingrove.com/2012/03/28/running-ocunit-and-specta-tests-from-command-line.html – fabb Oct 17 '12 at 11:20
Yup, patching the RunPlatformUnitTests is also required. I have a sample one here: https://github.com/victorlima/AutoBuild/blob/master/scripts/RunPlatformUnitTests.patched. Let me know if it works. – Victor Lima Oct 17 '12 at 15:33

score 2 · Answer 4 · answered Jan 18 '13 at 09:27

Also there's little hack that can help to run command-line tests on iOS6.0 simulator SDK

I'm Using Cedar and this tweak helped me :

First, you need to update your main file a little:

  // Faking up that workspace port
  CFMessagePortCreateLocal(NULL, (CFStringRef) @"PurpleWorkspacePort", NULL, NULL,NULL);
  return UIApplicationMain(argc, argv, nil, @"CedarApplicationDelegate");

Second, you need to add category to UIWindow:

@implementation UIWindow (Private)
- (void)_createContext {
   // Doing nothing here. Just for crash avoidance
}
@end

Cedar Unittest will run fine, with some runtime warnings, but, at least they'll be able to run :)

Xcode 4.5 command line unit testing

4 Answers4

Linked