1

TLDR: What are the limitations of running iOS unit tests without a host application?

It seems a lot of posts on the internet regarding iOS hostless tests (Logic vs Application Tests) may be out of date. For example, according to this and this Stack Overflow post you cannot instantiate a UIFont in logic tests. I just tried to do so with Xcode 8.2.1 and iPhone 7 simulator running iOS 10.2 and it seems to work fine.

Here is an example project with such a test: https://github.com/lyahdav/SnapshotLogicTest

According to this blog post:

This is actually a limitation of Xcode logic tests—they don't fire up a UIApplication instance and don't play nicely with UIKit

But if you look at my example project above it's creating a UIView and using FBSnapshotTestCase to assert views are rendered the same in a logic test and everything seems to work fine.

Also I can't find Apple's documentation for Logic vs Application tests anymore. For example, this post and this post link to Apple web pages which seems to be dead now and I can't find a newer page that replace it.

I wonder if Apple has minimized the distinction since the difference is simply setting a host application on your test target or not. But it's not clear to me if there are any limitations with Logic Tests. I really like them because it allows me to create smaller, more isolated test targets for parts of my app that can run much faster that an Application Test suite.

UPDATE: I figured I'd add a note about UI Testing. Per that Apple doc, starting in Xcode 7 they added UI Testing as part of XCTest. This seems to be orthogonal to the point about a host vs hostless unit test target.

Community
  • 1
  • 1
Liron Yahdav
  • 10,152
  • 8
  • 68
  • 104

1 Answers1

0

I wonder if Apple has minimized the distinction since the difference is simply setting a host application on your test target or not.

There are now Unit Tests and UI Tests. Both involve running the application. Apple no longer talks about logic tests separately at all.

matt
  • 515,959
  • 87
  • 875
  • 1,141
  • Your example never tests anything about the application; it's really a test on a separate framework (the pod). Thus it doesn't need to run the application. – matt Dec 31 '16 at 01:29
  • I just updated the test repo to have the test exercise a class that's part of the application, same result. – Liron Yahdav Dec 31 '16 at 01:51
  • Also, unit tests don't involve running the application if you remove the host application from the test target settings. It won't launch the app in the simulator (but it does launch the simulator itself for some reason). – Liron Yahdav Dec 31 '16 at 01:58
  • I'm not sure what you mean by "result". The part in your question about UIFont etc. seems to me like a red herring, so I disregarded it. I'm trying to distinguish what you mean by "logic tests". You seemed to be trying to draw a distinction between running the app and not running it, so I gave a suggestion about why the app wasn't running. And I explained that Apple no longer mentions the distinction. They just give you a project with unit tests built-in, and they _do_ run the application. Beyond that I don't see what your question is about. – matt Dec 31 '16 at 02:07
  • I updated the question to more concisely ask my question. By "result" I meant that I updated the test repo to have tests that exercise production code but I am still able to successfully run tests that use UIKit which I thought was a limitation of a hostless test target. Why is the default to have unit test suites run within the host application if it seems there are no limitations with a hostless test suite? I'm guessing there are some, but I'm trying to understand what they are. The benefit of a hostless test target is speed, no need to wait for the app to launch on each test run. – Liron Yahdav Jan 02 '17 at 22:00
  • "Why is the default to have unit test suites run within the host application if it seems there are no limitations with a hostless test suite" Well, you can go further. Why has Apple effectively _hidden_ the fact that, as you claim, a hostless test suite is even _possible_? Is it because in Swift it _isn't_ possible? Or because some other important feature of the testing tools doesn't work in that situation? Or is it just that they don't find any value in supporting the distinction? – matt Jan 02 '17 at 22:46