20

I am running test cases in application with enabled code coverage data Xcode 7 Beta 2. But I am able to get only few files coverage data while my all test cases are running successfully.

Some files has covered all codes by unit test cases but still showing 3% code coverage.

For example:

enter image description here This is the result of code coverage, as you can see on the right side, there is an info how many times these lines of code was called during tests. In this case - 0.

But...

enter image description here here is a place in tests where we can see that this function was called indeed. How many times? oh... at least once. This number is delivered by info on the right side.

So the code above should be marked as called, and not be grayed out:-)

Can anyone explain this? Why does this happen?

Bartłomiej Semańczyk
  • 59,234
  • 49
  • 233
  • 358
Sudhir Kumar
  • 301
  • 2
  • 9
  • Exactly the same question from me:) I covered a whole class, but code coverage gray out my methods, that were called, indeed, they should not be grayed out. (Only if grayed out means code that wasn't called while tests). Waiting for answer... – Bartłomiej Semańczyk Jun 29 '15 at 10:35
  • I think it is still broken (for iOS projects at least). I had a test project for OSX and there it works as expected, but for iOS custom code that gets triggered is not highlighted as such. – 0x416e746f6e Jul 07 '15 at 19:01

5 Answers5

27

IT WORKS.

  1. Since Apple released @testable keyword to import your project into test target, you don't have to add your files to both target anymore:

enter image description here

  1. So just remove every file from your test target:

enter image description here

  1. Wherever you need access to your file from your test target just import your target using: @testable import MyApp

enter image description here

  1. Do this for every file in your project.

Then code coverage will be working fine.

Read more from Swift 2 + Xcode 7: Unit Testing Access Made Easy!!!!

If you need to know how to work with code coverage read How to use code coverage in Xcode 7?

As @Gerd Castan mentioned earlier is: "So it appears to me that a tested method shows a coverage of 0 when there exists at least one target where this method is not tested."

Solution is simple. Do not let compiler think that this file is included in more that one target, instead import your module using @testable keyword.

Community
  • 1
  • 1
Bartłomiej Semańczyk
  • 59,234
  • 49
  • 233
  • 358
  • 1
    **IT WORKS NOT**. Using "@testable import" is a good idea, but it works only for the simplest special case for those who chose to compile into the test target. There are many good reasons to compile some classes into several (non test) targets (like main app + extension(s), or free app + paid app). And for those common cases your idea does not work. As you see in the source code of my answer, I used "@testable import" and still the source editor shows no coverage. – Gerd Castan Aug 02 '15 at 07:12
  • Because your files are (I suppose) targeted for both: app and test, since they should not. Apple released `@testable` keyword to avoid targeting files for both. – Bartłomiej Semańczyk Aug 02 '15 at 07:14
  • nope. I don't compile app classes into the test target. – Gerd Castan Aug 02 '15 at 07:16
  • Remove your test target, and create it again. One time it was good for me. – Bartłomiej Semańczyk Aug 02 '15 at 07:23
  • I created an additional completely fresh project to test my answer before I wrote it. Do you compile in more than one target? – Gerd Castan Aug 02 '15 at 07:29
  • No, I compile it in exactly one target, my app target. If you compile specific file for more than one target, then you were right, it has to be used in both targets, to mark coverage of that file as fulfilled. – Bartłomiej Semańczyk Aug 02 '15 at 07:30
  • 4
    agreed: don't compile into the test target and test each remaining target. – Gerd Castan Aug 02 '15 at 07:46
4

I think I found out what XCTest coverage ist doing and it makes some sense:

My setup:

class1 compiled into target1

class2 compiled into target1 and into target2

Test setup:

import XCTest
@testable import target1

class MyTests: XCTestCase {
    func testSomething() {
        someMethodFromClass1()
        someMethodFromClass2()
    }
}

What I find is that class1 (compiled into target1) shows test coverage and class2 (compiled into target1 and into target2) shows no test coverage.

So it appears to me that a tested method shows a coverage of 0 when there exists at least one target where this method is not tested.

And this makes a lot of sense, because testing a method in a target doesn't say anything about how it behaves in a different target.

Apple wants us to test all targets.

Update One more hint to back this theory:

go to the report navigator

report navigator

and click on coverage.

If you have more than one target, you see your files grouped by target.

And if you have one file in two targets, you see your file twice.

If you have one file in both targets, the code coverage of this one file is shown for both targets. And (at least in my projects) one file has different blue lines in each target:

coverage in target 1:

target 1

coverage of same file in the same project in the same test run in target 2:

target 2

If you look at your test coverage in the source editor, apple has to decide which coverage it shows to you. I think showing the target with the lowest coverage is the best apple can do in the source editor.

simple fix for a special case:

If your only second target is your test target: don't compile into your test target and use @testable import.

For all other cases you have to test each target.

Gerd Castan
  • 6,275
  • 3
  • 44
  • 89
2

I checked at Apple developers forums for the subject and after reading through various posts I guess I came across the solution.

In order for the thing to work it is necessary to:

  1. Remove all your application source files from the test target
  2. In your unit-test sources put @testable import <NameOfYourModule>
  3. Re-build and re-run tests

I tested this out with my current project, and the results are much better.

Original solution recipe can be found at: http://natashatherobot.com/swift-2-xcode-7-unit-testing-access/

Also it seems that the functionality is a bit raw-ish, hence bugs possible, and Apple suggests submitting bug reports when things do not work as expected:

I've personally seen code coverage results for some very large projects. Right now the support works best for applications and frameworks. If that's what you're testing, then it would be best if you could file a bug report at https://bugreport.apple.com so that we can investigate your particular circumstances. For that matter, a bug report would be a good thing no matter what type of project you have. If possible, it's best to actually in the project with the report. If you can't do that, describe its setup in as much detail as possible. Pictures are good.

Original thread: https://forums.developer.apple.com/message/9733#9733

0x416e746f6e
  • 9,872
  • 5
  • 40
  • 68
0

Bear in mind that there are multiple ways to cover code with tests, you may test all functions, or you may be covering all instructions in the functions, but you may not be covering all the execution paths that are possible.

Or Xcode coverage stuff may be broken, but it's hard to tell if you don't give details on what kind of coverage are you expecting it to check.

Bartłomiej Semańczyk
  • 59,234
  • 49
  • 233
  • 358
Dietr1ch
  • 178
  • 8
  • All functions and all instructions in the functions. Using coverage feature when I run tests, there in code the lines which were used are grayed out. But it is not working in a good way. There is too small lines grayed out. They should be. – Bartłomiej Semańczyk Jul 05 '15 at 16:00
0

This happens because .swift file of your project selected for both targets by default.

Manually select and remove test target for files works for me.

Rakesh Yembaram
  • 433
  • 4
  • 7