3

Question: For this pull request, the tests do not pass on CircleCI but the tests pass locally. Why?

The CircleCI test output shows failures for all FBSnapshotTestCase tests. For example:

✗ testAdjustsFontSizeToFitWidth, ((comparisonSuccess__) is true) failed - Snapshot comparison failed: Error Domain=FBSnapshotTestControllerErrorDomain Code=1 "Unable to load reference image." UserInfo=0x7f85f36b0a50 {NSLocalizedFailureReason=Reference image not found. You need to run the test in record mode, NSLocalizedDescription=Unable to load reference image., FBReferenceImageFilePathKey=/Users/distiller/TTTAttributedLabel/Example/TTTAttributedLabelTests/ReferenceImages_32/TTTAttributedLabelTests/testAdjustsFontSizeToFitWidth@2x.png}

[…]

✗ testAttributedTruncationToken, ((comparisonSuccess__) is true) failed - Snapshot comparison failed: Error Domain=FBSnapshotTestControllerErrorDomain Code=1 "Unable to load reference image." UserInfo=0x7f85f35b06d0 {NSLocalizedFailureReason=Reference image not found. You need to run the test in record mode, NSLocalizedDescription=Unable to load reference image., FBReferenceImageFilePathKey=/Users/distiller/TTTAttributedLabel/Example/TTTAttributedLabelTests/ReferenceImages_32/TTTAttributedLabelTests/testAttributedTruncationToken@2x.png}

However, the same tests pass locally:

image

On CircleCI the tests use the ReferenceImages_32 directory, which does not exist:

/Users/distiller/TTTAttributedLabel/Example/TTTAttributedLabelTests/ReferenceImages_32/TTTAttributedLabelTests/testAdjustsFontSizeToFitWidth@2x.png

However, the images do exist in ReferenceImages_64. I expect the _64 directory to be used because the circle.yml file specifies to use the iPhone 6 simulator:

dependencies:
  pre:
    - xcrun instruments -w "iPhone 6 (8.3 Simulator)" || exit 0
  override:    
    - sudo gem install cocoapods xcpretty obcd -N
    - pod install --project-directory=Example
test:
  override:
    - set -o pipefail && xcodebuild -workspace 'Example/Espressos.xcworkspace' -scheme 'Espressos'
      -sdk iphonesimulator -destination "platform=iOS Simulator,name=iPhone 6"
      GCC_INSTRUMENT_PROGRAM_FLOW_ARCS=YES GCC_GENERATE_TEST_COVERAGE_FILES=YES clean test | xcpretty -c
      --report junit --output ${CIRCLE_TEST_REPORTS}/junit.xml
Aaron Brager
  • 65,323
  • 19
  • 161
  • 287

1 Answers1

2

Have you tried running the build without the cache? This often fixes these kinds of issues for me. To build without cache, just click the button at the top right.

Cody
  • 36
  • 4