Is it possible to take a screenshot and test pixel values with espresso?

Question

I started using espresso and wrote some simple UI Tests for clicking on buttons, typing in text and checking if everything is there where it should be. Now I wanted to take a screenshot and test some pixel values, but I can't find a way to do this. Has anyone any suggestions? Is it even possible with espresso?

Answer 1

As espresso tests are simple InstrumentationTestCases, you could use paparazzo and celebrity to take screenshots of your application.

Spoon also as a very nice screenshot system. Sharper than paparazzi but that can only take a screenshot of activities, not dialogs for instance.

And last but not least, since android SDK 16, you can use screencap utility to take very fast and sharp screenshots of an app. I think this solution is really the best but it works only with SDK 16+.

Finally to test screenshots against a reference screenshot, there is no real tools. You can find great ideas of implementation on stack over flow, but up to now, noone came with a reference tool that can be used on android.

Answer 2

There's a tool made by Facebook for comparing screenshots if that's what you're looking for: http://facebook.github.io/screenshot-tests-for-android/

Iterating on UI code is hard. How do you quickly verify that your layout or view changes work correctly in all configurations? screenshot-tests-for-android can solve these problems by providing a test framework that checks for visual differences across changes.

Answer 3

I struggled a lot to set up screenshot tests for my custom view.
Here is how I managed to do that and everything I learned in the process.
It may also be used by regular applications and activities.

⚠ Caution #1

You can use JUnit 4 if you want. I'm using JUnit 5. Because JUnit 5 is built on Java 8 from the ground up, its instrumentation tests will only run on devices running Android 8.0 (API 26) or newer. Older phones or emulators will skip the execution of these tests completely, marking them as ignored.

If you want to run JUnit 5 tests on Android, refer to this answer for how to set it up.

⚠ Caution #2

The screenshot tests may not work on other devices even if they have the same screen DPI (they may not work at all on devices with different screen DPIs). For example, even when I use the same device in my local machine and on GitHub Actions to run the tests, they do not produce the same result (GitHub Actions assertions fail). So, I had to disable them on GitHub Actions.

If you want to disable your screenshot tests on GitHub Actions, see this answer.

⚠ Caution #3

If you have resources in instrumented tests (in subdirectories in androidTest directory) and you want to reference their id, you should use them like this:

my.package.name.test.R.id.an_id

For example, if your package name is com.example then to access the layout file in src/androidTest/res/layout/my_layout.xml in your tests, you use com.example.test.R.layout.my_layout.

⚠ Caution #4

Since we are saving our test screenshots on the external storage of the device/emulator, we need to make sure that we have both WRITE_EXTERNAL_STORAGE permission added in the manifest and adb install options -g and -r configured in the build script. When running on Marshmallow+, we also need to have those permissions granted before running a test. -g is for granting permissions when installing the app (works on Marshmallow+ only) while -r is to allow reinstalling of the app. These correspond to adb shell pm install options. Just be aware that this does not work with Android Studio yet.

Create an AndroidManifest.xml file in src/androidTest/ directory and add the following to it:

<manifest package="com.example">
  <!-- For saving screenshots in tests -->
  <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"
                   tools:ignore="ScopedStorage"
                   tools:remove="android:maxSdkVersion"/>
  <application android:requestLegacyExternalStorage="true">
   <!-- In case your test activity is in *androidTest* source set
        (I have it like this to test my custom view), define it here -->
    <activity android:name=".MyActivityThatContainsTheView"/>
  </application>
</manifest>

and add the adb install options in the app build file:

android {
    // adbOptions block is deprecated in newer version of Android Gradle Plugin;
    // replace adbOptions block with installation block
    adbOptions {
        installOptions("-g", "-r")
    }
}

⚠ Caution #5

I save the reference screenshot (the one I want to compare with the current screenshot) in src/androidTest/assets directory. So, specify that directory as an assets entry in the app build script:

android {
    sourceSets {
        get("debug").assets.srcDirs("src/androidTest/assets")
    }

⚠ Caution #6

To pass instrumentation arguments (like shouldSave and shouldAssert in the code below) when running the tests with:

• Gradle task:
  • Running the task from command line: pass your arguments after the task name
    ./gradlew myTask -Pandroid.testInstrumentationRunnerArguments.shouldSave=true
  • Running the task with Android Studio: pass your arguments in Arguments: field
    -Pandroid.testInstrumentationRunnerArguments.shouldSave=true
• Android Studio Android Instrumented Tests run configuration:
  Open the run Configuration, click ... in front of Instrumentation arguments: and then add a name-value like Name shouldSave Value true.

See this article and this post.

⚠ Caution #7

Make sure to use -ktx version of the AndroidX Core library.
The -ktx version contains useful Kotlin extension functions:

implementation("androidx.core:core-ktx:1.6.0")

⚠ Caution #8

Make sure the device screen is on and unlocked for the activity to go to resumed state.

The code

This is my activity under test in src/androidTest/java/com/example/ directory that has a property pointing to the view that I want to take its screenshot:

class MyActivityThatContainsTheView : AppCompatActivity() {

    lateinit var myView: MyView

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(com.example.test.R.layout.my_layout_that_contains_the_view)
        myView = findViewById(com.example.test.R.id.my_view_id_in_the_layout_file)
    }
}

And finally, this is my test and how I save, load, and compare the screenshots:

@DisabledIfBuildConfigValue(named = "CI", matches = "true")
class ScreenshotTestView {

    @JvmField
    @RegisterExtension
    val scenarioExtension = ActivityScenarioExtension.launch<MyActivityThatContainsTheView>()
    lateinit var scenario: ActivityScenario<MyActivityThatContainsTheView>
    // See ⚠ Caution #6 above in the post
    val shouldSave = InstrumentationRegistry.getArguments().getString("shouldSave", "false").toBoolean()
    val shouldAssert = InstrumentationRegistry.getArguments().getString("shouldAssert", "true").toBoolean()

    @BeforeEach fun setUp() {
        scenario = scenarioExtension.scenario
        scenario.moveToState(Lifecycle.State.RESUMED)
    }

    @Test fun test1() {
        val screenshotName = "screenshot-view-1"
        scenario.onActivity { activity ->
            val view = activity.myView
            view.drawToBitmap()
                    .saveIfNeeded(shouldSave, screenshotName)
                    .assertIfNeeded(shouldAssert, screenshotName)
        }
    }

    fun Bitmap.saveIfNeeded(shouldSave: Boolean, name: String): Bitmap {
        if (shouldSave) save(name)
        return this
    }
    
    fun Bitmap.assertIfNeeded(shouldCompare: Boolean, screenshotName: String) {
        if (shouldCompare) assert(screenshotName)
    }
    
    /**
     * The screenshots are saved in /Android/data/my.package.name.test/files/Pictures
     * on the external storage of the device.
     */
    private fun Bitmap.save(name: String) {
        val context = InstrumentationRegistry.getInstrumentation().targetContext
        val path = context.getExternalFilesDir(Environment.DIRECTORY_PICTURES)
        val file = File(path, "$name.png")
        file.outputStream().use { stream ->
            compress(Bitmap.CompressFormat.PNG, 100, stream)
        }
    }
    
    private fun Bitmap.assert(screenshotName: String) {
        val reference = loadReferenceScreenshot(screenshotName)
        // I'm using AssertJ library; you can simply use assertTrue(this.sameAs(reference))
        assertThat(this.sameAs(reference))
            .withFailMessage { "Screenshots are not the same: $screenshotName.png" }
            .isTrue()
    }
    
    private fun loadReferenceScreenshot(name: String): Bitmap {
        val context = InstrumentationRegistry.getInstrumentation().context
        val assets = context.resources.assets
        val reference = assets.open("compose/$name.png").use { stream ->
            BitmapFactory.decodeStream(stream)
        }
        return reference
    }
}

Answer 4

I fork a version to add Espresso run with Spoon to a great quality tools suit for android, check it out

Just run you will see:

mvn clean install -P espresso-spoon

Answer 5

As an alternative to doing screenshot comparison, you could consider comparing layout state itself (i.e. positions of the views) by using libraries such as LayoutVerifier.

While it will not give you a visual representation of the layout in form of a screenshot, it essentially performs the same function by comparing states of the views. Also it runs on Robolectric, removing the need for running it on a real device.