3

I am attempting to determine (to within 1 ms) when particular screen flips happen on Android. Choreographer fires every time a frame flips, but gives no way of determining which frame is actually being displayed. According to https://source.android.com/devices/graphics/architecture.html, there are several layers in the process: the user land buffer, which flips to a triple-buffered queue, which flips to the surface flinger, which flips to the hardware. Each of these layers can potentially drop a frame, but at this point I have only determined how to to monitor the user land buffer. Is there a way to monitor the other buffers/flips (in real time, on a non-rooted, non-custom phone)?

I have observed unexpected frame delays on the HTC M8 (about 1 every 5 minutes), but the Nexus 7 does not appear to have this problem. I measure the delays by using a Cedrus StimTracker (http://cedrus.com/stimtracker/) with a photo sensor and the Lab Streaming Layer (https://github.com/sccn/labstreaminglayer). I have tried using eglPresentationTimeANDROID to control when screens are flipped, and that has not fixed the problem.

Note that I'm using the ndk, but I can usually use the JNI to get access to non-ndk features when I need to.

The reason I care is in order to use Android for psychological and neurological experiments, where 1 ms precision is highly desirable.

Matthew Grivich
  • 431
  • 3
  • 12

1 Answers1

2

As far as accessible APIs go, it sounds like you've found the relevant bits and pieces. If you haven't yet, please read through this stackoverflow item.

Using Choreographer and extrapolation, you can guess at when the next display refresh will occur. Using eglPresentationTimeANDROID() on an Android 5.0+ device, you can tell SurfaceFlinger when you want a particular frame to be sent to the display. Assuming SurfaceFlinger is properly accounting for all latency (such as additional frames added by "smart" panels), that should get you reliable timing.

(Bear in mind that the timing is based on when the display latches the next frame, not when the next frame is fully visible on the display... the latency there will depend on the panel.)

Grafika's "scheduled swap" Activity uses this feature, but it sounds like you're already familiar.

The only way to get signaled by the display when it does the swap would be to dup() the display-retire fence fd from the previous frame, and wait on it. Some of the code in SurfaceFlinger does this, notably DispSync watches the retire fences to see if the software "VSYNC" is drifting. There is no public API for fences, and the user-space response time could certainly be more than 1ms anyway... it usually works out better to schedule ahead than it does to react. Your requirement for non-rooted non-custom devices makes this problematic.

If you're mostly seeing correct behavior, but occasionally seeing a miss, your best bet is to use systrace to track down the cause.

Community
  • 1
  • 1
fadden
  • 51,356
  • 5
  • 116
  • 166
  • Thanks Fadden. "Using Choregorapher and extrapolation .... Using eglPresentationTimeANDROID() .... that should get you reliable timing." This is the test that I did. Unfortunately, it did not result in reliable timing on the HTC M8. The typical latency changes with increasing extrapolation values, so I know that I'm using the function correctly. Also unfortunately, systrace does not work on the HTC M8. See http://stackoverflow.com/questions/32185412/unable-to-systrace-on-htc-one-m8-lollipop. Without more tools, we may have to put the HTC M8 on a not recommended for science list. – Matthew Grivich Feb 05 '16 at 18:03
  • 1
    Every Android device is a special snowflake. With your requirements it might be safer to create a whitelist. If you're particularly motivated you may be able to root the M8 and install a systrace-enabled kernel, but there's a chance that you'll be replacing whatever component is causing the instability, which would be great for that one device but somewhat defeat the purpose. – fadden Feb 05 '16 at 18:24