Preview callback in Camera2 is significantly slower than in Camera1

Question

It is 2017 and I am finally starting switching from Camera1 to Camera2. In Camera1 I was greatly relying on setPreviewCallbackWithBuffer() to perform a real time frame processing, however in Camera2 this works much much slower to the point where it becomes almost unusable.

To compare, on Moto G3 Camera1 can easily produce 30-40 FPS while on Camera2 I couldn't get more than 10-15 FPS.

Here is how I am creating ImageReader

imageReader = ImageReader
  .newInstance(
    previewSize.width,        // size is around 1280x720
    previewSize.height,
    ImageFormat.YUV_420_888,  // note, it is not JPEG
    2 // max number of images, does not really affect performance
  );

imageReader.setOnImageAvailableListener(
  callback,
  CameraThread.getInstance().createHandler()
);

Callback itself does the minimum possible job:

Image image = reader.acquireNextImage();
image.close();

I already checked similar answers, such as this one. However their problem is that they're using JPEG image format instead of YUV_420_888.

How to achieve a performance similar to Camera1?

Answer 1

I had the same performance problems on an app supporting both Camera1 and Camera2 APIs. When Android version was above Lollipop I used to switch to Camera2 API resulting in very bad performances (I had two target at time: an ImageReader and a Surface).

I ended up to use Camera2 API only when there was full Hardware support by the phone. You can check using the CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL

Hope it helps

Answer 2

This is just an observation but I'll post it anyway.

You say you are registering an OnImageAvailableListener. This listener does not deliver images but a reference to the same ImageReader you subscribed to. And then you must call either acquireLatestImage or acquireNextImage to get the actual image.

There is a paragraph in the docs that might be helpful to understand what is going on:

The image data is encapsulated in Image objects, and multiple such objects can be accessed at the same time, up to the number specified by the maxImages constructor parameter. New images sent to an ImageReader through its Surface are queued until accessed through the acquireLatestImage() or acquireNextImage() call. Due to memory limits, an image source will eventually stall or drop Images in trying to render to the Surface if the ImageReader does not obtain and release Images at a rate equal to the production rate.

So some things that might help:

• request large memory in the manifest
• Pass a large enough maxImages argument to the ImageReader constructor (You would get IllegalStateException if you exhaust the queue anyway).
• Prefer acquireLatestImage over acquireNextImage for real-time processing. This method releases older images automatically while the other one does not, and thus using acquireNextImage by mistake will increasingly slowdown image delivery until you run out of memory.