4

I'm trying to use Google Mobile Vision API with the camera2 module and I'm having a lot of trouble.

I'm using Google's android-Camera2Video example code as a base. I've modified it to include the following callback:

Camera2VideoFragment.java

OnCameraImageAvailable mCameraImageCallback;

public interface OnCameraImageAvailable {
    void onCameraImageAvailable(Image image);
}


ImageReader.OnImageAvailableListener mImageAvailable = new ImageReader.OnImageAvailableListener() {
    @Override
    public void onImageAvailable(ImageReader reader) {

        Image image = reader.acquireLatestImage();
        if (image == null)
            return;

        mCameraImageCallback.onCameraImageAvailable(image);

        image.close();
    }
};

That way any fragment including Camera2VideoFragment.java can get access to its images.

Now, The Barcode API only accepts Bitmap images, but I'm unable to convert YUV_420_888 to Bitmap. Instead, I changed the imageReader's file format to JPEG and ran the following conversion code:

    Image.Plane[] planes = image.getPlanes();
    ByteBuffer buffer = planes[0].getBuffer();
    buffer.rewind();
    byte[] data = new byte[buffer.capacity()];
    buffer.get(data);
    Bitmap bitmap = BitmapFactory.decodeByteArray(data, 0, data.length);

This worked but the framerate drop of feeding JPEG data to the imageReader was significant. I'm wondering if anyone has worked around this issue before.

aaa
  • 363
  • 4
  • 15
  • 1
    Your code helped me solve the problem that I could not otherwise. Thank you :) – Stefano Mtangoo Aug 09 '19 at 13:02
  • Yes I have the same problem and I was not able to find better solution. In addition to this, on some Qualcom Camera chips, this causes the camera capture to stop working at all and the only way to recover is the device reboot. Unfortunately, even with the Camera 2, the Android team only cares about the main stream users (who display the preview directly to the surface). They do not seem to case much about those who need the camera to be working completely without ANY UI (or I just failed to find such a solution). – Jiří Křivánek Apr 20 '23 at 07:05

1 Answers1

0

A late answer but hopefully still helpful.

As Ezequiel Adrian on his Example has explained the conversion of YUV_420_888 into one of the supported formats (In his case NV21), you can do the similar thing to get your Bitmap output:

private byte[] convertYUV420888ToNV21(Image imgYUV420) {
// Converting YUV_420_888 data to YUV_420_SP (NV21).
byte[] data;
ByteBuffer buffer0 = imgYUV420.getPlanes()[0].getBuffer();
ByteBuffer buffer2 = imgYUV420.getPlanes()[2].getBuffer();
int buffer0_size = buffer0.remaining();
int buffer2_size = buffer2.remaining();
data = new byte[buffer0_size + buffer2_size];
buffer0.get(data, 0, buffer0_size);
buffer2.get(data, buffer0_size, buffer2_size);
return data;}

Then you can convert the result into Bitmap:

Bitmap bitmap = BitmapFactory.decodeByteArray(bytes, 0, bytes.length);
Reza Hosseini
  • 154
  • 1
  • 10