12

I saw that in android vision api (the sample is here: https://github.com/googlesamples/android-vision) camera (camera1) is now deprecated and the recommend is to use camera2.

Do you guys have any idea how to re-write CameraSource to use camera2 on android vision?

Thanks in advance,

Vietnt134
  • 512
  • 7
  • 19

3 Answers3

3

It is possible to use Camera2 API with Google Vision API.

To start with, the Google Vision API Face Detector receives a Frame object that uses to analyze (detect faces and its landmarks).

The Camera1 API provides the preview frames in NV21 image format, which is ideal for us. The Google Vision Frame.Builder supports both setImageData (ByteBuffer in NV16, NV21 or YV12 image format) and setBitmap to use a Bitmap as the Preview Frames to process.

Your issue is that the Camera2 API provides the preview frames in a different format. It is YUV_420_888. To make everything work you have to convert the preview frames into one of the supported formats.

Once you get the Camera2 Preview Frames from your ImageReader as Image you can use this function to convert it to a supported format (NV21 in this case).

private byte[] convertYUV420888ToNV21(Image imgYUV420) {
    // Converting YUV_420_888 data to YUV_420_SP (NV21).
    byte[] data;
    ByteBuffer buffer0 = imgYUV420.getPlanes()[0].getBuffer();
    ByteBuffer buffer2 = imgYUV420.getPlanes()[2].getBuffer();
    int buffer0_size = buffer0.remaining();
    int buffer2_size = buffer2.remaining();
    data = new byte[buffer0_size + buffer2_size];
    buffer0.get(data, 0, buffer0_size);
    buffer2.get(data, buffer0_size, buffer2_size);
    return data;
}

Then you can use the returned byte[] to create a Google Vision Frame:

outputFrame = new Frame.Builder()
    .setImageData(nv21bytes, mPreviewSize.getWidth(), mPreviewSize.getHeight(), ImageFormat.NV21)
    .setId(mPendingFrameId)
    .setTimestampMillis(mPendingTimeMillis)
    .setRotation(mSensorOrientation)
    .build();

Finally, you call the detector with the created Frame:

mDetector.receiveFrame(outputFrame);

Anyway, if you want to know more about this you can test my working example available for free on GitHub: Camera2Vision. I hope I've helped :)

Ezequiel Adrian
  • 726
  • 1
  • 11
  • 29
  • 1
    I tried converting data using `convertYUV420888ToNV21()` but it doesn't scan barcode or qrcode at all – Lalit Poptani Sep 20 '17 at 09:24
  • 1
    @LalitPoptani If you are having issues, please feel free to open a new issue here: https://github.com/EzequielAdrianM/Camera2Vision/issues and give some more information so I can help you. – Ezequiel Adrian Sep 20 '17 at 16:58
2

Please have a look

camera2 with mobile vision? #65

Ok, I found this

There are no near term plans for a camera2 version of the CameraSource class in the official API. However, given how the API is structured, an alternate version of CameraSource could be written by the developer community that uses camera2. All of the existing APIs for working with frames and detectors are sufficient to support a camera2 implementation as well.

ashishdhiman2007
  • 807
  • 2
  • 13
  • 28
1

I haven't tried the link below because I stopped working on the Google Android Vision, but I think it is good for those who want:

https://medium.com/@mt1729/an-android-journey-barcode-scanning-with-mobile-vision-api-and-camera2-part-1-8a97cc0d6747

Vietnt134
  • 512
  • 7
  • 19