Android : Preview frame, converted from YCbCr_420_SP (NV21) format to RGB render correct picture but in green

Question

I post this question as I didn’t find answers solving my issues in the following posts :

Converting preview frame to bitmap

Android decodeYUV420SP results in green images?

Displaying YUV Image in Android

I got my data and cameraResolution (after selecting BestPreviewSize) from :

  public void onPreviewFrame(byte[] data, Camera camera) {
    Point cameraResolution = configManager.getCameraResolution();
    Handler thePreviewHandler = previewHandler;
    if (cameraResolution != null && thePreviewHandler != null) {
      Message message = thePreviewHandler.obtainMessage(previewMessage, cameraResolution.x,
          cameraResolution.y, data);
      message.sendToTarget();
      previewHandler = null;
    } else {
      Log.d(TAG, "Got preview callback, but no handler or resolution available");
    }
  }

Later from conversion to RGB, I use :

  int[] pixels=new int[yuvData.length];
  // int size = dataWidth*dataHeight;
 //  int[] pixels=new int[size]; replacing yuvData.length by size is not working too
  pixels=decodeYUV420SP(pixels,yuvData, dataWidth, dataHeight);
    Bitmap bitmap = Bitmap.createBitmap(dataWidth, dataHeight, Bitmap.Config.RGB_565);
    bitmap.setPixels(pixels, 0, dataWidth, 0, 0, dataWidth, dataHeight);

I tried 2 methods, one with RenderScript and one with decodeYUV420SP:

public Bitmap convertYUV420_NV21toRGB8888_RenderScript(byte [] data,int W, int H, CaptureActivityOCR fragment) { 
    // https://stackoverflow.com/questions/20358803/how-to-use-scriptintrinsicyuvtorgb-converting-byte-yuv-to-byte-rgba 
  RenderScript rs;
  ScriptIntrinsicYuvToRGB yuvToRgbIntrinsic;
  rs = RenderScript.create(fragment.getActivity());
  yuvToRgbIntrinsic = ScriptIntrinsicYuvToRGB.create(rs, Element.U8_4(rs)); //Create an intrinsic for converting YUV to RGB. 

  Type.Builder yuvType = new Type.Builder(rs, Element.U8(rs)).setX(data.length);
  Allocation in = Allocation.createTyped(rs, yuvType.create(), Allocation.USAGE_SCRIPT); //an Allocation will be populated with empty data when it is first created

  Type.Builder rgbaType = new Type.Builder(rs, Element.RGBA_8888(rs)).setX(W).setY(H);
  Allocation out = Allocation.createTyped(rs, rgbaType.create(), Allocation.USAGE_SCRIPT); //an Allocation will be populated with empty data when it is first created

  in.copyFrom(data);//Populate Allocations with data.

  yuvToRgbIntrinsic.setInput(in); //Set the input yuv allocation, must be U8(RenderScript). 
  yuvToRgbIntrinsic.forEach(out); //Launch the appropriate kernels,Convert the image to RGB. 


  Bitmap bmpout = Bitmap.createBitmap(W, H, Bitmap.Config.ARGB_8888);
  out.copyTo(bmpout); //Copy data out of Allocation objects.
  return bmpout;

}

or

int[] decodeYUV420SP(int[] rgb, byte[] yuv420sp, int width, int height) { 
  Log.e("camera", "   decodeYUV420SP  ");

  // TODO Auto-generated method stub
  final int frameSize = width * height;

    for (int j = 0, yp = 0; j < height; j++) {
        int uvp = frameSize + (j >> 1) * width, u = 0, v = 0;
        for (int i = 0; i < width; i++, yp++) {
            int y = (0xff & ((int) yuv420sp[yp])) - 16;
            if (y < 0) y = 0;
            if ((i & 1) == 0) {
                v = (0xff & yuv420sp[uvp++]) - 128;
                u = (0xff & yuv420sp[uvp++]) - 128;
            }

            int y1192 = 1192 * y;
            int r = (y1192 + 1634 * v);
            int g = (y1192 - 833 * v - 400 * u);
            int b = (y1192 + 2066 * u);

            if (r < 0) r = 0; else if (r > 262143) r = 262143;
            if (g < 0) g = 0; else if (g > 262143) g = 262143;
            if (b < 0) b = 0; else if (b > 262143) b = 262143;

            rgb[yp] = 0xff000000 | ((r << 6) & 0xff0000) | ((g >> 2) & 0xff00) | ((b >> 10) & 0xff);

      }
  }
  return rgb;
} 

But still I don't know why I got some good image but in green. (I have another similar method in black and white which is working) In addition of the links above,i tried all posts linked to this subject on SO , so If someone could help for another tip please? I may be confused with the size to apply?

Somewhere after OnPreviewFrame method and before conversion to RGB method, I use another method with instruction below as I need to rotate the data received. I am wondering if this is not the origin of issue :

  byte[] rotatedData = new byte[data.length];
  for (int y = 0; y < height; y++) {
      for (int x = 0; x < width; x++)
          rotatedData[x * height + height - y - 1] = data[x + y * width];
  }
  int tmp = width;
  width = height;
  height = tmp;

Please help me?

Answer 1

I think your rotation routine is wrong. It works for Luma (Y), hence you get good results for black and white picture but not for chroma. If width and height are dimensions of your picture you do not rotate chroma values at all. So you will need to add second loop for chroma and move pairs of bytes (V and U).

If you tried it without this rotation and you still have greenish picture the problem might be also in your decodeYUV420SP function.

There is not one universal formula for yuv to rgba conversion. It must match the opposite one.
Look here http://www.codeproject.com/Articles/402391/RGB-to-YUV-conversion-with-different-chroma-sampli and here http://www.fourcc.org/fccyvrgb.php

This one works for me

B = 1.164(Y - 16)                   + 2.018(U - 128)
G = 1.164(Y - 16) - 0.813(V - 128) - 0.391(U - 128)
R = 1.164(Y - 16) + 1.596(V - 128)

It seems to be the same as you are using only float version so you may try other.

Answer 2

I suggest to rotate the bitmap after RenderScript rotation:

if (bmpout == null) {
     bmpout = Bitmap.createBitmap(w, h, Bitmap.Config.ARGB_8888);
}
rgbaAllocation.copyTo(bmpout);

if (matrixPostRotate90 == null) {
    matrixPostRotate90 = new Matrix();
    matrixPostRotate90.postRotate(90);
}

Bitmap rotatedBitmap = Bitmap.createBitmap(bmpout, 0, 0, w, h, matrixPostRotate90, true);