31

I have done some google-ing around and couldn't find enough information about this format. It is the default format for camera preview. Can anyone suggest good sources of information about it and how to extract data from a photo/preview image with that format? To be more specific, I need the black and white image extracted.

EDIT: Seems like that format is also called YCbCr 420 Semi Planar

Adil Soomro
  • 37,609
  • 9
  • 103
  • 153
Ethan
  • 1,488
  • 3
  • 19
  • 24
  • Starting with Android API level 17, the best solution is to use an intrinsic RenderScript - `ScriptIntrinsicYuvToRGB`. This RenderScript is built-in, so you do not have to write a line of C-like code. See this question for details: [How to use ScriptIntrinsicYuvToRGB converting yuv to rgba](http://stackoverflow.com/q/20358803/377657) – rwong Jun 09 '14 at 22:06

6 Answers6

83

I developed the following code to convert the NV21 to RGB, and it is working.

/**
 * Converts YUV420 NV21 to RGB8888
 * 
 * @param data byte array on YUV420 NV21 format.
 * @param width pixels width
 * @param height pixels height
 * @return a RGB8888 pixels int array. Where each int is a pixels ARGB. 
 */
public static int[] convertYUV420_NV21toRGB8888(byte [] data, int width, int height) {
    int size = width*height;
    int offset = size;
    int[] pixels = new int[size];
    int u, v, y1, y2, y3, y4;

    // i percorre os Y and the final pixels
    // k percorre os pixles U e V
    for(int i=0, k=0; i < size; i+=2, k+=2) {
        y1 = data[i  ]&0xff;
        y2 = data[i+1]&0xff;
        y3 = data[width+i  ]&0xff;
        y4 = data[width+i+1]&0xff;

        u = data[offset+k  ]&0xff;
        v = data[offset+k+1]&0xff;
        u = u-128;
        v = v-128;

        pixels[i  ] = convertYUVtoRGB(y1, u, v);
        pixels[i+1] = convertYUVtoRGB(y2, u, v);
        pixels[width+i  ] = convertYUVtoRGB(y3, u, v);
        pixels[width+i+1] = convertYUVtoRGB(y4, u, v);

        if (i!=0 && (i+2)%width==0)
            i+=width;
    }

    return pixels;
}

private static int convertYUVtoRGB(int y, int u, int v) {
    int r,g,b;

    r = y + (int)(1.402f*v);
    g = y - (int)(0.344f*u +0.714f*v);
    b = y + (int)(1.772f*u);
    r = r>255? 255 : r<0 ? 0 : r;
    g = g>255? 255 : g<0 ? 0 : g;
    b = b>255? 255 : b<0 ? 0 : b;
    return 0xff000000 | (b<<16) | (g<<8) | r;
}

This image helps to understand. YUV420 NV21

If you wanna just grayscale image is easer. You can discard all the U and V info, and take just the Y info. The code would can be like this:

/**
 * Converts YUV420 NV21 to Y888 (RGB8888). The grayscale image still holds 3 bytes on the pixel.
 * 
 * @param pixels output array with the converted array o grayscale pixels
 * @param data byte array on YUV420 NV21 format.
 * @param width pixels width
 * @param height pixels height
 */
public static void applyGrayScale(int [] pixels, byte [] data, int width, int height) {
    int p;
    int size = width*height;
    for(int i = 0; i < size; i++) {
        p = data[i] & 0xFF;
        pixels[i] = 0xff000000 | p<<16 | p<<8 | p;
    }
}

To create your Bitmap just:

Bitmap bm = Bitmap.createBitmap(pixels, width, height, Bitmap.Config.ARGB_8888);

Where pixels is your int [] array.

Derzu
  • 7,011
  • 3
  • 57
  • 60
  • How to extract Cr channel from byte [] data ? – Yuriy Chernyshov Feb 18 '14 at 21:47
  • @YuriyChernyshov Supposing that YUV=YCrCb, you can extract Cr channel adapting the method convertYUV420_NV21toRGB8888 and putting the U values on an array. But why you wanna only the Cr? Pay attention that the U and V are interlaced, the first U1 from the picture is used with the Y1, Y2, Y7, Y8. – Derzu Feb 20 '14 at 01:00
  • Derzu, I am start to learn traffic sign detection. And the one of the first step is to take Cr channel and start to process it - adaptively thresholded using two thresholds to provide robust isolation of red scene component. – Yuriy Chernyshov Feb 21 '14 at 21:21
  • 2
    @Derzu You simply Rock !! \m/ – Salman Khakwani Mar 26 '14 at 14:46
  • 8
    There is an error in the graphic. The byte layout shown is NV12, not NV21. The only difference is the UV bytes are flipped. NV12: YYYYUV NV21: YYYYVU – Christopher Schneider Apr 10 '15 at 18:55
  • Derzu I want to add green toning (night vision) to my previewCallback. kindly help me out. – Zar E Ahmer Apr 15 '15 at 14:58
  • 1
    This image helped me a lot! Obrigado! =) – thiagolr May 15 '15 at 12:46
  • I was using openCV's methods and getting the highly chopped-up image. I used your method and got the same result. It is possible I'm not getting NV21 in the byte[]? I already am setting parameters.setPreviewFormat(ImageFormat.NV21). – jasxir Aug 27 '15 at 05:17
  • Derzu how can i change it's Y component to get these effect. https://en.wikipedia.org/wiki/File:Barn-yuv.png I primarily want emboss – Xar-e-ahmer Khan Aug 28 '15 at 08:10
  • @Xar-e-ahmerKhan The imagem you link show is divided in 4 Images. First is all components together, second just Y component, third Y+U, forth Y+V. To generate the two last ones, just set V or U to zero on the method convertYUVtoRGB(). – Derzu Aug 29 '15 at 15:08
  • Thank you so much for this answer, but is there a way to convert `int[] pixels` back into a `byte[]`? – JuiCe Jul 01 '16 at 01:13
  • 3
    I think the above answer is wrong. http://www.fourcc.org/pixel-format/yuv-nv21/ lists that NV21 has a V/U plane rather than U/V which is NV12 as depicted above. See http://stackoverflow.com/questions/12469730/confusion-on-yuv-nv21-conversion-to-rgb Another ref: https://linuxtv.org/downloads/v4l-dvb-apis/uapi/v4l/pixfmt-nv12.html?highlight=nv21 – Gooey Oct 28 '16 at 09:39
  • 1
    There is an error in the conversion: `r = y + (int)1.402f*v;` and `b = y + (int)1.772f*u;` should be `r = y + (int)(1.402f*v);` and `b = y + (int)(1.772f*u);` otherwise the float constant is first converted to `int` and then multiplied by `v` or `u`. See [this example](http://ideone.com/MtIh0A) – Lesque Apr 28 '17 at 09:27
  • 1
    @Lesque, thanks for detecting this precision error. I just correct this on the answer. – Derzu Apr 29 '17 at 12:23
  • @Derzu how about getting colored image? – Shishram Aug 24 '17 at 06:23
  • i have shifted from camera to camerax and i am getting index out of bound exception at this line. any guess u = data[offset + k] & 0xff; – Bilal Shahid Jun 03 '22 at 07:56
12

NV21 is basically YUV420 but instead of planar format where Y, U and V have independent planes, NV21 has 1 plane for Luma and 2nd plane for Chroma. The format looks like

YYYYYYYYYYYYYYYYYYYYYYYYYYYYY YYYYYYYYYYYYYYYYYYYYYYYYYYYYY . . . .

VUVUVUVUVUVUVUVUVUVUVUVUVUVUVU VUVUVUVUVUVUVUVUVUVUVUVUVUVUVU . . . . .

Mayank
  • 121
  • 1
  • 2
7

I also had lots of headache because of this preview format.

The best I could find are these:

http://www.fourcc.org/yuv.php#NV21

http://v4l2spec.bytesex.org/spec/r5470.htm

It seems that the Y component is the first width*height bytes int the array you get.

Some more informational links:

http://msdn.microsoft.com/en-us/library/ms867704.aspx#yuvformats_yuvsampling

http://msdn.microsoft.com/en-us/library/ff538197(v=vs.85).aspx

Hope this helps.

Max
  • 1,028
  • 2
  • 8
  • 28
4

The data is in YUV420 format.
If you are only interested in the monochrome channel, i.e. "black and white", then this the first width x height bytes of the data buffer you already have.
The Y channel is the first image plane. It is exactly the grey/intensity/luminosity etc. channel.

Adi Shavit
  • 16,743
  • 5
  • 67
  • 137
3

Here's code to just extract the greyscale image data:

private int[] decodeGreyscale(byte[] nv21, int width, int height) {
    int pixelCount = width * height;
    int[] out = new int[pixelCount];
    for (int i = 0; i < pixelCount; ++i) {
        int luminance = nv21[i] & 0xFF;
        out[i] = Color.argb(0xFF, luminance, luminance, luminance);
    }
    return out;
}
Sam
  • 40,644
  • 36
  • 176
  • 219
  • Sam how could I remove red and blue color from the nv21. I want to make my image to Greenish( night Vision Effect). – Zar E Ahmer Apr 17 '15 at 14:00
  • @Nepster, I'm not sure off the top of my head. You could use the luminance as the green value by making this change in the above code: `out[i] = Color.argb(0xFF, 0, luminance, 0);`. Or maybe you could alter the code of a full-colour implementation to discard the blue and red information. I think you should make a new StackOverflow question for that. – Sam Apr 20 '15 at 12:02
  • It works Sam. Thanks a lot. But when i open Front Camera. It rotate the image vertically.(up side down) – Zar E Ahmer Apr 20 '15 at 12:45
2

When you only need a grayscale camera preview, you could use a very simple renderscript:

# pragma version(1)
# pragma rs java_package_name(com.example.name)
# pragma rs_fp_relaxed

rs_allocation gIn;   // Allocation filled with camera preview data (byte[])
int previewwidth;    // camera preview width (int)

// the parallel executed kernel
void root(uchar4 *v_out, uint32_t x,uint32_t y){
   uchar c = rsGetElementAt_uchar(gIn,x+y*previewwidth); 
   *v_out = (uchar4){c,c,c,255};
}

Note : This is not faster than ScriptIntrinsicYuvToRGB (and a following ScriptIntrinsicColorMatrix to do the RGBA-> gray), but it runs with API 11+ (where the Intrinsics need Api 17+).

Matti81
  • 216
  • 3
  • 3