0

I want to use OpenGL to process the data from camera which format is NV21 on Android platform.

My code was below:

vertex shader:

attribute vec4 position;
attribute vec2 inputTextureCoordinate;
varying vec2 v_texCoord;
void main()
{
   gl_Position = position;
   v_texCoord = inputTextureCoordinate;
}

fragment shader:

precision mediump float;
varying vec2 v_texCoord;
uniform sampler2D yTexture; 
uniform sampler2D uvTexture;
const mat3 yuv2rgb = mat3(
                        1, 0, 1.2802,
                        1, -0.214821, -0.380589,
                        1, 2.127982, 0
                        );

void main() {    
    vec3 yuv = vec3(
                1.1643 * (texture2D(yTexture, v_texCoord).r - 0.0627),
                texture2D(uvTexture, v_texCoord).a - 0.5,
                texture2D(uvTexture, v_texCoord).r - 0.5
                );
    vec3 rgb = yuv * yuv2rgb;
    gl_FragColor = vec4(rgb, 1.0);
}

I send yTexture:

GLES20.glTexImage2D(GLES20.GL_TEXTURE_2D, 0, GLES20.GL_LUMINANCE, w, h, 0,                  GLES20.GL_LUMINANCE, GLES20.GL_UNSIGNED_BYTE,                           ByteBuffer.wrap(data));

where data was the nv21 format byte array of camera preview And I send uvTexture:

byte[] luminanceAlpha = new byte[w * h / 2];
System.arraycopy(data, w * h, luminanceAlpha, 0, w * h / 2);
GLES20.glTexImage2D(GLES20.GL_TEXTURE_2D, 0,            GLES20.GL_LUMINANCE_ALPHA, w / 2, h / 2, 0,                             GLES20.GL_LUMINANCE_ALPHA,                              GLES20.GL_UNSIGNED_BYTE, ByteBuffer.wrap(luminanceAlpha));

That's all the code which I thought was important. But when I run the program, I found the result in GLSurfaceView looks more bluer. Is there any wrong with my code. I was very trouble.

The Lord Of Ring
  • 155
  • 1
  • 3
  • 10
  • Why do you do 1.1643 * () in your shader? Looks like you are not using the Y texture. – BlueVoodoo Mar 26 '13 at 11:37
  • Oh, I'm sorry. That's just an input error. I have modified it already. – The Lord Of Ring Mar 26 '13 at 12:29
  • Looks all right to me. Are you sure data contains what you expect? – BlueVoodoo Mar 26 '13 at 14:29
  • I have solve this problem. My yuv data is ok. The problem is when I use the method "GLES20.glUniform1i(mGLUniformTexture, 0)" to send uvTexture to opengl, I forget to add the second parameter. So my uvTexture in fragment shader got the wrong value. Thank you very much. – The Lord Of Ring Mar 27 '13 at 02:03
  • @TheLordOfRing I know the comment is off topic, but can you please tell me how did you get the content of the camera preview in order to obtain the `yTexture` and `uvTexture`. I am working on a similar application and I cannot figure out how to accomplish this. Thank you. – niculare Apr 18 '13 at 13:49
  • 1
    Before the camera start preview, you can set PreviewCallback, then the camera data will return by the method onPreviewCallback. It's a byte array and it's format is NV21. You should look up to the NV21 format document and you will see how to copy y channel data and uv channel data. Call me if you have any problem. – The Lord Of Ring May 11 '13 at 02:36

1 Answers1

0

Why not using the GLSurfaceView and generate the SurfaceTexture, then passing it into camera using setPreviewTexture? If you want the pixel RGB data, using glReadPixels to get the buffer data.

Yu-Hsuan
  • 505
  • 4
  • 9
  • Thanks for your suggest. Because SurfaceTexture only support API 11 and above. And I also has a problem when using SurfaceTexture. It will prompt the warning message 'updateTexImage: clearing GL error: 0x502', but this does not affect the rendering. How could I remove the warning? – The Lord Of Ring Jun 17 '13 at 02:27
  • Did you try this post [Modifying camera output using SurfaceTexture and OpenGL](http://stackoverflow.com/questions/12519235/modifying-camera-output-using-surfacetexture-and-opengl)? I use the [VideoSurfaceview](https://android.googlesource.com/platform/cts/+/master/tests/tests/media/src/android/media/cts/VideoSurfaceView.java) as my render environment and change the MediaPlayer source to camera, and I don't have any warning message. – Yu-Hsuan Jun 17 '13 at 18:18
  • I have already saw that, but not the same problem. My warning will not affect the running at all. I just wondering. – The Lord Of Ring Jun 18 '13 at 05:59
  • I could only guess that there may be little wrong in your OpenGL ES render setting. Try that Google's test project as I described above. :) – Yu-Hsuan Jun 18 '13 at 14:14
  • I have found the reason. Because I send GL_TEXTURE_EXTERNAL_OES as GL_TEXTURE_2D, so – The Lord Of Ring Jun 20 '13 at 06:52