OpenGLES Rendering Color Issue

Question

I'm quite new to OpenGLES. I am trying to render the Android camera frames (YUV) to a quad. I am using a similar approach as seen here.

The issue is that I get this color issue as shown below. I don't think its a shader issue as I've tried other shaders and it has resulted in the same rendering.

Here's my rendering code:

GLES20.glActiveTexture(GLES20.GL_TEXTURE0);
        GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, mTextureYHandle);

GLES20.glTexImage2D(GLES20.GL_TEXTURE_2D, 0, GLES20.GL_LUMINANCE,
                mWidth, mHeight, 0, GLES20.GL_LUMINANCE, GLES20.GL_UNSIGNED_BYTE, mBufferY);

GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, mTextureUVHandle);
GLES20.glTexImage2D(GLES20.GL_TEXTURE_2D, 0, GLES20.GL_LUMINANCE_ALPHA,
                mWidth/2, mHeight/2, 0, GLES20.GL_LUMINANCE_ALPHA, GLES20.GL_UNSIGNED_BYTE, mBufferUV);

GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, GLES20.GL_NONE);




GLES20.glActiveTexture(GLES20.GL_TEXTURE0);
        GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, mTextureYHandle);
mTextureYHandle = GLES20.glGetUniformLocation(mProgramHandle, "u_InputImageTexture0");
GLES20.glUniform1i(mTextureYHandle, 0);

GLES20.glActiveTexture(GLES20.GL_TEXTURE1);
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, mTextureUVHandle);
mTextureUVHandle = GLES20.glGetUniformLocation(mProgramHandle, "u_InputImageTexture1");
GLES20.glUniform1i(mTextureUVHandle, 1);

GLES20.glGetAttribLocation(mProgramHandle, "a_Position");
GLES20.glEnableVertexAttribArray(mPositionHandle);
GLES20.glVertexAttribPointer(mPositionHandle, mPositionDataSize, GLES20.GL_FLOAT, false,
                0, quadVertices);

mTextureCoordinateHandle = GLES20.glGetAttribLocation(mProgramHandle, "a_TexCoordinate");
GLES20.glEnableVertexAttribArray(mTextureCoordinateHandle);
GLES20.glVertexAttribPointer(mTextureCoordinateHandle, mTextureCoordinateDataSize, GLES20.GL_FLOAT, false,
                0, quadTextureCoordinates);

GLES20.glDrawArrays(GLES20.GL_TRIANGLE_STRIP, 0, 4);

GLES20.glActiveTexture(GLES20.GL_TEXTURE0);
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, GLES20.GL_NONE);

GLES20.glActiveTexture(GLES20.GL_TEXTURE1);
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, GLES20.GL_NONE);

Thanks.

It was a simple fix. I was assigning the same handle for Y and UV. When I created a new handle for them, it worked. — Anil, Dec 19 '17 at 22:05

score 0 · Answer 1 · answered Dec 19 '17 at 18:23

0

You fail to switch texture unit before submitting your UV buffer — it looks like you've omitted a GLES20.glActiveTexture(GLES20.GL_TEXTURE1);.

Assuming that's not a copy and paste error, it would overwrite your luminance texture, so you'd end up sampling the UV texture for both luminance and chrominance. I would expect that to give you very grey images though, as U and V have a range of (-0.5, 0.5), meaning that when rendered as bytes 0 in that is 128 in luminance.

Suggested diagnostic check: skip UV entirely, just upload and show Y as greyscale.

You can also skip rebinding of active textures to samplers when establishing shader inputs.

answered Dec 19 '17 at 18:23

Tommy

99,986
12
185
204

Thanks for pointing out the texture switching part. I've added that line to the code but doesn't seem to fix the issue. Let me see what happens when I remove the UV upload part and stick to Y alone. Also, when I remove the rebinding code, I get a blue screen. – Anil Dec 19 '17 at 18:34
Then is anything else happening in between? OpenGL being the modal pain that it is, could somebody be trampling your textures? – Tommy Dec 19 '17 at 18:43
I tried creating a vec4 of y alone. But that's giving me a black screen. Shouldn't it change greys atleast? As I move my camera around? – Anil Dec 19 '17 at 18:43
I do a clearColor. Other than that, not doing anything else. – Anil Dec 19 '17 at 18:44
Yes, it should give you exactly a black and white image. UV are just extra information about colour. I assume that while debugging you've inspected `mBufferY` prior to upload and confirmed that it has a whole bunch of interesting values? What about if you set it entirely to e.g. 128 before upload? – Tommy Dec 19 '17 at 19:06
I don't get a black and white image. I get plain black.Nothing else. – Anil Dec 19 '17 at 19:16

score 0 · Answer 2 · answered Dec 19 '17 at 21:57

In general it's more efficient to use a SurfaceTexture to directly import the camera data as an EGLImage, and then use the GL_OES_EGL_image_external extension to access the YUV data natively via a samplerExternalOES in the fragment shader. This will handle color conversion and multi-plane sampling automatically and save you having to worry about it (as it can vary from platform to platform).

OpenGLES Rendering Color Issue

2 Answers2