Allow for Multiple-Pass Renders in GLSurfaceView Renderer with SurfaceTexture

Question

I am displaying a video on a GLSurfaceView with a custom renderer that requires that multiple shaders be applied in succession. Currently, it is working successfully with one shader, though I am not sure how to extend the rendering pipeline to apply multiple shaders in succession.

I know that there are some examples concerning applying multiple shaders (using FrameBuffers and RenderBuffers), but I have not found any that deal with an image passed in through a SurfaceTexture.

There is a specific concern I would like to address:

A SurfaceTexture must be bound to a GL_TEXTURE_EXTERNAL_OES texture. On the other hand, a FrameBuffer cannot be bound to a GL_TEXTURE_EXTERNAL_OES texture (typically a GL_TEXTURE_2D is used), so is it even possible to use a FrameBuffer for a multi-pass render when the input texture is of a different format than the output? If not, what are the other options for performing a multi-pass render?

Below is some relevant code in the onSurfaceCreated function of the renderer I am trying to extend to perform multiple passes::

            GLES20.glGenTextures(1, this.textureID, 0);
            GLES20.glBindTexture(GL_TEXTURE_EXTERNAL_OES, this.textureID[0]);
            GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_LINEAR);
            GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR);
            GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_S, GLES20.GL_CLAMP_TO_EDGE);
            GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_T, GLES20.GL_CLAMP_TO_EDGE);
            GLES20.glBindTexture(GL_TEXTURE_EXTERNAL_OES, 0);
            this.surfaceTexture = new SurfaceTexture(this.textureID[0]);

Below is some relevant code in the onDrawFrame function of that renderer:

        synchronized (this) {
            if (this.updateSurface) {
                this.surfaceTexture.updateTexImage();
                this.surfaceTexture.getTransformMatrix(this.stMatrix);
                this.updateSurface = false;
            }
        }

        GLES20.glBindTexture(GL_TEXTURE_EXTERNAL_OES, this.textureID[0]);
        GLES20.glClearColor(0.5f, 0.5f, 0.5f, 1.0f);
        GLES20.glClear(GLES20.GL_DEPTH_BUFFER_BIT | GLES20.GL_COLOR_BUFFER_BIT);

  //apply shader here and call glDrawArrays() at end

Answer 1

One way to approach this would be to use a SurfaceHolder rather than a SurfaceTexture.

From there, you can then get the Surface being held by the SurfaceHolder.

Then you can get the underlying Canvas being drawn to.

Note: Per this answer, you will need to use setBitmap(Bitmap canvas_bitmap) to specify the Bitmap being drawn into.

All the texture setup and parameter stuff will still need to be done basically the way you showed in you question.

GLES20.glGenTextures(1, this.textureID, 0);
GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, this.textureID[0]);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_LINEAR);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_S, GLES20.GL_CLAMP_TO_EDGE);
GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_T, GLES20.GL_CLAMP_TO_EDGE);

Then dump the bitmap into a texture using:

GLUtils.texImage2D( GLES20.GL_TEXTURE_2D, 0, canvas_bitmap, 0 );