2

I have coded my own ARGB image overlay code and it is as optimized to be as fast as "possible". However it isn't as fast as I would like it to be. Also, this process is called multiple times and uses a lot of the CPU.

I want to use the GPU to do the image overlay. I think this could potentially provide a huge boost in speed along with minimal CPU usage.

How can I go about using the GPU to overlay the images? From my research I can use a frame buffer to draw off screen. However, I'm not 100% sure how to go about setting that up. Essentially I'm passing a Bitmap from java down to JNI and then getting the Bitmap pixels where I would need to overlay the images into. The amount of images to be overlayed can range from 2 up to 12. These are large images.

UPDATE1:
So far this is what I have. I think is somewhat a step in the right direction.

GLuint arraybuffer;
glGenBuffers(GL_ARRAY_BUFFER, &arraybuffer);
glBindBuffer(GL_ARRAY_BUFFER, arraybuffer);

// draw into the array buffer the images
...

// read the buffer pixels from OpenGL
glReadPixels(0, 0, outputWidth, outputHeight, GL_RGBA, GL_UNSIGNED_BYTE, pixels);
Jona
  • 13,325
  • 15
  • 86
  • 129
  • read this: http://duriansoftware.com/joe/An-intro-to-modern-OpenGL.-Table-of-Contents.html it's about OpenGL itself (not OpenGL ES specific) but you can have a very good understanding about how graphics pipeline (GPU) works. – Wagner Patriota Nov 02 '13 at 05:17

1 Answers1

2

Yes, all you gotta do is to set up the OpenGL context. It's very straightforward in Android documentation. No secrets!

Displaying Graphics with OpenGL ES

Then, set up your framebuffer:

Check this link here: Android OpenGL ES Framebuffer objects - rendering depth-buffer to texture

All functions here are pretty much the same thing as in GLES20 from Android. Some methods have names a little bit different, but you can find all of them for sure.

When you reach the Drawing Shapes part. You gotta to write your vertex and fragment shaders. The vertex shader may be the same as in the example. Except that you need to treat your VBO (the vertexBuffer in the Android's sample). You must give the vertices (coordinates where you are drawing) and the UV's (coordinates of your texture) in order to draw the texture into the destiny image (one VBO for vertices and another to the UV's).

The fragment shader is the part you are looking for. It's gonna be executed by GPU for every pixel of your overlay process. This can be done with something like this:

Fragment shader for Overlay:

precision mediump float;
uniform float opacity;          // if you want to apply opacity
uniform sampler2D myTexture;    // the image you are drawing overlay
uniform sampler2D myBackground; // the destiny texture ID
varying vec2 texturePoint;      // here is a coordinate (U,V) coming from the VBO
                                // this one must come from the vertex shader.

void main()
{
    vec2 texturePointBG = gl_FragCoord.xy / textureSize( myBackground, 0 );
    vec4 A = texture2D( myTexture, texturePoint );
    vec4 B = texture2D( myBackground, texturePointBG );

    // this is one of the most traditional formula for overlay
    // http://en.wikipedia.org/wiki/Blend_modes
    gl_FragColor = vec4(
        B.x < 0.5 ? ( 2.0 * B.x * A.x ) : ( 1.0 - 2.0 * ( 1.0 - B.x ) * ( 1.0 - A.x ) ),
        B.y < 0.5 ? ( 2.0 * B.y * A.y ) : ( 1.0 - 2.0 * ( 1.0 - B.y ) * ( 1.0 - A.y ) ),
        B.z < 0.5 ? ( 2.0 * B.z * A.z ) : ( 1.0 - 2.0 * ( 1.0 - B.z ) * ( 1.0 - A.z ) ),
        A.a
    );

    gl_FragColor.a *= opacity; // if you want!
}

You can learn how to set up the uniform variables in the Android documentation... so you can give then some extra parameters like opacity, for example.

Community
  • 1
  • 1
Wagner Patriota
  • 5,494
  • 26
  • 49
  • So you are saying I need to loop by every x,y pixel to calculate the overlay or is this automatically done? How could I keep overlaying images? Say I have 6 images I want to overlay the quickest way. Do I need to generate a texture for every one of those images? I'm not 100% sure I understand your code... sorry kinda of a beginner with OpenGl. – Jona Nov 04 '13 at 16:44
  • No, you DON'T need to loop by every (x,y). The fragment shader is executed per pixel. It's like OpenGL does the loop for you, but it actualy parallelize it, doing all loop cycles together. You only write your fragment shader. But your question can't be easily answered in a quick answer. You gotta learn OpenGL. Try this out: http://duriansoftware.com/joe/An-intro-to-modern-OpenGL.-Table-of-Contents.html – Wagner Patriota Nov 04 '13 at 17:07
  • Completely agree I need to learn more OpenGL. Thought I could get away with shortcuts but it seems like I might not be able. – Jona Nov 04 '13 at 20:05