Images and mask in OpenGL ES 2.0

Question

I'm learning OpenGL ES 2.0 and I'd like to create an App to better understand how it works. The App has a set of filter that the user can apply on images (I know, nothing new :P).

One of this filter takes two images and a mask and it mixes the two images showing them through the mask (here an image to better explain what I want to obtain)

enter image description here

At the moment I'm really confused and I don't know where to start to create this effect. I can't understand wether I have to work with multiple textures and multiple FrameBuffers or I can just work with a single shader.

Do you have any hint to help me in doing this project?

EDIT--------

I've found this solution, but when I use as mask lines instead of circles the result is really "grungy", especially if lines are rotated.

precision highp float;

varying vec4 FragColor;
varying highp vec2 TexCoordOut;

uniform sampler2D textureA;
uniform sampler2D textureB;
uniform sampler2D mask;

void main(void){
    vec4 mask_color = texture2D(mask, TexCoordOut);

    if (mask_color.a > 0.0){
        gl_FragColor =  texture2D(textureA, TexCoordOut);
    }else {
        gl_FragColor =  texture2D(textureB, TexCoordOut);
    }
}

Is it probably better to use Stencil buffer or blending?

BTW, you don't have to use alpha (`mask_color.a`) channel for mask. You can use any other `r`,`g`,`b` channel and this way you'll save GPU memory by using mask texture w/o alpha channel. — keaukraine, Sep 02 '13 at 10:46
@MatterGoal May be you can answer this question, http://stackoverflow.com/questions/24486729/uiimage-masking-with-gesture — Prasad De Zoysa, Jul 04 '14 at 11:45

Sergey K. · Accepted Answer · 2013-09-07T18:45:42.840

21

You can apply the mask in one line without using the costly if:

gl_FragColor = step( 0.5, vMask.r ) * vColor_1 + ( 1.0 - step( 0.5, vMask.r ) ) * vColor_2;

Or, better just interpolate between two colors:

gl_FragColor = mix( vColor_1, vColor_2, vMask.r );

In this case the mask can be smoothed (i.e. with Gaussian blur) to produce less aliasing. This will yield very good results compared to a single value thresholding.

edited Sep 07 '13 at 18:45

answered Sep 07 '13 at 18:40

Sergey K.

24,894
13
106
174

Have I to use multipass to create a blurry mask? Or you are suggesting to create the blur with the same shader? – MatterGoal Sep 07 '13 at 20:07
2

You can create blur with the same shader by doing several (i.e. 5) mask texture fetches and averaging the result. Then using this average value instead of ``vMask.r``. – Sergey K. Sep 07 '13 at 20:09
2

Here you can find an implementation of a gaussing blur in a fragment shader: http://xissburg.com/faster-gaussian-blur-in-glsl/ – Sergey K. Sep 07 '13 at 20:10
have to remind: why would people think **if** is more expensive than **multiply** – zinking Sep 14 '13 at 04:18
@PsychoDad branch prediction takes more CPU cycles compared with multiply ? that's unbelievable to me. Any references ? – zinking Jan 12 '15 at 01:15
@zinking http://stackoverflow.com/questions/17095324/fastest-way-in-c-to-determine-if-an-integer-is-between-two-integers-inclusive – jjxtra Jan 12 '15 at 02:47

score 1 · Answer 2 · answered Aug 31 '13 at 17:33

There is no need for multiple shaders or framebuffers, just multiple texture units. Simply use 3 texture units which are all indexed by the same texture coordinates and use the Mask texture to select between the other two textures. The fragment shader would look like this:

uniform sampler2D uTextureUnit_1;
uniform sampler2D uTextureUnit_2;
uniform sampler2D uTextureMask;
varying vec2 vTextureCoordinates;

void main()
{
    vec4 vColor_1 = texture2D(uTextureUnit_1, vTextureCoordinates);
    vec4 vColor_2 = texture2D(uTextureUnit_2, vTextureCoordinates);
    vec4 vMask = texture2D(uTextureMask, vTextureCoordinates);

    if (vMask.r > 0.5)
        gl_FragColor = vColor_1;
    else
        gl_FragColor = vColor_2;
}

You can see that using a third texture unit just to do a binary test on the Red channel is not very efficient, so it would be better to encode the mask into the alpha channels of Textures 1 or 2, but this should get you started.

Do you suggest to apply any particular settings to the scene to obtain a perfect result? For some "non-linear" masks I get a terrible final result. — MatterGoal, Aug 31 '13 at 20:16
OpenGL specifically does not provide pixel-perfect results, but you can improve the quality by increasing the dimensions of the textures. For glTexParameteri(), use GL_LINEAR for the MAG_FILTER and MIN_FILTER. MIP_MAP only helps if scaling down. — ClayMontgomery, Aug 31 '13 at 22:43

Images and mask in OpenGL ES 2.0

2 Answers2

Linked