How to create Stencil buffer with texture (Image) in OpenGL-ES 2.0

Question

Can I have Stencil prepared with a texture (Image) in OpenGL 2.0 So that some part of the Image will be transparent and as a result it will be transfered as is
to Stencil buffer and then will use this use this stencil buffer for further drawing.

EDIT by datenwolf to account for OPs question update in a answer:

By @InfiniteLoop:

@datenwolf thanks a lot for ur reply but no success :( here is my code

- (void)render 
{
    [EAGLContext setCurrentContext:context];
    glBindFramebuffer(GL_FRAMEBUFFER, defaultFramebuffer);
    glUseProgram(program);
    glClearColor(0, 104.0/255.0, 55.0/255.0, 1.0);
    glViewport(0, 0, backingWidth, backingHeight);
    glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT | GL_STENCIL_BUFFER_BIT);
    // Set the stencil clear value
    glClearStencil ( 0x1 );

    // Set the depth clear value
    glClearDepthf( 0.75f );



    ////////
    ////
    GLfloat threshold = 0.5f;//half transparency
    /* the order in which orthogonal OpenGL state is set doesn't matter */
    glEnable(GL_STENCIL_TEST);
    glEnable(GL_ALPHA_TEST);

    /* don't write to color or depth buffer */
    glColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE);
    glDepthMask(GL_FALSE);

    glStencilOp(GL_KEEP, GL_KEEP, GL_REPLACE);

    /* first set alpha test so that fragments greater than the threshold
     will pass thus will set nonzero bits masked by 1 in stencil */
    glStencilFunc(GL_ALWAYS, 1, 1);//set one
    glAlphaFunc(GL_GREATER, threshold);//greater than half transparency

    glVertexAttribPointer(positionLoc, 3, GL_FLOAT, GL_FALSE, 0, fullVertices);
    glVertexAttribPointer(colorLoc, 4, GL_FLOAT, GL_FALSE, 0, colorVertices);
    glVertexAttribPointer(textureLoc, 2, GL_FLOAT, GL_FALSE, 0, textureVertices);
    [self drawTexture:texture2DMaskImage];
    /* second pass of the fragments less or equal than the threshold
     will pass thus will set zero bits masked by 1 in stencil */
    glStencilFunc(GL_ALWAYS, 0, 1);//set zero
    glAlphaFunc(GL_LEQUAL, threshold);//Less than Half transparency
    glVertexAttribPointer(positionLoc, 3, GL_FLOAT, GL_FALSE, 0, fullVertices);
    glVertexAttribPointer(colorLoc, 4, GL_FLOAT, GL_FALSE, 0, colorVertices);
    glVertexAttribPointer(textureLoc, 2, GL_FLOAT, GL_FALSE, 0, textureVertices);
    [self drawTexture:texture2DMaskImage];


// till here as suggested by u after this I have added to draw
// the next Image which will use the created Stencil
    glDepthMask(GL_TRUE);
    glDisable(GL_ALPHA_TEST);
    glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);
    //
    glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP);//stop further write to stencil buffer
    glStencilFunc(GL_EQUAL, 0, 1);//pass if 0
    glVertexAttribPointer(positionLoc, 3, GL_FLOAT, GL_FALSE, 0, fullVertices);
    glVertexAttribPointer(colorLoc, 4, GL_FLOAT, GL_FALSE, 0, colorVertices);
    glVertexAttribPointer(textureLoc, 2, GL_FLOAT, GL_FALSE, 0, textureVertices);
    [self drawTexture:texture2D];
    ////
    ///////



    // Validate program before drawing. This is a good check, 
    // but only really necessary in a debug build.
    // DEBUG macro must be defined in your debug configurations
    //  if that's not already the case.
#if defined(DEBUG)
    if (![self validateProgram:program])
    {
        NSLog(@"Failed to validate program: %d", program);
        return;
    }
#endif
    // draw
    [context presentRenderbuffer:GL_RENDERBUFFER];//Present
}

- (void) drawTexture:(Texture2D*)texture
{
    // handle viewing matrices
    GLfloat proj[16], modelview[16], modelviewProj[16];
    // setup projection matrix (orthographic)
    mat4f_LoadOrtho(-1.0f, 1.0f, -1.0f, 1.0f, -1.0f, 1.0f, proj);
    //   glUniformMatrix4fv(uniforms[UNIFORM_PROJECTION], 1, 0, proj);
    // setup modelview matrix (rotate around z)
    mat4f_LoadIdentity(modelview);
    mat4f_LoadZRotation(0.0, modelview);
    mat4f_LoadTranslation3D(0.0, 0.0, 0.0, modelview);
    // projection matrix * modelview matrix
    mat4f_MultiplyMat4f(proj, modelview, modelviewProj);
    glUniformMatrix4fv(_modelViewUniform, 1, 0, modelviewProj);
    // update uniform values
    glActiveTexture(GL_TEXTURE0);
    glBindTexture(GL_TEXTURE_2D, texture.name);
    glUniform1i(_textureUniform, 0);
    glDrawElements(GL_TRIANGLES, 6, GL_UNSIGNED_SHORT, cubeIndices);
}

please shed some light I am in desperate need :( thanks once again.

score 8 · Answer 1 · edited Jan 01 '22 at 11:27

8

Yes, you can use alpha testing (glEnable(GL_ALPHA_TEST); glAlphaFunc(COMPARISION, VALUE);) to discard fragments; discarded fragments will not be stencil tested, so you can use the passing fragments to build the stencil mask.

EDIT code example

/* the order in which orthogonal OpenGL state is set doesn't matter */
glEnable(GL_STENCIL_TEST);
glEnable(GL_ALPHA_TEST);

/* don't write to color or depth buffer */
glColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE);
glDepthMask(GL_FALSE);

glStencilOp(GL_KEEP, GL_KEEP, GL_REPLACE);

/* first set alpha test so that fragments greater than the threshold
   will pass thus will set nonzero bits masked by 1 in stencil */
glStencilFunc(GL_ALWAYS, 1, 1);
glAlphaFunc(GL_GREATER, threshold);
draw_textured_primitive();


/* second pass of the fragments less or equal than the threshold
   will pass thus will set zero bits masked by 1 in stencil */
glStencilFunc(GL_ALWAYS, 0, 1);
glAlphaFunc(GL_LEQUAL, threshold);
draw_textured_primitive();

Don't forget to revert OpenGL state for drawing afterwards.

EDIT to account for updated question:

Since you're actually using iOS, thus OpenGL-ES 2.0 things get a bit different. There's no alpha test in OpenGL-ES2. Instead you're expected to discard the fragment in the fragment shader, if it's below/above a chosen threshold, by using the discard keyword/statement.

edited Jan 01 '22 at 11:27

hata

11,633
6
46
69

answered Sep 27 '11 at 15:10

datenwolf

159,371
13
185
298

Thanks for ur Reply ,Ya I tried it but confused with the flow Like 1. disable depth 2. Enable Alpha test 3. set alpha test function 4. set stencil test function 5. set stencil OP (Operation). 6. Draw the texture . Right??????? but this is not giving the result as expected.What should be the StencilOp ?? and stencilFunc values ?? so that if alpha test passes then the '1' or '0' is written to stencil buffer and the rest will be as previously set to Clear the Stencil buffer. Please help thanks once again. – infiniteLoop Sep 28 '11 at 07:43
@infiniteLoop: See my code edit, which should work (I didn't test it). – datenwolf Sep 28 '11 at 08:38
Wasn't alpha test removed from ES 2.0? So you need to do it in the fragment shader using your own comparison and the `discard` keyword. – Christian Rau Oct 19 '11 at 18:58
@ChristianRau: Question title spoke of OpenGL 2.0 (no ES). However since must supply a fragment shader in OpenGL-ES2 the change becomes trivial. – datenwolf Oct 19 '11 at 20:31
@datenwolf Oh yes, it seems Brad Larson has completely changed the question to OpenGL ES and added a code sample. If he is not the same person or doesn't stand in contact with the OP, then this would be a serious misuse. – Christian Rau Oct 19 '11 at 21:00
@ChristianRau: I reverted the question to the version before. If one makes such an change/addition, then it should be expected to be clearly indicated as such. Also since the original question was about OpenGL, but the changes for OpenGL-ES2, thus covering a different topic, this qualifies for a separate question. – datenwolf Oct 19 '11 at 21:34
@Christian - You can't see it, because it was deleted (you're not yet at 10k), but the asker provided follow-on code in response to this answer in another answer. Standard policy is to edit that code into the question if it elaborates on what was being asked, and then to have the moderators delete the non-answer. I did this. In the process, I saw that this was clearly an OpenGL ES question that had been mistakenly identified as OpenGL (see the use of iOS's EAGLContext in the follow-on sample code). I did not change the core of what was being asked here, only organized things better. – Brad Larson Oct 20 '11 at 01:04
@Christian - I also looked at this user's previous questions, which all were iOS-related, so it's a safe bet that they meant OpenGL ES 2.0 here, not OpenGL 2.0. I've seen several people make this mistake before. If anything else they had written indicated desktop OpenGL, I would have left the wording / tagging alone. The follow-on code shows one attempt by the user to implement something along the lines of their original question, so it does not change the tone of the question, it merely provides more detail. – Brad Larson Oct 20 '11 at 01:13
@BradLarson: Anyway, these kind of changes should be clearly indicated. Also you may want to add a comment to given answers, since such a change, like OpenGL-2 → OpenGL-ES2 has a significant effect on the answer. – datenwolf Oct 20 '11 at 05:44

score 1 · Accepted Answer · answered Oct 20 '11 at 10:40

1

Finally I have managed to do it using conditional passing of colors using discard keyword in shader. I also used 2 textures in same shader and combined them accordingly.

Thanks all.

answered Oct 20 '11 at 10:40

infiniteLoop

2,135
1
25
29

2

Please, could you add some more details about how you combine the 2 textures? – MatterGoal Sep 07 '13 at 15:04

How to create Stencil buffer with texture (Image) in OpenGL-ES 2.0

EDIT by datenwolf to account for OPs question update in a answer:

By @InfiniteLoop:

2 Answers2

EDIT to account for updated question:

Linked