0

I'm looking for a way to mask my entire viewport using a texture. In my case I would like to use a checkered black and white pattern (or any 2 colors) and only show the parts that are black on the scene.

Would the best way to do this be with a cliping mask, a fragment shaders, or an alpha blending. I've seen on SO this post: How to create Stencil buffer with texture (Image) in OpenGL-ES 2.0 which seems similar to what I need, but I don't completely understand what to do with the discard keyword. Would it apply to my situation.

Community
  • 1
  • 1
PPR
  • 39
  • 5

1 Answers1

0

Let's assume you have a checkered texture of black and white squares. First, you'll want to setup the stencil test to draw the mask:

glEnable(GL_STENCIL_TEST);
glDisable(GL_DEPTH_TEST);
glColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE);
glDepthMask(GL_FALSE);

glStencilFunc(GL_ALWAYS, 1, -1);
glStencilOp(GL_KEEP, GL_KEEP, GL_REPLACE);

Next draw the checkered texture. The key step here (where discard comes in), is that you set up your fragment shader to only draw fragments where the checkerboard is black (or flip it for white if you prefer). The discard keyword skips rendering of any pixels that don't match your criteria.

//in the fragment shader
if(sampleColor.r > 0.5) { discard; }

After this rendering step you will have a stencil buffer with a checkerboard image where half of the buffer has a stencil value of 0 and the other half has a stencil value of 1.

Then render normally with stencil test enabled to pass when the stencil buffer is == 1.

glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP);
glStencilFunc(GL_EQUAL, 1, -1);
Tim
  • 35,413
  • 11
  • 95
  • 121
  • ok, but if I want my texture to be a rectangle that is in front of the screen without it being affected by the different transformations done to the modelview matrix (rotate translate). Would I need a second viewport that would always stay on top, and then affect the stencil based on that viewport so it affects both? if that makes any sense at all... – PPR Jul 06 '12 at 20:39
  • I don't think you need to do anything with the viewport. If you don't want your texture to be effected by the current MVP matrix, then just don't multiply it by the matrix in your vertex shader. @PPR – Tim Jul 06 '12 at 20:51