5

Me and a friend have been having an ongoing argument about the stencil buffer. In short I haven't been able to find a situation where the stencil buffer would provide any advantage over the programmable pipeline tools in OpenGL 3.2+. Are there any uses to the stencil buffer in modern OpenGL?

[EDIT]

Thanks everyone for all the inputs on the subject.

claudehenry
  • 736
  • 8
  • 17
  • 2
    [Early fragment test](https://www.opengl.org/wiki/Early_Fragment_Test) to avoid running expensive fragment shaders? – genpfault Aug 10 '16 at 22:41
  • I thought this link is related http://gamedev.stackexchange.com/questions/105667/stencil-buffer-vs-conditional-discard-in-fragment-shader – Harish Aug 11 '16 at 00:26
  • 2
    It's just as useful as it ever was, unrelated to whether you use the programmable pipeline or not. Stencil tests are applied as part of the fixed function per-fragment operations, after the fragment shader executes. That part of the rendering pipeline is still not programmable. – Reto Koradi Aug 11 '16 at 02:35

3 Answers3

14

It is more useful than ever since you can sample stencil index textures from fragment shaders. It should not even be argued that the stencil buffer is not part of the programmable pipeline.

The depth buffer is used for simple pass/fail fragment rejection, which the stencil buffer can also do as suggested in comments. However, the stencil buffer can also accumulate information about test results over multiple passes. All sorts of logic and counting applications exist such as measuring a scene's depth complexity, constructive solid geometry, etc.

Andon M. Coleman
  • 42,359
  • 2
  • 81
  • 106
10

To add a recent example to Andon's answer, GTA V uses the stencil buffer kinda like an ID buffer to mark the player character, cars, vegetation etc.

It subsequently uses the stencil buffer to e.g. apply subsurface scattering only to the character or exclude him from motion blur.

See the GTA V Graphics Study (highly recommended, it's a great read!)

Edit: sure you can do this in software. But you can do rasterization or tessellation in software just as well... In the end it's about performance I guess. With depth24stencil8 you have a nice hardware-supported format, and the stencil test is most likely faster then doing discards in the fragment shader.

karyon
  • 1,957
  • 1
  • 11
  • 16
  • thanks for the study, its a very interesting read and one i hadn't seen before. Ill leave this open for a few more days in order to gather more ideas and opinions but this is a very solid answer. thanks. – claudehenry Aug 12 '16 at 04:54
  • 1
    @claudehenry: There's an interesting technique for order independent transparency called [stencil-routed A-buffering](http://www.humus.name/index.php?page=3D&ID=76) that uses the stencil buffer to defer sorting fragments until final resolve in a multi-sampled framebuffer. _This is something you cannot do in software that complements karyon's example of assigning identities to pixels._ – Andon M. Coleman Aug 12 '16 at 09:23
-1

Just to provide one other use case, shadow volumes (aka "stencil shadows") are still very relevant: https://en.wikipedia.org/wiki/Shadow_volume

They're useful for indoor scenes where shadows are supposed to be pixel perfect, and you're less likely to have alpha-tested foliage messing up the extruded shadow volumes.

It's true that shadow maps are more common, but I suspect that stencil shadows will have a comeback once the brain dead Createive/3DLabs patent expires on the zfail method.

fieldtensor
  • 3,972
  • 4
  • 27
  • 43
  • 3
    It's not the patent -- it's the excessive fill-rate they require. Shadow maps simply scale better, and crisp shadows aren't photo-realistic anyway. – Yakov Galka May 15 '17 at 08:45