0

I'd like to render concave polygons using C, OpenGL on a Linux system. Can anyone provide sample code how this would be done in the latest version of OpenGL (Let's say OpenGL 4.5 which according to glxinfo NVIDIA driver 352.41 supports, but whatever 3+ version you like that supports extensions implemented in hardware on Linux at least by NVIDIA).

This question on stackoverflow points to use GLU tesselation which is deprecated. Using stencil buffers approach which I don't fully understand, but if this is the "industry accepted standard" of doing this now, I'll put more effort into understanding.

This post refers to a .NET library which I don't want to use on Linux and C, tesselation shader which I would love an example how to use but all examples I can find start with convex patches, and the deprecated library moved for posterity into another repository which I'd like to move away from.

Thanks.

Community
  • 1
  • 1
Budric
  • 3,599
  • 8
  • 35
  • 38
  • 1
    Are you rendering isolated concave polygons, or a bunch of them that are for example part of a mesh? – Reto Koradi Sep 23 '15 at 15:11
  • OpenGL is a system for rendering triangles. That's the only primitive there is. If you want to render concave or convex polygons you have to provide the decomposition yourself. You *could* do it with a geometry shader but you have to accept the input vertices (with restrictions), decompose into triangles and output the primitives. Graphics Gems I (available online) had some algorithms in it for tessellating arbitrary polygons if I remember correctly. – Robinson Sep 23 '15 at 15:21
  • 3
    While GLU is deprecated, the tessellation code isn't too bad, and isn't actually tied to OpenGL. [libtess2](https://github.com/memononen/libtess2) is effectively the tessellation functions ripped out of Mesa's GLU implementation. – Colonel Thirty Two Sep 23 '15 at 15:21
  • Thanks, I'll give libtess2 a try. – Budric Sep 23 '15 at 15:45

1 Answers1

1

The vast, overwhelming majority of OpenGL applications simply never draws concave polygons; it's not something the API or hardware attempts directly to support and hence there's no industry standard approach. Stencil will achieve it in the frame buffer, any sort of convex decomposition will achieve it at the geometry level.

The stencil approach actually isn't that hard to understand. The metaphor is fence jumping: the edges of the polygon are a fence and the polygon itself is enclosed land. To determine whether a point is inside the land, start anywhere outside and walk to it in a straight line. Count how many times you had to jump a fence to get there. Every time you jump a fence you switch from inside to outside or vice versa. So if you jumped an even number of fences then the point must be inside (e.g. you went from outside to inside once, or -> inside -> outside -> inside, or -> inside -> outside -> inside -> outside -> inside, etc). If you jumped an odd number of fences then it must be outside.

The stencil approach keeps a per-pixel count of fence jumps.

It draws a triangle fan relative to a fixed point that is a vertex of the polygon. So it's one triangle per fence, with the third coordinate being a fixed position you know to be inside.

The pixels inside the triangle are every point that, in order for you to have reached on your straight line towards the fixed point, would require you to have jumped that fence.

Therefore the stencil total at any location is the number of fences you would need to jump to reach the polygon from there.

So all odd stencil results are points that are inside. All even results are points that are outside.

Tommy
  • 99,986
  • 12
  • 185
  • 204