1

I have a mesh whose vertex positions are generated dynamically by the vertex shader. I've been using https://www.khronos.org/opengl/wiki/Calculating_a_Surface_Normal to calculate the surface normal for each primitive in the geometry shader, which seems to work fine.

Unfortunately, I'm planning on switching to an environment where using a geometry shader is not possible. I'm looking for alternative ways to calculate surface normals. I've considered:

  • Using compute shaders in two passes. One to generate the vertex positions, another (using the generated vertex positions) to calculate the surface normals, and then passing that data into the shader pipeline.
  • Using ARB_shader_image_load_store (or related) to write the vertex positions to a texture (in the vertex shader), which can then be read from the fragment shader. The fragment shader should be able to safely access the vertex positions (since it will only ever access the vertices used to invoke the fragment), and can then calculate the surface normal per fragment.

I believe both of these methods should work, but I'm wondering if there is a less complicated way of doing this, especially considering that this seems like a fairly common task. I'm also wondering if there are any problems with either of the ideas I've proposed, as I've had little experience with both compute shaders and image_load_store.

User12345
  • 96
  • 1
  • 6
  • You said that you wouldn't be able to use a geometry shader. What about tessellation shaders? Or are those also a no-go? Another option would be to use transform feedback; you could potentially use that in combination with buffer textures to bypass the incoherent image load/store routines if that seems desireable. – jh100 May 04 '22 at 23:46
  • 2
    Are you trying to calculate a *vertex* normal or a *face* normal? – Nicol Bolas May 05 '22 at 02:02
  • 1
    @jh100 If it can't use a geometry shader, then it can't use a tessellation shader either. – Rabbid76 May 05 '22 at 04:59
  • @NicolBolas The face normal. – User12345 May 06 '22 at 15:45
  • @jh100 I hadn't thought of transform feedback -- that's a great idea, thanks! I think I'll probably go with the partial derivative (as per Rabbid76's answer) but it's definitely worth nothing for the future, thanks. And yeah, unfortunately my only options for shaders are vertex, fragment, and compute -- no tessellation either. – User12345 May 06 '22 at 15:46

1 Answers1

2

See Diffuse light with OpenGL GLSL. If you just want the face normals, you can use the partial derivative dFdx, dFdy. Basic fragment shader that calculates the normal vector (N) in the same space as the position:

in vec3 position; 

void main()
{
    vec3 dx = dFdx(position);
    vec3 dy = dFdy(position);
    vec3 N = normalize(cross(dx, dy));
    
    // [...]
}
Rabbid76
  • 202,892
  • 27
  • 131
  • 174