What is wrong with my normal calculations?

Question

I attempted to use the algorithm described at OpenGL - How to calculate normals in a terrain height grid?. I popped it in my vertex shader and using some perlin noise functions to test it, and it worked like a charm, however when I ported it to Java, it didn't work so well.

float nx = 0;
float ny = 0;
float nz = 0;
Vector3 P = new Vector3(vpx,vpy,vpz);
Vector3 off = new Vector3(1,1,0);
float hL = tryGetHeight(_mempoints2,P.x - off.x,P.z - off.z);
float hR = tryGetHeight(_mempoints2,P.x + off.x,P.z + off.z);
float hD = tryGetHeight(_mempoints2,P.x - off.z,P.z - off.y);
float hU = tryGetHeight(_mempoints2,P.x + off.z,P.z + off.y);
nx = hL - hR;
ny = hD - hU;
nz = 2.0f;
Vector3 v = new Vector3(nx,ny,nz);
v = v.nor();
nx = v.x;
ny = v.y;
nz = v.z;

The results with the algorithm in the vertex shader:

vertex shader

The results with the algorithm in my buffer setup:

buffer setup

(Sorry about the blur, I was testing some depth of field stuff when I snapped these.)

You do not evaluate at small displacements of P.xy but of of P.xz, thus you build that evaluation patch not along the tangent plane, thus its normal is not the surface normal searched for. — Solkar, Jun 24 '14 at 21:01

score 0 · Answer 1 · answered Jun 24 '14 at 21:14

0

The results with the algorithm in the vertex shader:

Normals should not be calculated in any shader stage. A vertex shader doesn't have the information about neighboring vertices, the geometry shader is a performance hog, tesselation shaders could deliver normals only for the local tesselation. And doing screen space normal calculations in the fragment shader is just weird. Precalculate the normals on the CPU and pass them as another vertex attribute.

v = v.nor();

The nor function does not what you think it does. It normalizes the vector to unit length. (vector normalization =/= surface normal calculation). To calculate a normal you must calculate the cross product of the local slope vectors.

answered Jun 24 '14 at 21:14

datenwolf

159,371
13
185
298

I agree with you on the comment in regards to the vertex shader. Do you have any resources I could reference that you favor that could help explain the normal calculation further? – seesharper Jun 24 '14 at 21:24
@seesharper: This should be a good primer: http://stackoverflow.com/a/6661242/524368 – then I recommend you to get an undergraduate textbook on linear algebra. You must become comfortable with using vector spaces and the operations on them. When it comes to mathematics for computer graphics, there are so many books on it, that I completely lost track which are good ones and which are bad ones. The definitive text-to-have as a graphics coder is "Computer Graphics – Principle and Practice" by Foley et al. It got a new edition released in 2009 (which I yet have to buy). – datenwolf Jun 24 '14 at 22:18
You don't need the neighboring vertices for the cited glsl method. you evaluate the height map in a small calculated topological neighborhood of P. In terms of efficiency it's advisable to bake that result once into a normal map and reuse that map. – Solkar Jun 24 '14 at 22:25
I don't think this is shader code -- this is client code (see my answer). – wcochran Jun 25 '14 at 20:56
@wcochran: OP explicitly mentioned calculations being done in the vertex shader. I QFT-ed that part in my answer at the very beginning. – datenwolf Jun 26 '14 at 08:36
@datenwolf I think the OP tried to mimic some shader code in Java.. it certainly isn't GLSL... – wcochran Jun 26 '14 at 13:29
@wcochran: I was referring to this "Normals should not be calculated in any shader stage. A vertex shader doesn't have the information about neighboring vertices, " part of datenwolf's answer and the glsl method at http://stackoverflow.com/questions/13983189/opengl-how-to-calculate-normals-in-a-terrain-height-grid – Solkar Jun 26 '14 at 14:18
@Solkar: Well in that case the vertex shader doesn't really gets the information from vertex attributes (not for the heightmap displancement, nor for the neighbors) but from a texture that provides the heightmap. Yes, in that case the vertex shader does have the full information. However this mandates 5 texture fetches for a single vertex; precalculating the normal and placing it into a "surface normal" texture would reduce this to 2 texture fetches. Anyway, if you do the math, insert the heights into the differential vectors and do the cross product you'll get *exactly* the same result. – datenwolf Jun 26 '14 at 15:01
It's undisputed that one can calculate and provide normals by different means; point is, that explicit vertex adjacency information is not needed for the given heightmap based approach, so this "A vertex shader doesn't have the information about neighboring vertices" of your is true, but can be left out of ansatz. One more word about efficiency - for an once-an-for-all static heightmap I wouldn't spend much effort on optimization. But for dynamic heightmaps , or in situations in which on get's a height map for other reasons cheaply, this alternative approach is very well to be considered. – Solkar Jun 26 '14 at 19:20

wcochran · Answer 2 · 2014-06-26T03:52:49.047

You are on the right track. The cross product you are attempting to compute is

   (1, 0, (R-L)/2)
 x (0, 1, (U-D)/2)
 -----------------
(-(R-L)/2, -(U-D)/2, 1)

which is has the same direction as the vector

(L-R,D-U,2)

When normalized you have

N = norm(L-R,D-U,2)

Your left, right, up, and down surface heights are

L = height(P.x - 1, P.y)
R = height(P.x + 1, P.y)
U = height(P.x, P.y + 1)
D = height(P.x, P.y - 1)

If you look you have some signs wrong and mismatching subscripts.

What is wrong with my normal calculations?

2 Answers2