I have a list of faces [index], that a list of vertices and normals are mapped to. I generate offsets at each vertex to produce terrain/variation, but by doing so the vertice normals stop working like they should. This is what I do to make the shading work again, namely try to redo the vertice normals (vn);
for (auto &x : normals)x = vec3(0); //zero normals first
vec3 facenormal; //buffer
for (size_t i = 0; i<indices.size();) //iterate 3 points per face
{
//find the face normal (b-a x c-a)
facenormal = cross(
(shape[indices[i + 1]] - shape[indices[i]]),
(shape[indices[i + 2]] - shape[indices[i]])
);
//add this face normal, to each of the 3 vn slots nearby
normals[indices[i++]] += facenormal; //note +=
normals[indices[i++]] += facenormal;
normals[indices[i++]] += facenormal;
}
for (auto &x : normals)x = normalize(x); //then normalize them
According to this reply it should do the trick. But, something is wrong with the approach.
It causes artifacting like shown on in the lower half of the image, while the shading mostly seems to work.
The question is: how should I calculate the vn's, to avoid these lines?
Remaking the parser: The way I was parsing (loading) the model was taking out duplicate index-values, which can save reference faces that I have to pass to the GPU, but will cause some of the values to stack/be reused. Atleast I suspect this was the problem, since it seems alot cleaner now. The only problem however is I'm getting flat shading all of a sudden. The flat shading makes it hard to see if the problem is really gone too, but I suspect it is. Why the new parsing produces flat shading after recalculation is beyond me. But these are the results im looking at.
I still have to find out the correct formula to calculate vertex normals? I dont understand why the shading was round before, but now its flat, all because I stopped stacking 1/4th of the index values?
Finding the right solution, finally! As NicoSchertler pointed out, the vertex normals were not of the base mesh. The way I load the vertices/uv/normals to fit the index for the glDrawElementsBaseVertex render-call, means cycling through the vertices and normals as they are loaded, does not refer to the base, but copies designed to fit indexing for uv+normals.
Soo, what I end up getting is weird artifacting and flat shading, as the values I'm modifying are post-parsing. Making every face not-unique does not solve anything, but clarify part of the problem, namely that I needed to modify the base index-values, not the loaded index ones.
After I recalculate for the base mesh (pre indexing) I get this result; smooth shading and visible differences for the mountainsides. The model is higher poly (and serialized in binary) and the coloring is still in the its early stages. It is however, shadily correct.