Mesh color is messed up probably due to bad vertex normal computation

Question

I have vertex position and index and I want vertex normal:

// input
vector<Vec3f> points = ... // position
vector<Vec3i> facets = ... // index (triangles)

// output
vector<Vec3f> norms; // normal

Method 1

I compute normal like this:

    norms.resize(points.size()); // for each vertex there is a normal

    for (Vec3i f : facets) {
        int i0 = f.x();
        int i1 = f.y(); // index
        int i2 = f.z();
        Vec3d pos0 = points.at(i0);
        Vec3d pos1 = points.at(i1); // position
        Vec3d pos2 = points.at(i2);

        Vec3d N = triangleNormal(pos0, pos1, pos2); // face/triangle normal

        norms[i0] = N;
        norms[i1] = N; // Use the same normal for all 3 vertices
        norms[i2] = N;
    }

Then, the output mesh is rendered like this with a Phong material:

Method 1 with reversed normal

When I reverse normal direction in method 1:

        norms[i0] = -N;
        norms[i1] = -N;
        norms[i2] = -N;

The dark and light regions are swapped:

The same happens by swapping position 0 with position 1 by:

        // Vec3d N = triangleNormal(pos0, pos1, pos2);
        Vec3d N = triangleNormal(pos1, pos0, pos2); // Swap pos0 with pos1

Method 2

I compute the normal by this method:

    // Count how many faces/triangles a vertex is shared by
    vector<int> counters;
    counters.resize(points.size());

    norms.resize(points.size());
    for (Vec3i f : facets) {
        int i0 = f.x();
        int i1 = f.y(); // index
        int i2 = f.z();
        Vec3d pos0 = points.at(i0);
        Vec3d pos1 = points.at(i1); // position
        Vec3d pos2 = points.at(i2);
        
        Vec3d N = triangleNormal(pos0, pos1, pos2);

        // Must be normalized
        // https://stackoverflow.com/a/21930058/3405291
        N.normalize();

        norms[i0] += N;
        norms[i1] += N; // add normal to all vertices used in face
        norms[i2] += N;

        counters[i0]++;
        counters[i1]++; // increment count for all vertices used in face
        counters[i2]++;

    }

    // https://stackoverflow.com/a/21930058/3405291
    for (int i = 0; i < static_cast<int>(norms.size()); ++i) {
        if (counters[i] > 0)
            norms[i] /= counters[i];
        else
            norms[i].normalize();
    }

This method yields a totally dark final render by a Phong material:

I also tried methods suggested here and there which are similar to method 2. They all result in a final render which looks like that of method 2 i.e. all dark regions without any light one.

Method 2 with reversed normal

I used method 2, but at the end, I reversed the normal direction by:

    for (Vec3d & n : norms) {
        n = -n;
    }

To my surprise, the final render is all darK:

Also in method 2, I tried swapping position 0 with position 1:

        // Vec3d N = triangleNormal(pos0, pos1, pos2);
        Vec3d N = triangleNormal(pos1, pos0, pos2); // swap pos0 with pos1

The final render is all dark regions without any light ones.

How?

Any idea how I can get my final render to be all light without any dark region?

btw is your `triangleNormal` working properly (assuming its using cross product to obtain the normal) and or are you passing 3D position into them? as your cylinders has the same color along circumference which hints they might not be 3D or correct — Spektre, Oct 14 '20 at 14:42
@Spektre `Vec3d MyClass::triangleNormal(Vec3d pos0, Vec3d pos1, Vec3d pos2) { Vec3d V = pos1 - pos0; Vec3d W = pos2 - pos0; Vec3d N = V.cross(W); return N; }` — Megidd, Oct 14 '20 at 15:24
That looks ok but I would add normalization so the normal is unit vector ... right now the returned vector has magnitude equal to double of the triangle's area which might also mess your lighting computations if not handled — Spektre, Oct 14 '20 at 17:10

Spektre · Accepted Answer · 2020-10-14T10:16:58.733

That looks like your mesh does not have consistent winding rule. So some triangles/faces are defined CW other in CCW order of vertexes causing that some of your normals are facing in opposite direction. There are few things you can do to remedy:

use double sided normals lighting

this is easiest... somwhere in fragment or wherever you are computing the shading something like this:
```
out_color = face_color*(ambient_light+diffuse_light*max(0.0,dot(face_normal,light_direction)));
```
when the normal is in wrong direction the result of dot is negative leading to dark color so just use abs value instead:
```
out_color = face_color*(ambient_light+diffuse_light*abs(dot(face_normal,light_direction)));
```
In fixed function pipeline there is even switch for this IIRC:
```
glLightModeli(GL_LIGHT_MODEL_TWO_SIDE, GL_TRUE);
```
repair mesh winding

there must be 3D tools to do this (Blender,3DS,...) or if your mesh is generated on the fly you could update your code to create consistent winding on your own.

Correct winding enables you the use of GL_CULL_FACE which speeds up rendering considerably. Also it enables more advanced stuff like this:
- OpenGL - How to create Order Independent transparency?
repair normals

In some cases there are ways to detect if the normal is pointing outwards or inwards to mesh for example like this:
- Determing the direction of face normals consistently?
So just negate the wrong ones during computation of normal and that is it. However if your mesh is too complicated (too far from convex) is this not so easily done as you need to use local "centers" of mesh or even inside polygon tests which are expensive.

The averaging method of generating normals gives you dark colors for both directions of normals which means you wrongly computed them and they are most likely zero. For more info about such approach see:

How to achieve smooth tangent space normals?

Anyway to debug problems like this its best to render your normals as lines going from the vertexes of your mesh (use wireframe). Then you would see directly what normals are good and bad. Here example:

normals

My problem is not fixed yet, but your post just makes sense =) — Megidd, Oct 14 '20 at 15:26