2

I'm doing per-pixel lighting(phong shading) on my terrain. I'm using a heightmap to generate the terrain height and then calculating the normal for each vertex. The normals are interpolated in the fragment shader and also normalized.

I am getting some weird dark lines near the edges of triangles where there shouldn't be. https://i.stack.imgur.com/OP9gF.jpg

I checked if the normals were correct using a geometry shader to draw the normals on the terrain and they seem to be correct. https://i.stack.imgur.com/xLNNR.jpg

There is no point using a normal map for the terrain it will just give pretty much the same normals. The problem lies with the way the normals are interpolated across a triangle.

I am out of idea's how to solve this. I couldn't find any working solution online.

Terrain Vertex Shader:

#version 330 core

layout (location = 0) in vec3 position;
layout (location = 1) in vec3 normal;
layout (location = 2) in vec2 textureCoords;

out vec2 pass_textureCoords;
out vec3 surfaceNormal;
out vec3 toLightVector;
out float visibility;

uniform mat4 transformationMatrix;
uniform mat4 viewMatrix;
uniform mat4 projectionMatrix;

uniform vec3 lightPosition;

const float density = 0.0035;
const float gradient = 5.0;

void main()
{
    vec4 worldPosition = transformationMatrix * vec4(position, 1.0f);
    vec4 positionRelativeToCam = viewMatrix * worldPosition;

    gl_Position = projectionMatrix * positionRelativeToCam;
    pass_textureCoords = textureCoords;

    surfaceNormal = (transformationMatrix * vec4(normal, 0.0f)).xyz;
    toLightVector = lightPosition - worldPosition.xyz;

    float distance = length(positionRelativeToCam.xyz);
    visibility = exp(-pow((distance * density), gradient));
    visibility = clamp(visibility, 0.0, 1.0);
}

Terrain Fragment Shader:

#version 330 core

in vec2 pass_textureCoords;
in vec3 surfaceNormal;
in vec3 toLightVector;
in float visibility;

out vec4 colour;

uniform vec3 lightColour;
uniform vec3 fogColour;

uniform sampler2DArray blendMap;
uniform sampler2DArray diffuseMap;

void main()
{
    vec4 blendMapColour = texture(blendMap, vec3(pass_textureCoords, 0));

    float backTextureAmount = 1 - (blendMapColour.r + blendMapColour.g + blendMapColour.b);
    vec2 tiledCoords = pass_textureCoords * 255.0;
    vec4 backgroundTextureColour = texture(diffuseMap, vec3(tiledCoords, 0)) * backTextureAmount;
    vec4 rTextureColour = texture(diffuseMap, vec3(tiledCoords, 1)) * blendMapColour.r;
    vec4 gTextureColour = texture(diffuseMap, vec3(tiledCoords, 2)) * blendMapColour.g;
    vec4 bTextureColour = texture(diffuseMap, vec3(tiledCoords, 3)) * blendMapColour.b;

    vec4 diffuseColour = backgroundTextureColour + rTextureColour + gTextureColour + bTextureColour;

    vec3 unitSurfaceNormal = normalize(surfaceNormal);
    vec3 unitToLightVector = normalize(toLightVector);

    float brightness = dot(unitSurfaceNormal, unitToLightVector);
    float ambient = 0.2;
    brightness = max(brightness, ambient);
    vec3 diffuse = brightness * lightColour;

    colour = vec4(diffuse, 1.0) * diffuseColour;
    colour = mix(vec4(fogColour, 1.0), colour, visibility);
}
Simon Mathay
  • 141
  • 1
  • 7
  • The way my terrain tiles are triangulated is the following: http://imgur.com/t8DOkTJ I tried another triangulation method but i just got different dark lines near the edges of the triangles. The effect I'm looking for on my terrain is like this: http://imgur.com/9Jx2g24 This is from vanilla wow. – Simon Mathay Feb 24 '16 at 15:29
  • Please supply two more images: geometry wireframe + normals encoded into color from the pixel shader. This should bring us closer to the solution. – mrVoid Feb 24 '16 at 17:21
  • I'm sry mrVoid what do you mean? The geometry shader and fragment shader code for drawing the normal vectors? – Simon Mathay Feb 24 '16 at 21:28
  • He means to turn the normals of the vertices into the color component by using `surfaceNormal` as the `color` value. – Colin Basnett Feb 24 '16 at 21:42
  • I used this code to visualize the normal vectors: http://learnopengl.com/#!Advanced-OpenGL/Geometry-Shader Scroll down to "Visualizing normal vectors" – Simon Mathay Feb 24 '16 at 21:46
  • I suspect the problem is in the normals not being calculated correctly. That is why I wanted to see them color encoded. Ussually you want to have weighted average from adjacent faces to be in conformance with chosen tirangulation scheme. – mrVoid Feb 25 '16 at 09:19
  • I use this method: http://stackoverflow.com/questions/13983189/opengl-how-to-calculate-normals-in-a-terrain-height-grid To calculate the normals, I even used another method with cross products and i get exact same result, I wouldn't post this question if I wasn't sure my normals were correct. – Simon Mathay Feb 25 '16 at 12:08

2 Answers2

0

This can be either two issues :

1. Incorrect normals : There is different types of shading : Flat shading, Gouraud shading and Phong shading (different of Phong specular) example :

You usually want to do a Phong shading. To do that, OpenGL make your life easier and interpolate for you the normals between each vertex of each triangle, so at each pixel you have the correct normal for this point: but you still need to feed it proper normal values, that are the average of the normals of every triangles attached to this vertex. So in your function that create the vertex, the normals and the UVs, you need to compute the normal at each vertex by averaging every triangle normal attached to this vertex. illustration

2. Subdivision problem : The other possible issue is that your terrain is not subdivided enough, or your heightmap resolution is too low, resulting to this kind of glitch because of the difference of height between two vertex in one triangle (so between two pixels in your heightmap).

Maybe if you can provide some of your code and shaders, maybe even the heightmap so we can pin exactly what is happening in your case.

jubertal
  • 41
  • 6
  • Yes I am doing Phong shading, however I'm not using any specular lighting yet, I'm doing that later it isn't hard. To calculate the normal vector's I am using this method: http://stackoverflow.com/questions/13983189/opengl-how-to-calculate-normals-in-a-terrain-height-grid – Simon Mathay Feb 24 '16 at 21:10
  • I'm using a 256x256 heightmap: http://imgur.com/GHm9GbD One pixel on the heightmap represents one vertex on the terrain so my terrain has 256x256 vertices. I have a tile size of 3.0 so the terrain width and height will be 256 x 3 – Simon Mathay Feb 24 '16 at 21:18
  • I defined a max height value of 50, so the highest point (completely white) is 50 and lowest point (completely black) is -50. Which means that the difference in height of one pixel color is 0.39. So the height of a pixel with color(128, 128, 128) is 0 and (129,129,129) is 0.39 – Simon Mathay Feb 24 '16 at 21:35
  • By the way this method for calculating the normals: http://stackoverflow.com/questions/13983189/opengl-how-to-calculate-normals-in-a-terrain-height-grid They switched the y with z axis just so u know ... might be confusing at first, so basically i'm using: vertices[i].normal = glm::normalize(glm::vec3(heightL - heightR, 2.0f, heightB - heightF)); – Simon Mathay Feb 24 '16 at 21:57
  • Ok, maybe try another heightmap with a higher dimension (1024x1024) to see if you still have the same problem. – jubertal Feb 25 '16 at 00:18
  • And reduce tile size I imagine? Or else using bigger heightmap will just make terrain bigger. Ok will try. – Simon Mathay Feb 25 '16 at 01:15
  • Still got same problem, but with this heightmap 1024x1024 i just get more difference of height between one pixel. http://imgur.com/JJLjWiz – Simon Mathay Feb 25 '16 at 14:18
0

This is old, but I suspect you're not transforming your normal using the transposed inverse of the upper 3x3 part of your modelview matrix. See this. Not sure what's in "transformationMatrix", but if you're using it to transform the vertex and the normal something is probably fishy...

Bim
  • 1,008
  • 1
  • 10
  • 29