GLSL normal vector transformation

Question

In this glsl shader sample some vertex normal converted to the view space by transform: vec4(vertNormal,1.0)).xyz Why we need to do this kind of the transformation?

#version 430
layout (location=0) in vec3 vertPos;
layout (location=1) in vec3 vertNormal;

void main(void)
{ 
...


// convert vertex position to view space
vec4 P = mv_matrix * vec4(vertPos,1.0); // looks good

// convert normal to view space
vec3 N = normalize((norm_matrix * vec4(vertNormal,1.0)).xyz); // why to vec4 and back to vec3?
...
}

Answer 1

The model view matrix looks like this:

( X-axis.x, X-axis.y, X-axis.z, 0 )
( Y-axis.x, Y-axis.y, Y-axis.z, 0 )
( Z-axis.x, Z-axis.y, Z-axis.z, 0 )
( trans.x,  trans.y,  trans.z,  1 )

The upper let 3*3 is contains the orientation and the scale. The 4th row contains the translation.
While for the transformation of a point the full matrix has to take into account, for the transformation of a direction vector only the orientation is of interest.

In general the normal matrix is a mat3 and the inverse, transposed of the upper left 3*3 of the model view matrix. See

• Why is the transposed inverse of the model view matrix used to transform the normal vectors?
• Why transforming normals with the transpose of the inverse of the modelview matrix?

 mat3 norm_matrix;

 vec3 = normalize(norm_matrix * vertNormal);

The normal matrix can be calculated from the model view matrix:

mat4 mv_matrix;

mat3 norm_matrix = transpose(inverse(mat3(mv_matrix));
vec3 N = normalize(norm_matrix * vertNormal);

If the model view matrix is an Orthogonal matrix then the inverse, transpose can be skipped, because the inverse matrix is equal to the transposed matrix.
See In which cases is the inverse matrix equal to the transpose?

vec3 N = normalize(mat3(mv_matrix)* vertNormal);

If you want to do the calculations in view space, then you have to transform the vertex coordinate from model space to view space:

vec4 P = mv_matrix * vec4(vertPos,1.0);

and you have to transform the direction of the normal vector form model space to view space:

vec3 N = normalize(norm_matrix * vertNormal);