OpenGL, geometry shader with orthographic projection

Question

This is a simple program to draw anti aliased lines. The program works, I have only a problem if I add a projection. I have to use this program in another context, I have to use there an orthographic projection. I think I have to replace the parameter "win_scale" in geometry shader, but I don't exactly how I have to scale the result or how I have to rewrite the function if I'm using a projection. Here are the shaders. Vertex Shader:

#version 330 core
layout (location = 0) in vec2 position;
layout (location = 1) in vec3 color;

uniform mat4 model;
//uniform mat4 view;
uniform mat4 projection;

out VS_OUT {
    vec3 color;
} vs_out;

void main() {
    gl_Position = projection * vec4(position.x, position.y, 0.0f, 1.0f);
    vs_out.color = color;
}

Geometry shader

#version 330 core
layout (points) in;
layout (triangle_strip, max_vertices = 4) out;

uniform vec2 target;

uniform float thickness;
uniform vec2 win_scale; //window width and height

in VS_OUT {
    vec3 color;
} gs_in[];

out vec3 fColor;

vec2 screen_space(vec4 vertex) {
    return vec2(vertex.xy/vertex.w) * win_scale;

}

void main() {
    fColor = gs_in[0].color;
    vec2 p0 = screen_space(gl_in[0].gl_Position); // gs_in[0] since there's only one input vertex
    vec2 p1 = target * win_scale;

    // determine the direction of each of the  segments
    vec2 v0 = normalize(p1-p0);
    // determine the normal of each of the segments
    vec2 n0 = vec2(-v0.y, v0.x);


    gl_Position = vec4((p0 - thickness * n0)/win_scale, 0.0, 1.0);
    EmitVertex();
    gl_Position = vec4((p0 + thickness * n0)/win_scale, 0.0, 1.0);
    EmitVertex();
    gl_Position = vec4((p1 - thickness * n0)/win_scale, 0.0, 1.0);
    EmitVertex();
    gl_Position = vec4((p1 + thickness * n0)/win_scale, 0.0, 1.0);
    EmitVertex();        

    EndPrimitive();
}

And in the main I'm passing the projection values in the following ways,:

glm::mat4 proj = glm::ortho(0.f, (float)width, 0.f, (float)height, -1.f, 1.f);
glUniformMatrix4fv(glGetUniformLocation(shader.Program, "projection"), 1, GL_FALSE, glm::value_ptr(proj));

Of course, if someone has suggestions about how can I implement better antialiased lines, these are welcome.

Answer 1

You should pass the vertex position to the geometry shader

out VS_OUT {
    vec2 pos;
    vec3 color;
} vs_out;

void main()
{
    vs.out.pos = position;

    .....
}

Then you can do the calculations in the geometry shader and use the projection matrix in the geometry shader the same way as you do it in the vertex shader:

in VS_OUT {
    vec2 pos;
    vec3 color;
} gs_in[];


uniform mat4 projection;
uniform vec2 win_scale; // compensation for the view matrix 

void main()
{
    .....

    vec2 p0 = gs_in[0].pos
    vec2 p1 = target;

    // determine the direction of each of the  segments
    vec2 v0 = normalize(p1-p0);
    // determine the normal of each of the segments
    vec2 n0 = vec2(-v0.y, v0.x);

    ....

    gl_Position = projection * vec4((p0 - thickness*n0)/win_scale, 0.0, 1.0); EmitVertex();  
    gl_Position = projection * vec4((p0 + thickness*n0)/win_scale, 0.0, 1.0); EmitVertex();  
    gl_Position = projection * vec4((p1 - thickness*n0)/win_scale, 0.0, 1.0); EmitVertex();  
    gl_Position = projection * vec4((p1 + thickness*n0)/win_scale, 0.0, 1.0); EmitVertex();

    ....
}

Note, the common solution would be to draw GL_LINES, GL_LINE_STRIP, or GL_LINE_LIST and to use a geometry shader with the input layout lines:

#version 330 core
layout (lines) in;
layout (triangle_strip, max_vertices = 4) out;

uniform float thickness;
uniform mat4 projection;
uniform vec2 win_scale; // compensation for the view matrix 

in VS_OUT {
    vec2 pos;
    vec3 color;
} gs_in[];

out vec3 fColor;

void main() {
    fColor = gs_in[0].color;

    vec2 p0 = gs_in[0].pos;
    vec2 p1 = gs_in[1].pos;

    vec2 v0 = normalize(p1-p0);
    vec2 n0 = vec2(-v0.y, v0.x);

    gl_Position = projection * vec4((p0 - thickness*n0)/win_scale, 0.0, 1.0); EmitVertex();  
    gl_Position = projection * vec4((p0 + thickness*n0)/win_scale, 0.0, 1.0); EmitVertex();  
    gl_Position = projection * vec4((p1 - thickness*n0)/win_scale, 0.0, 1.0); EmitVertex();  
    gl_Position = projection * vec4((p1 + thickness*n0)/win_scale, 0.0, 1.0); EmitVertex();  

    EndPrimitive();
}

See further Khronos OpenGL wiki - Geometry Shader:

Note, in common in a rendering, each object of the scene usually is transformed by the model matrix, the view matrix and the projection matrix.

• Projection matrix:
  The projection matrix describes the mapping from 3D points of a scene, to 2D points of the viewport. The projection matrix transforms from view space to the clip space, and the coordinates in the clip space are transformed to the normalized device coordinates (NDC) in the range (-1, -1, -1) to (1, 1, 1) by dividing with the w component of the clip coordinates.

• View matrix:
  The view matrix describes the direction and position from which the scene is looked at. The view matrix transforms from the wolrd space to the view (eye) space. In the coordinat system on the viewport, the X-axis points to the left, the Y-axis up and the Z-axis out of the view (Note in a right hand system the Z-Axis is the cross product of the X-Axis and the Y-Axis).

• Model matrix:
  The model matrix defines the location, oriantation and the relative size of a mesh in the scene. The model matrix transforms the vertex positions from of the mesh to the world space.

This means a transformation of a vertex to clip space works like this:

gl_Position = projection * view * model * vec4( pos.xyz, 1.0 );

Since you have a 2D scene with and a orthographic projection, you skip the transformation with the model matrix and the view matrix. In your case you compensate this, by dividing by win_scale:

gl_Position = projection * vec4( pos.xy/win_scale, 0.0, 1.0 ); 

See further OpenGL - Mouse coordinates to Space coordinates: