Implementing RayPicking in a Fragment Shader

Question

I am having trouble implementing RayPicking in a Fragment-Shader, I understand I must start from my mouse coordinates, but I am not sure what to multiply my origin with.

I have tried creating a 3 component vector, with x and y as my mouse coordinates divided by my resolution, and in z I have tried using my p(for point in space, calculated as rayOrigin + rayDirection * t) with no luck. Here is a Shadertoy that tries what I am looking for.

float ray( vec3 ro, vec3 rd, out float d )
{

   float t = 0.0; d = 0.0;

   for( int i = 0; i < STEPS; ++i )
   {

       vec3 p = ro + rd * t;
       d = map( p );

       if( d < EPS || t > FAR ) break;

       t += d;

    }

    return t;

}

vec3 shad( vec3 ro, vec3 rd, vec2 uv )
{

    float t = 0.0, d = 0.0;
    t = ray( ro, rd, d );

    float x = ( 2.0 * iMouse.x ) / iResolution.x - 1.0;
    float y = 1.0 - ( 2.0 * iMouse.y ) / iResolution.y;
    float z = 1.0;

    vec3 p = ro + rd * t;
    vec3 n = nor( p );
    vec3 lig = ( vec3( x, -y, z ) );
    lig += ro + rd;
    lig = normalize( lig );
    vec3 ref = reflect( rd, n );

    float amb = 0.5 + 0.5 * n.y;
    float dif = max( 0.0, dot( n, lig ) );
    float spe = pow( clamp( dot( ref, lig ), 0.0, 1.0 ), 16.0 );

    vec3 col = vec3( 0 );

    col += 0.1 * amb;
    col += 0.2 * dif;
    col += spe;

    return col;

}

I expect to get a light that moves as if I was shooting a ray from my mouse coordinates to the SDF.

what does ray picking to do with dynamic light controlled by mouse? take a look at this: [OpenGL 3D-raypicking with high poly meshes](https://stackoverflow.com/a/51764105/2521214) The result of ray picking is the index of selected object and world coordinate of "clicked" point on it. The linked QA do it in old style GL without shaders however , in GLSL its easier you can even directly use linear depth ... the problem is what for you want to use the result as you would have it only in fragment ... and its usually needed on CPU side instead. Yes there are ways to do it but linked QA is simpler — Spektre, Jan 29 '19 at 12:43
Thanks @Spektre, it has to do with what you said, getting the world coordinate of clicked point, I am using a dynamic light as it was the easiest example to setup, but if I could get an index to modify my sdf's it would be great! I will take a look. — Felipe Gutierrez, Jan 29 '19 at 14:12
The dynamic light will not help you because it does not do what you need ... if you want just the coordinate you just read the pixel at mouse position with `glReadPixels` from the depth buffer and convert to your world coordinate system by unproject or by your own matrix math ... If you need also the index of selected object then you need the additional buffer (like the Stencil in the link of mine) and add it to the rendering too. — Spektre, Jan 29 '19 at 14:29
@Spektre I don't have access to the host, as everything is done on the client side so glReadPixels won't help here. — Felipe Gutierrez, Jan 29 '19 at 14:39

Felipe Gutierrez · Answer 1 · 2019-03-09T06:55:37.050

This is the correct code:

// Our sphere-tracing algorithm.
float ray( vec3 ro, vec3 rd, out float d )
{

    float t = 0.0; d = 0.0;

    for( int i = 0; i < STEPS; ++i )
    {

        vec3 p = ro + rd * t;
        d = map( p );

        if( d < EPS || t > FAR ) break;

        t += d;

    }

    return t;

}

// Here we compute all our lighting calculations.
vec3 shad( vec3 ro, vec3 rd, vec2 uv )
{

    float t = 0.0, d = 0.0;
    t = ray( ro, rd, d );

    vec3 p = ro + rd * t;
    vec3 n = nor( p );
    // The values of the variable lig are not random they are in the same position as our rayOrigin for our sphere tracing algo, that goes in main's body.
    vec3 lig = ( vec3( 0, 0, 2 ) );
    // Here is where we "shoot" our ray from the mouse position. Our ray's origin.
    vec2 uvl = ( -iResolution.xy + 2.0 * iMouse.xy ) / iResolution.y;
    // This is our ray's direction.
    vec3 lir = normalize( vec3( uvl, -1 ) );
    // Here we get our SDF(dO) and our incrementing value(tO).
    float dO = 0.0, tO = ray( lig, lir, dO );
    // Now we update our vector with the direction and incrementing steps.
    lig += lir * tO;
    // We must normalize lights as they are just a direction, the magnitude screws the lighting calculations.
    lig = normalize( lig );

    vec3 ref = reflect( rd, n );

    float amb = 0.5 + 0.5 * n.y;
    float dif = max( 0.0, dot( n, lig ) );
    float spe = pow( clamp( dot( ref, lig ), 0.0, 1.0 ), 16.0 );

    vec3 col = vec3( 0 );

    col += 0.1 * amb;
    col += 0.2 * dif;
    col += spe;

    return col;

}

// Last step, here we create the origin and direction of our rays that we shoot against the SDF.
void mainImage( out vec4 fragColor, in vec2 fragCoord )
{

    //Normalize the coordinates.
    vec2 uv = ( -iResolution.xy + 2.0 * fragCoord.xy ) / iResolution.y;

    // This is our ray's origin. We must use the same values for our lig's origin.
    vec3 ro = vec3( 0, 0, 2 );
    // This is our ray's direction.
    vec3 rd = normalize( vec3( uv, -1 ) );

    // Our SDF(d) and our incrementing steps(t), we only need our SDF(d) to bail the shading calculations according to our epsilon(EPS).
    float t = 0.0, d = 0.0;
    t = ray( ro, rd, d );

    vec3 col = d < EPS ? shad( ro, rd, uv ) : vec3( 0 );

    fragColor = vec4( col, 1 );

}

Implementing RayPicking in a Fragment Shader

1 Answers1