0

after my original post was flagged as "too broad" by other stack overflow users. I will rephrase my question in less lines. I have implemented a ray marcher in shadertoy, and i understood all the math about the ray-object intersection. And i want to make the next step to ray trace with meshes. I found out that doing so needs to be done with deferred rendering and framebuffers(2 pass rendering, one for the geometry and the other for the lighting calculations, the ray tracing) and the intersections will happen with ray-triangle intersection.

Of course i will post some code in the next weeks as i will start implementing this. and i will need some specific help but till then i would appreciate the help in order to now waste time searching the web. This is why i posted here..

original post as it was flagged "too broad"

this is my first ever post to stack overflow. My goal is to write a ray tracer with mesh objects.

So far the only thing i have accomplished is make a ray tracer with spheres and planes. This is done with forward rendering, and while creating all the geometry in the fragment shader(which is easy to produce the spheres and the plane).

After all the research that i have made i finally found the solution. This is accomplished with deferred shading technique where you create the geometry and pass it to the G-buffer as a texture and then with the second pass you start the calculations for the lighting and the ray tracing.

I am seeking guidance cause i have spent many days/months(3-4) in order to come to this conclusion since i was not familiar with neither shaders nor the graphics libraries. My questions are the following:

1) Is this approach the correct one? 2) And second can someone guide me to the triangle intersections? By the term triangle intersections i don't mean the math and the procedure of ray-triangle intersection; i know about that. What i don't know is after the first intersection, how to check for the next collision, how to test the next triangle. In what way do i need to pass the vertexes in the G-buffer in order to traverse through them and check for intersections(some sample glsl code would be nice to understand)?

here are the links that i have read in order to come to that conclution

  1. https://www.opengl.org/discussion_boards/showthread.php/200487-Ray-intersection-with-GLSL
  2. https://www.imgtec.com/blog/five-steps-to-adding-ray-tracing-to-an-opengl-es-based-deferred-lighting-system/
  3. https://learnopengl.com/Advanced-Lighting/Deferred-Shading
  4. GPU PRO 6 book, page 351-368

If i missed any link in the stack overflow community please post them here.

I would appreciate any answer relative to the answer. Thank you in advance.

Thodoris Koutsis
  • 109
  • 1
  • 12

1 Answers1

1

I'm also self thought, had a hard time starting. But it is very easy.
You need to pack your mesh data into an RGB(A) texture and then uppack it inside the fragment shader
some really good pfds on this technique, here is one example: http://www.cs.harvard.edu/~sjg/papers/gim.pdf

exampel of texture layout:
using RGB ( Alpha Channel can be used for meterial index ) 

             colums       colums      colums
        |      0      |     1     |      2
        |  R   G   B  | R   G   B |  R   G   B
        |-------------|-----------|-----------
    rows|  1   0   0  |           |

CPU side:
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGB32F, dataLenght/channels, 1, 0, gl.RGB, gl.FLOAT, meshVerts);
texImage2D reference
GPU side:
texelFetch(uMeshData, ivec2(vertIndex, 0), 0);
texelFetch reference

Some Resources/LINKS:
Written in C++ but actually the conversation is not that far away to an #version 300 es glsl shader:
scratchapixel
OpenGL Development Cookbook has some expampels reading obj to image to fragment
the source on github
i personaly think this is the best pathtracer on the web:
with me little knowledge i can see that this guy read the Physically Based Rendering book to the very end
here is really good tutorial to understand the concept of computer graphics
UPDATE/EDIT: Peter Shirleys Raytracing in a Weekend is free for download

my own little attempt 

(() => {
'use strict';
// vertex
const vs = `#version 300 es\nin vec2 p;out vec2 vuv;void main(){gl_Position = vec4(vuv = p, 0, 1);}`;

// fragment
const fs = `#version 300 es\n
precision highp float;
precision highp int;
precision highp sampler2D; 
in vec2 vuv;
uniform float time;
uniform vec2 Res, mouse;
uniform sampler2D uMeshData;
uniform int vertsCount;
layout(location = 0) out lowp vec4 fragColor;

struct Ray {
  vec3 orig, dir;
}R_;

mat4 rotate() {
float x = mouse.x, y=mouse.y+sin(time*2.),z=0.;
  float a = sin(x), b = cos(x), c = sin(y), d = cos(y), e = sin(z), f = cos(z), ac = a * c, bc = b * c;
  return mat4(d * f, d * e, -c, 0.0, ac * f - b * e, ac * e + b * f, a * d, 0.0, bc * f + a * e, bc * e - a * f, b * d, 0.0, 0.0, 0.0, 0.0, 1.0);
}

// https://github.com/Jojendersie/gpugi/blob/5d18526c864bbf09baca02bfab6bcec97b7e1210/gpugi/shader/intersectiontests.glsl#L63
bool isTriangle(Ray ray, in vec3 p0, in vec3 p1, in vec3 p2, out vec3 N) {
  vec3 e0 = p1 - p0, e1 = p0 - p2;
  N = cross(e1, e0);
  vec3 e2 = (1.0 / dot(N, ray.dir)) * (p0 - ray.orig);
  vec3 i = cross(ray.dir, e2);
  vec3 b = vec3(0.0, dot(i, e1), dot(i, e0));
  b.x = 1.0 - (b.z + b.y);
  return (dot(N, e2) > 1e-8) && all(greaterThanEqual(b, vec3(0.0)));
}

void Camera(out Ray ray, vec3 lookAt, vec3 up, float angle, float aspect) {
  vec3 g = normalize(lookAt - ray.orig);
  vec3 u = normalize(cross(g, up));
  vec3 v = normalize(cross(u, g));
  u = u * tan(radians(angle * .5));
  v = v * tan(radians(angle * .5)) / aspect;
  ray.dir = normalize(g + ray.dir.x * u + ray.dir.y * v);
}

void main() {
  vec3 SceneCol = vec3(0.5);
  
  vec3 hit = vec3(0.);
  vec4 a = vec4(0.0), b = vec4(0.0), c = vec4(0.0);
  
  R_ = Ray(vec3(0.0, 0.0, 3.0), vec3(vuv, -1.));
  
  Camera(R_, vec3(0., 0., 1.), vec3(0., 1., 0.), 90.0, (Res.x / Res.y));
  
  float mindist = -1000.0;

 // here comes this importend part unpack the texture
  for (int i = 0; i < vertsCount; i += 3) 
  {

    a = rotate() * texelFetch(uMeshData, ivec2(i, 0), 0);
    b = rotate() * texelFetchOffset(uMeshData, ivec2(i, 0), 0, ivec2(1, 0));
    c = rotate() * texelFetchOffset(uMeshData, ivec2(i, 0), 0, ivec2(2, 0));
    
    if (isTriangle(R_, a.xyz, b.xyz, c.xyz, hit))
    {
      float z = hit.z;
      if (z > mindist) {
        mindist = z;
        SceneCol.rgb = vec3(hit.x, hit.y, 1. - (hit.x - hit.y));
      };
    }
  }
  
  vec3 sky = vec3(0.5, 0.25, 0.1) * (-R_.dir.y - 0.1);
  fragColor.rgb = SceneCol + sky;
  fragColor.a = 1.0;
}`;

    const canvas = document.getElementById('c');
    const gl = canvas.getContext('webgl2', {
        alpha: !1,
        depth: !1,
        stencil: !1,
        antialias: !1,
        premultipliedAlpha: !1,
        presereDrawingBuffer: !1,
        failIfMajorPerformanceCaveat: !1
    });

    const { width, height } = canvas.getBoundingClientRect();
    gl.canvas.width = width;
    gl.canvas.height = height;

    // init
    const P = gl.createProgram();

    const Fp = gl.createShader(gl.FRAGMENT_SHADER);
    gl.shaderSource(Fp, fs);
    gl.compileShader(Fp);
    if (!gl.getShaderParameter(Fp, gl.COMPILE_STATUS)) throw '! F r a g: ' + gl.getShaderInfoLog(Fp);
    gl.attachShader(P, Fp);

    const Vp = gl.createShader(gl.VERTEX_SHADER);
    gl.shaderSource(Vp, vs);
    gl.compileShader(Vp);
    if (!gl.getShaderParameter(Vp, gl.COMPILE_STATUS)) throw '! V e r t: ' + gl.getShaderInfoLog(Vp);
    gl.attachShader(P, Vp);

    // link use program
    gl.linkProgram(P);
    gl.useProgram(P);

    // uniform location
    const time_loc = gl.getUniformLocation(P, 'time');
    const res_loc = gl.getUniformLocation(P, 'Res');
    const uLvertices = gl.getUniformLocation(P, 'vertsCount');
    const uLSr = gl.getUniformLocation(P, 'uMeshData');
    const mouse_loc = gl.getUniformLocation(P, 'mouse');

    // free resources
    gl.detachShader(P, Fp);
    gl.detachShader(P, Vp);
    gl.deleteProgram(P);

    // fullscreen quad
    gl.bindBuffer(gl.ARRAY_BUFFER, gl.createBuffer());
    gl.bufferData(gl.ARRAY_BUFFER, new Int8Array([-3, 1, 1, -3, 1, 1]), gl.STATIC_DRAW);
    gl.enableVertexAttribArray(0);
    gl.vertexAttribPointer(0, 2, gl.BYTE, !1, 0, 0);
    gl.bindVertexArray(null);

    // bind texture
    const texture = gl.createTexture();
    gl.bindTexture(gl.TEXTURE_2D, texture);


    // blender 2.79 Icosphere
    // export -> *.raw (needs to be enable first under settings)
    const verts = [
        0.000000, 0.000000, -1.000000, 0.425323, -0.309011, -0.850654, -0.162456, -0.499995, -0.850654,
        0.723607, -0.525725, -0.447220, 0.425323, -0.309011, -0.850654, 0.850648, 0.000000, -0.525736,
        0.000000, 0.000000, -1.000000, -0.162456, -0.499995, -0.850654, -0.525730, 0.000000, -0.850652,
        0.000000, 0.000000, -1.000000, -0.525730, 0.000000, -0.850652, -0.162456, 0.499995, -0.850654,
        0.000000, 0.000000, -1.000000, -0.162456, 0.499995, -0.850654, 0.425323, 0.309011, -0.850654,
        0.723607, -0.525725, -0.447220, 0.850648, 0.000000, -0.525736, 0.951058, -0.309013, 0.000000,
        -0.276388, -0.850649, -0.447220, 0.262869, -0.809012, -0.525738, 0.000000, -1.000000, 0.000000,
        -0.894426, 0.000000, -0.447216, -0.688189, -0.499997, -0.525736, -0.951058, -0.309013, 0.000000,
        -0.276388, 0.850649, -0.447220, -0.688189, 0.499997, -0.525736, -0.587786, 0.809017, 0.000000,
        0.723607, 0.525725, -0.447220, 0.262869, 0.809012, -0.525738, 0.587786, 0.809017, 0.000000,
        0.723607, -0.525725, -0.447220, 0.951058, -0.309013, 0.000000, 0.587786, -0.809017, 0.000000,
        -0.276388, -0.850649, -0.447220, 0.000000, -1.000000, 0.000000, -0.587786, -0.809017, 0.000000,
        -0.894426, 0.000000, -0.447216, -0.951058, -0.309013, 0.000000, -0.951058, 0.309013, 0.000000,
        -0.276388, 0.850649, -0.447220, -0.587786, 0.809017, 0.000000, 0.000000, 1.000000, 0.000000,
        0.723607, 0.525725, -0.447220, 0.587786, 0.809017, 0.000000, 0.951058, 0.309013, 0.000000,
        0.276388, -0.850649, 0.447220, 0.688189, -0.499997, 0.525736, 0.162456, -0.499995, 0.850654,
        -0.723607, -0.525725, 0.447220, -0.262869, -0.809012, 0.525738, -0.425323, -0.309011, 0.850654,
        -0.723607, 0.525725, 0.447220, -0.850648, 0.000000, 0.525736, -0.425323, 0.309011, 0.850654,
        0.276388, 0.850649, 0.447220, -0.262869, 0.809012, 0.525738, 0.162456, 0.499995, 0.850654,
        0.894426, 0.000000, 0.447216, 0.688189, 0.499997, 0.525736, 0.525730, 0.000000, 0.850652,
        0.525730, 0.000000, 0.850652, 0.162456, 0.499995, 0.850654, 0.000000, 0.000000, 1.000000,
        0.525730, 0.000000, 0.850652, 0.688189, 0.499997, 0.525736, 0.162456, 0.499995, 0.850654,
        0.688189, 0.499997, 0.525736, 0.276388, 0.850649, 0.447220, 0.162456, 0.499995, 0.850654,
        0.162456, 0.499995, 0.850654, -0.425323, 0.309011, 0.850654, 0.000000, 0.000000, 1.000000,
        0.162456, 0.499995, 0.850654, -0.262869, 0.809012, 0.525738, -0.425323, 0.309011, 0.850654,
        -0.262869, 0.809012, 0.525738, -0.723607, 0.525725, 0.447220, -0.425323, 0.309011, 0.850654,
        -0.425323, 0.309011, 0.850654, -0.425323, -0.309011, 0.850654, 0.000000, 0.000000, 1.000000,
        -0.425323, 0.309011, 0.850654, -0.850648, 0.000000, 0.525736, -0.425323, -0.309011, 0.850654,
        -0.850648, 0.000000, 0.525736, -0.723607, -0.525725, 0.447220, -0.425323, -0.309011, 0.850654,
        -0.425323, -0.309011, 0.850654, 0.162456, -0.499995, 0.850654, 0.000000, 0.000000, 1.000000,
        -0.425323, -0.309011, 0.850654, -0.262869, -0.809012, 0.525738, 0.162456, -0.499995, 0.850654,
        -0.262869, -0.809012, 0.525738, 0.276388, -0.850649, 0.447220, 0.162456, -0.499995, 0.850654,
        0.162456, -0.499995, 0.850654, 0.525730, 0.000000, 0.850652, 0.000000, 0.000000, 1.000000,
        0.162456, -0.499995, 0.850654, 0.688189, -0.499997, 0.525736, 0.525730, 0.000000, 0.850652,
        0.688189, -0.499997, 0.525736, 0.894426, 0.000000, 0.447216, 0.525730, 0.000000, 0.850652,
        0.951058, 0.309013, 0.000000, 0.688189, 0.499997, 0.525736, 0.894426, 0.000000, 0.447216,
        0.951058, 0.309013, 0.000000, 0.587786, 0.809017, 0.000000, 0.688189, 0.499997, 0.525736,
        0.587786, 0.809017, 0.000000, 0.276388, 0.850649, 0.447220, 0.688189, 0.499997, 0.525736,
        0.000000, 1.000000, 0.000000, -0.262869, 0.809012, 0.525738, 0.276388, 0.850649, 0.447220,
        0.000000, 1.000000, 0.000000, -0.587786, 0.809017, 0.000000, -0.262869, 0.809012, 0.525738,
        -0.587786, 0.809017, 0.000000, -0.723607, 0.525725, 0.447220, -0.262869, 0.809012, 0.525738,
        -0.951058, 0.309013, 0.000000, -0.850648, 0.000000, 0.525736, -0.723607, 0.525725, 0.447220,
        -0.951058, 0.309013, 0.000000, -0.951058, -0.309013, 0.000000, -0.850648, 0.000000, 0.525736,
        -0.951058, -0.309013, 0.000000, -0.723607, -0.525725, 0.447220, -0.850648, 0.000000, 0.525736,
        -0.587786, -0.809017, 0.000000, -0.262869, -0.809012, 0.525738, -0.723607, -0.525725, 0.447220,
        -0.587786, -0.809017, 0.000000, 0.000000, -1.000000, 0.000000, -0.262869, -0.809012, 0.525738,
        0.000000, -1.000000, 0.000000, 0.276388, -0.850649, 0.447220, -0.262869, -0.809012, 0.525738,
        0.587786, -0.809017, 0.000000, 0.688189, -0.499997, 0.525736, 0.276388, -0.850649, 0.447220,
        0.587786, -0.809017, 0.000000, 0.951058, -0.309013, 0.000000, 0.688189, -0.499997, 0.525736,
        0.951058, -0.309013, 0.000000, 0.894426, 0.000000, 0.447216, 0.688189, -0.499997, 0.525736,
        0.587786, 0.809017, 0.000000, 0.000000, 1.000000, 0.000000, 0.276388, 0.850649, 0.447220,
        0.587786, 0.809017, 0.000000, 0.262869, 0.809012, -0.525738, 0.000000, 1.000000, 0.000000,
        0.262869, 0.809012, -0.525738, -0.276388, 0.850649, -0.447220, 0.000000, 1.000000, 0.000000,
        -0.587786, 0.809017, 0.000000, -0.951058, 0.309013, 0.000000, -0.723607, 0.525725, 0.447220,
        -0.587786, 0.809017, 0.000000, -0.688189, 0.499997, -0.525736, -0.951058, 0.309013, 0.000000,
        -0.688189, 0.499997, -0.525736, -0.894426, 0.000000, -0.447216, -0.951058, 0.309013, 0.000000,
        -0.951058, -0.309013, 0.000000, -0.587786, -0.809017, 0.000000, -0.723607, -0.525725, 0.447220,
        -0.951058, -0.309013, 0.000000, -0.688189, -0.499997, -0.525736, -0.587786, -0.809017, 0.000000,
        -0.688189, -0.499997, -0.525736, -0.276388, -0.850649, -0.447220, -0.587786, -0.809017, 0.000000,
        0.000000, -1.000000, 0.000000, 0.587786, -0.809017, 0.000000, 0.276388, -0.850649, 0.447220,
        0.000000, -1.000000, 0.000000, 0.262869, -0.809012, -0.525738, 0.587786, -0.809017, 0.000000,
        0.262869, -0.809012, -0.525738, 0.723607, -0.525725, -0.447220, 0.587786, -0.809017, 0.000000,
        0.951058, -0.309013, 0.000000, 0.951058, 0.309013, 0.000000, 0.894426, 0.000000, 0.447216,
        0.951058, -0.309013, 0.000000, 0.850648, 0.000000, -0.525736, 0.951058, 0.309013, 0.000000,
        0.850648, 0.000000, -0.525736, 0.723607, 0.525725, -0.447220, 0.951058, 0.309013, 0.000000,
        0.425323, 0.309011, -0.850654, 0.262869, 0.809012, -0.525738, 0.723607, 0.525725, -0.447220,
        0.425323, 0.309011, -0.850654, -0.162456, 0.499995, -0.850654, 0.262869, 0.809012, -0.525738,
        -0.162456, 0.499995, -0.850654, -0.276388, 0.850649, -0.447220, 0.262869, 0.809012, -0.525738,
        -0.162456, 0.499995, -0.850654, -0.688189, 0.499997, -0.525736, -0.276388, 0.850649, -0.447220,
        -0.162456, 0.499995, -0.850654, -0.525730, 0.000000, -0.850652, -0.688189, 0.499997, -0.525736,
        -0.525730, 0.000000, -0.850652, -0.894426, 0.000000, -0.447216, -0.688189, 0.499997, -0.525736,
        -0.525730, 0.000000, -0.850652, -0.688189, -0.499997, -0.525736, -0.894426, 0.000000, -0.447216,
        -0.525730, 0.000000, -0.850652, -0.162456, -0.499995, -0.850654, -0.688189, -0.499997, -0.525736,
        -0.162456, -0.499995, -0.850654, -0.276388, -0.850649, -0.447220, -0.688189, -0.499997, -0.525736,
        0.850648, 0.000000, -0.525736, 0.425323, 0.309011, -0.850654, 0.723607, 0.525725, -0.447220,
        0.850648, 0.000000, -0.525736, 0.425323, -0.309011, -0.850654, 0.425323, 0.309011, -0.850654,
        0.425323, -0.309011, -0.850654, 0.000000, 0.000000, -1.000000, 0.425323, 0.309011, -0.850654,
        -0.162456, -0.499995, -0.850654, 0.262869, -0.809012, -0.525738, -0.276388, -0.850649, -0.447220,
        -0.162456, -0.499995, -0.850654, 0.425323, -0.309011, -0.850654, 0.262869, -0.809012, -0.525738,
        0.425323, -0.309011, -0.850654, 0.723607, -0.525725, -0.447220, 0.262869, -0.809012, -0.525738,
    ];
    const meshVerts = new Float32Array(verts);
    const vertsLenght = meshVerts.length / 3;
    gl.uniform1i(uLvertices, vertsLenght);
    gl.pixelStorei(gl.UNPACK_ALIGNMENT, 1);
    gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGB32F, vertsLenght, 1, 0, gl.RGB, gl.FLOAT, meshVerts);
    gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.NEAREST);
    gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST);
    gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
    gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);

    // mouse click
    let mousePosition = [0, 0];
    gl.canvas.addEventListener('mousemove', (e) => {
        if (!e.buttons) return;
        if (e.buttons == 1) mousePosition = [e.clientX * .01, e.clientY * .01];
    }, !1);

    // animation
    const draw = (clock) => {
        clock *= 0.001;
        gl.viewport(0.0, 0.0, gl.drawingBufferWidth, gl.drawingBufferHeight);
        gl.activeTexture(gl.TEXTURE0);
        gl.bindTexture(gl.TEXTURE_2D, texture);
        gl.uniform1i(uLSr, 0);
        gl.uniform1f(time_loc, clock);
        gl.uniform2f(mouse_loc, mousePosition[0], mousePosition[1]);
        gl.uniform2f(res_loc, width, height);
        gl.drawArrays(gl.TRIANGLE_STRIP, 0, 3);
        requestAnimationFrame(draw);
    };
    requestAnimationFrame(draw);
})()
<canvas id="c"></canvas>
nabr
  • 131
  • 5
  • thank you for this detailed answer. I will start right away, cause now at least you pointed a direction. – Thodoris Koutsis Dec 14 '18 at 13:24
  • Hello again, I want some clarification about your post, since the time you posted it i made a lot of progress with this task. I read about the intersections, i learned a bit more about glsl. Straight to the point. With all the tutorials i had there where like vertices, normals, texture coordinates that they create a VAO and thei send them like that to the gpu, and then the gl_Position = projectionMatrix * viewMatrix * modelMatrix * vec4(position, 1 ); position is the vertices you bound with the VAO. – Thodoris Koutsis Feb 03 '19 at 18:44
  • So you send each each vertex to the gpu and then the gpu interpolates them and renders the result. and in the renderLoop then you render the same object(eg cube) 3 times by multiplying with the view matrix with a different translation. So you do not have access to the vertices in order to make an intersection test. My question is what do i have to do in order to reender a second sphere? What i did was add to the verts array the other blender export, but after that how do i give to the second sphere another rotation? – Thodoris Koutsis Feb 03 '19 at 18:55
  • and last thing ` gl.bindBuffer(gl.ARRAY_BUFFER, gl.createBuffer()); gl.bufferData(gl.ARRAY_BUFFER, new Int8Array([-4, 2, 2, -4, 2, 2, ]), gl.STATIC_DRAW);` these 3 lines they produce a triangle which the phere is rendered in? – Thodoris Koutsis Feb 03 '19 at 18:56
  • @ThodorisKoutsis Hello the last question first: "and last thing...". No, some blogs on the web, sorry i can't find right now, claims that a fullscreen quad is more buggy, then render to a giant fullscreen triangle and let the graphic card clip it into screen resolution quad. "So you send..." You reading your vertex data inside the fragment shader you if you have multiple objects you need to specify/tell the program where you find the data in the loop. Think about it, how would you implement multiple objects on the cpu? Pseudocode: for( vertexdata ) if(vertexdata==1234) objectid = 2; //sphere2 – nabr Feb 04 '19 at 21:31
  • @ThodorisKoutsis "Hello again, I want ... ": VAO, yes you CANNOT do it becourse the usual pipeline pass the data first to the vertex shader then to fragment, but you need the actual values in fragment that's why you have to bypass the vertexshader and send the array with vertices direct to fragment. Yes, view, projection matrix is fine, but in raytracing theirs an easier way to create a raytracing camera. https://www.youtube.com/watch?v=PBxuVlp7nuM – nabr Feb 04 '19 at 21:38
  • just a little update to the post above, the raytracing in one weekend is now free for download https://twitter.com/Peter_shirley/status/984947257035243520 – nabr Feb 04 '19 at 21:40