0

What I am trying to achieve

So I'm a fractal enthusiast and decided to build a 2D/3D fractal generator in WebGL using raymarching, with Typescript as scripting language. I've been a C#/Typescript dev for several years but having zero experience with 3d programming, I used Michael Walczyk's blog as a starting point. Some of my code I use here is derived from his tutorial.

I added the functionality that you can move through the object using WASDQEZC keys. WS = strafe forward-back, AD = strafe left-right, QE = strafe up-down, ZC = roll left-right. I combine this with a mouse look function which moves in the direction the mouse pointer is located on the rendering canvas. So what I want is total freedom of movement like in a spacesim. For this I am using a separate camera rotation matrix together with translation values and send them to the shader like this:

  setCameraMatrix(): void {
    let cameraRotationMatrixLocation: WebGLUniformLocation = this.currentContext.getUniformLocation(this.currentProgram, "u_cameraRotation");
    let cameraTranslationLocation: WebGLUniformLocation = this.currentContext.getUniformLocation(this.currentProgram, "u_cameraTranslation");
    let foVLocation: WebGLUniformLocation = this.currentContext.getUniformLocation(this.currentProgram, "u_foV");

    //add point of camera rotation at beginning
    let cameraRotationMatrix: Array<number> = Matrix3D.identity();

    //set camera rotation and translation, Z-axis (heading) first, then X-axis (pitch), then Y-axis (roll)
    cameraRotationMatrix = Matrix3D.multiply(cameraRotationMatrix, Matrix3D.rotateZ(this.cameraRotateZ * Math.PI / 180));
    cameraRotationMatrix = Matrix3D.multiply(cameraRotationMatrix, Matrix3D.rotateX(this.cameraRotateX * Math.PI / 180));
    cameraRotationMatrix = Matrix3D.multiply(cameraRotationMatrix, Matrix3D.rotateY(this.cameraRotateY * Math.PI / 180));
    //cameraRotationMatrix = Matrix3D.multiply(cameraRotationMatrix, Matrix3D.translate(this.cameraTranslateX, this.cameraTranslateY, this.cameraTranslateZ));
    cameraRotationMatrix = Matrix3D.inverse(cameraRotationMatrix);

    let cameraPosition: Array<number> = [
      this.cameraTranslateX,
      this.cameraTranslateY,
      -this.cameraTranslateZ,
    ];

    this.currentContext.uniformMatrix4fv(cameraRotationMatrixLocation, false, cameraRotationMatrix);
    this.currentContext.uniform3fv(cameraTranslationLocation, cameraPosition);
    this.currentContext.uniform1f(foVLocation, this.foV);
  }

I tried adding the camera translation values to the camera matrix but that didn't work. I got weird distortion effects and couldn't get it right so I commented that line out and left it there for now for clarity. The reason I did it this way is because of the way my GLSL code is constructed:

The main function from the fragment shader with the call to the ray_march function. v_position is a vec2 with x,y coordinates coming from the vertex shader.:

    void main() {
      outColor = vec4(ray_march(u_cameraTranslation, u_cameraRotation * vec4(rayDirection(u_foV,v_position),1), u_world, vec3(u_light * vec4(0,0,0,1)).xyz ).xyz,1);
    }

The ray_march function I am using. This derives from the sample code in Michael Walczyk's blog.

    vec3 ray_march(in vec3 ro, in vec4 rd, in mat4 wm, in vec3 lightPosition) //ro = ray origin, rd = ray direction gt = geometry position after matrix multiplication
    {
        float total_distance_traveled = 0.0;
        const int NUMBER_OF_STEPS = 1024;

        float MINIMUM_HIT_DISTANCE = 0.001 * min_hit_distance_correction;

        const float MAXIMUM_TRACE_DISTANCE = 1000.0;

        for (int i = 0; i < NUMBER_OF_STEPS; i++)
        {
            vec3 current_position = ro + total_distance_traveled * vec3(rd);

            float distance_to_closest = map(current_position, wm);

            if (distance_to_closest < MINIMUM_HIT_DISTANCE) 
            {
              vec3 normal = calculate_normal(current_position, wm);
              vec3 outColor = vec3(1.0,0,0);
              vec3 v_surfaceToLight = lightPosition - current_position;
              vec3 v_surfaceToView = ro - current_position;

              //insert lighting code below this line

              return outColor;
            }

            if (total_distance_traveled > MAXIMUM_TRACE_DISTANCE)
            {
              break;
            }

            total_distance_traveled += distance_to_closest;
        }

        return vec3(0.25);//gray background
    }

The rayDirection function I am using.

  vec3 rayDirection(float fieldOfView, vec2 p) {
    float z = 1.0 / (tan(radians(fieldOfView) / 2.0));
    return normalize(vec3(p.xy, -z));
  }

My problem

I have issues moving and rotating my camera correctly in 3d world. I'm doing this by applying some trigonometry to get the movement right. E.g., when I move forward, that is the Z-axis. But when I make a 90 degree turn to the right the X-axis now becomes the Z-axis. I am using trigonometry to correct this and actually got something working but now I am stuck in a quagmire of trigonometry with no end in sight and I have a feeling there must be a better and less complicated way. To see what I'm talking about, here is the code of the 'move' function:

  move(event: KeyboardEvent): void {
    
    //strafe forward-back
    let tXForwardBack: number = (Math.sin(this.cameraRotateY * Math.PI / 180) * Math.cos(this.cameraRotateX * Math.PI / 180)) * this.clipSpaceFactor * this.speed;
    let tYForwardBack: number = Math.sin(this.cameraRotateX * Math.PI / 180) * this.speed;
    let tZForwardBack: number = (Math.cos(this.cameraRotateY * Math.PI / 180) * Math.cos(this.cameraRotateX * Math.PI / 180)) * this.clipSpaceFactor * this.speed;

    //strafe up-down
    let tXUpDown: number = ((Math.sin(this.cameraRotateX * Math.PI / 180) * Math.sin(this.cameraRotateY * Math.PI / 180)) * this.clipSpaceFactor * this.speed);
    let tYUpDown: number = Math.cos(this.cameraRotateX * Math.PI / 180) * this.speed;
    let tZUpDown: number = Math.sin(this.cameraRotateX * Math.PI / 180) * Math.cos(this.cameraRotateY * Math.PI / 180) * this.clipSpaceFactor * this.speed;

    //strafe left-right without roll. TODO: implement roll
    let tXLeftRight: number = Math.cos(this.cameraRotateY * Math.PI / 180) * this.clipSpaceFactor * this.speed;
    let tYLeftRight: number = 0;
    let tZLeftRight: number = Math.sin(this.cameraRotateY * Math.PI / 180) * this.clipSpaceFactor * this.speed;

    switch (event.key) {
      case "w": { //strafe forward
        this.cameraTranslateX = this.cameraTranslateX + tXForwardBack;
        this.cameraTranslateY = this.cameraTranslateY - tYForwardBack;
        this.cameraTranslateZ = this.cameraTranslateZ + tZForwardBack;
        //this.cameraTranslateZ = this.cameraTranslateZ + (this.clipSpaceFactor * this.speed);
        break;
      }
      case "s": { //strafe back
        this.cameraTranslateX = this.cameraTranslateX - tXForwardBack;
        this.cameraTranslateY = this.cameraTranslateY + tYForwardBack;
        this.cameraTranslateZ = this.cameraTranslateZ - tZForwardBack;
        break;
      }
      case "a": {//strafe left
        this.cameraTranslateX = this.cameraTranslateX - tXLeftRight;
        this.cameraTranslateY = this.cameraTranslateY + tYLeftRight;
        this.cameraTranslateZ = this.cameraTranslateZ + tZLeftRight;
        break;
      }
      case "d": { //strafe right
        this.cameraTranslateX = this.cameraTranslateX + tXLeftRight;
        this.cameraTranslateY = this.cameraTranslateY - tYLeftRight;
        this.cameraTranslateZ = this.cameraTranslateZ - tZLeftRight;
        break;
      }
      case "q": { //strafe up
        this.cameraTranslateX = this.cameraTranslateX + tXUpDown;
        this.cameraTranslateY = this.cameraTranslateY + tYUpDown;
        this.cameraTranslateZ = this.cameraTranslateZ + tZUpDown;
        break;
      }
      case "e": { //strafe down
        this.cameraTranslateX = this.cameraTranslateX - tXUpDown;
        this.cameraTranslateY = this.cameraTranslateY - tYUpDown;
        this.cameraTranslateZ = this.cameraTranslateZ - tZUpDown;
        break;
      }
      case "z": { //roll left
        this.cameraRotateZ = (this.cameraRotateZ + (this.sensitivity * this.speed)) % 360;
        break;
      }
      case "c": { //roll right
        this.cameraRotateZ = (this.cameraRotateZ - (this.sensitivity * this.speed)) % 360;
        break;
      }
    }

It actually works to some degree, but you can see where this is going :( Also, I get a 'dead' zone when I look up and down along the Y-axis. I found This thread which seems to describe my problem and says 'The trick is to apply the translation to the z-axis but in the local coordinate system of the camera.'

But how do I do that with my existing code? I tried multiplying the world matrix u_world by the rotationmatrix u_rotationMatrix but then the lighting changes as well and it's just an object rotation instead of a separate camera rotation. In the thread I posted there is no lighting so multiplying the camera matrix with the world matrix works for them. But it doesn't for me because of the lighting I implemented. Also, I can't seem to apply the normals separately this way so that I only apply the normals to the world matrix and not to the camera rotation matrix, so that the lighting stays in place when I rotate/translate the camera.

The only way I can get correct normals to the world matrix and a separate cameramatrix is by multiplying the rotationMatrix with the rayDirection like so u_cameraRotation * vec4(rayDirection(u_foV,v_position),1). But when I do this I have to apply all this horrible, partially working trigonometry mess to get something decent. What I want is getting it to work like 'The trick is to apply the translation to the z-axis but in the local coordinate system of the camera.'

But I don't know how. I tried all kinds of things but I'm currently stuck. Any help would be greatly appreciated. I think I've outlined my problem sufficiently enough, if you miss anything please let me know. Thanks in advance.

Arthur Visser
  • 51
  • 6
  • SO is not a forum... -> [How do I ask a good question?](https://stackoverflow.com/help/how-to-ask) - The whole _"Introduction"_ part is not relevant for the question – Andreas Oct 21 '21 at 16:12
  • 1
    The best place to put this question (and increase the quality) is in Computer Graphics exchange. – UserOfStackOverFlow Oct 21 '21 at 16:17
  • 1
    See if is it that you are searching for -> https://www.youtube.com/watch?v=FkCv5Mh2Qck – UserOfStackOverFlow Oct 21 '21 at 16:19
  • 1
    Thanks I'll use the Computer Graphics exchange. – Arthur Visser Oct 21 '21 at 16:23
  • 1
    How I see on your code, you're in correct track, continue trying (modifying the phormulas). You checked my video? If you need know how to movement a camera in 3D space I can release the code for you, with the mouse feature if you need. – UserOfStackOverFlow Oct 21 '21 at 16:30
  • 1
    Yes I just checked. It indeed looks like what I want - working freedom of movement. If you want to release the code that would be appreciated. Somehow working examples using my trigonometry methodology is somehow hard to find, or I'm not looking good enough. – Arthur Visser Oct 21 '21 at 16:34
  • 1
    Really is hard to find a standardized phormulas, I achieved the result changing the calculus till get right. In moment I can't release the code, later today I let you know here when I release, is in Java and it's well compressed (only a few lines). In the video I use the polar coordinate system even in 3D (instead of Spherical). – UserOfStackOverFlow Oct 21 '21 at 16:55
  • As @UserOfStackOverFlow recommended I asked this question on [computergraphics.stackexchange.com](https://computergraphics.stackexchange.com/questions/12269/is-there-a-better-more-elegant-way-of-translating-rotating-my-camera-in-my-3d-r). See the link for the answer provided. Although it's a bit different than what I originally had in mind, it's a more elegant solution which works with roll rotation too and now I can drop all the trigonometry code. – Arthur Visser Oct 24 '21 at 08:33

1 Answers1

1

Looks like I found the answer myself. I applied part of Adisak's answer from this question which is similar to mine. I applied his EulerAnglesToMatrix function with rotation order ZXY, then extracted the x, y and z-axis like so:

    let mx: Array<number> = Matrix3D.eulerAnglesToMatrix(pitch, yaw, roll, "ZXY");

    let xAxis: Array<number> = mx.slice(0, 3); //x,y,z
    let yAxis: Array<number> = mx.slice(3, 6); //x,y,z
    let zAxis: Array<number> = mx.slice(6, 9); //x,y,z

I then applied the translation like so, setting the [this.cameraTranslateX,this.cameraTranslateY,this.cameraTranslateZ] as the uniform vec3 u_cameraTranslation variable for the fragmentshader:

    switch (event.key) {
      case "w": { //strafe forward
        this.cameraTranslateX = this.cameraTranslateX - ((zAxis[0]) * this.clipSpaceFactor * this.speed);
        this.cameraTranslateY = this.cameraTranslateY - ((zAxis[1]  ) * this.clipSpaceFactor * this.speed);
        this.cameraTranslateZ = this.cameraTranslateZ + ((zAxis[2] ) * this.clipSpaceFactor * this.speed);
        break;
      }
      case "s": { //strafe back
        this.cameraTranslateX = this.cameraTranslateX + ((zAxis[0] ) * this.clipSpaceFactor * this.speed);
        this.cameraTranslateY = this.cameraTranslateY + ((zAxis[1] ) * this.clipSpaceFactor * this.speed);
        this.cameraTranslateZ = this.cameraTranslateZ - ((zAxis[2] ) * this.clipSpaceFactor * this.speed);
        break;
      }
      case "a": {//strafe left
        this.cameraTranslateX = this.cameraTranslateX - (xAxis[0] * this.clipSpaceFactor * this.speed);
        this.cameraTranslateY = this.cameraTranslateY - (xAxis[1] * this.clipSpaceFactor * this.speed);
        this.cameraTranslateZ = this.cameraTranslateZ + (xAxis[2] * this.clipSpaceFactor * this.speed);
        break;
      }
      case "d": { //strafe right
        this.cameraTranslateX = this.cameraTranslateX + (xAxis[0] * this.clipSpaceFactor * this.speed);
        this.cameraTranslateY = this.cameraTranslateY + (xAxis[1] * this.clipSpaceFactor * this.speed);
        this.cameraTranslateZ = this.cameraTranslateZ - (xAxis[2] * this.clipSpaceFactor * this.speed);
        break;
      }
      case "q": { //strafe up
        this.cameraTranslateX = this.cameraTranslateX + (yAxis[0] * this.clipSpaceFactor * this.speed);
        this.cameraTranslateY = this.cameraTranslateY + (yAxis[1] * this.clipSpaceFactor * this.speed);
        this.cameraTranslateZ = this.cameraTranslateZ - (yAxis[2] * this.clipSpaceFactor * this.speed);
        break;
      }
      case "e": { //strafe down
        this.cameraTranslateX = this.cameraTranslateX - (yAxis[0] * this.clipSpaceFactor * this.speed);
        this.cameraTranslateY = this.cameraTranslateY - (yAxis[1] * this.clipSpaceFactor * this.speed);
        this.cameraTranslateZ = this.cameraTranslateZ + (yAxis[2] * this.clipSpaceFactor * this.speed);
        break;
      }
      case "z": { //roll left
        this.cameraRotateZ = (this.cameraRotateZ + (this.sensitivity * this.speed)) % 360;
        break;
      }
      case "c": { //roll right
        this.cameraRotateZ = (this.cameraRotateZ - (this.sensitivity * this.speed)) % 360;
        break;
      }
    }

I left the raymarching function intact. This gave me exactly what I wanted.

Arthur Visser
  • 51
  • 6