1

I'm working on a 3D project in OpenGL. Picking objects is my thing to worry about, but right now I can't get the main thing working. Which is casting a ray from my camera using the mouse. I've been digging through too many examples including raycasting without "unproject", but they literally give me the same result! They are almost working, and now I am pretty much stuck and do not know what to do.

Consider my method I've picked last:

function CCamera.GetRay(x: Single; y: Single;) : TVector3;
var
  mouseX: single;
  mouseY: single;
  invVP: TMatrix4;
  screenPos: TVector4;
  worldPos: TVector4;
  worldPos3: TVector3;
  dir: TVector3;
begin

  // NDC
  mouseX := x / (fViewportWidth * 0.5) - 1.0;
  mouseY := y / (fViewportHeight * 0.5) - 1.0;

  invVP := mView * mProjection;
  // invVP.SetInversed; // Inversed doesn't even cast the ray close to where I want it cast at.
  // mProjection * mView would also generate behavior that isn't close to what I want.

  screenPos.Init(mouseX, -mouseY, 1.0, 1.0);
  worldPos := invVP * screenPos;
  worldPos3.Init(worldPos.X, worldPos.Y, worldPos.Z);

  dir := worldPos3.Normalize();

  Exit(dir);
end;

Literally. Every. Single. Example gives me the same result! This is the one I will stick to.

And then, I would determine the camera ray's end position as such:

UPDATED code:

procedure TOpenGLControl.OnMouseDown(Sender: TObject; Button: 
TMouseButton; Shift: TShiftState; X: Integer; Y: Integer);
var
  ray_start: TVector3;
  ray_end: TVector3;
  ray_dir: TVector3;
begin

    ray_start := GetCamera.GetPosition(); // Position of the camera
    ray_dir := GetCamera.GetRay(X, Y); // Pass 2D mouse coordinates
    ray_end := ray_start + ray_dir * 50.0; // Max distance of 50 units

    Invalidate();
end;

Using Neslib.FastMath for the math functions and attributes. So, basically, what happens is that the further I click from the center of the world (where the triangle is), the ray strives closer to the center. Check image.

Img1

Img2

IOviSpot
  • 358
  • 3
  • 19
  • When passing the mouse coordinates which values do your use? X and Y position from the component/form OnMouseDown event or perhaps X and Y values from mouse cursor. It if is the latter do note that mouse cursor X and Y values represent their respective positions relative to the screen size/position and not relative to the window that is your Wiewport. – SilverWarior Feb 25 '22 at 09:39
  • @SilverWarior I'm using the X and Y values from the OnMouseDown call. Updated my code. – IOviSpot Feb 25 '22 at 12:48

2 Answers2

2

Not a Delphi coder but here some insights from BCBs and VCL (should be the same):

  1. use local mouse position (OnMouseMove)

  2. convert mouse position to OpenGL position

    beware mouse y axis is in opposite direction to OpenGL and the (0,0) is in center of screen instead of in top left corner. And the range is usually <-1.0,+1.0> From What I see you are not inverting y and have range <0.0,1.0> if I see it right ...

    Also beware if you have panels in your Form then you should also check your glViewbox settings as VCL sometimes wrongly calculates ClientWidth/Height and you need to substrack what is missing like - Panel1->Width; ...

  3. beware of you ray origin

    in perspective views it has to start from camera focal point. Depending on your projection matrix it might be shifted by znear (focal length).

    If not correct this will create bigger error on edges of view so I bet its the cause of your problems.

  4. beware of depth buffer precision

    you can tweak perspective so zfar/znear matches your depthbuffer precision.

    if not enough or not possible use linear depth buffer

Take a look at these:

Spektre
  • 49,595
  • 11
  • 110
  • 380
  • 1
    Posted my solution. The problem was bad at maths. – IOviSpot Feb 27 '22 at 12:26
  • @wEight You are using transposed / Inversed notation ? `pos2 := pos * mat;` ... OpenGL native is direct matrix and order `matrix*vector` ... If not then more correct would be `pos2 := mat * pos;` ... as unprojecting would be `Inverse(matrix)*vector` – Spektre Mar 03 '22 at 15:19
  • I'm using a different approach in a .NET application and glut/glfw (For Visual C++) and they work. However, when implemented in Delphi, they do not. I am not sure what the issue is, but the one I am using at the moment (Check updated post) works. – IOviSpot Mar 12 '22 at 20:18
  • @IOviSpot I did and that is why I commented ... btw from second look `screenPos.Init(mouseX, -mouseY, 1.0, 1.0);` I am unsure about the `z` I think it should be zero or +/- focal length of perspective projection. btw see similar [Computing&Plotting 3D Points on Surface of a Mesh from mouse position](https://stackoverflow.com/a/71382169/2521214) its using native old OpenGL notations and as you can see `pos=(_mv*vec4(pos,1.0)).xyz;` vertex is last operand in muliplication instead of first. – Spektre Mar 13 '22 at 06:50
  • @IOviSpot If you use the same notation then your code is wrong and works only because you got multiple of such notation mismatches that cancel each other in some circumstances (but might fail in others) ... That is why I asked about the notations you use... IIRC there are 4 possibilities of combining direct/inverse and transposed matrices versus multiplication order (vertex * matrix / matrix * vertex) – Spektre Mar 13 '22 at 06:51
0

SOLUTION! This is the new unproject method. It is accurate 100%. I managed to put it all together. Weird.

pos.Init(0.0, 0.0, 0.0, 0.0);
pos.X := (x - 0.0) / fViewportWidth * 2.0 - 1.0;
pos.Y := 1 - (y - 0.0) / fViewportHeight * 2.0;
pos.Z := z * 2.0 - 1.0;
pos.W := 1;

mat := mView * mProjection;
mat.SetInversed();

pos2 := pos * mat;

pos_out.Init(pos2.X, pos2.Y, pos2.Z);

Exit(pos_out / pos2.W);
IOviSpot
  • 358
  • 3
  • 19