THREE.JS - How to set the length of vector based on angle between camera position and mouse cursor position?

Question

In my project, using Three.JS I have implemented the functionality where I want my camera to zoom to the cursor position. Below is the function for that. In this function, I am using a factor as static value 5.

              onWheelMove = (event: WheelEvent): void => {

              let rect: ClientRect = this.canvas.getBoundingClientRect();

              var factor = 5;

              this.mousePos.x = (event.clientX - rect.left) / this.renderer.domElement.width * 2 - 1;
              this.mousePos.y = -(event.clientY - rect.top) / this.renderer.domElement.height * 2 + 1;

              console.log("X-->", this.mousePos.x);
              console.log("Y-->", this.mousePos.y);
              var vector = new THREE.Vector3(this.mousePos.x, this.mousePos.y, 0.5);
              vector.unproject(this.camera);
              console.log("mouse wheel vector -- >", vector);
              vector.sub(this.camera.position);

              console.log("vector.setLength(factor)-- > ", vector.setLength(factor));
              if (event.deltaY < 0) {
                 this.camera.position.addVectors(this.camera.position, vector);
              } else {
                 this.camera.position.subVectors(this.camera.position, vector);
              }

              this.camera.updateMatrixWorld(true);
              this.enableRender();
              this.render(true);
          }

Now Look at the above picture, this is my project viewer and the camera at initialization time is not perpendicular to the model. at this point, if I am keeping my cursor exactly on axis-helper and do mouse wheel up/down, then I am getting below the value of setLength(vector).

vector.setLength(factor)-- > Vector3 {x: 1.6930115334358085, y: -3.3042632885205743, z: -3.348963431838058} -3.30 instead of 5.

Now look at another picture below

Here, you can see that I moved the camera on top view, now when I am doing mouse wheel up/down, then below value is coming vector.setLength(factor)-- > Vector3 {x: 0.02806398289696144, y: -4.998806591956658, z: -0.10557020921938588}

see -4.99 which is closer to 5, that I am setting as a factor. but in this case also, if I am keeping my cursor somewhere else, let say the top right corner of the canvas and do mouse wheel up/down, the value gets changed

vector.setLength(factor)-- > Vector3 {x: -1.724124575016607, y: -4.527894564140576, z: -1.2351377517710418}

-.4.52

now all these values of 3.30, -4.52 which are not exactly 5, messing up my other function. look below

private _setPivotPointOnMouseHit(mouseEvent: MouseEvent, shouldRerender: boolean = true): void {
        this._setRaycasterFromMouse(mouseEvent);

        let rect: ClientRect = this.canvas.getBoundingClientRect();

        let intersects: THREE.Intersection[] = this.octree.raycast(this.raycaster);
        let minTargetToCameraDistanceAllowed = 10;

        intersects.sort(function (a: THREE.Intersection, b: THREE.Intersection): number {
            return a.distance - b.distance;
        });

        if (!intersects || intersects.length === 0) {

            var distance = new THREE.Vector3().subVectors(this.camera.position, this.controls.target).length();

            console.log("camera position-->", this.camera.position);
            console.log("this.controls.target.distanceTo(this.camera.position)",this.controls.target.distanceTo(this.camera.position));
            var targetToCameraDistance = Math.max(minTargetToCameraDistanceAllowed,this.controls.target.distanceTo(this.camera.position));

            console.log("target to camera dstce-->",targetToCameraDistance);
            var vc2 = new THREE.Vector3();
            this.camera.getWorldDirection(vc2);
            var newTarget = vc2.setLength( targetToCameraDistance ).add(this.camera.position);
            console.log("New Target-->",newTarget);

            this.controls.target = newTarget;
            this.controls.update();
            this.camera.updateProjectionMatrix();

            return;
        }

        this._setPivotPoint(intersects[0].point);

        if(shouldRerender) {
            this.render();
        }
    }

In the above function, I am trying to set the orbit controls target even if intersects value is 0. if intersection happening, then everything is working, but if intersection not happening, then my controls target in setting in the middle of object area, because of mismatch between factor 5 and length of vector-based on the cursor position. if I check camera distance in above function, I clearly see 5 is getting add/subtract from an old distance but if I check the camera position, then it is variable based on cursor position and camera look At an angle.

It has something to do with the angle between camera and mouse cursor, but I am not able to understand what.? How can I make sure that no matter the angle/position/cursor, it should always add/subtract factor 5 to camera-target distance and camera position.?