THREE.JS: Conversion from 3D Perspective scene to 2D Orthographic hud scene

Question

I have objects (very far away) in a 3D scene, using a perspective camera and a 2D HUD set up using a orthographic camera:

this.scene = new THREE.Scene();
this.hud = new THREE.Scene();

this.camera = new THREE.PerspectiveCamera( 30, aspect, front, back );
this.camera.position.set(0,0,0);

this.hudCamera = new THREE.OrthographicCamera (-this.windowHalfX,this.windowHalfX, this.windowHalfY, -this.windowHalfY, 1, 10);
this.hudCamera.position.set(0,0,10);

Here is my render loop:

  updateFrame : function () {
    this.renderer.clear();
    this.renderer.render( this.scene, this.camera );
    this.renderer.clearDepth();
    this.renderer.render( this.hud, this.hudCamera );
  },

How can I find the position of the objects in the HUD, using their position in the 3D scene?

Your question does not make any sense because your orthographic camera parameters are in pixels. Read http://stackoverflow.com/questions/17558085/three-js-orthographic-camera/17567292#17567292. — WestLangley, May 15 '15 at 15:55
I have realized this. I have a solution which I'll be posting promptly. — Boogiechillin, May 18 '15 at 20:12

score 3 · Accepted Answer · edited May 23 '17 at 12:33

In order to find the 2D HUD position of a 3D object (using three.js version r71), you may do the following, which I have modified from this post:

  findHUDPosition : function (obj) {
      var vector = new THREE.Vector3();

      obj.updateMatrixWorld();
      vector.setFromMatrixPosition(obj.matrixWorld);
      vector.project(this.camera);

      vector.x = ( vector.x * this.windowHalfX );
      vector.y = ( vector.y * this.windowHalfY );

      return {
          x: vector.x,
          y: vector.y,
          z: vector.z
      }
  }

The parameter obj is the object you are trying to find the position of in the hud.

vector.project(this.camera); draws a vector from the object to the position of this.camera, through the near plane of the camera.

The new value of vector's components are the intersection of the projected vector and this.camera's near plane.

The coordinates are in three.js' world coordinate system though, so we have to do a quick conversion to pixel coordinates, to scale up to the size of our canvas.

  vector.x = ( vector.x * this.windowHalfX );
  vector.y = ( vector.y * this.windowHalfY );

The above conversion is for a setup where the HUD's coordinate system has an origin (0,0) at the center of the screen, and has a maximum value of half the canvas' resolution. For example, if your canvas is 1024 x 768 pixels, the position of the upper right corner would be (512, 384).

For a typical screen coordinate system, the bottom right corner would be (1024, 768), and the middle of the screen would be (512, 384). To have this setup, you can use the following conversion, as seen in this post.

    vector.x = ( vector.x * widthHalf ) + widthHalf;
    vector.y = - ( vector.y * heightHalf ) + heightHalf;

Notice, the z coordinate doesn't matter now, since we are in 2D.

The last thing you'll want to do is make sure that the object you're showing in 2D is actually visible to the perspective camera. This is as simple as checking to see if the object falls within the frustum of this.camera. source for following code

checkFrustrum : function (obj) {
    var frustum = new THREE.Frustum();
    var projScreenMatrix = new THREE.Matrix4();

    this.camera.updateMatrix();
    this.camera.updateMatrixWorld();

    projScreenMatrix.multiplyMatrices( this.camera.projectionMatrix, this.camera.matrixWorldInverse );

    frustum.setFromMatrix( new THREE.Matrix4().multiplyMatrices( this.camera.projectionMatrix,
                                                     this.camera.matrixWorldInverse ) );
    return frustum.containsPoint ( obj.position );
}

If this is not done, you can have an object which is behind the camera being registered as visible in the 2D scene (this poses problems for object tracking). It's also good practice to update obj's matrix and matrix world.

THREE.JS: Conversion from 3D Perspective scene to 2D Orthographic hud scene

1 Answers1