I have a scene where I want to combine perspective objects (ie. objects that appear smaller when they are far away) with orthogographic objects (ie. objects that appear the same size irrespective of distance). The perspective objects are part of the rendered "world", while the orthogographic objects are adornments, like labels or icons. Unlike a HUD, I want the orthogographic objects to be rendered "within" the world, which means that they can be covered by world objects (imagine a plane passing before a label).
My solution is to use one renderer, but two scenes, one with a PerspectiveCamera and one with an OrthogographicCamera. I render them in sequence without clearing the z buffer (the renderer's autoClear property is set to false). The problem that I am facing is that I need to synchronize the placement of the objects in each scene so that an object in one scene is assigned a z-position that is behind objects in the other scene that are before it, but before objects that are behind it.
To do that, I am designating my perspective scene as the "leading" scene, ie. all coordinates of all objects (perspective and orthogographic) are assigned based on this scene. The perspective objects use these coordinates directly and are rendered within that scene and with the perspective camera. The coordinates of the orthogographic objects are transformed to the coordinates in the orthogographic scene and then rendered in that scene with the orthogographic camera. I do the transformation by projecting the coordinates in the perspective scene to the perspective camera's view pane and then back to the orthogonal scene with the orthogographic camera:
position.project(perspectiveCamera).unproject(orthogographicCamera);
Alas, this is not working as indended. The orthogographic objects are always rendered before the perspective objects even if they should be between them. Consider this example, in which the blue circle should be displayed behind the red square, but before the green square (which it isn't):
var pScene = new THREE.Scene();
var oScene = new THREE.Scene();
var pCam = new THREE.PerspectiveCamera(40, window.innerWidth / window.innerHeight, 1, 1000);
pCam.position.set(0, 40, 50);
pCam.lookAt(new THREE.Vector3(0, 0, -50));
var oCam = new THREE.OrthographicCamera(window.innerWidth / -2, window.innerWidth / 2, window.innerHeight / 2, window.innerHeight / -2, 1, 500);
oCam.Position = pCam.position.clone();
pScene.add(pCam);
pScene.add(new THREE.AmbientLight(0xFFFFFF));
oScene.add(oCam);
oScene.add(new THREE.AmbientLight(0xFFFFFF));
var frontPlane = new THREE.Mesh(new THREE.PlaneGeometry(20, 20), new THREE.MeshBasicMaterial( { color: 0x990000 }));
frontPlane.position.z = -50;
pScene.add(frontPlane);
var backPlane = new THREE.Mesh(new THREE.PlaneGeometry(20, 20), new THREE.MeshBasicMaterial( { color: 0x009900 }));
backPlane.position.z = -100;
pScene.add(backPlane);
var circle = new THREE.Mesh(new THREE.CircleGeometry(60, 20), new THREE.MeshBasicMaterial( { color: 0x000099 }));
circle.position.z = -75;
//Transform position from perspective camera to orthogonal camera -> doesn't work, the circle is displayed in front
circle.position.project(pCam).unproject(oCam);
oScene.add(circle);
var renderer = new THREE.WebGLRenderer();
renderer.setSize(window.innerWidth, window.innerHeight);
document.body.appendChild(renderer.domElement);
renderer.autoClear = false;
renderer.render(oScene, oCam);
renderer.render(pScene, pCam);
You can try out the code here.
In the perspective world the (world) z-position of the circle is -75, which is between the squares (-50 and -100). But it is actually displayed in front of both squares. If you manually set the circles z-position (in the orthogographic scene) to -500 it is displayed between the squares, so with the right positioning, what I'm trying should be possible in principle.
I know that I can not render a scene the same with orthogographic and perspective cameras. My intention is to reposition all orthogographic objects before each rendering so that they appear to be at the right position.
What do I have to do to calculate the orthogographic coordinates from the perspective coordinates so that the objects are rendered with the right depth values?
UPDATE:
I have added an answer with my current solution to the problem in case someone has a similar problem. However, since this solution does not provide the same quality as the orthogographic camera. So I would still be happy if somoeone could explain why the orthogographic camera does not work as expected and/or provide a solution to the problem.
