How to correctly position html elements in three js coordinate system?

Question

I hopefully have a simple problem I can't get an answer to.

I have three js geometric spheres which move in a box. I place this box at the centre of the scene. The mechanics of how the spheres stay in the box is irrelevant. What is important is the spheres move about the origin (0,0) and the canvas always fills the page.

I want to draw a line from the moving spheres to a div or img element on the page. To do this I would assume I have to transform the css coordinates to three js coordinates. I found something I thought did something like this (Note: Over use of somethings to signify I am probably mistaken)

I can add a html element to the same scene/camera as webgl renderer but obviously using a different renderer but I am unsure how to proceed from there?

Basically I want to know:

• How should I change the size of the div preserving aspect ratio if need be?
  • In essence I want the div or element to fill screen at some camera depth.
• How to place the div at the centre of the scene by default?
  • Mines seems to be shifted 1000 in z direction but this might be the size of the div(img) which I have to bring into view.
• How to draw a line between the webgl sphere and html div/img?

thanks in advance!

Answer 1

Unfortunately you have asked 3 questions, it is tricky to address them all at once.

I will explain how to position DIV element on top of some 3D object. My example would be a tooltip that appears when you hover the object by mouse: http://jsfiddle.net/mmalex/ycnh0wze/

So let's get started,

First of all you need to subscribe mouse events and convert 2D coordinates of a mouse to relative coordinates on the viewport. Very well explained you will find it here: Get mouse clicked point's 3D coordinate in three.js

Having 2D coordinates, raycast the object. These steps are quite trivial, but for completeness I provide the code chunk.

var raycaster = new THREE.Raycaster();

function handleManipulationUpdate() {
    // cleanup previous results, mouse moved and they're obsolete now
    latestMouseIntersection = undefined;
    hoveredObj = undefined;

    raycaster.setFromCamera(mouse, camera);
    {
        var intersects = raycaster.intersectObjects(tooltipEnabledObjects);
        if (intersects.length > 0) {
            // keep point in 3D for next steps
            latestMouseIntersection = intersects[0].point;
            // remember what object was hovered, as we will need to extract tooltip text from it
            hoveredObj = intersects[0].object;
        }
    }
    ... // do anything else

    //with some conditions it may show or hide tooltip
    showTooltip();
}

// Following two functions will convert mouse coordinates
// from screen to three.js system (where [0,0] is in the middle of the screen)
function updateMouseCoords(event, coordsObj) {
    coordsObj.x = ((event.clientX - renderer.domElement.offsetLeft + 0.5) / window.innerWidth) * 2 - 1;
    coordsObj.y = -((event.clientY - renderer.domElement.offsetTop + 0.5) / window.innerHeight) * 2 + 1;
}

function onMouseMove(event) {
    updateMouseCoords(event, mouse);
    handleManipulationUpdate();
}

 window.addEventListener('mousemove', onMouseMove, false);

And finally see the most important part, DIV element placement. To understand the code it is essential to get convenient with Vector3.project method.

The sequence of calculations is as follows:

1. Get 2D mouse coordinates,
2. Raycast object and remember 3D coordinate of intersection (if any),
3. Project 3D coordinate back into 2D (this step may seem redundant here, but what if you want to trigger object tooltip programmatically? You won't have mouse coordinates)
4. Mess around to place DIV centered above 2D point, with nice margin.

// This will move tooltip to the current mouse position and show it by timer.
function showTooltip() {
    var divElement = $("#tooltip");

    //element found and mouse hovers some object?
    if (divElement && latestMouseIntersection) {
        //hide until tooltip is ready (prevents some visual artifacts)
        divElement.css({
            display: "block",
            opacity: 0.0
        });

        //!!! === IMPORTANT === 
        // DIV element is positioned here
        var canvasHalfWidth = renderer.domElement.offsetWidth / 2;
        var canvasHalfHeight = renderer.domElement.offsetHeight / 2;

        var tooltipPosition = latestMouseProjection.clone().project(camera);
        tooltipPosition.x = (tooltipPosition.x * canvasHalfWidth) + canvasHalfWidth + renderer.domElement.offsetLeft;
        tooltipPosition.y = -(tooltipPosition.y * canvasHalfHeight) + canvasHalfHeight + renderer.domElement.offsetTop;

        var tootipWidth = divElement[0].offsetWidth;
        var tootipHeight = divElement[0].offsetHeight;

        divElement.css({
            left: `${tooltipPosition.x - tootipWidth/2}px`,
            top: `${tooltipPosition.y - tootipHeight - 5}px`
        });

        //get text from hovered object (we store it in .userData)
        divElement.text(hoveredObj.userData.tooltipText);

        divElement.css({
            opacity: 1.0
        });
    }
}