Unity function to access the 2D box immediately from the 3D pipeline?

Question

In Unity, say you have a 3D object,

Of course, it's trivial to get the AABB, Unity has direct functions for that,

(You might have to "add up all the bounding boxes of the renderers" in the usual way, no issue.)

So Unity does indeed have a direct function to give you the 3D AABB box instantly, out of the internal mesh/render pipeline every frame.

Now, for the Camera in question, as positioned, that AABB indeed covers a certain 2D bounding box ...

In fact ... is there some sort of built-in direct way to find that orange 2D box in Unity??

Question - does Unity have a function which immediately gives that 2D frustrum box from the pipeline?

(Note that to do it manually you just make rays (or use world to screen space as Draco mentions, same) for the 8 points of the AABB; encapsulate those in 2D to make the orange box.)

I don't need a manual solution, I'm asking if the engine gives this somehow from the pipeline every frame?

Is there a call?

(Indeed, it would be even better to have this ...)

My feeling is that one or all of the

• occlusion system in particular
• the shaders
• the renderer

would surely know the orange box, and perhaps even the blue box inside the pipeline, right off the graphics card, just as it knows the AABB for a given mesh.

We know that Unity lets you tap the AABB 3D box instantly every frame for a given mesh: In fact does Unity give the "2D frustrum bound" as shown here?

Answer 1

As far as I am aware, there is no built in for this.

However, finding the extremes yourself is really pretty easy. Getting the mesh's bounding box (the cuboid shown in the screenshot) is just how this is done, you're just doing it in a transformed space.

1. Loop through all the verticies of the mesh, doing the following:
2. Transform the point from local to world space (this handles dealing with scale and rotation)
3. Transform the point from world space to screen space
4. Determine if the new point's X and Y are above/below the stored min/max values, if so, update the stored min/max with the new value
5. After looping over all vertices, you'll have 4 values: min-X, min-Y, max-X, and max-Y. Now you can construct your bounding rectangle

You may also wish to first perform a Gift Wrapping of the model first, and only deal with the resulting convex hull (as no points not part of the convex hull will ever be outside the bounds of the convex hull). If you intend to draw this screen space rectangle while the model moves, scales, or rotates on screen, and have to recompute the bounding box, then you'll want to do this and cache the result.

Note that this does not work if the model animates (e.g. if your humanoid stands up and does jumping jacks). Solving for the animated case is much more difficult, as you would have to treat every frame of every animation as part of the original mesh for the purposes of the convex hull solving (to insure that none of your animations ever move a part of the mesh outside the convex hull), increasing the complexity by a power.

Answer 2

3D bounding box

1. Get given GameObject 3D bounding box's center and size
2. Compute 8 corners
3. Transform positions to GUI space (screen space)

Function GUI3dRectWithObject will return the 3D bounding box of given GameObject on screen.

2D bounding box

1. Iterate through every vertex in a given GameObject
2. Transform every vertex's position to world space, and transform to GUI space (screen space)
3. Find 4 corner value: x1, x2, y1, y2

Function GUI2dRectWithObject will return the 2D bounding box of given GameObject on screen.

Code

public static Rect GUI3dRectWithObject(GameObject go)
{

    Vector3 cen = go.GetComponent<Renderer>().bounds.center;
    Vector3 ext = go.GetComponent<Renderer>().bounds.extents;
    Vector2[] extentPoints = new Vector2[8]
    {
            WorldToGUIPoint(new Vector3(cen.x-ext.x, cen.y-ext.y, cen.z-ext.z)),
            WorldToGUIPoint(new Vector3(cen.x+ext.x, cen.y-ext.y, cen.z-ext.z)),
            WorldToGUIPoint(new Vector3(cen.x-ext.x, cen.y-ext.y, cen.z+ext.z)),
            WorldToGUIPoint(new Vector3(cen.x+ext.x, cen.y-ext.y, cen.z+ext.z)),
            WorldToGUIPoint(new Vector3(cen.x-ext.x, cen.y+ext.y, cen.z-ext.z)),
            WorldToGUIPoint(new Vector3(cen.x+ext.x, cen.y+ext.y, cen.z-ext.z)),
            WorldToGUIPoint(new Vector3(cen.x-ext.x, cen.y+ext.y, cen.z+ext.z)),
            WorldToGUIPoint(new Vector3(cen.x+ext.x, cen.y+ext.y, cen.z+ext.z))
    };
    Vector2 min = extentPoints[0];
    Vector2 max = extentPoints[0];
    foreach (Vector2 v in extentPoints)
    {
        min = Vector2.Min(min, v);
        max = Vector2.Max(max, v);
    }
    return new Rect(min.x, min.y, max.x - min.x, max.y - min.y);
}

public static Rect GUI2dRectWithObject(GameObject go)
{
    Vector3[] vertices = go.GetComponent<MeshFilter>().mesh.vertices;

    float x1 = float.MaxValue, y1 = float.MaxValue, x2 = 0.0f, y2 = 0.0f;

    foreach (Vector3 vert in vertices)
    {
        Vector2 tmp = WorldToGUIPoint(go.transform.TransformPoint(vert));

        if (tmp.x < x1) x1 = tmp.x;
        if (tmp.x > x2) x2 = tmp.x;
        if (tmp.y < y1) y1 = tmp.y;
        if (tmp.y > y2) y2 = tmp.y;
    }

    Rect bbox = new Rect(x1, y1, x2 - x1, y2 - y1);
    Debug.Log(bbox);
    return bbox;
}

public static Vector2 WorldToGUIPoint(Vector3 world)
{
    Vector2 screenPoint = Camera.main.WorldToScreenPoint(world);
    screenPoint.y = (float)Screen.height - screenPoint.y;
    return screenPoint;
}

Reference: Is there an easy way to get on-screen render size (bounds)?

Answer 3

refer to this

It needs the game object with skinnedMeshRenderer.

Camera camera = GetComponent(); 
SkinnedMeshRenderer skinnedMeshRenderer = target.GetComponent(); 
// Get the real time vertices 
Mesh mesh = new Mesh(); 
skinnedMeshRenderer.BakeMesh(mesh); 
Vector3[] vertices = mesh.vertices; 
for (int i = 0; i < vertices.Length; i++) 
{ 
    // World space 
    vertices[i] = target.transform.TransformPoint(vertices[i]); 
    // GUI space 
    vertices[i] = camera.WorldToScreenPoint(vertices[i]); 
    vertices[i].y = Screen.height - vertices[i].y; 
} 
Vector3 min = vertices[0]; 
Vector3 max = vertices[0]; 
for (int i = 1; i < vertices.Length; i++) 
{ 
    min = Vector3.Min(min, vertices[i]); 
    max = Vector3.Max(max, vertices[i]); 
} 
Destroy(mesh); 
// Construct a rect of the min and max positions 
Rect r = Rect.MinMaxRect(min.x, min.y, max.x, max.y); 
GUI.Box(r, "");