2

I need to sort a selection of 3D coordinates in a winding order as seen in the image below. The bottom-right vertex should be the first element of the array and the bottom-left vertex should be the last element of the array. This needs to work given any direction that the camera is facing the points and at any orientation of those points. Since "top-left","bottom-right", etc is relative, I assume I can use the camera as a reference point? We can also assume all 4 points will be coplanar.

I am using the Blender API (writing a Blender plugin) and have access to the camera's view matrix if that is even necessary. Mathematically speaking is this even possible if so how? Maybe I am overcomplicating things?

4 corners in winding order

Since the Blender API is in Python I tagged this as Python, but I am fine with pseudo-code or no code at all. I'm mainly concerned with how to approach this mathematically as I have no idea where to start.

Colton Fox
  • 41
  • 7
  • 1
    Are you guaranteed that the four points are coplanar? – Stef Jan 07 '22 at 10:27
  • 1
    Do you have a reference to explain what the camera's "view matrix" represent exactly? I assume it's a rotation matrix, but understanding "what it rotates" precisely is important to determine where are up, left, right, down. – Stef Jan 07 '22 at 10:29
  • 1
    Yes in this case we can guarantee the points are coplanar and I will add that into the post. It appears the `view_matrix` is indeed a rotation matrix, but the [docs](https://docs.blender.org/api/current/bpy.types.RegionView3D.html#bpy.types.RegionView3D.view_matrix) also give me access to the `view_rotation` which is a quaternion that is always normalized. I can also access the `perspective_matrix` which the docs state is the `window_matrix * view_matrix`, maybe that would also be helpful? – Colton Fox Jan 07 '22 at 14:51
  • 1
    Apparently, I am wrong about the `view_matrix`. I asked one of the Blender folks and they said it essentially allows you to transform from world space into camera space. – Colton Fox Jan 07 '22 at 14:53
  • 1
    This [article](https://www.3dgep.com/understanding-the-view-matrix/) explains it well. "The View Matrix: This matrix will transform vertices from world-space to view-space. This matrix is the inverse of the camera’s transformation matrix." – Colton Fox Jan 07 '22 at 14:58

3 Answers3

2

Since you assume the four points are coplanar, all you need to do is find the centroid, calculate the vector from the centroid to each point, and sort the points by the angle of the vector.

import numpy as np

def sort_points(pts):
    centroid = np.sum(pts, axis=0) / pts.shape[0]
    vector_from_centroid = pts - centroid
    vector_angle = np.arctan2(vector_from_centroid[:, 1], vector_from_centroid[:, 0]) 
    sort_order = np.argsort(vector_angle) # Find the indices that give a sorted vector_angle array

    # Apply sort_order to original pts array. 
    # Also returning centroid and angles so I can plot it for illustration. 
    return (pts[sort_order, :], centroid, vector_angle[sort_order])

This function calculates the angle assuming that the points are two-dimensional, but if you have coplanar points then it should be easy enough to find the coordinates in the common plane and eliminate the third coordinate.

Let's write a quick plot function to plot our points:

from matplotlib import pyplot as plt

def plot_points(pts, centroid=None, angles=None, fignum=None):
    fig = plt.figure(fignum)
    plt.plot(pts[:, 0], pts[:, 1], 'or')
    if centroid is not None:
        plt.plot(centroid[0], centroid[1], 'ok')
        
    for i in range(pts.shape[0]):
        lstr = f"pt{i}"
        if angles is not None:
            lstr += f" ang: {angles[i]:.3f}"
        plt.text(pts[i, 0], pts[i, 1], lstr)
    
    return fig

And now let's test this:

With random points:

pts = np.random.random((4, 2))
spts, centroid, angles = sort_points(pts)
plot_points(spts, centroid, angles)

enter image description here

With points in a rectangle:

pts = np.array([[0, 0],  # pt0
                [10, 5], # pt2
                [10, 0], # pt1
                [0, 5]]) # pt3
spts, centroid, angles = sort_points(pts)
plot_points(spts, centroid, angles)

enter image description here


It's easy enough to find the normal vector of the plane containing our points, it's simply the (normalized) cross product of the vectors joining two pairs of points:

plane_normal = np.cross(pts[1, :] - pts[0, :], pts[2, :] - pts[0, :])
plane_normal = plane_normal / np.linalg.norm(plane_normal)

Now, to find the projections of all points in this plane, we need to know the "origin" and basis of the new coordinate system in this plane. Let's assume that the first point is the origin, the x axis joins the first point to the second, and since we know the z axis (plane normal) and x axis, we can calculate the y axis.

new_origin = pts[0, :]
new_x = pts[1, :] - pts[0, :]
new_x = new_x / np.linalg.norm(new_x)

new_y = np.cross(plane_normal, new_x)

Now, the projections of the points onto the new plane are given by this answer:

proj_x = np.dot(pts - new_origin, new_x)
proj_y = np.dot(pts - new_origin, new_y)

Now you have two-dimensional points. Run the code above to sort them.

Pranav Hosangadi
  • 23,755
  • 7
  • 44
  • 70
  • *"This function calculates the angle assuming that the points are two-dimensional, but if you have coplanar points then it should be easy enough to find the coordinates in the common plane and eliminate the third coordinate."* I find this dubious. I think ordering points in 2d is the easier part of this problem; correctly identifying the plane and its orientation given the 3d coordinates of the points and the camera matrix is the trickier part of this problem. – Stef Jan 07 '22 at 16:46
  • Your approach makes a lot of sense and is really interesting. I am stuck, however, with eliminating the third coordinate. Could you explain that a little more? I'm also confused about what the "common plane" would be. – Colton Fox Jan 07 '22 at 17:07
  • @Stef I did not see your comment before posting mine. I guess what you mention here: "correctly identifying the plane and its orientation given the 3d coordinates of the points and the camera matrix" is the part I am struggling on now. That step needs to be taken first before you can even eliminate the third coordinate, correct? – Colton Fox Jan 07 '22 at 17:10
  • @ColtonFox I assumed you knew how to do this already since you said the points are coplanar. I'm working on an edit to show how to do this. If you want to do some reading in the meantime, here's a reference: https://stackoverflow.com/q/9605556/843953. You can find the normal vector to the plane by calculating the cross product `(p3 - p1) x (p2 - p1)`. – Pranav Hosangadi Jan 07 '22 at 17:12
  • @ColtonFox can you confirm that the points you are starting with are coplanar? Or would you like me to also discuss how to project the points onto a plane? – Pranav Hosangadi Jan 07 '22 at 17:32
  • Yes, the points will always be coplanar to start with. Since this tool takes a user's selection of points before ordering them, I have a function that checks if their selection of points is coplanar before running the rest of the code. For the sake of this tool, there is no reason why they would need to select non-coplanar points. I still am not sure how to eliminate the third coordinate so I appreciate you taking your time to make that edit. I'll give that post you linked a read as well. – Colton Fox Jan 07 '22 at 17:43
  • So I implemented your solution. I projected the points to a 2d plane then sorted the points all while keeping track of what the original 3d points were. However, it is not behaving as intended and I think, as Stef mentioned, this is due to the fact that we still need to use the camera as a point of reference. I have created a gif of me running the tool. I have `pts[0]` and `pts[2]` highlighted in green after the tool is ran. Notice how in the first selection the points are at the bottom right and top left, but when I go to the other side of the mesh and run the tool the points are at the... – Colton Fox Jan 07 '22 at 18:33
  • bottom left and top right. https://gyazo.com/685c329d6e82661f635b7a50813ee3ad . In both instances they should be at the top left and bottom right. This seems only possible if we use the camera as a reference point, or perhaps I am understanding incorrectly. – Colton Fox Jan 07 '22 at 18:34
  • I figured your coplanar points already in the camera's coordinate system. Since you have the `view_matrix`, you could transform your points using this matrix, and use those transformed points? I don't quite understand what that gif is showing @ColtonFox – Pranav Hosangadi Jan 07 '22 at 18:37
  • Transforming the points using the camera's view_matrix makes no difference. In the gif, "Create Portal from Verts" will transform the points by the view_matrix, project the 4 selected points onto a 2d plane, and sort them using the code you sent all while keeping track of the original corresponding 3d points. I then displayed the 1st and 3rd points in green (the two corner points). Those two corner points should always be the bottom right and top left, but notice when I move the camera and run the tool on a different set of points, the two corners are on the bottom left and top right. – Colton Fox Jan 07 '22 at 18:52
  • This is my code so far if you want to take a look: https://pastebin.com/2aYZJbPV – Colton Fox Jan 07 '22 at 18:57
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/240841/discussion-between-pranav-hosangadi-and-colton-fox). – Pranav Hosangadi Jan 07 '22 at 18:59
1

After many hours, I finally found a solution. @Pranav Hosangadi's solution worked for the 2D side of things. However, I was having trouble projecting the 3D coordinates to 2D coordinates using the second part of his solution. I also tried projecting the coordinates as described in this answer, but it did not work as intended. I then discovered an API function called location_3d_to_region_2d() (see docs) which, as the name implies, gets the 2D screen coordinates in pixels of the given 3D coordinate. I didn't need to necessarily "project" anything into 2D in the first place, getting the screen coordinates worked perfectly fine and is much more simple. From that point, I could sort the coordinates using Pranav's function with some slight adjustments to get it in the order illustrated in the screenshot of my first post and I wanted it returned as a list instead of a NumPy array.

import bpy
from bpy_extras.view3d_utils import location_3d_to_region_2d
import numpy

def sort_points(pts):
    """Sort 4 points in a winding order"""
    pts = numpy.array(pts)
    centroid = numpy.sum(pts, axis=0) / pts.shape[0]
    vector_from_centroid = pts - centroid
    vector_angle = numpy.arctan2(
        vector_from_centroid[:, 1], vector_from_centroid[:, 0])
    # Find the indices that give a sorted vector_angle array
    sort_order = numpy.argsort(-vector_angle)

    # Apply sort_order to original pts array.
    return list(sort_order)

# Get 2D screen coords of selected vertices
region = bpy.context.region
region_3d = bpy.context.space_data.region_3d

corners2d = []
for corner in selected_verts:
    corners2d.append(location_3d_to_region_2d(
        region, region_3d, corner))

# Sort the 2d points in a winding order
sort_order = sort_points(corners2d)
sorted_corners = [selected_verts[i] for i in sort_order]

Thanks, Pranav for your time and patience in helping me solve this problem!

Colton Fox
  • 41
  • 7
0

There is a simpler and faster solution for the Blender case:

1.) The following code sorts 4 planar points in 2D (vertices of the plane object in Blender) very efficiently:

def sort_clockwise(pts):
    rect = np.zeros((4, 2), dtype="float32")
    s = pts.sum(axis=1)
    rect[0] = pts[np.argmin(s)]
    rect[2] = pts[np.argmax(s)]
    diff = np.diff(pts, axis=1)
    rect[1] = pts[np.argmin(diff)]
    rect[3] = pts[np.argmax(diff)]
    return rect

2.) Blender keeps vertices related data, such as the translation, rotation and scale in the world matrix. If you query for vertices.co(ordinates) only, you just get the original coordinates, without translation, rotation and scaling. But that does not affect the order of vertices. That simplifies the problem because what you get is actually a 2D (with z's = 0) mesh data. If you sort that 2D data (excluding z's) you will get the information, the sort indices for the 3D sorted data. You can modify the code above to get the indices from that 2D array. For the plane object of Blender, for some reason the order is always [0,1,3,2], not [0,1,2,3]. The following modified code gives the sorted indices for the vertices data in 2D.

def sorted_ix_clockwise(pts):
    #rect = zeros((4, 2), dtype="float32")
    ix = array([0,0,0,0])
    s = pts.sum(axis=1)
    #rect[0] = pts[argmin(s)]
    #rect[2] = pts[argmax(s)]
    ix[0] = argmin(s)
    ix[2] = argmax(s)
    dif = diff(pts, axis=1)
    #rect[1] = pts[argmin(dif)]
    #rect[3] = pts[argmax(dif)]
    ix[1] = argmin(dif)
    ix[3] = argmax(dif)
    return ix

You can use these indices to get the actual 3D sorted data, which you can obtain by multiplying vertices coordinates with the world matrix to include any translation, rotation and scaling.

user2800464
  • 113
  • 3
  • 11