0

I have a 2x2 matrix of distances from a depth sensor. The matrix is cropped so only the points we are interested in is in the frame(All the points in the cropped image contains the object). My question is how can we determine if this object is flat or not?

The depth image is acquired from Realsense d435. I read the depth image and then multiply it by depth_scale. The object is recognized using AI for the rgb image that is aligned with the depth image. And I have 4 points on the object. So, all the distances in that rectangle contains the distance of the object from the sensor.

My first idea was standard deviation of all the points. But then this falls apart if the image is taken from an angle. (since the standard deviation won't be 0)

From an angle the distance of a flat object is changing uniformly on the y axis. Maybe somehow, we can use this information?

The 2x2 matrix is a numpy array in python. Maybe there are some libraries which do this already.

MROJ4N
  • 1
  • 2
    With a PCA (principal component analysis) you can compute the oriented bounding box of a set of points. If one of the dimensions is nearly 0, the points shape a plane – Micka Dec 11 '22 at 20:09

2 Answers2

1

After reprojecting your four depth measurements to the 3D space, it becomes a problem of deciding if your set of points is coplanar. There are several ways you can go about it.

One way to do it is to reproject the points to 3D and fit a plane to all four of them there. Since you're fitting a plane to four points in three dimensions, you get an over-determined system, and it's very unlikely that all points would lie exactly on the estimated plane. At this stage, you could prescribe some tolerance to determine "goodness of fit". For instance, you could look at the R^2 coefficient. To fit the plane you can use scipy.linalg.lstsq. Here's a good description of how it can be done: Fit plane to a set of points in 3D.

Another way to approach the problem is by calculating the volume of a tetrahedron spanned by the four points in 3D. If they are coplanar (or close to coplanar), the volume of such a tatrahedron should be equal to (or close to) 0. Assuming your pointa reprojected to 3D can be described by (x_0, y_0, z_0), ..., (x_3, y_3, z_3), the volume of the tetrahedron is equal to:

volume = abs(numpy.linalg.det(tetrahedron)) / 6, where

tetrahedron = np.array([[x_0, y_0, z_0, 1], [x_1, y_1, z_1, 1], [x_2, y_2, z_2, 1], [x_3, y_3, z_3, 1]])

To check if your points are on the same plane, (equivalently - if the tetrahedron has a small enough volume), it is now sufficient to check if

volume < TOL

for some defined small tolerance value, which must be determined experimentally.

pwswierczynski
  • 23
  • 4
0

You can define a surface by choosing three of the four 3D points. Evaluate the distance from the remaining point to the surface.

How to choose the three points is... it may be good to choose the pattern that maximizes the area of the triangle.

fana
  • 1,370
  • 2
  • 7
  • I don't think it's a good idea to use a constant value as threshold for this evaluation value. Maybe, it should decide the threshold so that it is proportional to the size of the triangle...? – fana Dec 12 '22 at 05:37