Wall detection using PCL and RANSAC

Question

I have been working on wall detection in PCD (Point Cloud Data) file using PCL (Point Cloud Library). The PCD file has been generated through a depth camera. I found that in many of the similar applications e.g. floor detection, RANSAC has been used. So, I thought of applying RANSAC here as well and I tried my best to understand RANSAC in-general but I still have certain questions pertaining to my application:

1. In brief, RANSAC tries to remove the outliers in the given data and generalize the inliers through a model iteratively. So, in the case of floor detection, would it consider the rest of the point clouds corresponding to other objects as just outliers and the floor as inlier? The same is the case for the walls?

2. As per Plane model segmentation tutorial by PCL, RANSAC is giving the coefficients of the model plane i.e. a, b, c, and d in the equation of plane: a*x + b*y + c*z + d = 0 through coefficients->values vector. So, I assume that in the case of wall detection, it would try to give the equation of the plane corresponding to the wall. However, what if the depth camera is at the corner of a room and the top view of walls look like this:

             wall 1
       ______________
       |
       |
       | wall 2
       |
       |

       So, in this case, what would be the resultant model plane look like? Would it be kind of a        hypotenuse (making a triangle)?

             wall 1
       ---------------------
                                |
                                | wall 2
                                |
                                 ----------------------
                                          wall 3

       Even in this case, how would it look like?

3. As per the Extracting indices from a PointCloud tutorial by PCL, ExtractIndices <pcl::ExtractIndices> filter is used to extract a subset of points from a point cloud based on the indices output by a segmentation algorithm. But, what exactly this filter is doing? In fact, in the case of floor detection or wall detection (assuming there is only one straight wall), RANSAC is already giving an equation of one plane. So, is there any need of using that filter? If yes then why and how?

4. How can I detect multiple walls in the following case? ExtractIndices <pcl::ExtractIndices> filter can do this? If yes then how?

             wall 1
       ---------------------
                                |
                                | wall 2
                                |
                                 ----------------------
                                          wall 3

If you think that there are better ways than using RANSAC then also please let me know.

Answer 1

Answers for a few question you asked:

As fas as I know, in case of using plane model, RANSAC chooses 3 points from the cloud randomly, and considers it as a plane.(this is a provisional statement that will be substantiated later) All the points that are closer to this plane that the given threshold as a perpendicular distance are considered as inliers. The algorithm gives back the plane which contains the most points in it (the found plane also depends on the iteration number you choose, if it is too low maybe it misses the largest plane). In case of walls the story is the same. You can search for planes, but should choose the searching directions well. Walls are casually perpendicular to plane x-y. The parameters should be set considered this. Example:

pcl::SACSegmentation<pcl::PointXYZI> seg;
Eigen::Vector3f axis;

//HELPER VARIABLES
float angle = 12.0;
void set_segmentation(float threshold, int max_iteration, float probability) {
    seg.setModelType(pcl::SACMODEL_PERPENDICULAR_PLANE);
    seg.setMethodType(pcl::SAC_RANSAC);
    // set cloud, threshold, and other paramatres
    seg.setDistanceThreshold(threshold);//Distance need to be adjusted according 
    to the obj
    seg.setMaxIterations(max_iteration);
    seg.setProbability(probability);
}  
PlaneSegment(float x, float y, float z, float set_angle, float threshold, int 
max_iteration, float probability) {
    axis = Eigen::Vector3f(x, y, z);
    angle = set_angle;
    seg.setAxis(axis);
    seg.setEpsAngle(angle * (3.1415 / 180.0f));
    //SET SEGMENTATION
    set_segmentation(threshold, max_iteration, probability);
}
pcl::PointIndices::Ptr segment_plane(pcl::PointCloud<pcl::PointXYZI>::Ptr cloud, 
pcl::PointIndices::Ptr inliers) {
    seg.setInputCloud(cloud);
    seg.segment(*inliers, *coefficients);
    if (inliers->indices.size() == 0)
    {
        PCL_ERROR("COULD NOT ESTIMATE PLANAR MODEL.\n");
        exit(-1);
    }
    return inliers;
}

pcl::PointCloud<pcl::PointXYZI>::Ptr 
extraction(pcl::PointCloud<pcl::PointXYZI>::Ptr cloud, pcl::PointIndices::Ptr 
inliers) {
    extract.setInputCloud(cloud);
    extract.setIndices(inliers);
    extract.setNegative(true);
    extract.filter(*cloud);
    return cloud;
}

You can set an acceptance angle, in this case 12 degrees, and also the searching directions based on the axis.

For your second point:

In case of multiple walls, it will give back the one which contains the most points. But you should be able to extract the other planes also if needed. (advice, save all planes which contains more points than a threshold you choose) I chekced after your problem, this is also a solution for that: pcl::RANSAC segmentation, get all planes in cloud?. First comment gives very good answer.

Third point:

Check the example code. Note, this is a class so thats why there is a constructor. The segment_plane function returns inliers. Based on that you can call the extraction function, and it removes the inliers from the cloud. This is a very simple and fast soulition for this. You can avoid the suffering with the coefficients values. Also, if you dont want to remove them just colour them by iterating through the inliers and set its intensity to a chosen value.

RANSAC algorithm can be robust, but sometimes it just really does not work. Also it can be slow because of the iteration number. There are multiple ways to solve this problem on another way. Just an example: consider a grid below the cloud. A lot of equal sized square cells. In each cell you check the minimum and maximum point heights. Based on this you can get the ground plane (If these values are just slightly different, and are close to each other. The difference of the maximum heigth and the minimum height is very low in case of ground cell) or the walls. With walls you can assume that the points have an even distribution in each cells and the difference of the maximum - minimum values are high. Best regards.