6

I'm working on a 3D reconstruction system and want to generate a triangular mesh from the registered point cloud data using Python 3. My objects are not convex, so the marching cubes algorithm seems to be the solution.

I prefer to use an existing implementation of such method, so I tried scikit-image and Open3d but both the APIs do not accept raw point clouds as input (note that I'm not expert of those libraries). My attempts to convert my data failed and I'm running out of ideas since the documentation does not clarify the input format of the functions.

These are my desired snippets where pcd_to_volume is what I need.

scikit-image

import numpy as np
from skimage.measure import marching_cubes_lewiner

N = 10000
pcd = np.random.rand(N,3)

def pcd_to_volume(pcd, voxel_size):
    #TODO

volume = pcd_to_volume(pcd, voxel_size=0.05)

verts, faces, normals, values = marching_cubes_lewiner(volume, 0)

open3d

import numpy as np
import open3d

N = 10000
pcd = np.random.rand(N,3)

def pcd_to_volume(pcd, voxel_size):
    #TODO

volume = pcd_to_volume(pcd, voxel_size=0.05)

mesh = volume.extract_triangle_mesh()

I'm not able to find a way to properly write the pcd_to_volume function. I do not prefer a library over the other, so both the solutions are fine to me.

Do you have any suggestions to properly convert my data? A point cloud is a Nx3 matrix where dtype=float.

Do you know another implementation [of the marching cube algorithm] that works on raw point cloud data? I would prefer libraries like scikit and open3d, but I will also take into account github projects.

Marco Carletti
  • 318
  • 2
  • 5
  • 16
  • Thanks for this question. Actually I'm having the same problem now. Just curious, have you found any solutions? Thank you! – CodingNow Dec 10 '19 at 18:58
  • 1
    @CodingNow Not yet, unfortunately. As a workaround I've tried the [alphaShape](https://www.mathworks.com/help/matlab/ref/alphashape.html) in Matlab and it works decently. However, I haven't found any good alternative in Python. – Marco Carletti Dec 11 '19 at 11:50
  • Thank you for the info. Matlab is always the winner. – CodingNow Dec 11 '19 at 18:13

1 Answers1

5

Do you know another implementation [of the marching cube algorithm] that works on raw point cloud data?

Hoppe's paper Surface reconstruction from unorganized points might contain the information you needed and it's open sourced.

And latest Open3D seems to be containing surface reconstruction algorithms like alphaShape, ballPivoting and PoissonReconstruction.

From what I know, marching cubes is usually used for extracting a polygonal mesh of an isosurface from a three-dimensional discrete scalar field (that's what you mean by volume). The algorithm does not work on raw point cloud data.

Hoppe's algorithm works by first generating a signed distance function field (a SDF volume), and then passing it to marching cubes. This can be seen as an implementation to you pcd_to_volume and it's not the only way!

If the raw point cloud is all you have, then the situation is a little bit constrained. As you might see, the Poisson reconstruction and Screened Poisson reconstruction algorithm both implement pcd_to_volume in their own way (they are highly related). However, they needs additional point normal information, and the normals have to be consistently oriented. (For consistent orientation you can read this question).

While some Delaunay based algorithm (they do not use marching cubes) like alphaShape and this may not need point normals as input, for surfaces with complex topology, it's hard to get a satisfactory result due to orientation problem. And the graph cuts method can use visibility information to solve that.

Having said that, if your data comes from depth images, you will usually have visibility information. And you can use TSDF to build a good surface mesh. Open3D have already implemented that.

Jing Zhao
  • 2,420
  • 18
  • 21
  • I will definitely look at those papers. Thank you! I'm not expert of 3D reconstruction and your answer is very instructive. Concerning my data, I generate the point clouds from depth images so the visibility information is available: I will check also TSDF. – Marco Carletti Mar 16 '20 at 09:55
  • FYI if the depth data comes from a continuously scanning equipment, tsdf is used the most (my personal experience). And if that is the case, you may want to look at [KinectFusion](https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/ismar2011.pdf) and a series of following work. – Jing Zhao Mar 16 '20 at 11:14
  • And if you depth pictures are just taken at separate spots at discrete time, then after normal estimation for your point cloud, screened Poisson might just be good enough for you. Hope it helps! – Jing Zhao Mar 16 '20 at 11:17