3

I have several questions on 3D reconstruction with a set of 2D image slices using OpenCV:

  • What are the steps or the process on recreating a 3D model from a set of 2D image slices?

  • How would I start with 3D reconstruction using OpenCV? I heard OpenCV Viz works but I'm new to this so I'm uncertain.

Take note that the camera is not needed because I am given a set of images and I am only limited to OpenCV.

Thank in advance!

navy_marble
  • 41
  • 1
  • 1
  • 5

1 Answers1

5

One of the most authoritative books on this is Zisserman.

The (very) high level steps are:

  1. For each image, identify a set of features (e.g. SIFT, ORB, SURF or similar)
  2. For a pair of images, find correspondences between the features located in 1
  3. Using these correspondences, calculate the Fundamental matrix F which described the relationship between the two images
  4. From F (and ideally some internal camera parameters) , calculate the relative camera pose in the second image with respect to the first
  5. Using F, rectify image pairs such that corresponding points lie on the same scan lines
  6. Calculate a dense disparity map using the two images and then convert this to a depth map for each pixel in an image
  7. Given the camera pose, and depth of each pixel, back project points 3D point in space
  8. Use some technique such as bundle adjustment to optimise the derived values of 3D coordinates and camera poses across the entire set of calculated points and images

The OpenCV calib3d library contains many useful functions for this process and Google is your friend for more details on how to apply them.

Also see: OpenCV with stereo 3D reconstruction, OpenCV 3D reconstruction using shipped images and examples

Community
  • 1
  • 1
Dave Durbin
  • 3,562
  • 23
  • 33
  • If I did look up on Google about my question that uses camera to attempt 3D reconstruction, I won't be posting up here. Because my problem deals with recreating 3D model with **no camera**, and it is true that looking up will only or, if not, almost all of them yields results that utilises cameras to reconstruct 3D model and not based on image slices. Suppose I'm provided with **image slices** and **no cameras** and requested to recreate a 3D model based from [**2D slices**](http://i.imgur.com/y1vcOoc.jpg), how would I go around that? Google didn't really helped me this time. – navy_marble May 19 '15 at 11:24
  • So do you mean you have e.g. 2D histological slices through a 3D object and want to build a 3D model by recombining the data from each slice ? – Dave Durbin May 19 '15 at 23:55
  • Yes sir. You can imagine that it is like simulating a virtual model out from the medical MRI scanner except I'm only given 1 set of image slices that scans through the kneecap in one direction. So I have to simulate that in OpenCV without the need of the camera because image slices are provided. The problem is that I can't seem to get much information out from the internet because almost every result that I got deals with stereo imaging with cameras, which is irrelevant to my question. So I figured I should seek help from here since the information is limited and scarce. Sorry for late reply. – navy_marble May 27 '15 at 00:51
  • It depends on what you want to use the model for. Do you need a solid model or just a skin? One approach would be to consider your stack of slices as a set of voxels at whatever resolution you have available. Then you can threshold each slice to try to separate bone from soft tissue or at least the tissue of interest from the background. You then end up with a 3D space with some voxels filled and others empty. You could use http://en.wikipedia.org/wiki/Marching_cubes algorithm to generate a skin for this. If your slice resolution is much lower than your XY resolution maybe you can interpolate – Dave Durbin May 27 '15 at 01:27
  • FYI I was told to reconstruct the model in terms of solid volume, so is the marching cube algorithm still necessary for this ? – navy_marble May 29 '15 at 05:18
  • It depends on what you intend to do with the model. For example, if you want to visualise it, what tools will you use ? Do they expect a voxel body or a triangulated mesh ? – Dave Durbin Jun 02 '15 at 09:18
  • I reckon we will be visualising the model in terms of voxel body. I'm told to use OpenCV for model recreation, I heard [Viz](http://docs.opencv.org/modules/viz/doc/viz3d.html) might work but somehow it needs VTK to work. – navy_marble Jun 02 '15 at 15:49