1

Anyone know a good starting point to check out, for 3D reconstruction from X-ray images/2D images using OpenCV. Im trying to do this project using nodejs and the js version of OpenCV. I basically trying to recreate the bone structure from a 2D X-ray image(multiple views are available). Open source codes(in python/c/c++)/algorithms/guides/anything is appreciated.

Thank you

dvenkatsagar
  • 936
  • 7
  • 22
  • Can you post example of data-set you are working upon – nbsrujan Jan 21 '16 at 05:29
  • @nbsrujan Lets say we have an xray of the forearm taken in three different angles. My doubt is, how to achieve an accurate depth map out of this?? – dvenkatsagar Jan 21 '16 at 06:37
  • Yes, there are standard methods to achieve this. Some importand methods are given here http://www.zib.de/projects/3d-reconstruction-anatomical-structures-2d-x-ray-images – nbsrujan Jan 21 '16 at 06:39
  • @nbsrujan I have checked this out, but I cant seem to find an example or any algorithm that they used to make it possible. – dvenkatsagar Jan 21 '16 at 06:45
  • can you please show some images (best would be 3 consecutive image layers where different kind of thicknesses can be seen)? – Micka Jan 21 '16 at 10:04
  • @micka Lets take an example where we have 2 different views of a forearm bone and wrist, and I need to 3D reconstruct this. (image here : [http://image.wikifoundry.com/image/1/BGeE_Na6CsS1DIAxpuffTw203771/GW720H503](http://image.wikifoundry.com/image/1/BGeE_Na6CsS1DIAxpuffTw203771/GW720H503)) – dvenkatsagar Jan 21 '16 at 21:08
  • no, it doesnt work that way. You would need slices of the target, not single unorganized scans. – Micka Jan 21 '16 at 21:11
  • @micka Sorry I gave the links to the wrong one, I edited the above link – dvenkatsagar Jan 21 '16 at 21:12
  • You don't need slices of images. It is possible to generate images using various methods. See here: https://en.wikipedia.org/wiki/3D_reconstruction_from_multiple_images#3D_reconstruction_of_Medical_Images – Gokhan Dilek Aug 01 '17 at 13:47

2 Answers2

3

In a nutshell, to get a good 3D reconstruction image, you need to know some information about both camera (lens distortion, distance, calibration, etc)

There's a bunch of information online, here are some for good starting point

  1. http://docs.opencv.org/master/dd/d53/tutorial_py_depthmap.html#gsc.tab=0 This is an example of creating disparity map. OpenCV has 3 stereo matching algorithm - Block Matching (stereoBM), GraphCut (stereoGC), and Semi Global Block Matching (stereoSGBM) - For more in-depth explanation, try http://scholar.google.com.

  2. After you generate the disparity map, you can generate the point in 3d space. However, as I said, you need some information from the camera. The code example in this question might help 3d reconstruction from 2 images without info about the camera

  3. Look for the sample from openCV sample folder it self https://github.com/Itseez/opencv/blob/master/samples/cpp/stereo_match.cpp

    and the documentation
    http://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html

Side Note:

I was using EmguCV, which is an openCV wrapper for C#, to create 3d reconstruction of human face. You might find this example useful, however, you need to find the corresponding method in openCV (they usually have the same name) http://www.emgu.com/wiki/index.php/Stereo_Imaging

Hope it helps!

Community
  • 1
  • 1
Joshua H
  • 754
  • 8
  • 18
  • Thank you for the reply, but the thing is, if im using normal images, I am able to do the above procedure and am able to create a point cloud, but when it comes to xrays, they are, in one way, depth maps(Im just not able to find the right threshold values to get a good disparity map). So would the normal method of finding a depth map work in this situation ?? – dvenkatsagar Jan 21 '16 at 06:34
2

Typically you have the following:

  • N image layers with each image dimension = width * height
  • Interpretation from a color value to some "thickness value"

The general idea is: Create a 3D map of dimension N * width * height with either floating point or byte values. Then just add your image layers to that map, giving you something like a huge 3D texture. Now you can define the tissue-thickness you are interested in, for example bones. Then search each cell in your 3D map where the values differ from "less than bone-thickness" to "bigger or equal to bone-thickness" (or just cells which have the exact thickness value stored) and mark those cells as "bones". Then you have some voxel grid of your bones :)

A better approach is to use something like marching cubes and interpolate between thickness changes.

Probably, if you google "marching cubes" and "x-ray" you'll find some more detailed information (and university lecture notes) about different ways to solve the approach. For example: http://www.eecs.berkeley.edu/~jrs/meshpapers/LorensenCline.pdf and from those papers you might find more tags to search for.

Micka
  • 19,585
  • 4
  • 56
  • 74
  • How many images would you need for marching cubes? Do you need many many slices? – Gokhan Dilek Aug 02 '17 at 14:40
  • @GokhanDilek probably depends on the precision/resolution you want to reach. – Micka Aug 02 '17 at 14:43
  • in fact if you know the size of a pixel and you want the same resolution in Z direction, you can compute the number of slices (or the distance between slices) you 'll need. See http://www.cs.carleton.edu/cs_comps/0405/shape/marching_cubes.html for an example – Micka Aug 02 '17 at 14:47
  • @Micka I am trying to reduce the number of images. If I had only 2 xray images to work with(1 is the front view of the object and the other is the side view of the object), what are my options to create a 3d reconstruction? – Gokhan Dilek Aug 02 '17 at 14:50
  • with alot of assumptions, maybe you could generate a single front-view slice from each x-axis pixel of your side-view image – Micka Aug 02 '17 at 15:25
  • Like these guys here: http://www.eos-imaging.com/sites/default/files/2016-02/EOS%20Solutions%20video%20%28R22-BRO-100-A-EN%29_0.mp4 – Gokhan Dilek Aug 02 '17 at 15:51