2

I have a stereo system with 2 cameras. I calibrated these cameras. I try to calculate distance between each fingertip.On left image, I find fingertips using convex hull. I calculate epipoolar line for these points.I draw epipolar lines on right image. How can I calculate 3d position each fingertips? I used c++ and opencv.

There are 5 window below image. They are : right image, left image, find fingertips using convex hull on right image, draw epipolar lines on left image, find correspondences points on left images

enter image description here

My .yml file is below after stereo calibration

%YAML:1.0
CM1: !!opencv-matrix
 rows: 3
 cols: 3
 dt: d
 data: [ 1.4947330489959640e+02, 0., 8.5026435902438408e+01, 0.,
   1.7045159164506524e+02, 6.8513237416979280e+01, 0., 0., 1. ]
CM2: !!opencv-matrix
rows: 3
cols: 3
dt: d
data: [ 1.4947330489959640e+02, 0., 7.6063817190941975e+01, 0.,
   1.7045159164506524e+02, 6.9869364400956655e+01, 0., 0., 1. ]
D1: !!opencv-matrix
rows: 1
cols: 5
dt: d
data: [ 4.6664660489275862e+00, -9.5605452982913761e+01, 0., 0.,
   4.4411083031870203e+02 ]
D2: !!opencv-matrix
rows: 1
cols: 5
dt: d
data: [ -2.6243438145377401e-01, 3.1158182596121313e+00, 0., 0.,
   -6.9555261934841601e+00 ]
R: !!opencv-matrix
rows: 3
cols: 3
dt: d
data: [ -9.9870707407742809e-01, 5.0820157566619700e-02,
   1.2213814337059467e-03, -4.6584627039081256e-02,
   -9.2456021193091820e-01, 3.7817758664136281e-01,
   2.0348285218473684e-02, 3.7763173343769685e-01,
   9.2573226215224258e-01 ]
T: !!opencv-matrix
rows: 3
cols: 1
dt: d
data: [ -5.0257191774306198e-01, -5.1791340062890008e+00,
   -1.7104054803114692e+00 ]
E: !!opencv-matrix
rows: 3
cols: 3
dt: d
data: [ -1.8506509733057530e-01, -3.5371782058656147e+00,
   -4.1476544229091719e+00, 1.7184205294528965e+00,
   1.0286402846218139e-01, 4.6315798080871423e-01,
   -5.1490256443274198e+00, 7.2786240503729882e-01,
   -1.8373573684783620e-01 ]
 F: !!opencv-matrix
  rows: 3
 cols: 3
 dt: d
 data: [ -2.0635586643392613e-06, -3.4586914187982223e-05,
   -4.3677532717492718e-03, 1.6802903312164187e-05,
   8.8202517402136951e-07, -8.1218529743132760e-04,
   -9.5988974549000728e-03, 3.6330053228360980e-03, 1. ]
zakjma
  • 2,030
  • 12
  • 40
  • 81

3 Answers3

0

Since you can't be sure to really get the exact finger tips with your method for both images, I'd use a different approach:

  • First use 3D image reconstruction to get a depth image of your hand.
  • Then use your existing algorithm to get the finger tips in your depth image.
  • Finally use the depth information to reconstruct the 3D position of the finger tips.
Mario
  • 35,726
  • 5
  • 62
  • 78
  • I don't need depth only. I want to calculate x,y,z coordinates. – zakjma Dec 14 '14 at 21:15
  • @holazollil That's just the first step. Once you've got the depth for each pixel, you're able to restore the actual world coordinates given you've got your camera transforms, since you know the distance to the depicted pixel as well as the angle (vertical and horizontal based on the coordinates and field of view). – Mario Dec 15 '14 at 07:59
0

You may need some very quick and simple solution based on 3D-2D point correspondence. Then you do fitting of the points to the 3D model, a lot of 3D models of hand can be found freely, i.e: http://www.turbosquid.com/3d-model/anatomy/hand OpenCV provides a nice method - solvePnP - that can do the fitting step. We need to do the following:

  1. download a free hand model.
  2. Use any 3D editor to edit the model (i.e: MeshLab)
  3. Mark the position of each fingertip, we will get five 3D points
  4. save them in a file as a reference model
  5. your opencv program will find you five 2D points.
  6. Now you have Model points (3d) and real image points (2D), you have to fit them using solvePnP function: solvePnP (model_points,image_points,cameraMatrix ..some_calibration_params)
  7. we will get a 3x3 matrix, that you can define it as: double calMat[9] = { x_center, 0, x_shift, 0, y_center, y_shift, 0, 0, 1 }; //"calibration matrix": x_center,y_center point is to consider the center of image with focal length ie. x_center=y_center=30 cameraMatrix = Mat(3,3,CV_64FC1,calMat);
  8. By these steps you can estimate the 3D position of each fingertip at any moment when moving hand
Y.AL
  • 1,808
  • 13
  • 27
0

Undistort your 2D points with cv::undistortPoints(). Pass your undistorted points from both cameras to cv::triangulatePoints() (along with camera projection matrices) and cv::Mat to store (homogenous) 3D coordinates. Call convertPointsFromHomogenous() to get ordinary (inhomogeneous) 3D points. Note: the projection matrices are P1 and P2 from cv::stereoRectify and/or <opencv-dir>/samples/cpp/stereo_calib.cpp. You may find How to correctly use cv::triangulatePoints() to be useful.

mannyglover
  • 2,139
  • 14
  • 18