1
             Localization of an object specified in the image.

I am working on the project of computer vision to find the distance of an object using stereo images.I followed the following steps using OpenCV to achieve my objective 1. Calibration of camera 2. Surf matching to find fundamental matrix 3. Rotation and Translation vector using svd as method is described in Zisserman and Hartley book. 4. StereoRectify to get the projection matrix P1, P2 and Rotation matrices R1, R2. The Rotation matrices can also be find using Homography R=CameraMatrix.inv() H Camera Matrix.

Problems: i triangulated point using least square triangulation method to find the real distance to the object. it returns value in the form of [ 0.79856 , .354541 .258] . How will i map it to real world coordinates to find the distance to an object.

http://www.morethantechnical.com/2012/01/04/simple-triangulation-with-opencv-from-harley-zisserman-w-code/

Alternative approach: Find the disparity between the object in two images and find the depth using the given formula Depth= ( focal length * baseline ) / disparity

for disparity we have to perform the rectification first and the points must be undistorted. My rectification images are black.

Please help me out.It is important

Here is the detail explanation of how i implemented the code.

  • Calibration of Camera using Circles grid to get the camera matrix and Distortion coefficient. The code is given on the Github (Andriod).

    2.Take two pictures of a car. First from Left and other from Right. Take the sub-image and calculate the -fundmental matrix- essential matrix- Rotation matrix- Translation Matrix....

    3.I have tried to projection in two ways.

  • Take the first image projection as identity matrix and make a second project 3x4d through rotation and translation matrix and perform Triangulation.

  • Get the Projection matrix P1 and P2 from Stereo Rectify to perform Triangulation.

My object is 65 meters away from the camera and i dont know how to calculate this true this based on the result of triangulation in the form of [ 0.79856 , .354541 .258]

Question: Do i have to do some extra calibration to get the result. My code is not based to know the detail of geometric size of the object.

Sameed Tariq
  • 81
  • 3
  • 10

1 Answers1

1

So you already computed the triangulation? Well, then you have points in camera coordinates, i.e. in the coordinate frame centered on one of the cameras (the left or right one depending on how your code is written and the order in which you feed your images to it).

What more do you want? The vector length (square root of the sum of the square coordinates) of those points is their estimated distance from the same camera. If you want their position in some other "world" coordinate system, you need to give the coordinate transform between that system and the camera - presumably through a calibration procedure.

Francesco Callari
  • 11,300
  • 2
  • 25
  • 40
  • I added more details in the above question that how i implemented the code. I want to know the real distance of an object that is 65 meters away from my camera. I want to know that how these coordinates will help me to approximate distance. any calculations or reference ? – Sameed Tariq Feb 01 '14 at 05:58
  • @SameedTariq If you didn't get your answer then why you marked as answer? and if you got what you want than don't edit the question and use the new thread for it – AHF Feb 01 '14 at 20:23
  • @Ahmad . As far as i know he is pointing in right direction. I didn't get the answer. My object was 65 meters away from the camera. how these coordinates will help me to show to estimate 65 meter distance. – Sameed Tariq Feb 01 '14 at 20:47
  • Your cameras are calibrated and, from the essential matrix, you have computed the relative pose of the cameras. Now, do you need a dense disparity map, or only the location of a few corresponding points? If the latter, you don't need to rectify the images. Just triangulate the corresponding pixels. If the results still don't make sense (i.e. their distance is too small, like in your example), it probably means that you have a wrong scaling in the calibration. Did you input the correct physical size for your calibration object? – Francesco Callari Feb 01 '14 at 22:42
  • @SameedTariq So you actually want to know that how you can calculate the distance of the object from the camera ? like if the man is 4 meter away from the camera than your app will show that object is 4 meter from the camera ? – AHF Feb 02 '14 at 13:21
  • @Francesco : i want to compute the triangulation without knowing the physical size of the object. – Sameed Tariq Feb 02 '14 at 14:51
  • I meant the calibration object. If you do not know it, you need to input the size of at least one physical object in the scene. Otherwise your calibration is determined only up to scale, and you can recover physical measurements (including distances) only up to scale. There is no free lunch. – Francesco Callari Feb 02 '14 at 17:24
  • @FrancescoCallari. you meant there must be at least one calibrated object in the scene to estimate the true distance ? Here are some quotes of research paper "PositionIt – An Image-based Remote Target Localization System on Smartphones" my next two comments will describe the two images case briefly as it is mentioned in the research paper. – Sameed Tariq Feb 03 '14 at 19:43
  • however GPS location of the phone for both images taken is required. In case only the remote target’s distance is of interest, then it is sufficient to estimate the distance between the two phones, e.g., using the single image based system. When user collects the two images, she/he can choose to identify the remote target using a bounding box on both photos, after which our application will perform image feature matching, camera relative position estimation, and remote target position estimation based on two view triangulation. – Sameed Tariq Feb 03 '14 at 19:48
  • We select the best feature matches and use them to calculate the fundamental matrix and essential matrix based on the 8 Point Algorithm [1]. Following the procedure outlined in [5], we obtain the camera’s relative pose, Rotation matrix R, and Translation Vector T with the L-2 norm equaling one. After T’s norm is scaled based on the true distance, we perform Two-View Triangulation to estimate the target’s position and the distances from the target to the two camera positions. For such cases i have to calibrate the camera for the specific object too ? – Sameed Tariq Feb 03 '14 at 19:49
  • Godness, this is becoming tiresome. Your essential matrix E is known only up to scale. Why? Because your calibration matrices are known only up to scale. Why? Because you don't calibrate using an object of known size (or, equivalently, you don't know the physical size of the pixels on your sensor) so the ratio between pixels and meters is unknown. Therefore your cameras' relative coordinate transform is known only up to scale. Therefore your inter-camera distance is known only up to scale. Therefore every triangulation you compute is up to scale. This is also exactly what that article says. – Francesco Callari Feb 04 '14 at 01:30
  • Note also that the sensor size is, sometimes, given to you for free in the EXIF header attached by digital cameras to the images. Using a calibration target of known size usually produces more accurate results (because the calibration errors are smeared over all camera parameters), but if you don't have anything else... – Francesco Callari Feb 04 '14 at 01:37
  • my cameras are tilted so as the origin of the landmark must be left camera and the depth axis direction is not normal to my surface? that s why for a flat surface i have a "linear" variation of depth? thanks – user3601754 Mar 16 '16 at 11:17
  • 1
    @user3601754 I can't make head or tails of you comment. Please open a new question instead of commenting on this old one. – Francesco Callari Mar 16 '16 at 19:19
  • Hi i created one http://stackoverflow.com/questions/36034849/opencv-tilted-camera-and-triangulation-landmark-for-stereovision – user3601754 Mar 17 '16 at 08:04