OpenCV: How-to calculate distance between camera and object using image?

Question

I am a newbie in OpenCV. I am working with the following formula to calculate distance:

distance to object (mm) = focal length (mm) * real height of the object (mm) * image height (pixels)
                          ----------------------------------------------------------------
                                object height (pixels) * sensor height (mm)

Is there a function in OpenCV that can determine object distance? If not, any reference to sample code?

+1 looking for the same, have you found any solutions? – Mangesh May 20 '13 at 14:35 — Mangesh, May 20 '13 at 14:35

Cameron Lowell Palmer · Answer 1 · 2017-03-24T11:58:22.270

How to calculate distance given an object of known size

You need to know one of 2 things up front

Focal-length (in mm and pixels per mm)
Physical size of the image sensor (to calculate pixels per mm)

I'm going to use focal-length since I don't want to google for the sensor datasheet.

Calibrate the camera

Use the OpenCV calibrate.py tool and the Chessboard pattern PNG provided in the source code to generate a calibration matrix. I took about 2 dozen photos of the chessboard from as many angles as I could and exported the files to my Mac. For more detail check OpenCV's camera calibration docs.

Camera Calibration Matrix (iPhone 5S Rear Camera)

RMS: 1.13707201375
camera matrix:
[[  2.80360356e+03   0.00000000e+00   1.63679133e+03]
 [  0.00000000e+00   2.80521893e+03   1.27078235e+03]
 [  0.00000000e+00   0.00000000e+00   1.00000000e+00]]
distortion coefficients:  [ 0.03716712  0.29130959  0.00289784 -0.00262589 -1.73944359]

f_x = 2803
f_y = 2805
c_x = 1637
c_y = 1271

Checking the details of the series of chessboard photos you took, you will find the native resolution (3264x2448) of the photos and in their JPEG EXIF headers, visible in iPhoto, you can find the Focal Length value (4.15mm). These items should vary depending on camera.

Pixels per millimeter

We need to know the pixels per millimeter (px/mm) on the image sensor. From the page on camera resectioning we know that f_x and f_y are focal-length times a scaling factor.

f_x = f * m_x
f_y = f * m_y

Since we have two of the variables for each formula we can solve for m_x and m_y. I just averaged 2803 and 2805 to get 2804.

m = 2804px / 4.15mm = 676px/mm

Object size in pixels

I used OpenCV (C++) to grab out the Rotated Rect of the points and determined the size of the object to be 41px. Notice I have already retrieved the corners of the object and I ask the bounding rectangle for its size.

cv::RotatedRect box = cv::minAreaRect(cv::Mat(points));

Small wrinkle

The object is 41px in a video shot on the camera @ 640x480.

Convert px/mm in the lower resolution

3264/676 = 640/x
x = 133 px/mm

So given 41px/133px/mm we see that the size of the object on the image sensor is .308mm .

Distance formula

distance_mm = object_real_world_mm * focal-length_mm / object_image_sensor_mm
distance_mm = 70mm * 4.15mm / .308mm
distance_mm = 943mm

This happens to be pretty good. I measured 910mm and with some refinements I can probably reduce the error.

Feedback is appreciated.

Similar triangles approach

Adrian at pyimagesearch.com demonstrated a different technique using similar triangles. We discussed this topic beforehand and he took the similar triangles approach and I did camera intrinsics.

Very nice explanation. But can you also do this if you cannot calibrate your camera? Would you then use your native camera resolution? How reliable is this when not correcting for distortion? How would I apply this method using only native resolution? Sorry for the many questions ;) — Sander, Jan 22 '15 at 15:30
Without a camera calibration you're going to be off. I suppose you could use the sensor spec to get the measurements or just put together a database of cameras in various devices. We can probably safely assume all iPhone 5s back-side cameras fall within some tight tolerances. Also, figuring out how to dump the Qualcomm Vuforia libraries database of camera calibrations would be a task someone could investigate. — Cameron Lowell Palmer, Jan 22 '15 at 18:50
Ok, let me elaborate on my questions. I want to calculate the distance to specific objects in given images, without having the camera. This is a Canon EOS 60D with distortion correction. From the EXIF tags in the JPEG images I can retrieve the focal length. I know the object's dimensions in millimeters. Do you think your method is applicable to this problem? — Sander, Jan 23 '15 at 09:42
No. The formula: distance = size_obj * focal_length / size_obj_on_sensor. The whole method depends on figuring out the size of the object as it appears on the sensor given the focal length and the measured object size. Otherwise you have two unknowns. — Cameron Lowell Palmer, Jan 23 '15 at 09:48
@Sander you could consider a completely different approach by using similar triangles. For that, refer to http://www.pyimagesearch.com/2015/01/19/find-distance-camera-objectmarker-using-python-opencv/ — Cameron Lowell Palmer, Jan 23 '15 at 09:50
I checked that method out before, but then I would need a reference image taken from known distance. Anyway, thanks! — Sander, Jan 23 '15 at 10:11
@Sander yes, and these are the only two ways of doing this. Good luck! — Cameron Lowell Palmer, Jan 23 '15 at 13:32
@CameronLowellPalmer Great answer. One question. When you say `object_real_world_mm` it means we need to know the physical size of the object? Also, is this size along a particular dimension i.e. an object length or width or height? — Luca, Mar 19 '19 at 06:40
A bit late to the party, but this is pretty much the same as answer 40 here https://photo.stackexchange.com/questions/12434/how-do-i-calculate-the-distance-of-an-object-in-a-photo/12437#12437. Just that the story is told differently. But the same :) — decades, Feb 11 '21 at 17:55

score 1 · Answer 2 · edited May 23 '17 at 11:33

1

there is no such function available in opencv to calculate the distance between object and the camera. see this : Finding distance from camera to object of known size

You should know that the parameters depend on the camera and will change if the camera is changed.

edited May 23 '17 at 11:33

Community

1
1

answered Dec 26 '12 at 08:42

Abhishek Thakur

16,337
15
66
97

camera parameters i am keeping to be constant and i knew them – swapna Dec 26 '12 at 09:13

score 0 · Answer 3 · answered Dec 27 '12 at 05:00

0

To get a mapping between the real world and camera without any prior information of the camera you need to calibrate the camera...here you can find some theory
For calculating the depth i.e. distance between camera and object you need at least two images of the same object taken by two different cameras...which is popularly called the stereo vision technique..

answered Dec 27 '12 at 05:00

rotating_image

3,046
4
28
46

is it necessary to have two cameras? – swapna Dec 31 '12 at 08:00
1

the formula what you have given is an ideal case when the principal axis of the camera is perpendicular to the object plane...what if the principal axis of the camera is tilted to the object plane(which is generally the real world scenario...)then we get a projection of the object in the image...the projected image doesnt give you the correct idea of the object dimensions in real world...so with one camera it is difficult if you dont have any prior idea about the orientation of the camera with respect to the the object... – rotating_image Dec 31 '12 at 08:50
do you still need to calibrate the camera if you know all the details about the camera already? – Crashalot Nov 01 '16 at 00:36