OpenCV - How to get real world distance from a 2D image using a chessboard as reference

Question

After checking several pieces of codes, I took several shots, found the chessboard corners and use them to get the camera matrix, distortion coefficients, rotation, and translation vectors. Now, can someone tell me which python opencv function do I need to calculate the distance in the real world from the 2D image? project points? For example, using a chessboard as a reference (see picture), if the tile size is 5cm, the distance for 4 tiles should be 20 cm. I saw some functions like projectPoints,findHomography, solvePnP but I am not sure which one do I need to solve my problem and get the transformation matrix between the camera world and the chessboard world. 1 single camera, same position of the camera for all cases but not exactly over the chessboard, and chessboard is placed over a planar object (table)

# prepare object points, like (0,0,0), (1,0,0), (2,0,0) ....,(6,5,0)
    objp = np.zeros((nx * ny, 3), np.float32)
    objp[:, :2] = np.mgrid[0:nx, 0:ny].T.reshape(-1, 2)

    # Arrays to store object points and image points from all the images.
    objpoints = []  # 3d points in real world space
    imgpoints = []  # 2d points in image plane.

    # Make a list of calibration images
    images = glob.glob(path.join(calib_images_dir, 'calibration*.jpg'))
    print(images)
    # Step through the list and search for chessboard corners
    for filename in images:

        img = cv2.imread(filename)

        imgScale = 0.5
        newX,newY = img.shape[1]*imgScale, img.shape[0]*imgScale
        res = cv2.resize(img,(int(newX),int(newY)))

        gray = cv2.cvtColor(res, cv2.COLOR_BGR2GRAY)

        # Find the chessboard corners
        pattern_found, corners = cv2.findChessboardCorners(gray, (nx,ny), None)

        # If found, add object points, image points (after refining them)
        if pattern_found is True:
            objpoints.append(objp)

            # Increase accuracy using subpixel corner refinement
            cv2.cornerSubPix(gray,corners,(5,5),(-1,-1),(cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 30, 0.1 ))
            imgpoints.append(corners)

            if verbose:
                # Draw and display the corners
                draw = cv2.drawChessboardCorners(res, (nx, ny), corners, pattern_found)
                cv2.imshow('img',draw)
                cv2.waitKey(500)

    if verbose:
        cv2.destroyAllWindows()

    #Now we have our object points and image points, we are ready to go for calibration
    # Get the camera matrix, distortion coefficients, rotation and translation vectors
    ret, mtx, dist, rvecs, tvecs = cv2.calibrateCamera(objpoints, imgpoints, gray.shape[::-1], None, None)
    print(mtx)
    print(dist)
    print('rvecs:', type(rvecs),' ',len(rvecs),' ',rvecs)
    print('tvecs:', type(tvecs),' ',len(tvecs),' ',tvecs)

    mean_error = 0
    for i in range(len(objpoints)):
        imgpoints2, _ = cv2.projectPoints(objpoints[i], rvecs[i], tvecs[i], mtx, dist)
        error = cv2.norm(imgpoints[i],imgpoints2, cv2.NORM_L2)/len(imgpoints2)
        mean_error += error

    print("total error: ", mean_error/len(objpoints))


    imagePoints,jacobian = cv2.projectPoints(objpoints[0], rvecs[0], tvecs[0], mtx, dist)
    print('Image points: ',imagePoints)

Have a look at [this](http://answers.opencv.org/question/2902/how-to-calculate-the-distance-from-the-camera-origin-to-any-of-the-corners-square-chessboard-calibration/) — Rick M., Mar 20 '19 at 07:29
So, just to make sure I understood, you calibrated with several images of a chessboard and now you want to take another image of the *same* chessboard and compute the distance between the camera and the board? — Milo, Mar 21 '19 at 16:15
@Milo I calibrated the camera to obtain the camera parameters. With cam matrix and distortion coefficients,My new idea is to use the getPerspective method to obtain a better view of the chessboard, so how can I get the size in real world if the chessboard can be seen as if the camera is exactly over the chessboard. Is there any opencv method for that? I found info about use a ratio px/mm but I wanna know if there is a better method for that. Now, is there any good way to convert from pixels to a unit in meters? Any method inside opencv? — Pablo Gonzalez, Mar 25 '19 at 05:49
@PabloGonzalez To help you, I really need to understand your final goal. You have a calibrated camera over a table, in which will be a chessboard (but not directly below the camera). You would like to generate a "flat" image of the chessboard, as the one you posted, but all of this with which goal? Just generate the "flat" image? Compute the size of a new (unknown) board? Compute the position of the board with respect to the camera? Maybe you can find some information in this answer: https://stackoverflow.com/questions/55220229/extrinsic-matrix-computation-with-opencv/55284535#55284535 — Milo, Mar 26 '19 at 08:50
@Milo I want to measure the size of the tile so I can know the mm/pixel ratio, and finally calculate lengths and sizes in real world using the camera.Similar to https://stackoverflow.com/questions/49530738/precision-measurement-with-opencv-python — Pablo Gonzalez, Mar 26 '19 at 08:58

score 9 · Answer 1 · answered Mar 21 '19 at 13:17

You are indeed right, and I think you should use solvePnP for this problem. (Read more on perspective-n-point problems here: https://en.wikipedia.org/wiki/Perspective-n-Point.)

The Python OpenCV solvePnP function takes the following parameters and returns an ouput rotation and output translation vector which converts the model coordinate system to the camera coordinate system.

cv2.solvePnP(objectPoints, imagePoints, cameraMatrix, distCoeffs[, rvec[, tvec[, useExtrinsicGuess[, flags]]]]) → retval, rvec, tvec

In your case the imagePoints will be the corners of the chessboard so it would look something like:

ret, rvec, tvec = cv2.solvePnP(objpoints, corners, mtx, dist)

With the returned translation vector you can calculate the distance from the camera to the chessboard. The output translation from solvePnP is in the same units as specified in objectPoints.

Finally, you can compute the real distance from the tvec as the euclidean distance:

d = math.sqrt(tx*tx + ty*ty + tz*tz).

Thank you for your contribution. This "d" result will output the distance from the camera to the object right? If yes, however, my goal is to determinate the size in the real world of the size of the tiles. My new idea is to use getPerspective method to obtain a better view of the chessboard, so how can I get the size in real world if the chessboard can be seen as if the camera is exactly over the chessboard. Is there any opencv method for that? I found info about use a ratio px/mm but I wanna know if there is a better method for that — Pablo Gonzalez, Mar 25 '19 at 05:47
@PabloGonzalez You have to know the size of the chess tiles in advance to be able to use `solvePnP`. Checkout this example: https://docs.opencv.org/3.4.2/d7/d53/tutorial_py_pose.html — Yonatan Simson, Mar 26 '19 at 09:09

score 0 · Answer 2 · answered Mar 26 '19 at 09:06

Your problem relates mainly to camera-calibration especially in poor implementation of resolving camera distortion in opencv. You must approximate distortion function of your camera len by taking a few probes of distance in different coordinates of your chessboard. The good idea will be taking first small distance in the center of the len, then one square far taking second a little longer distance and repeat operation to the border. It will give you coefficients of your distortion function. Matlab has own library to solve your problem with big accuracy, unfortunately it's quite expensive.
According to:

Now, can someone tell me which python opencv function do I need to calculate the distance in real world from the 2D image?

I think this article takes good explenation of python opencv set of functions to generate real measure. With resolving coefficients as I said above you could make good accuracy. Anyway I don't think so if it is an open source implementation of function like

cv2.GetRealDistance(...)

thank you for your comment. About a GetRealDistance, I just found https://github.com/DIPlib/diplib so it will give a some hints for the task. — Pablo Gonzalez, Mar 27 '19 at 00:40

OpenCV - How to get real world distance from a 2D image using a chessboard as reference

2 Answers2