OpenCV - Estimating Box dimensions in Python

Question

This is the continuation of my previous question. I now have an image like this

Here the corners are detected. Now I am trying to estimate the dimensions of the bigger box while smaller black box dimensions are known.

Can anyone guide me what is the best way to estimate the dimensions of the box? I can do it with simple Euclidean distance but I don't know if it is the correct way or not. Or even if it is the correct way then from a list of tuples (coordinates) how can I find distances like A-B or A-D or G-H but not like A-C or A-F?

The sequence has to be preserved in order to get correct dimensions. Also I have two boxes here so when I create list of corners coordinates then it should contain all coordinates from A-J and I don't know which coordinates belong to which box. So how can I preserve that for two different boxes because I want to run this code for more similar images.

Note: The corners in this image is not a single point but a set of points so I clustered the set of the corner and average them to get a single (x,y) coordinate for each corner.

I have tried my best to explain my questions. Will be extremely glad to have some answers :) Thanks.

Answer 1

For the

How can I find distances like A-B or A-D or G-H but not like A-C or A-F

part

Here's a quick code, not efficient for images with lots of corners, but for your case it's OK. The idea is to start from the dilated edge image you got in your other question (with only the big box, but the idea is the same for the image where there is also the small box)

then for every possible combination of corners, you look at a few points on an imaginary line between them, and then you check if these points actually fall on a real line in the image.

import cv2
import numpy as np

#getting intermediate points on the line between point1 and point2
#for example, calling this function with (p1,p2,3) will return the point
#on the line between p1 and p2, at 1/3 distance from p2
def get_intermediate_point(p1,p2,ratio):
    return [p1[0]+(p2[0]-p1[0])/ratio,p1[1]+(p2[1]-p1[1])/ratio]

#open dilated edge images
img=cv2.imread(dilated_edges,0)

#corners you got from your segmentation and other question
corners=[[29,94],[102,21],[184,52],[183,547],[101,576],[27,509]]
nb_corners=len(corners)

#intermediate points between corners you are going to test
ratios=[2,4,6,8] #in this example, the middle point, the quarter point, etc
nb_ratios=len(ratios)

#list which will contain all connected corners
connected_corners=[]

#double loop for going through all possible corners
for i in range(nb_corners-1):
    for j in range(i+1,nb_corners):
        cpt=0
        c1=corners[i]; c2=corners[j]

        #testing every intermediate points between the selected corners
        for ratio in ratios:
            p=get_intermediate_point(c1,c2,ratio)

            #checking if these points fall on a white pixel in the image
            if img[p[0],p[1]]==255:
                cpt+=1

        #if enough of the intermediate points fall on a white pixel
        if cpt>=int(nb_ratios*0.75):
            #then we assume that the 2 corners are indeed connected by a line
            connected_corners.append([i,j])

print(connected_corners)

Answer 2

In general you cannot, since any reconstruction is only up to scale.

Basically, given a calibrated camera and 6 2D-points (6x2=12) you want to find 6 3D points + scale = 6x3+1=19. There aren't enough equations.

In order to do so, you will have to make some assumptions and insert them into the equations.

Form example:

1. The box edges are perpendicular to each other (which means that every 2 neighboring points share at least one coordinate value).
2. You need to assume that you know the height of the bottom points, i.e. they are on the same plane as your calibration box (this will give you the Z of the visible bottom points).

Hopefully, these constraints are enough to given you less equations that unknown and you can solve the linear equation set.