1

I trained a model with Detectron2 to detect windows on an image of building. As output I also get bounding boxes around each window with x,y-coordinates of top-right and bottom-left corner. enter image description here

I want to derive info about the number of floors from this output. The idea is to get centroids of each bbox and to draw a line connecting centroids on one line. The number of lines means then the number of floors. See the desirable output: enter image description here

I obtained x,y-coordinates of each centroid, but can't get how to draw lines. Please, any ideas?

 #for snapshot-10.png choose only bboxes for class 0 (fenster)
bbox = output['instances'][output['instances'].pred_classes==0].pred_boxes
box_cent = bbox.get_centers()
box_cent = box_cent.cpu().numpy()
#sort centroids by y-coordinates
box_cent[box_cent[:,1].argsort()]

The sorted by y-coordinates output is:

[[540.9429, 233.10698],
[406.09546, 236.09201],
[769.1001, 236.3072],
[655.85944, 236.33795],
[883.7689, 241.99356],
[998.5099, 244.86237],
[532.45764, 381.28082],
[653.0167, 386.0125],
[891.9795, 387.96347],
[1140.524, 388.55304],
[1008.31445, 390.77576],
[771.84033, 390.8385],
[390.84647, 394.80884],
[522.44476, 559.79565],
[899.99866, 560.30835],
[1022.31305, 562.9693],
[650.5117, 563.4512],
[772.411, 565.10144],
[1162.2795, 565.864],
[1269.4834, 566.28735],
[373.81348, 575.3587],
[907.84314, 768.63293],
[509.22638, 770.1798],
[643.81964, 770.6924],
[775.07874, 771.9369],
[234.46616, 775.37305],
[117.903625, 776.7884],
[350.1, 776.8995]]

UPD: I tried a simple solution which clusters y-coordinates of centroids, but there is a specific threshold here. And I would like to find an universal algorithm that counts the number of floors regardless of the height of the windows or the distance between them. Here's my try:

y_cent = []
for b in box_cent:
    (x, y) = (int(b[0]), int(b[1]))
    y_cent.append(y)

#group values of centroid y-coordinates
def cluster(array, maxdiff):
    tmp = array.copy()
    groups = []
    while len(tmp):
        # select seed
        seed = tmp.min()
        mask = (tmp - seed) <= maxdiff
        groups.append(tmp[mask, None])
        tmp = tmp[~mask]
    return groups
Julia Koncha
  • 85
  • 1
  • 12
  • A not so good way to do that is to take a look into each window at a time and find with the y coordinate in the image the other windows that are in the same floor. Then delete all the windows of that floor from the list of the windows and repeat. You can set a threshold and if the difference of the y-coordinates of the two windows are less that the threshold then they are considered to be in the same floor. – kostis 1101 Mar 11 '22 at 14:34
  • Does this answer your question? [Python: Sorting items from top left to bottom right with OpenCV](https://stackoverflow.com/questions/66946804/python-sorting-items-from-top-left-to-bottom-right-with-opencv) – Christoph Rackwitz Mar 11 '22 at 15:10
  • 1
    I'm not saying it's a duplicate, and that thing is impossible to search for, and it might not contain any robust algorithms for your situation, but basically you need to associate those objects to horizontal neighbors. you'll need to look for the closest candidate in a certain direction (let's say in a sector of 90 or 45 or whatever degrees), then add an edge between those two. – Christoph Rackwitz Mar 11 '22 at 15:12
  • you might even want to associate **columns** of windows, because you might have a staircase with windows, which are on the landings between stories, so that'd destroy any row-wise association of windows. – Christoph Rackwitz Mar 11 '22 at 15:15
  • I think you would have better luck looking for bounding boxes whose y-coordinates overlap. Unless, of course, you have images that are rotated to the point that the y-coordinates of different floors overlap. – beaker Mar 11 '22 at 16:35
  • @ChristophRackwitz thank you for the link. My problem is, that I don't want to predefine the number of rows myself. I want to find a universal algorithm that can count the number of floors regardless of the height of the windows or the distance between them – Julia Koncha Mar 14 '22 at 08:30

1 Answers1

0

I gave it a try and here is what I came up with. It is not the best algorithm in the word, but it gets the job done. Keep in mind though that it only works when the photo is upright.

# you don't have to do that if the list is already sorted
windows = sorted(windows, key = lambda x: x[1]) 

thresshold = 30
floors_amount = 0

while windows:
    for w in windows[1:].copy():
        if w[1] - windows[0][1] > thresshold:
            break
        windows.remove(w)
    windows.remove(windows[0])
    floors_amount += 1

print(floors)

This counts the number of floors. If you want to draw lines going through those points you can store the windows of each floor separately:

windows = sorted(windows, key = lambda x: x[1])

thresshold = 30

floors = []

while windows:
    floors.append([windows[0]])
    for w in windows[1:].copy():
        if w[1] - windows[0][1] > thresshold:
            break
        floors.append(w)
        windows.remove(w)
    windows.remove(windows[0])

The loop though every list in the floors list and sort the list with respect to the x-coordinates and for every neighboring window draw a line with the cv2.line method. (https://python.engineering/python-opencv-cv2-line-method/)

kostis 1101
  • 171
  • 9