3

My Question is related to OpenCV / Matplotlib only. However, to understand the problem bear with me for a few lines of ML / Computer vision side of stuff :

I am working on a Image segmentation problem on floor plan dataset. I would be using Fully Convolutional Networks (FCN) for the same.

Now, FCNs require for an image, a corresponding segmented image. For example :

image and its groundtruth

The image on the left is actual image, while image on the right is the proper "ANNOTATION" of the image. Essentially, every object (class) is completely filled with its own colour - car, road, buildings, etc.

Now, in my case, I'm working with floor plan data. An example floorplan image looks like this :

Example floorplan image

The relevant entities in such images are walls, doors, etc.

Problem I have gotten data annotated for a HUGE bunch of such images, however it is not annotated in the above form. Walls are annotated as simple lines --- which means they're 1px thick only and do not actually fill the area/thickness of the walls in actual image. See the following example :

Annotated Data I have

See the 2 lines representing walls on top left in above image. While wall is thick, what I have as annotated data is just those pink 1px thick lines.

However, what I would require is actually filled region of walls completely. For example :

Ground truth I want

What I want to do now is, programmatically, convert these lines to filled regions, i.e., based on having data about lines for every wall, I wish to create a filled version of walls inside which these lines reside.

So, essentially the question is this : If I have a thin line inside a rectangular region, can I somehow get the whole rectangular region that it is representing? If I can I could fill it with its colour and I can annotate the way it's needed.

Some assumptions we can make : - Lines will always be representing the walls inside which they are - Walls would usually be thicker lines only having some dark colour compared to rest of the image

It would be very difficult and expensive to get data annotated again, hence I'm asking this in case this could be programmatically achieved.

I have very thin knowledge of opencv and matplotlib and hence this could be a newbie question. It might be a very very simple thing to do. In such a case, please let me know the algorithm, or function I need to read up on.

Thanks.

Bhavul
  • 418
  • 4
  • 8
  • I'm sorry I didn't understand the reason for down-voting the question. It would be good if you guys let me know the reason for the same. I would be happy if someone could even just point me in the right direction. – Bhavul Aug 15 '18 at 21:06

2 Answers2

2

Editing my answer to cover @Prune condition. I would probably try some logic such as:

  1. Find pink lines. In order to do that, you can take a look at: how to detect lines in opencv. The result of this part is an array containing endpoints of detected line segments.
  2. Find each line orientation. Consider the endpoints of one line: [x1;y1] and [x2;y2]. If (x2-x1)>(y2-y1), the line is horizontal. Else, the line is vertical.
  3. Find new endpoints. If line is horizontal, move left from your "leftiest" (smaller x) endpoint, until you find the dark gray pixel. It will be your new left endpoint (x1') once you find it. Do the same for the "rightiest" (larger x) endpoint, this time moving right, so you find x2'. If line is vertical, move down from your lower (smaller y) endpoint, until you find the dark gray pixel. It will be your new lower endpoint (y1') once you find it. Do the same for the upper (larger y) endpoint, this time moving up, so you find y2'.
  4. Draw a rectangle, in order to paint your new wall. If the wall width is constant and the pink line is centralized, from here you can simply draw a rectangle. If the line is horizontal, the top-left coordinate of the rectangle is [x1';y1+half of wall width] and the bottom-right coordinate is [x2'; y2-half of wall width]. If the line is vertical, the top-left coordinate of the rectangle is [x1-half of wall width;y1'] and the bottom-right coordinate is [x2; y2'-half of wall width].

Let me know if the 4th condition is not met. In this case, for horizontal lines, for example, we would need to iterate up/down along the horizontal line, looking for the smaller distance between the dark pixels, and then get this y-coordinates to help draw the rectangle.

Mariana Mascarenhas de Carvalh
  • 106
  • 1
  • 4
  • 2
    Thanks. I had initially thought the same but this didn't seem so much of an optimised approach to me. In any case, even if I go ahead with something like this, is there a nice way to use some existing matplotlib or opencv2 functions to help me through this? Like are there methods through which I could get pink pixel points directly, and all their neighbouring pixel points as well? Is there anything to keep filling neighbour points until you reach a certain condition? – Bhavul Aug 15 '18 at 21:12
  • In order to get the pink pixels, you can use an InRange sort of function. https://docs.opencv.org/3.4/da/d97/tutorial_threshold_inRange.html. Now, to find pixels that meet certain condition, I believe iterating until you find image borders will be the best method, specifically for the solution I gave in my edited answer. – Mariana Mascarenhas de Carvalh Aug 23 '18 at 06:17
  • 1
    Hey thanks for an updated and cleaner answer. I happened to use the approach wherein, for vertical lines, I checked if endpoints one pixel to the left would also have same background color...and if they do, I would draw a line here. Then move 1 px left again. So on... Similarly, to the right. Then for horizontal lines, I similarly iterate and keep plotting neighbouring lines above and below. This approach also works. And yours would do too. I'm accepting your answer. For anyone in future referring to this, keep in mind you need to have your pink line clearly inside the wall region. – Bhavul Aug 23 '18 at 19:14
2

Normally, you could use a simple "fill" algorithm for this, as suggest in Mariana's answer. However, you need to extend only so far as the given line -- you cannot continue down the west wall of that office area. I'm assuming that each end result will be a rectangle aligned to the drawing axes.

Instead, you need a "wave-front" fill: extend the entire line north and south (the line dimension that is more than a single pixel) so far as all of the adjacent pixels are gray. Then do the same with the east-west direction. You could perform the extension by replicating the line to the adjacent pixel row/column.

I don't know of a package that will support this directly, but most will do a vectorized copy-paste or equivalent change.

Prune
  • 76,765
  • 14
  • 60
  • 81