1

I do have different images which all have some kind of border around the "real" image. What I would like to achieve is to find the "real" image (size and location in pixels).

real image showing water with shark

For me the challenge is that the border is not always black (can be any kind of black or grey with a lot of noise) and the "real" image (water with shark in this example) can have any combination of color, saturation, ...

Now in general I'm aware of algorithms like Canny, Blob detection, hough lines, ..., but I have just started using them. So far I managed to find the border for a specific image, but as soon as I try to apply the same algorithms and parameters to the next image it doesn't work. My current approach looks like this (pseudo code):

  1. convert to gray CvInvoke.CvtColor(_processedImage, tempMat, CvEnum.ColorConversion.Rgb2Gray)
  2. downsample with CvInvoke.PyrDown(srcImage, targetImage) and CvInvoke.PyrUp(srcImage, targetImage)
  3. blur image with CvInvoke.GaussianBlur(_processedImage, bluredImage, New Drawing.Size(5, 5), 0)
  4. Binarize with CvInvoke.Threshold(_processedImage, blackWhiteImage, _parameters.BinarizeThreshold, 255, CvEnum.ThresholdType.Binary)
  5. Detect Edges with CvInvoke.Canny(_processedImage, imgEdges, 60, 100)
  6. Find Contours with CvInvoke.FindContours(_processedImage, contours, Nothing, CvEnum.RetrType.External, CvEnum.ChainApproxMethod.ChainApproxSimple)
  7. Assume that largest contour is the real image

I already tried different approaches based on for example:

Any hint especially on how to find proper parameters (that apply for all images) for algorithms like (adaptive) threshold and canny as well as ideas for improving the processing pipeline would be highly appreciated.

Jan Suchotzki
  • 1,406
  • 12
  • 24

2 Answers2

1

you can try to subtract black image from this image , and you will get the inside image , way to do this: Use image subtraction to compare images in C# ,

Nofar Eliasi
  • 109
  • 5
  • Thanks for the idea and the link. Do I understand this correctly, that I start with a completely black image of same size then my original image and substract the original image from the black? As mentioned in the question the border is not always black, but a kind of grey. So I would end up with the border in another grey, correct? The next Problem is that I need coordinates and size of the inside image. How to do that? Sorry for the beginner questions, but this stuft is really not clear to me. – Jan Suchotzki Oct 29 '17 at 18:51
  • 1
    Subtracting a totally black image is of course a no-op, but subtracting something else might possibly be useful. BTW OpenCV has a built-in functions for computing absolute difference between images, and finding maxima. – NickJH Oct 30 '17 at 11:19
1

If the border was uniform, this would be easy. Use cv::reduce to find MIN and MAX of each row and column; then count the top,left,bottom,right rows/columns whose MIN and MAX are equal (or very close) to the pixel value in a nearby corner. For sanity, maybe check the border colour is the same on all sides.

In your example the border contains faint red stuff, but a row/column approach might still be a useful way to simplify the problem. Maybe, as Nofar suggests, take an absolute difference with what you think is the background colour; square it, convert to grey, then reduce to Sums of rows and columns. You still need to find edges, but have reduced the data from two dimensions to one.

If there's a large border and lots of noise, maybe iterate: in the second pass, exclude the rows you think comprise the border, from statistics on columns (and vice versa).

EDIT: The above only works for an upright rectangle! If it could be rotated then the row/column projection method won't work. In that case I might go for sum-of-squared differences as above (don't start by converting to grey as it could throw away information), followed by blurring or some morphology, edge detection then some kind of Hough transform to find straight edges.

NickJH
  • 561
  • 3
  • 7
  • good ideas! Thanks. Indeed the rectangles are slightly rotated. Just researched about sum-of-squared-differences. However, it is not clear to me how to use this on a single image. I could assume that a certain part of the overall image will be the border. Is this what you mean? Or are you thinking more fine grained and compare certain parts of the image with each other? – Jan Suchotzki Oct 29 '17 at 16:04
  • @Jan Well I didn't have a complete solution but tried to contribute some ideas.In the end it will depend on your particular image content. If the rotation is slight I might still start by trying to cut off rows and columns. Is there any region that's always background; or is the background always dark? If not then you might need some other way to indentify background, such as its texture. (Looks like it's a movie, in which case you could maybe do something with multiple frames too...) – NickJH Oct 30 '17 at 10:37
  • Thank you so much for your time and effort! Will check your ideas out next week once I‘m back on my machine. – Jan Suchotzki Oct 31 '17 at 14:09