2

I need to segment the images of cans and bottles in order to classify them later by the shape of the area. To do this, I'm trying to find a closed contour of the object. I use the Canny algorithm and closing morphological operation. As a result of this, I don't always get a closed contour to find the mask.
There are three features:

  1. It is acceptable if the contour is not quite accurate. The main goal is classification into a can and a bottle, which is feasible even with inaccurate contours.
  2. The final project should be rewritten in c++, so it is necessary to use the libraries that are available in this language.
  3. The segmentation process should not be too long. 40-45 seconds are allocated for the entire photo processing cycle. Therefore, no more than 20-25 should be spent on segmentation.

Based on requirements 2 and 3, OpenCV is the preferred development tool, but it is not necessary.

Examples

  • Expected

Closed contour ** What I get **

  • Correct

Source

Canny

Closing

  • Incorrect

Canny

Closing

What I've tried

  1. Сanny with closing (described above)
canny = cv.Canny(cropped,t1,t2)
kernel_close = cv.getStructuringElement(cv.MORPH_RECT,ksize = (45,45))
closed = cv.morphologyEx(canny,cv.MORPH_CLOSE,kernel_close)
  1. Canny with pre-blurring I got the best results with a gaussian kernel size of 30, but the contour is not closed here either.
    Result
def get_edges(gray,t1k,t2k,ksize = 3):
    Gk = cv.getGaussianKernel(ksize,-1)
    kx,ky =cv.getDerivKernels(1,0,3,normalize=True)
    blured = cv.sepFilter2D(gray,cv.CV_64F,Gk,Gk)
    dx = cv.sepFilter2D(blured,-1,kx,ky)
    dy = cv.sepFilter2D(blured,-1,ky,kx)
    max_val = max(dx.max(),dy.max())
    t1 = max_val*t1k
    t2 = max_val*t2k
    edges = cv.Canny(np.int16(dx),np.int16(dy),t1,t2)
    return blured,edges
    
    t1=0.2
    t2 = t1*2
    ksize = 30
    blured,edges = get_edges(gray,t1,t2,ksize)
  1. Active contour models algorithm by scikit-image
    Gives a closed contour, which is what I need. However, the final project should be written in c++. I have not found a suitable implementation of this method in c++.

Result (red is the initial snake, blue is the result).

Update.
Bottles can vary greatly in color and transparency. Here are examples

example 1

example 2

  • you can apply [YOLO](https://github.com/ultralytics/yolov5) before filtering and binarizing. – Bilal Aug 08 '22 at 19:48
  • Does this question with answers help: https://stackoverflow.com/q/17538366/18667225 ? – Markus Aug 08 '22 at 20:21
  • @Markus, it uses the active contour method. To date, the authors of OpenCv have removed the implementation of this method from the library. – Захар Гостев Aug 08 '22 at 20:44
  • Ah, ok. Another possibility could be working with [alpha shape](https://en.wikipedia.org/wiki/Alpha_shape) algorithm (e.g. CGAL lib). – Markus Aug 08 '22 at 21:08
  • 1
    Canny detects regions where there is a strong edge. It does not find closed contours. Do not use it if you need closed contours. If you have a fixed environment (fixed camera, fixed background, fixed illumination), take a picture without the bottle, then put the bottle in and take another picture. The difference is the bottle. This will work well even for mostly transparant bottles, as the color of the pixels will change because of the glass or plastic in the way. – Cris Luengo Aug 09 '22 at 15:21
  • @CrisLuengo, I'll ask for a photo of the background, thanks for the idea. However, I have already tried to apply this method with another batch of photos, without a strictly fixed camera. In this case, nothing good came out. I hope it will work with a fixed camera. – Захар Гостев Aug 09 '22 at 19:23
  • @CrisLuengo, I think I can try to remove the background using morphological operations. – Захар Гостев Aug 09 '22 at 19:25
  • If the everything is fixed, this works, and it's the simplest, fastest and best way to solve the problem. If things change, such as when the camera is hand-held, or you illuminate the scene using a window, this will not work. – Cris Luengo Aug 09 '22 at 19:25

0 Answers0