How to accurately measure the contact ear orientation on a flat battery using diplib

Question

Our production line want to automate the fabrication of flat battery, I want to accurately measure the contact ear orientation of the flat battery and send the center of the battery coordinate plus the ear orientation degrees to linuxCNC motion controller. I follow a very naive and ad hoc way to implement it using python binding of diplib, I am not sure it is accurate enough and bearing the coarse light source on production line. Here is my code and test images

import diplib as dip

img = dip.ImageRead('tail2.png')
dip.viewer.Show(img)
img2 = dip.ColorSpaceManager.Convert(img, 'grey')

# blend the ultimate results on img3
img3 = img.Copy()

gm = dip.Norm(dip.GradientMagnitude(img2))
dip.viewer.Show(gm)

# detect main circle using watershed
wlab = dip.Watershed(gm, connectivity=1, maxDepth=1, flags={'correct', 'labels'})
dip.viewer.Show(wlab)

wlab = dip.SmallObjectsRemove(wlab, 8000)
dip.viewer.Show(wlab)

# select main circle lab
pp = (wlab == 18179)

lab = dip.Label(pp)
dip.viewer.Show(lab)

# lab = dip.GrowRegions(lab, connectivity=1, iterations = 3)
result = dip.MeasurementTool.Measure(lab, img, features=['Center', 'Radius'])
print(result)

circle = dip.Image(img.Sizes(), 1, 'SFLOAT')
circle.Fill(0)

# draw a solid circle as a mask, subtracted from wlab(id = 18179), obtain the tail(or ear)
dip.DrawBandlimitedBall(circle, diameter=result[1]['Radius'][1]*2, origin=result[1]['Center'], value=1)
circle /= dip.Maximum(circle)

img *= 1 - circle
img += circle * dip.Create0D([0,255,0])

dip.viewer.Show(img)
img[img == [0,255,0]] = 0

mymask = img.Copy()
mymask[mymask == [0,255,0]] = 1000
dip.viewer.Show(mymask)

mainCircle = dip.Threshold(dip.ColorSpaceManager.Convert(mymask, 'grey'))[0]
dip.viewer.Show(mainCircle)

mainCircle = dip.Dilation(mainCircle, dip.SE(1))

# obtain the ear, open by reconstruction and opening
tail = pp - mainCircle
dip.viewer.Show(tail)


tail = dip.OpeningByReconstruction(tail,15)
tail = dip.Opening(tail,5)

mylab = dip.Label(tail)
dip.viewer.Show(mylab)

# obtain the center of the ear, connect to center of the main cirle and blend it to the original image
result2 = dip.MeasurementTool.Measure(mylab, img, features=['Center'])
print(result2)

imgline = dip.Image(img3.Sizes(), 1, 'SFLOAT')
imgline.Fill(0)

dip.DrawBandlimitedLine(imgline, start = result2[1]['Center'], end = result[1]['Center'])

imgline /= dip.Maximum(imgline)

img3 *= 1 - imgline
img3 += imgline * dip.Create0D([0,255,0])

dip.viewer.Show(img3)

dip.ImageWrite(img3, 'mylab.jpg')

I draw a subpixels line between the centers of ear and the main circle, looks good. But the procedure of finding the two points is so ad hoc, I am not sure how accurate it is and how sensitive it is to the light source and how many effort should be involved tuning those parameters. For I have not converted this code into C++ and load into my OpenGL framework and evaluate using a GiGE camera

I am not sure binary dilation, opening could contribute errors, I am still a Apprentice in computer graphics, computer graphics is hard

Convert those code to C++ and integrate into my OpenGL based test platform is so labor intensive for me, before I do this, I want to hear from some more elegant and robust solution

Finally, I share the cpp code converted from python

Mat orientation(Mat input) {
  try {
    dip::Image gray, battery;
    dip::Image src  = dip_opencv::MatToDip(input);
    src.SetColorSpace("RGB");
    src.ResetPixelSize();
    dip::Gauss(src, gray, {1});
    gray = dip::MinimumTensorElement(gray);

    dip::Threshold(gray, battery);
    battery = dip::Label(battery, 0, 0, 0, {"remove"});
    battery = dip::SmallObjectsRemove(battery, 80000);
    battery.Convert(dip::DT_BIN);
    battery = dip::FillHoles(battery);

    dip::Image body = dip::Opening(battery, 60);
    dip::Image ear = battery - body;
    ear = dip::OpeningByReconstruction(ear, 7);

    dip::Image lab = dip::Convert(body, dip::DT_UINT8);
    lab.At(ear) = 2;

    lab.Convert(dip::DT_UINT8);
    lab.At(lab == 2) = 100;
    lab.At(lab == 1) = 160;

    dip::MeasurementTool msr;
    dip::Measurement sizes = msr.Measure(lab, {}, {"Center"}, {}, 1);

    std::cout << sizes << '\n';

    dip::Image imgline = dip::Image(src.Sizes(), 1, dip::DT_DFLOAT);
    imgline.Fill(0);

    dip::DrawBandlimitedLine(imgline, {sizes[100]["Center"][0], sizes[100]["Center"][1]}, {sizes[160]["Center"][0], sizes[160]["Center"][1]});

    imgline /= dip::Maximum(imgline);
    dip::Image output = src.Copy();
    output *= 1 - imgline;
    dip::Image my0d({0,255,0});
    output += imgline * my0d;

    return dip_opencv::CopyDipToMat(output);
  }
  catch (dip::Error const& e) {
    DIP_ADD_STACK_TRACE(e);
    std::cout << "exception: waterPreview" << '\n';
    return input;
  }
}

add this line after dip::SmallObjectsRemove, greatly smooth the ear finding procedure

battery = dip::FillHoles(battery);

pretty impressive, diplib is really light unbias, works very well on our production line

This is what I really want, the light source on production line is really sucks

"the coarse light source on production line." How much does your illumination change? If the battery is always overexposed like this, and the background is darker, then a simple threshold should suffice to find that battery. Your code is not only overly complex, but things like `wlab == 18179` will not work at all as soon as you put in a second image. Make sure you test your code on a series of images, as many as you can get your hands on, and from all the different cameras if there is more than one. You cannot write code for one image and expect it to work on new images. — Cris Luengo, Apr 06 '23 at 16:32
To find the ear, I would use a morphological opening with a circular structuring element that is larger than the ear. This would remove the ear but keep the rest of the battery. The difference would be the ear and possibly some noise around the perimeter of the battery that should be easy to discard. — Cris Luengo, Apr 06 '23 at 16:35
Oh, I see what you're doing with `lab == 18179`. You only have one object at this point. You can replace `pp = (wlab == 18179); lab = dip.Label(pp)` with `lab = dip.Relabel(wlab)` if you're sure you have only one label. You could also do `lab = +(wlab > 0)`, which will take all remaining labels and turn them into label #1, if that is what you need. — Cris Luengo, Apr 06 '23 at 16:43

score 1 · Accepted Answer · answered Apr 06 '23 at 17:13

I simplified your code as follows. I don't think it'll be more precise, but simpler usually means more robust.

import diplib as dip

img = dip.ImageRead('tail2.png')

# A little bit of noise reduction
gray = dip.Gauss(img, 1)

# Conversion to gray assumes the battery shows white in the image
gray = dip.MinimumTensorElement(gray)

# Binarization assumes the battery is surrounded by darker background
# We're keeping the largest segment not connected to the image edge
battery = dip.Threshold(gray)[0]
battery = dip.Label(battery, boundaryCondition=["remove"], mode="largest") > 0

# We separate the ear by opening
body = dip.Opening(battery, 60)
ear = dip.OpeningByReconstruction(battery - body, 7)

# Obtain the center of the body and ear
lab = dip.Convert(body, "UINT8")
lab[ear] = 2  # label for body = 1, label for ear = 2
msr = dip.MeasurementTool.Measure(lab, features=['Center'])

# Draw the line (code unchanged from OP)
imgline = dip.Image(img.Sizes(), 1, 'SFLOAT')
imgline.Fill(0)
dip.DrawBandlimitedLine(imgline, start=msr[1]['Center'], end=msr[2]['Center'])
imgline /= dip.Maximum(imgline)
img *= 1 - imgline
img += imgline * dip.Create0D([0,255,0])

img.Show()

For the line battery = dip.Threshold(gray)[0], you might want to try other threshold methods. In particular the "triangle" or "background" methods could be good here.

Very helpful, I found this code works extremely well. For our rough light source, I add dip.FillHoles(battery) after battery = dip.Label(battery, boundaryCondition=["remove"], mode="largest") > 0 to stabilize the ear label #2 location — HotCat, Apr 07 '23 at 11:28

How to accurately measure the contact ear orientation on a flat battery using diplib

1 Answers1