Python/OpenCV — Intelligent Centroid Tracking in Bacterial Images?

Question

I'm currently working on an algorithm to detect bacterial centroids in microscopy images.

This question is a continuation of: OpenCV/Python — Matching Centroid Points of Bacteria in Two Images: Python/OpenCV — Matching Centroid Points of Bacteria in Two Images

I am using a modified version of the program proposed by Rahul Kedia. https://stackoverflow.com/a/63049277/13696853

Currently, the issues in segmentation I am working on are:

1. Low Contrast
2. Clustering

The images below are sampled a second apart. However, in the latter image, one of the bacteria does not get detected.

Bright-field Image #1

Bright-Field Image #2

Bright-Field Contour Image #1

Bright-Field Contour Image #2

Bright-Field Image #1 (Unsegmented)

Bright-Field Image #2 (Unsegmented)

I want to know, given that I can successfully determine bacterial centroids in an image, can I use the data to intelligently look for the same bacteria in the subsequent image?

I haven't been able to find anything substantial online; I believe SIFT/SURF would likely be ineffective as the bacteria have the same appearance. Moreover, I am looking for specific points in the images. You can view my program below. Insert a specific path as indicated if you'd like to run the program.

import cv2
import numpy as np
import os

kernel = np.array([[0, 0, 1, 0, 0],
                   [0, 1, 1, 1, 0],
                   [1, 1, 1, 1, 1],
                   [0, 1, 1, 1, 0],
                   [0, 0, 1, 0, 0]], dtype=np.uint8)


def e_d(image, it):
    image = cv2.erode(image, kernel, iterations=it)
    image = cv2.dilate(image, kernel, iterations=it)
    return image


path = r"[INSERT PATH]"
img_files = [file for file in os.listdir(path)]


def segment_index(index: int):
    segment_file(img_files[index])


def segment_file(img_file: str):
    img_path = path + "\\" + img_file
    print(img_path)
    img = cv2.imread(img_path)
    img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
    # Applying adaptive mean thresholding
    th = cv2.adaptiveThreshold(img, 255, cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY_INV, 11, 2)
    # Removing small noise
    th = e_d(th.copy(), 1)

    # Finding contours with RETR_EXTERNAL flag and removing undesired contours and
    # drawing them on a new image.
    cnt, hie = cv2.findContours(th, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
    cntImg = th.copy()
    for contour in cnt:
        x, y, w, h = cv2.boundingRect(contour)
        # Eliminating the contour if its width is more than half of image width
        # (bacteria will not be that big).
        if w > img.shape[1] / 2:
            continue
        cntImg = cv2.drawContours(cntImg, [cv2.convexHull(contour)], -1, 255, -1)

    # Removing almost all the remaining noise.
    # (Some big circular noise will remain along with bacteria contours)
    cntImg = e_d(cntImg, 3)

    # Finding new filtered contours again
    cnt2, hie2 = cv2.findContours(cntImg, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)

    # Now eliminating circular type noise contours by comparing each contour's
    # extent of overlap with its enclosing circle.
    finalContours = []  # This will contain the final bacteria contours
    for contour in cnt2:
        # Finding minimum enclosing circle
        (x, y), radius = cv2.minEnclosingCircle(contour)
        center = (int(x), int(y))
        radius = int(radius)

        # creating a image with only this circle drawn on it(filled with white colour)
        circleImg = np.zeros(img.shape, dtype=np.uint8)
        circleImg = cv2.circle(circleImg, center, radius, 255, -1)

        # creating a image with only the contour drawn on it(filled with white colour)
        contourImg = np.zeros(img.shape, dtype=np.uint8)
        contourImg = cv2.drawContours(contourImg, [contour], -1, 255, -1)

        # White pixels not common in both contour and circle will remain white
        # else will become black.
        union_inter = cv2.bitwise_xor(circleImg, contourImg)

        # Finding ratio of the extent of overlap of contour to its enclosing circle.
        # Smaller the ratio, more circular the contour.
        ratio = np.sum(union_inter == 255) / np.sum(circleImg == 255)

        # Storing only non circular contours(bacteria)
        if ratio > 0.55:
            finalContours.append(contour)

    finalContours = np.asarray(finalContours)

    # Finding center of bacteria and showing it.
    bacteriaImg = cv2.cvtColor(img, cv2.COLOR_GRAY2BGR)

    for bacteria in finalContours:
        M = cv2.moments(bacteria)
        cx = int(M['m10'] / M['m00'])
        cy = int(M['m01'] / M['m00'])

        bacteriaImg = cv2.circle(bacteriaImg, (cx, cy), 5, (0, 0, 255), -1)

    cv2.imshow("bacteriaImg", bacteriaImg)
    cv2.waitKey(0)


# Segment Each Image
for i in range(len(img_files)):
    segment_index(i)

Edit #1: Applying frmw42's approach, this image seems to get lost. I have tried adjusting a number of parameters but the image does not seem to show up.

Bright-Field Image #3

Bright-Field Image #4

Answer 1

Here is my Python/OpenCV code to extract your bacteria. I simply threshold, then get the contours and draw filled contours for those within a certain area range. I will let you do any further processing that you want. I simply viewed each step to make sure I have tuned the arguments appropriately before moving to the next step.

Input 1:

Input 2:

import cv2
import numpy as np

# read image
#img = cv2.imread("bacteria1.png")
img = cv2.imread("bacteria2.png")

# convert img to grayscale
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
gray = 255 - gray

# do adaptive threshold on inverted gray image
thresh = cv2.adaptiveThreshold(gray, 255, cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY, 21, 5)

result = np.zeros_like(img)
contours = cv2.findContours(thresh , cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
for cntr in contours:
    area = cv2.contourArea(cntr)
    if area > 600 and area < 1100:
        cv2.drawContours(result, [cntr], 0, (255,255,255), -1)


# write results to disk
#cv2.imwrite("bacteria_filled_contours1.png", result)
cv2.imwrite("bacteria_filled_contours2.png", result)

# display it
cv2.imshow("thresh", thresh)
cv2.imshow("result", result)
cv2.waitKey(0)

Result 1:

Result 2:

Adjust as desired.

Answer 2

It would seem that adaptive threshold is not able to handle all your various images. I suspect nothing simple will. You may need to use AI with training. Nevertheless, this works for your images: 1, 2 and 4 in Python/OpenCV. I make no guarantee that it will work for any of your other images.

First I found a simple threshold that seems to work, but brings in other regions. So since all your bacteria have similar shapes and range of orientations, I fit and ellipse to your bacteria and get the orientation of the major axis and filter the contours with area and angle.

import cv2
import numpy as np

# read image
#img = cv2.imread("bacteria1.png")
#img = cv2.imread("bacteria2.png")
img = cv2.imread("bacteria4.png")

# convert img to grayscale
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
gray = 255 - gray

# median filter
#gray = cv2.medianBlur(gray, 1)

# do simple threshold on inverted gray image
thresh = cv2.threshold(gray, 170, 255, cv2.THRESH_BINARY)[1]

result = np.zeros_like(img)
contours = cv2.findContours(thresh , cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
for cntr in contours:
    area = cv2.contourArea(cntr)
    if area > 600 and area < 1100:
        ellipse = cv2.fitEllipse(cntr)
        (xc,yc),(d1,d2),angle = ellipse
        if angle > 90:
            angle = angle - 90
        else:
            angle = angle + 90
        print(angle,area)
        if angle >= 150 and angle <= 250:
            cv2.drawContours(result, [cntr], 0, (255,255,255), -1)
  
# write results to disk
#cv2.imwrite("bacteria_filled_contours1.png", result)
#cv2.imwrite("bacteria_filled_contours2.png", result)
cv2.imwrite("bacteria_filled_contours4.png", result)

# display it
cv2.imshow("thresh", thresh)
cv2.imshow("result", result)
cv2.waitKey(0)

Result for image 1:

Result for image 2:

Result for image 4:

You might explore noise reduction before thresholding. I had some success with using some of ImageMagick tools and there is a Python version called Python Wand that uses ImageMagick.