How to reliably detect a barcode's 4 corners?

Question

I'm trying to detect this Code128 barcode with Python + zbar module:

(Image download link here).

This works:

import cv2, numpy
import zbar
from PIL import Image 
import matplotlib.pyplot as plt

scanner = zbar.ImageScanner()
pil = Image.open("000.jpg").convert('L')
width, height = pil.size    
plt.imshow(pil); plt.show()
image = zbar.Image(width, height, 'Y800', pil.tobytes())
result = scanner.scan(image)

for symbol in image:
    print symbol.data, symbol.type, symbol.quality, symbol.location, symbol.count, symbol.orientation

but only one point is detected: (596, 210).

If I apply a black and white thresholding:

pil = Image.open("000.jpg").convert('L')
pil = pil .point(lambda x: 0 if x<100 else 255, '1').convert('L')    

it's better, and we have 3 points: (596, 210), (482, 211), (596, 212). But it adds one more difficulty (finding the optimal threshold - here 100 - automatically for every new image).

Still, we don't have the 4 corners of the barcode.

Question: how to reliably find the 4 corners of a barcode on an image, with Python? (and maybe OpenCV, or another library?)

Notes:

• It is possible, this is a great example (but sadly not open-source as mentioned in the comments):

  Object detection, very fast and robust blurry 1D barcode detection for real-time applications

  The corners detection seems to be excellent and very fast, even if the barcode is only a small part of the whole image (this is important for me).

  

• Interesting solution: Real-time barcode detection in video with Python and OpenCV but there are limitations of the method (see in the article: the barcode should be close up, etc.) that limit the potential use. Also I'm more looking for a ready-to-use library for this.

• Interesting solution 2: Detecting Barcodes in Images with Python and OpenCV but again, it does not seem like a production-ready solution, but more a research in progress. Indeed, I tried their code on this image but the detection does not yield successful result. It has to be noted that it doesn't take any spec of the barcode in consideration for the detection (the fact there's a start/stop symbol, etc.)

  import numpy as np
  import cv2
  image = cv2.imread("000.jpg")
  gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
  gradX = cv2.Sobel(gray, ddepth = cv2.CV_32F, dx = 1, dy = 0, ksize = -1)
  gradY = cv2.Sobel(gray, ddepth = cv2.CV_32F, dx = 0, dy = 1, ksize = -1)
  gradient = cv2.subtract(gradX, gradY)
  gradient = cv2.convertScaleAbs(gradient)
  blurred = cv2.blur(gradient, (9, 9))
  (_, thresh) = cv2.threshold(blurred, 225, 255, cv2.THRESH_BINARY)
  kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (21, 7))
  closed = cv2.morphologyEx(thresh, cv2.MORPH_CLOSE, kernel)
  closed = cv2.erode(closed, None, iterations = 4)
  closed = cv2.dilate(closed, None, iterations = 4)
  (_, cnts, _) = cv2.findContours(closed.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
  c = sorted(cnts, key = cv2.contourArea, reverse = True)[0]
  rect = cv2.minAreaRect(c)
  box = np.int0(cv2.boxPoints(rect))
  cv2.drawContours(image, [box], -1, (0, 255, 0), 3)
  cv2.imshow("Image", image)
  cv2.waitKey(0)
  

Answer 1

Solution 2 is pretty good. The critical factor that made it fail on your image was the thresholding. If you drop the parameter 225 way down to 55, you'll get much better results.

I've reworked the code, making some tweaks here and there. The original code is fine if you prefer. The documentation for OpenCV is quite good, and there are very good Python tutorials.

import numpy as np
import cv2

image = cv2.imread("barcode.jpg")
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

# equalize lighting
clahe = cv2.createCLAHE(clipLimit=2.0, tileGridSize=(8,8))
gray = clahe.apply(gray)

# edge enhancement
edge_enh = cv2.Laplacian(gray, ddepth = cv2.CV_8U, 
                         ksize = 3, scale = 1, delta = 0)
cv2.imshow("Edges", edge_enh)
cv2.waitKey(0)
retval = cv2.imwrite("edge_enh.jpg", edge_enh)

# bilateral blur, which keeps edges
blurred = cv2.bilateralFilter(edge_enh, 13, 50, 50)

# use simple thresholding. adaptive thresholding might be more robust
(_, thresh) = cv2.threshold(blurred, 55, 255, cv2.THRESH_BINARY)
cv2.imshow("Thresholded", thresh)
cv2.waitKey(0)
retval = cv2.imwrite("thresh.jpg", thresh)

# do some morphology to isolate just the barcode blob
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (9, 9))
closed = cv2.morphologyEx(thresh, cv2.MORPH_CLOSE, kernel)
closed = cv2.erode(closed, None, iterations = 4)
closed = cv2.dilate(closed, None, iterations = 4)
cv2.imshow("After morphology", closed)
cv2.waitKey(0)
retval = cv2.imwrite("closed.jpg", closed)

# find contours left in the image
(_, cnts, _) = cv2.findContours(closed.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
c = sorted(cnts, key = cv2.contourArea, reverse = True)[0]
rect = cv2.minAreaRect(c)
box = np.int0(cv2.boxPoints(rect))
cv2.drawContours(image, [box], -1, (0, 255, 0), 3)
print(box)
cv2.imshow("found barcode", image)
cv2.waitKey(0)
retval = cv2.imwrite("found.jpg", image)

edge.jpg

thresh.jpg

closed.jpg

found.jpg

output from console:

[[596 249]
 [470 213]
 [482 172]
 [608 209]]

Answer 2

For the following to work, you need to have contrib package installed using pip install opencv-contrib-python

Your OpenCV version would now have a separate class for detecting barcodes.

cv2.barcode_BarcodeDetector() comes equipped with 3 in-built functions:

• decode(): returns decoded information and type
• detect(): returns the 4 corner points enclosing each detected barcode
• detectAndDecode(): returns all the above

Sample Image used is from pyimagesearch blog:

The 4 corners are captured in points.

Code:

img = cv2.imread('barcode.jpg')

barcode_detector = cv2.barcode_BarcodeDetector()

# 'retval' is boolean mentioning whether barcode has been detected or not
retval, decoded_info, decoded_type, points = barcode_detector.detectAndDecode(img)

# copy of original image
img2 = img.copy()

# proceed further only if at least one barcode is detected:
if retval:
    points = points.astype(np.int)
    for i, point in enumerate(points):
        img2 = cv2.drawContours(img2,[point],0,(0, 255, 0),2)

        # uncomment the following to print decoded information
        #x1, y1 = point[1]
        #y1 = y1 - 10
        #cv2.putText(img2, decoded_info[i], (x1, y1), cv2.FONT_HERSHEY_SIMPLEX, 1, (255, 255, 0), 3, 2)

Result:

Detected barcode:

Detected barcode and information:

Answer 3

One solution not discussed here is PyZbar.

It is helpful to know there are a number of different types of barcode so reading this can be helpful. Now each solution for decoding might have limitations for the types it can decode. @Jeru Luke's solution seems to be only support EAN-13 barcodes currently see docs here.

Now using PyZbar a simple solution for getting the rect object (4 corners) with the decoding and the bonus of finding out which type the barcode it is can be done with this script.

Using this barcode

import cv2
from pyzbar.pyzbar import decode

file_path = r'c:\my_file'
img = cv2.imread(file_path)
detectedBarcodes = decode(img)
for barcode in detectedBarcodes:
    (x, y, w, h) = barcode.rect
    cv2.rectangle(img, (x, y), (x + w, y + h), (255, 0, 0), 5)
    print(barcode.rect)
    print(barcode.data)
    print(barcode.type)

output

Rect(left=77, top=1, width=665, height=516)
b'9771234567003'
EAN13

Using @Jeru Luke's code you can drawContours and putText.

ZBar supports

• EAN-13/UPC-A,
• UPC-E, EAN-8,
• Code 128,
• Code 93,
• Code 39,
• Codabar,
• Interleaved 2 of 5,
• QR Code
• SQ Code.

So I think PyZbar will also support these types.