Color thresholding of red rimmed traffic signs

Question

I want to detect all red color rimmed traffic signs (Triangular and Circular). The algorithm has to be efficient and robust to work in real world situations, so i decided to use HSV space since it is light invariant.

I came across this question of detecting red objects and the answer was to use this value ranges for HSV: The code is in C++:

inRange(hsv, Scalar(0, 70, 50), Scalar(10, 255, 255), mask1);
inRange(hsv, Scalar(170, 70, 50), Scalar(180, 255, 255), mask2);

Mat1b mask = mask1 | mask2;

Since I am using Java's OpenCV I tried that, but I found out it is not possible to do a Bitwise OR Operation.

So I tried to implement it manually instead of using OpenCV. I also tried same red color value ranges that is provided and sadly the results were horrible:

Here is my code

Mat hsv = new Mat();
Mat rgb = Highgui.imread(scene, Highgui.CV_LOAD_IMAGE_COLOR);
Imgproc.cvtColor(rgb, hsv, Imgproc.COLOR_RGB2HSV);
Mat thresh = new Mat(hsv.size(), CvType.CV_8UC1);


for(int x=0;x<hsv.rows();x++){
    for(int y=0;y<hsv.cols();y++)
    {
        double[] data = hsv.get(x,y);

        double H = data[0];
        double S = data[1];
        double V = data[2];
        if((( 0.0>=H && H<=10.0) && (70.0>=S && S<=255.0) && (50.0>=V && V<=255.0)) || (( 170.0>=H && H<=180.0) && (70.0>=S && S<=255.0) && (50.0>=V && V<=255.0)) ) {

            thresh.put(x,y, 255);
        }
        else
        {
            thresh.put(x,y, 0);
        }
    }
}

Here is the results before and after thresholding

Can someone provide me with right values?

Answer 1

The crucial mistake is right in the beginning:

Mat rgb = Highgui.imread(scene, Highgui.CV_LOAD_IMAGE_COLOR);
Imgproc.cvtColor(rgb, hsv, Imgproc.COLOR_RGB2HSV);

The OpenCV C++ API reference is generally the most complete and detailed, so it never hurts to refer to it. If you look at cv::imread you will notice the following note:

In the case of color images, the decoded images will have the channels stored in B G R order.

However, in your code you treat the image as RGB, i.e. swapping blue and red. This is fatal for your algorithm -- you're looking for red things, but anything that was red is actually blue.

The fix is simple -- rename rgb to bgr (to avoid misleading variable names) and change the conversion code to Imgproc.COLOR_BGR2HSV.

I believe your earlier problems with bitwise_or were just another symptom of this mistake. (I don't really see a reason why it wouldn't work).

See the following example (using OpenCV 3.4.0):

import org.opencv.core.Mat;
import org.opencv.core.Scalar;
import org.opencv.imgcodecs.Imgcodecs;
import org.opencv.imgproc.Imgproc;
import org.opencv.core.Core;

public class test
{
    public static void main(String[] args)
    {
        System.loadLibrary(Core.NATIVE_LIBRARY_NAME);

        Mat image = Imgcodecs.imread("test.jpg", Imgcodecs.CV_LOAD_IMAGE_COLOR);
        if ((image == null) || image.empty()) {
            System.out.println("Failed to load input image.");
            System.exit(-1);
        }

        Mat image_hsv = new Mat();
        Imgproc.cvtColor(image, image_hsv, Imgproc.COLOR_BGR2HSV);

        Mat mask1 = new Mat();
        Mat mask2 = new Mat();
        Core.inRange(image_hsv, new Scalar(0, 70, 50), new Scalar(10, 255, 255), mask1);
        Core.inRange(image_hsv, new Scalar(170, 70, 50), new Scalar(180, 255, 255), mask2);

        Mat mask_combined = new Mat();
        Core.bitwise_or(mask1, mask2, mask_combined);

        Mat image_masked = new Mat();
        Core.bitwise_and(image, image, image_masked, mask_combined);

        Imgcodecs.imwrite("test-mask.jpg", mask_combined);        
        Imgcodecs.imwrite("test-masked.jpg", image_masked);

        System.out.println("Done!");
    }
}

Which produces the following combined mask from your sample input image:

If we use this mask on the original image, we can see we really do get the red bits:

Answer 2

I have learned OpenCV for C++/Python, and I'm learning OpenCV for Java these days, and find this question is a good practice for pixel loop.

I'm use OpenCV 4.0.0-pre, so some functions maybe in other modules/packages/headers...

---

As @Dan Mašek Suggests, you should convert the image using COLOR_BGR2HSV. Except this, I also find that you have wrote the wrong hsv ranges.

For this code in C++:

inRange(hsv, Scalar(0, 70, 50), Scalar(10, 255, 255), mask1);
inRange(hsv, Scalar(170, 70, 50), Scalar(180, 255, 255), mask2);

Mat1b mask = mask1 | mask2;

Your range condition in Java:

(( 0.0>=H && H<=10.0) && (70.0>=S && S<=255.0) && (50.0>=V && V<=255.0)) ||
(( 170.0>=H && H<=180.0) && (70.0>=S && S<=255.0) && (50.0>=V && V<=255.0)) 

It should be:

(( 0.0<=H && H<=10.0) && (70.0<=S && S<=255.0) && (50.0<=V && V<=255.0)) ||
(( 170.0<=H && H<=180.0) && (70.0<=S && S<=255.0) && (50.0<=V && V<=255.0)) 

Or this:

///!(1) Create mask by loop with condition
(( 0.0<=H && H<=10.0) || ( 170.0<=H && H<=180.0)) && (70.0<=S && S<=255.0) && (50.0<=V && V<=255.0)

Or this:

///!(2) Create mask by calling API
Mat mask1 = new Mat();
Mat mask2 = new Mat();
Core.inRange(hsv, new Scalar(0, 70, 50), new Scalar(10, 255, 255), mask1);
Core.inRange(hsv, new Scalar(170, 70, 50), new Scalar(180, 255, 255), mask2);

Mat mask= new Mat();
Core.bitwise_or(mask1, mask2, mask);

---

In Java:

//! 2018.05.08 18:50:59 CST
//! 2018.05.08 20:53:48 CST
import org.opencv.imgcodecs.Imgcodecs;
import org.opencv.imgproc.Imgproc;
import org.opencv.core.*;
import java.util.*;

public class test
{
    public static void main(String[] args)
    {
        System.loadLibrary(Core.NATIVE_LIBRARY_NAME);
        Mat img = Imgcodecs.imread("test.jpg", Imgcodecs.CV_LOAD_IMAGE_COLOR);
        if ((img == null) || img.empty()) {
            System.out.println("Failed to load input img.");
            System.exit(-1);

        }
        Mat hsv = new Mat();
        Imgproc.cvtColor(img, hsv, Imgproc.COLOR_BGR2HSV);

        ///! (1) Create the mask by loop
        Mat mask = new Mat(hsv.size(), CvType.CV_8UC1, new Scalar(0,0,0));
        for(int i=0;i<hsv.rows();++i){
            for(int j=0;j<hsv.cols();++j){
                double[] data = hsv.get(i,j);
                double H = data[0];
                double S = data[1];
                double V = data[2];
                //mask[np.where(((h<10) | ((h>=170) & (h<=180)) ) & ((s>=70) & (s<=255)) & ((v>=50) & (v<=255)) )] = 255
                if((( 0.0<=H && H<=10.0) || ( 170.0<=H && H<=180.0)) && (70.0<=S && S<=255.0) && (50.0<=V && V<=255.0)) {
                    mask.put(i,j, 255);
                }
            }
        }

        ///!(2) Create mask by calling API
        Mat mask1 = new Mat();
        Mat mask2 = new Mat();
        Core.inRange(hsv, new Scalar(0, 70, 50), new Scalar(10, 255, 255), mask1);
        Core.inRange(hsv, new Scalar(170, 70, 50), new Scalar(180, 255, 255), mask2);

        Mat mask_combined = new Mat();
        Core.bitwise_or(mask1, mask2, mask_combined);

        ///! Get the masked
        Mat masked = new Mat();
        Core.bitwise_and(img, img, masked, mask);
        //Core.bitwise_and(img, img, masked, mask_combined);

        Imgcodecs.imwrite("test_mask.jpg", mask);
        Imgcodecs.imwrite("test_masked.jpg", masked);
    }
}

---

In Python, it can be wrote as:

import cv2
import numpy as np

img = cv2.imread("test.jpg")
hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
h,s,v = cv2.split(hsv)

## (1) by hand 
mask = np.zeros_like(s)
mask[np.where(((h<10) | ((h>=170) & (h<=180)) ) & ((s>=70) & (s<=255)) & ((v>=50) & (v<=255)) )] = 255

## (2) call api 
mask1 = cv2.inRange(hsv, (0, 70, 50), (10, 255, 255))
mask2 = cv2.inRange(hsv, (170, 70, 50), (180, 255, 255))
mask = cv2.bitwise_or(mask1, mask2)

masked = cv2.bitwise_and(img, img, mask=mask)

---

Here is my result: