Checking if specific pixels fall within color ranges?

Question

I am using OpenCV in python to make a program that solves a Rubik's Cube. I am not creating my own solving algorithm, but I am using the python implementation of kociemba.

Anyways, in order to use this algorithm, I have to pass a valid string that represents the colors of the cube. So, as of right now, I am trying to detect the colors of the faces of the Rubik's Cube.

I have tried:

        _, img = self.cap.read()
        print('frame captured!')

        hsv = cv.cvtColor(img, cv.COLOR_BGR2HSV)
        #scan each pixel in the list
        for a in self.pixelsToScan:
            hue = hsv[a[1],a[0],0]
            print(hue)
            sat = hsv[a[1],a[0],1]/255
            print(sat)
            val = hsv[a[1],a[0],2]/255
            print(val)

            #CHANGE COLOR CHECKING TO USE inRange()#

            if sat<.15 and val>.85:
                #WHITE
                self.colors = self.colors + 'U'
            elif hue<7 or hue>=173:
                #RED
                self.colors = self.colors + 'F'
            elif hue>=7 and hue<22:
                #ORANGE
                self.colors = self.colors + 'B'
            elif hue>=22 and hue<37:
                #YELLOW
                self.colors = self.colors + 'D'
            elif hue>=37 and hue<67:
                #GREEN
                self.colors = self.colors + 'L'
            elif hue>=67 and hue<135:
                #BLUE
                self.colors = self.colors + 'R'
            else:
                #BROKEN
                self.colors = self.colors + 'E'

As well as:

        _, img = self.cap.read()
        print('frame captured!')

        #scan each pixel in the list
        for a in self.pixelsToScan:

            if cv.inRange(numpy.copy(img), self.WHITE_MIN, self.WHITE_MAX)[a[1],a[0]]==255:
                #WHITE
                self.colors = self.colors + 'U'
            elif cv.inRange(numpy.copy(img), self.RED_LOWER_MIN, self.RED_LOWER_MAX)[a[1],a[0]]==255 or cv.inRange(numpy.copy(img), self.RED_UPPER_MIN, self.RED_UPPER_MAX)[a[1],a[0]]==255:
                #RED
                self.colors = self.colors + 'F'
            elif cv.inRange(numpy.copy(img), self.ORANGE_MIN, self.ORANGE_MAX)[a[1],a[0]]==255:
                #ORANGE
                self.colors = self.colors + 'B'
            elif cv.inRange(numpy.copy(img), self.YELLOW_MIN, self.YELLOW_MAX)[a[1],a[0]]==255:
                #YELLOW
                self.colors = self.colors + 'D'
            elif cv.inRange(numpy.copy(img), self.GREEN_MIN, self.GREEN_MAX)[a[1],a[0]]==255:
                #GREEN
                self.colors = self.colors + 'L'
            elif cv.inRange(numpy.copy(img), self.BLUE_MIN, self.BLUE_MAX)[a[1],a[0]]==255:
                #BLUE
                self.colors = self.colors + 'R'
            else:
                #BROKEN
                self.colors = self.colors + 'E'

I have no idea how I would reliably check what color each piece is, since lighting is always changing. I thought using the inRange() functions would be what I needed, but it just doesn't work for some reason. And eachrange is different, so I don't know of any way to check the same pixel from the same image against all of them.

Seriously, any help would be greatly appreciated. Thanks in advance!

EDIT: I have now also tried the following:

        _, img = self.cap.read()
        print('frame captured!')

        hsv = cv.cvtColor(img, cv.COLOR_BGR2HSV)
        #scan each pixel in the list
        for a in self.pixelsToScan:
            r = img[a[1],a[0],2]
            g = img[a[1],a[0],1]
            b = img[a[1],a[0],0]

            if r>220 and g>220 and b>220:
                #WHITE
                self.colors = self.colors + 'U'
            elif r>=175 and g<=60 and b<=60:
                #RED
                self.colors = self.colors + 'F'
            elif r>=175 and g>=96 and g<=171 and b<=54:
                #ORANGE
                self.colors = self.colors + 'B'
            elif r>=205 and r <= 213 and g>=175 and b<=41:
                #YELLOW
                self.colors = self.colors + 'D'
            elif r<=96 and g>=175 and b<=128:
                #GREEN
                self.colors = self.colors + 'L'
            elif r>=54 and r<=85 and g<=116 and b >=179:
                #BLUE
                self.colors = self.colors + 'R'
            else:
                #BROKEN
                self.colors = self.colors + 'E'

The rgb limits were created using my hsv boundaries from before.

Here is an image with the desired output:

I have this image of one face of a Rubik’s Cube:

This should produce the output LDLLFLBRR but it will either produce an output of EEEEEEEEE, representing all errors, or `LULLFRFRR’, meaning that orange is detected as red, yellow as white, and green occasionally as blue.

What I need is a reliable way to always be able to tell which color is which.

Thanks for any help!

Maybe your choose the wrong HSV range. Example image is better to under stand you question. https://stackoverflow.com/questions/47483951/how-to-define-a-threshold-value-to-detect-only-green-colour-objects-in-an-image/47483966#47483966 and https://stackoverflow.com/questions/10948589/choosing-the-correct-upper-and-lower-hsv-boundaries-for-color-detection-withcv/48367205#48367205 — Kinght 金, Jun 13 '18 at 06:01
Depending on what IDE you are using, I would suggest to use GDB in order to verify what values your pixels are showing; With that you can then also follow up what ausk is recommending, and see if your ranges do make sense. If you do not know how to use GDB, there is a great tutorial (and generally easy to use implementation) for Spyder. If you do not want to do that, you can always check with print statements... — dennlinger, Jun 13 '18 at 06:39

score 0 · Answer 1 · answered Jun 13 '18 at 09:09

0

I suspect you are looking at one, single pixel to determine the color of the sticker and there are lots of variations within the area of the sticker.

Each sticker seems to be around 510x510 pixels, so I would suggest you try making a box average of at least 100x100 pixels before colour testing.

answered Jun 13 '18 at 09:09

Mark Setchell

191,897
31
273
432

How would you suggest I do that? Do you mean that I should take the average of all of the pixels in that box, and compare the average to my range? Or do you mean compare all of the pixels, and if the majority match, then continue? – MattChris Jun 13 '18 at 13:28
Probably just apply a `boxFilter()` to your whole image and then when you take an individual pixel, it is actually the average of a larger number of pixels. Try with something like 9x9 and see if that helps. https://docs.opencv.org/3.0-beta/modules/imgproc/doc/filtering.html#boxfilter – Mark Setchell Jun 13 '18 at 13:34

Checking if specific pixels fall within color ranges?

1 Answers1