wrong colors when saving image with openCV

Question

With openCV, when I save my processed image in .png format with openCV, I get differents colors than when I display it on screen.

(code is at the end of message)

Here's what I get by displaying the image on screen with cv2.imshow('after', after) (that is what I want):

Here's what I get by saving the same image in .png with cv2.imwrite("debug.png", after) (that is not what I want):

The box and center seems to be transparent, because when I open the .png on vs code, they are blackish like vs code background (see picture above), but when I open it in windows Photos software, they are white (see picture below)

Here's my code that is kind of a fusion between this post and this post. It draws green boxes around differences between two images and draws the center of each difference :

def find_diff(before, after):
    before = np.array(before)
    after = np.array(after)

    # Convert images to grayscale
    before_gray = cv2.cvtColor(before, cv2.COLOR_BGR2GRAY)
    after_gray = cv2.cvtColor(after, cv2.COLOR_BGR2GRAY)

    # Compute SSIM between two images
    (score, diff) = structural_similarity(before_gray, after_gray, full=True)
    diff = (diff * 255).astype("uint8")

    # Threshold the difference image, followed by finding contours to
    # obtain the regions of the two input images that differ
    thresh = cv2.threshold(diff, 0, 255, cv2.THRESH_BINARY_INV | cv2.THRESH_OTSU)[1]
    contours = cv2.findContours(thresh.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
    contours = contours[0] if len(contours) == 2 else contours[1]

    centers = []
    for c in contours:
        area = cv2.contourArea(c)
        if area > 40:
            # Find centroid
            M = cv2.moments(c)
            cX = int(M["m10"] / M["m00"])
            cY = int(M["m01"] / M["m00"])
            centers.append((cY,cY)) 
            cv2.circle(after, (cX, cY), 2, (320, 159, 22), -1) 
            
            #draw boxes on diffs
            x, y, w, h = cv2.boundingRect(c)
            cv2.rectangle(after, (x, y), (x + w, y + h), (36, 255, 12), 2)

    cv2.imwrite("debug.png", after)
    cv2.imshow('after', after)
    cv2.waitKey(0)
    return centers


before = mss.mss().grab((466, 325, 1461, 783))
sleep(3)
after = mss.mss().grab((466, 325, 1461, 783))
relative_diffs = find_diff(before, after)

I can't figure out at which step it messes up. I do give green and blue colors when calling .circle() and .rectangle().

Where is "after" coming from? What is its shape and dtype? How many channels and does it contain alpha? — fmw42, Sep 03 '21 at 22:38
Are you sure you have referenced your two images correctly. I would expect that the imshow() version might not show color properly if it has transparency, whereas the saved PNG should be fine if you are writing your box on an image of a proper number of color channels. — fmw42, Sep 03 '21 at 22:41
@fmw42 I added the script part that calls my function. `after` comes from mss.mss().grab() and is ScreenShot instance. `np.dtype(after)` returns `Cannot interpret '' as a data type`. It seems ScreenShot holds data in BGRA format, from what I understand ? So you were right I wrongly used ScreenShot type as a compatible image format for numpy — vincenthavinh, Sep 03 '21 at 23:12
would this call be enough to rightly convert `ScreenShot` to usable image by numpy ? `after = Image.frombytes("RGB", after.size, after.bgra, "raw", "BGRX")` from https://python-mss.readthedocs.io/examples.html#pil. I am a beginner with openCV and image processing and don't grasp fully the image format handling between different libs (openCV, mss, PIL, numpy, scikit) — vincenthavinh, Sep 03 '21 at 23:18
@fmw42 OK i tested by adding 2 calls to `.frombytes()` at the beginning of my function before creating numpy arrays and it fixed the problem. So it was a problem of ScreenShot class not having the expected channels for the process function. Thank you a lot ! if you ever write a separate answer summarizing your comments I' ll approve it — vincenthavinh, Sep 03 '21 at 23:30

score 1 · Accepted Answer · answered Sep 08 '21 at 21:59

As @fmw42 pointed at me in the question comments section, I was calling np.array() on mss ScreenShot objects and expecting them to it to work out of the box. It turns out ScreenShot class stores image data on a specific format, and when np.array() manipulates an instance object, it uses a different format for managing image channels, specifically the alpha channel.

So the solution was to first convert mss ScreenShot objects in Image objects that hold image data in compatible way for np.array() to handle them :

def find_diff(before, after):
    before = Image.frombytes("RGB", before.size, before.bgra, "raw", "BGRX")
    after = Image.frombytes("RGB", after.size, after.bgra, "raw", "BGRX")
    before = np.array(before)
    after = np.array(after)
    ...

wrong colors when saving image with openCV

1 Answers1