0

This is a follow-up to my previous question here

I've been trying to convert the color data in a heatmap to RGB values.

source image

In the below image, to the left is a subplot present in panel D of the source image. This has 6 x 6 cells (6 rows and 6 columns). On the right, we see the binarized image, with white color highlighted in the cell that is clicked after running the code below. The input for running the code is the below image. The ouput is(mean = [ 27.72 26.83 144.17])is the mean of BGR color in the cell that is highlighted in white on the right image below.

enter image description here

A really nice solution that was provided as an answer to my previous question is the following (ref)

import cv2
import numpy as np


# print pixel value on click
def mouse_callback(event, x, y, flags, params):
    if event == cv2.EVENT_LBUTTONDOWN:
        # get specified color
        row = y
        column = x
        color = image[row, column]
        print('color = ', color)

        # calculate range
        thr = 20  # ± color range
        up_thr = color + thr
        up_thr[up_thr < color] = 255
        down_thr = color - thr
        down_thr[down_thr > color] = 0

        # find points in range
        img_thr = cv2.inRange(image, down_thr, up_thr)  # accepted range
        height, width, _ = image.shape
        left_bound = x - (x % round(width/6))
        right_bound = left_bound + round(width/6)
        up_bound = y - (y % round(height/6))
        down_bound = up_bound + round(height/6)
        img_rect = np.zeros((height, width), np.uint8)  # bounded by rectangle
        cv2.rectangle(img_rect, (left_bound, up_bound), (right_bound, down_bound), (255,255,255), -1)
        img_thr = cv2.bitwise_and(img_thr, img_rect)

        # get points around specified point
        img_spec = np.zeros((height, width), np.uint8)  # specified mask
        last_img_spec = np.copy(img_spec)
        img_spec[row, column] = 255
        kernel = np.ones((3,3), np.uint8)  # dilation structuring element
        while cv2.bitwise_xor(img_spec, last_img_spec).any():
            last_img_spec = np.copy(img_spec)
            img_spec = cv2.dilate(img_spec, kernel)
            img_spec = cv2.bitwise_and(img_spec, img_thr)
            cv2.imshow('mask', img_spec)
            cv2.waitKey(10)
        avg = cv2.mean(image, img_spec)[:3]
        mean.append(np.around(np.array(avg), 2))
        print('mean = ', np.around(np.array(avg), 2))
        # print(mean) # appends data to variable mean


if __name__ == '__main__':

    mean = []  #np.zeros((6, 6))
    # create window and callback
    winname = 'img'
    cv2.namedWindow(winname)
    cv2.setMouseCallback(winname, mouse_callback)

    # read & display image
    image = cv2.imread('ip2.png', 1)
    #image = image[3:62, 2:118]  # crop the image to 6x6 cells

    #---- resize image--------------------------------------------------
    # appended this to the original code

    print('Original Dimensions : ', image.shape)

    scale_percent = 220  # percent of original size
    width = int(image.shape[1] * scale_percent / 100)
    height = int(image.shape[0] * scale_percent / 100)
    dim = (width, height)
    # resize image
    image = cv2.resize(image, dim, interpolation=cv2.INTER_AREA)

    # ----------------------------------------------------------------------
    cv2.imshow(winname, image)
    cv2.waitKey()  # press any key to exit
    cv2.destroyAllWindows()

What do I want to do next?

The mean of the RGB values thus obtained has to be mapped to the values in the following legend provided in the source image,

enter image description here

I would like to ask for suggestions on how to map the RGB data to the values in the legend.

Note: In my previous post it has been suggested that one could

fit the RGB values into an equation which gives continuous results.

Any suggestions in this direction will also be helpful.

EDIT: Answering the comment below

I did the following to measure the RGB values of legend Input image: enter image description here

This image has 8 cells in columns width and 1 cell in rows height

Changed these lines of code:

left_bound = x - (x % round(width/8)) # 6 replaced with 8
right_bound = left_bound + round(width/8) # 6 replaced with 8
up_bound = y - (y % round(height/1)) # 6 replaced with 1
down_bound = up_bound + round(height/1) # 6 replaced with 1

Mean obtained for each cell/ each color in legend from left to right:

mean =  [ 82.15 174.95  33.66]
mean =  [45.55 87.01 17.51]
mean =  [8.88 8.61 5.97]
mean =  [16.79 17.96 74.46]
mean =  [ 35.59  30.53 167.14]
mean =  [ 37.9   32.39 233.74]
mean =  [120.29 118.   240.34]
mean =  [238.33 239.56 248.04]
Natasha
  • 1,111
  • 5
  • 28
  • 66
  • Did you measure the RGB values of the colorbar (legend) also? If so, then this question simplifies enormously to a question of how to find the nearest RGB value in the legen set to each RGB value in the data set. Or maybe even how to interpolate. But you get rid of all the image data since that has already been solved. – Cris Luengo Mar 29 '20 at 12:53
  • @CrisLuengo Thanks a lot for your response. Please check the edit – Natasha Mar 29 '20 at 13:45

1 Answers1

1

You can try to apply piece wise approach, make pair wise transitions between colors:

c[i->i+1](t)=t*(R[i+1],G[i+1],B[i+1])+(1-t)*(R[i],G[i],B[i]) 

Do the same for these values:

val[i->i+1](t)=t*val[i+1]+(1-t)*val[i]

Where i - index of color in legend scale, t - parameter in [0:1] range.

So, you have continuous mapping of 2 values, and just need to find color parameters i and t closest to sample and find value from mapping.

Update:

To find the color parameters you can think about every pair of neighbour legend colors as a pair of 3d points, and your queried color as external 3d point. Now you just meed to find a length of perpendicular from the external point to a line, then, iterating over legend color pairs, find the shortest perpendicular (now you have i).

Then find intersection point of the perpendicular and the line. This point will be located at the distance A from line start and if line length is L then parameter value t=A/L.

Update2:

Simple brutforce solution to illustrate piece wise approach:

#include "opencv2/opencv.hpp"
#include <string>
#include <iostream>

using namespace std;
using namespace cv;

int main(int argc, char* argv[])
{
    Mat Image=cv::Mat::zeros(100,250,CV_32FC3);
    std::vector<cv::Scalar> Legend;
    Legend.push_back(cv::Scalar(82.15,174.95,33.66));
    Legend.push_back(cv::Scalar(45.55, 87.01, 17.51));
    Legend.push_back(cv::Scalar(8.88, 8.61, 5.97));
    Legend.push_back(cv::Scalar(16.79, 17.96, 74.46));
    Legend.push_back(cv::Scalar(35.59, 30.53, 167.14));
    Legend.push_back(cv::Scalar(37.9, 32.39, 233.74));
    Legend.push_back(cv::Scalar(120.29, 118., 240.34));
    Legend.push_back(cv::Scalar(238.33, 239.56, 248.04));

    std::vector<float> Values;
    Values.push_back(-4);
    Values.push_back(-2);
    Values.push_back(0);
    Values.push_back(2);
    Values.push_back(4);
    Values.push_back(8);
    Values.push_back(16);
    Values.push_back(32);

    int w = 30;
    int h = 10;

    for (int i = 0; i < Legend.size(); ++i)
    {
        cv::rectangle(Image, Rect(i * w, 0, w, h), Legend[i]/255, -1);
    }

    std::vector<cv::Scalar> Smooth_Legend;
    std::vector<float> Smooth_Values;
    for (int i = 0; i < Legend.size()-1; ++i)
    {
        cv::Scalar c1 = Legend[i];
        cv::Scalar c2 = Legend[i + 1];
        float v1 = Values[i];
        float v2 = Values[i+1];
        for (int j = 0; j < w; ++j)
        {
            float t = (float)j / (float)w;
            Scalar c = c2 * t + c1 * (1 - t);
            float v = v2 * t + v1 * (1 - t);
            float x = i * w + j;
            line(Image, Point(x, h), Point(x, h + h), c/255, 1);
            Smooth_Values.push_back(v);
            Smooth_Legend.push_back(c);
        }
    }

    Scalar qp = cv::Scalar(5, 0, 200);
    float d_min = FLT_MAX;
    int ind = -1;
    for (int i = 0; i < Smooth_Legend.size(); ++i)
    {
        float d = cv::norm(qp- Smooth_Legend[i]);
        if (d < d_min)
        {
            ind = i;
            d_min = d;
        }
    }
    std::cout << Smooth_Values[ind] << std::endl;

    line(Image, Point(ind, 3 * h), Point(ind, 4 * h), Scalar::all(255), 2);
    circle(Image, Point(ind, 4 * h), 3, qp/255,-1);
    putText(Image, std::to_string(Smooth_Values[ind]), Point(ind, 70), FONT_HERSHEY_DUPLEX, 1, Scalar(0, 0.5, 0.5), 0.002);


    cv::imshow("Legend", Image);
    cv::imwrite("result.png", Image*255);
    cv::waitKey();

}

The result:

enter image description here

Python:

import cv2
import numpy as np
height=100
width=250
Image = np.zeros((height, width,3), np.float)
legend =  np.array([ (82.15,174.95,33.66),
          (45.55,87.01,17.51),
          (8.88,8.61,5.97),
          (16.79,17.96,74.46),
          ( 35.59,0.53,167.14),
          ( 37.9,32.39,233.74),
          (120.29,118.,240.34),
          (238.33,239.56,248.04)], np.float)

values = np.array([-4,-2,0,2,4,8,16,32], np.float)

# width of cell, also defines number 
# of one segment transituin subdivisions.
# Larger values will give more accuracy, but will woek slower.
w = 30 
# Only fo displaying purpose. Height of bars in result image.
h = 10


# Plot legend cells ( to check correcrness only )
for i in range(len(legend)):
    col=legend[i]
    cv2.rectangle(Image, (i * w, 0, w, h), col/255, -1)

# Start form smoorhed scales for color and according values
Smooth_Legend=[]
Smooth_Values=[]
for i in range(len(legend)-1): # iterate known knots
    c1 = legend[i] # start color point
    c2 = legend[i + 1] # end color point
    v1 = values[i] # start value 
    v2 = values[i+1] # emd va;ie
    for j in range(w): # slide inside [start:end] interval.
        t = float(j) / float(w) # map it to [0:1] interval
        c = c2 * t + c1 * (1 - t) # transition between c1 and c2
        v = v2 * t + v1 * (1 - t) # transition between v1 and v2
        x = i * w + j # global scale coordinate (for drawing)
        cv2.line(Image, (x, h), (x, h + h), c/255, 1) # draw one tick of smoothed scale
        Smooth_Values.append(v) # append smoothed values for next step
        Smooth_Legend.append(c) # append smoothed color for next step

# queried color    
qp = np.array([5, 0, 200])
# initial value for minimal distance set to large value
d_min = 1e7
# index for clolor search
ind = -1
# search for minimal distance from queried color to smoothed scale color
for i in range(len(Smooth_Legend)):
    # distance
    d = cv2.norm(qp-Smooth_Legend[i])
    if (d < d_min):    
        ind = i
        d_min = d
# ind contains index of the closest color in smoothed scale
# and now we can extract according value from smoothed values scale
print(Smooth_Values[ind]) # value mapped to queried color.
# plot pointer (to check ourself)
cv2.line(Image, (ind, 3 * h), (ind, 4 * h), (255,255,255), 2);
cv2.circle(Image, (ind, 4 * h), 3, qp/255,-1);
cv2.putText(Image, str(Smooth_Values[ind]), (ind, 70), cv2.FONT_HERSHEY_DUPLEX, 1, (0, 0.5, 0.5), 1);
# show window
cv2.imshow("Legend", Image)
# save to file
cv2.imwrite("result.png", Image*255)
cv2.waitKey()
marc_s
  • 732,580
  • 175
  • 1,330
  • 1,459
Andrey Smorodov
  • 10,649
  • 2
  • 35
  • 42
  • Smorodov Could you please explain a bit more on how to find the color parameters? – Natasha Mar 30 '20 at 09:42
  • May be this my answer will also be useful https://stackoverflow.com/questions/23641208/how-to-draw-curve-on-control-points-using-opencv/23659732#23659732 – Andrey Smorodov Mar 30 '20 at 10:28
  • I'm sorry I didn't completely understand. Would it help if I start from a matrix of RGB values of the cells in the image and another array with RGB values of legend? I wish to do it step by step. All this is new to me, so it's a bit difficult for me to understand in one go – Natasha Mar 30 '20 at 17:51
  • 1
    Updated. Added a simple brutforce sample. Varying w you can tune approximation accuracy. If you want more accuracy and speed, move to analytical way instead of brutforce. – Andrey Smorodov Mar 31 '20 at 09:41
  • Thanks a lot. Unfortunately, I don't know c++ . I'll try and figure out if there is a way to translate this to python code – Natasha Mar 31 '20 at 09:46
  • 1
    It is easy convertible, check updete. – Andrey Smorodov Mar 31 '20 at 13:02
  • Thank you so much. Could you please add comments for the python code? I'm trying to understand from line 19 of the code i.e w and h values. Also, could you please explain the result a bit? I could follow that the legend has been smoothed. But I fail to understand how to link this to the output from the code that I have posted – Natasha Mar 31 '20 at 14:14