OpenCV using k-means to posterize an image

Question

I want to posterize an image with k-means and OpenCV in C++ interface (cv namespace) and I get weird results. I need it for reduce some noise. This is my code:

#include "cv.h"
#include "highgui.h"

using namespace cv;

int main() {
    Mat imageBGR, imageHSV, planeH, planeS, planeV;

    imageBGR = imread("fruits.jpg");
    imshow("original", imageBGR);
    
    cv::Mat labels, data;
    cv::Mat centers(8, 1, CV_32FC1);
    imageBGR.convertTo(data, CV_32F);

    cv::kmeans(data, 8, labels,
            cv::TermCriteria(CV_TERMCRIT_ITER, 10, 1.0),
            3, cv::KMEANS_PP_CENTERS, &centers);
    imshow("posterized hue", data);
    data.convertTo(data, CV_32FC3);

    waitKey();
    return 0;
}

But I get a weird result

First image: original

Second image: after k-means.

Any advice?

---

Update: the right solution. maybe someone can help me in optimize the code?

#include "cv.h"
#include "highgui.h"

#include <iostream>

using namespace cv;
using namespace std;

int main() {
    Mat src;

    src = imread("fruits.jpg");
    imshow("original", src);

    blur(src, src, Size(15,15));
    imshow("blurred", src);

    Mat p = Mat::zeros(src.cols*src.rows, 5, CV_32F);
    Mat bestLabels, centers, clustered;
    vector<Mat> bgr;
    cv::split(src, bgr);
    // i think there is a better way to split pixel bgr color
    for(int i=0; i<src.cols*src.rows; i++) {
        p.at<float>(i,0) = (i/src.cols) / src.rows;
        p.at<float>(i,1) = (i%src.cols) / src.cols;
        p.at<float>(i,2) = bgr[0].data[i] / 255.0;
        p.at<float>(i,3) = bgr[1].data[i] / 255.0;
        p.at<float>(i,4) = bgr[2].data[i] / 255.0;
    }

    int K = 8;
    cv::kmeans(p, K, bestLabels,
            TermCriteria( CV_TERMCRIT_EPS+CV_TERMCRIT_ITER, 10, 1.0),
            3, KMEANS_PP_CENTERS, centers);

    int colors[K];
    for(int i=0; i<K; i++) {
        colors[i] = 255/(i+1);
    }
    // i think there is a better way to do this mayebe some Mat::reshape?
    clustered = Mat(src.rows, src.cols, CV_32F);
    for(int i=0; i<src.cols*src.rows; i++) {
        clustered.at<float>(i/src.cols, i%src.cols) = (float)(colors[bestLabels.at<int>(0,i)]);
//      cout << bestLabels.at<int>(0,i) << " " << 
//              colors[bestLabels.at<int>(0,i)] << " " << 
//              clustered.at<float>(i/src.cols, i%src.cols) << " " <<
//              endl;
    }

    clustered.convertTo(clustered, CV_8U);
    imshow("clustered", clustered);

    waitKey();
    return 0;
}

Result:

Answer 1

If you don't need to x,y coordinates in your k-means, you can arrange the data much quicker as follows using the reshape command:

  int origRows = img.rows;
  notes << "original image is: " << img.rows << "x" << img.cols << endl;
  Mat colVec = img.reshape(1, img.rows*img.cols); // change to a Nx3 column vector
  cout << "colVec is of size: " << colVec.rows << "x" << colVec.cols << endl;
  Mat colVecD, bestLabels, centers, clustered;
  int attempts = 5;
  int clusts = 8;
  double eps = 0.001;
  colVec.convertTo(colVecD, CV_32FC3, 1.0/255.0); // convert to floating point
  double compactness = kmeans(colVecD, clusts, bestLabels, 
        TermCriteria(CV_TERMCRIT_EPS+CV_TERMCRIT_ITER, attempts, eps), 
        attempts, KMEANS_PP_CENTERS, centers);
  Mat labelsImg = bestLabels.reshape(1, origRows); // single channel image of labels
  cout << "Compactness = " << compactness << endl;

Answer 2

I am no expert in OpenCV so I will give a general advice that relates to your question K-means takes list of vectors which is essentially a matrix:

[x0, y0, r0, g0, b0]
[x1, y1, r1, g1, b1]
[x2, y2, r2, g2, b2]
.
.
.

You are giving it an image which is not going to work. You first have to convert the image to this k-means matrix format. For each pixel of the source image you have one row in the resulting matrix. Also note that you should scale the values so that they all have similar values. If you don't do that, x and y coordinates will usually have much higher "gravity" than the color which leads to unsatisfactory results. C++ pseudocode:

int pixel_index = 0;
for (int y = 0; y < image height; y++)  {
  for (int x = 0; x < image width; x++)  {
     matrix[pixel_index][0] = (float)x / image width;
     matrix[pixel_index][1] = (float)y / image height;
     matrix[pixel_index][2] = (float)pixel(x, y).r / 255.0f;
     matrix[pixel_index][3] = (float)pixel(x, y).g / 255.0f;
     matrix[pixel_index][4] = (float)pixel(x, y).b / 255.0f;
  }
}
// Pass the matrix to kmeans...

As a result, you get labels of each individual pixel which corresponds to the cluster it has been assigned to. You then need to determine the color of the clusters - this can vary from taking the center pixel color value to computing an average/median color of the cluster. After you determine the color, just walk the image and set pixels to their cluster colors:

for (int y = 0; y < image height; y++)  {
  for (int x = 0; x < image width; x++)  {
     int index = y * image width + x;  // This corresponds to pixel_index above
     int cluster_index = labels[index]; // 0 to 7 in your case
     Color color = colors[cluster_index];  // Colors is an array of 8 colors of the clusters
     image.setpixel(x, y, color)
  }
}

If you prefer to use HSV instead of RGB, just use HSV values instead of RGB ones.

It is possible that OpenCV has functions that perform exactly the conversion I described above but I was unable to quick find them using Google.

Answer 3

I believe I made it a little bit more understandable, I got your example and replaced the part where you only use shades of gray to replace the color and used the color of the centers to make it use the "dominant" colors of the image.

cv::Mat MyClass::quantizeImage(cv::Mat _inputImage, int _quantizationColors){

Mat src = _inputImage.clone();  //cloning mat data
Mat data = Mat::zeros(src.cols * src.rows, 3, CV_32F);  //Creating the matrix that holds all pixel data
Mat bestLabels, centers, clustered; //Returns from the K Means
vector<Mat> bgr;    //Holds the BGR channels
cv::split(src, bgr);

//Getting all pixels in the Data row column to be compatible with K Means
for (int i = 0; i < src.cols * src.rows; i++) {
    data.at<float>(i, 0) = bgr[0].data[i] / 255.0;
    data.at<float>(i, 1) = bgr[1].data[i] / 255.0;
    data.at<float>(i, 2) = bgr[2].data[i] / 255.0;
}

int K = _quantizationColors;    //Number of clusters
cv::kmeans(data, K, bestLabels,
    TermCriteria(TermCriteria::EPS + TermCriteria::MAX_ITER, 10, 1.0),
    3, KMEANS_PP_CENTERS, centers);

centers = centers.reshape(3, centers.rows);
data = data.reshape(3, data.rows);

clustered = Mat(src.rows, src.cols, CV_32F);


Vec3f* p = data.ptr<Vec3f>();
for (size_t i = 0; i < data.rows; i++) {
    int center_id = bestLabels.at<int>(i);
    p[i] = centers.at<Vec3f>(center_id);
}

clustered = data.reshape(3, src.rows);
return clustered;}