OpenCV: Output of the predict function of Expectation Maximization

Question

Background: I have 2 sets of color pixels from an image, one corresponding to the background, another corresponding to the foreground. Next, I train 2 Gaussian Mixture Models using EM from OpenCV for each set. My aim is to find the probability of a random pixel to belong to the foreground and to the background. Thus, I use the function "predict" for each EM on my pixel.

Question:

• I don't understand the values returned by this function. In the documentation of OpenCV, it is written:

The method returns a two-element double vector. Zero element is a likelihood logarithm value for the sample. First element is an index of the most probable mixture component for the given sample.

http://docs.opencv.org/modules/ml/doc/expectation_maximization.html?highlight=predict#Vec2d%20EM::predict%28InputArray%20sample,%20OutputArray%20probs%29%20const

I don't understand what means "likehood logarithm". In my results, I have sometimes negative values and values > 1. Is anyone who used the same function has this kind of results or resuts between 0 and 1 ? What can I conclude from my results ?

• How can I get the probability of a pixel to belong to the whole GMM (not the probality to belong to each cluster of the GMM) ?

Here is my code:

Mat mask = imread("mask.tif", 0);
Mat formerImage = imread("ImageFormer.tif");
Mat currentImage = imread("ImageCurrent.tif");

// number of cluster in the GMM 
int nClusters = 5;

int countB=0, countF=0;

Vec3b color;

Vec2d probFg, probBg; // probabilities to belong to the foreground or background from GMMs

//count the number of pixels for each training data
for(int c=0; c<=40;c++) {
    for(int l=0; l<=40;l++) {
        if(mask.at<BYTE>(l, c)==255) {
            countF++;
        } else if(mask.at<BYTE>(l, c)==0) {
            countB++;
        }
    }
}


printf("countB %d countF %d \n", countB, countF);

Mat samplesForeground = Mat(countF,3, CV_64F);

Mat samplesBackground = Mat(countB,3, CV_64F);


// Expectation-Maximisation able to resolve the GMM and to predict the probability for a pixel to belong to the GMM.
EM em_foreground= EM(nClusters);
EM em_background= EM(nClusters);

countB=0;
countF=0;

// fill the training data from the former image depending of the mask
for(int c=0; c<=40;c++) {
    for(int l=0; l<=40;l++) {
        if(mask.at<BYTE>(l, c)==255) {
            color = formerImage.at<Vec3b>(l, c);
            samplesForeground.at<double>(countF,0)=color[0];
            samplesForeground.at<double>(countF,1)=color[1];
            samplesForeground.at<double>(countF,2)=color[2];
            countF++;
        } else if(mask.at<BYTE>(l, c)==0) {
            color = formerImage.at<Vec3b>(l, c);
            samplesBackground.at<double>(countB, 0)=color[0];
            samplesBackground.at<double>(countB, 1)=color[1];
            samplesBackground.at<double>(countB, 2)=color[2];
            countB++;
        }
    }
}

printf("countB %d countF %d \n", countB, countF);
em_foreground.train(samplesForeground);
em_background.train(samplesBackground);

Mat sample(1, 3, CV_64F);

// try every pixel of the current image and get the log likelihood
for(int c=0; c<=40;c++) {
    for(int l=0; l<=40;l++) {
        color = currentImage.at<Vec3b>(l,c);
        sample.at<double>(0)=color[0];
        sample.at<double>(1)=color[1];
        sample.at<double>(2)=color[2];
        probFg=em_foreground.predict(sample);
        probBg=em_background.predict(sample);
        if(probFg[0]>0 || probBg[0]>0)
            printf("probFg[0] %f probBg[0] %f \n", probFg[0], probBg[0]);
    }
}

EDIT

After @BrianL explained, I now understand the log likelihood.

My problem is the log probability of the predict function is sometimes >0. But it should be <=0. Has anyone met this problem before?

I have edited the code above to show the problem. I have tried the program with images below:

The first image is the ImageCurrent.tif, the second is the ImageFormer.tif and the last one is mask.tif.

Is this can be considered a bug in OpenCV? Should I open a ticket on OpenCV bug tracker?

Answer 1

The "likelihood logarithm" means the log of the probability. Since for a probability p we expect 0 ≤ p ≤ 1, I would expect the values to be negative: log(p) ≤ 0. Larger negative numbers imply smaller probabilities.

This form is helpful when you are dealing with products of very small probabilities: if you multiplied the normal way, you could easily get underflow and lose precision because the probability becomes very small. But in log space the multiplication turns into an addition, which improves the accuracy and also potentially the speed of the calculation.

The predict function is for classifying a data point. If you want to give a point a score for how likely it is to belong to any component in the model, you can use the model parameters (see the get documentation) to calculate it yourself.

Answer 2

I notice that you are doing graphcut based image segmentation.

You might want to take a look at the following blog post which use OpenCV and its GMM class in a very similar way as what you are doing to perform graph cut-based image segmentation. Code is given in C++ with detailed explanations. Here is the link: link

Basically, I can only say that the log probability, whether it is correct or not, is not what you are looking for. Check out the above link for details.

Answer 3

As I understand you have two separate GMMs for the foreground and background part of the image.The total probability of a sample pixel 'x' in the test image when evaluated in the foreground GMM is

P_fg(x) = sum_over_j_1_to_k ( Wj_fg * Pj_fg( x ))
where 
k = number of clusters in foreground GMM
x = test sample
Pj_fg(x) = probability that sample x is in j-th  cluster according to the foreground GMM
Wj_fg = weight of the j-th cluster in foreground GMM
also, sum of all weights should be 1 for each GMM.

We can do a similar calculation for the background GMM.

From looking at the EM code in opencv, it looks like the first part of the 2 values that EM returns is the log likelihood. For the foreground GMM this is

log(P_fg(x_i))

I implemented your algorithm and for each pixel in the test image, I compared the log-likelihoods returned for each of the two GMM-s and classified the pixel with the GMM with higher value. I got decent results.

In that respect, yes this value is an indication of the pixel to be belonging to the entire GMM.

2) In my implementation of your problem, I always got the log likelihoods of all GMMS of all test-sample pixels under 0.