4

I am trying to extract the entropy from co-occurence matrices with zero entries in Matlab. From the definition of entropy of a co-occurence matrix:

definition of entropy
has to be calculated, where cij stands for the (i,j) entry of the co-occurence matrix. Thus it seems to me that if there is a single zero entry, the entropy will be undefined. Do you set some sort of lower limit to log(x) when x = 0, or how do you deal with it?

Link to a pdf with the definition of entropy for the GLCM: http://www.code.ucsd.edu/pcosman/glcm.pdf

EDIT: Thanks for the suggestions on how to deal with log(0), but the equation actually calls for evaluating 0*log(0) which is 0 anyway. It would have been easier to explain if I could use formulas, but maybe my question was more mathematical anyway, and thus on the wrong forum.

Tonechas
  • 13,398
  • 16
  • 46
  • 80
Postermaestro
  • 85
  • 7
  • 1
    99.99999% of questions do not need math, or are of topic if they are too math related, so no latex is supported. You can always attach them as images. – Ander Biguri Jan 30 '18 at 11:25
  • how does [matlab's own entropy function](http://uk.mathworks.com/help/images/ref/entropy.html) deal with it? – ldgorman Jan 30 '18 at 11:50
  • @Idgorman thanks for the suggestions, but Matlab's entropy function expects a grayscale image, and converts the values of the image into 256 bin normalized histogram before calculating the entropy. I was trying to calculate the entropy straight from the co-occurence matrix. The values don't represent pixel intensities. – Postermaestro Jan 31 '18 at 10:02

2 Answers2

4

I always do this if I don't want a -Inf when I log something.

set an epsilon which is very, very little and deal your matrix C like

e = 1e-99;
C = C + e;

then you could run your old code and the answer will not be -Inf.

Thank for @CrisLuengo's useful advice in comment

Hunter Jiang
  • 1,300
  • 3
  • 14
  • 23
  • Or you can simply do `C=C+e`, since `e` is so small it will not affect "normal" numbers. A good way to pick `e` is using `e=eps(mean(C(:)))`, assuming the mean is representative for the majority of values in `C`. – Cris Luengo Jan 30 '18 at 14:16
  • Yeah! Adding something to both two matrices will not change the sequence of them (Since log is a monotonic function). And in my point of view, 1e-99 could work in most cases and needn't any assuming. It is also the shorter one so I prefer it. @CrisLuengo – Hunter Jiang Jan 30 '18 at 14:36
2

I generally use the following workaround to avoid this issue:

X = C .* log2(C + (C == 0));
entropy = -sum(X(:));

For those entries of C (the co-occurrence matrix) that are 0, the argument of the logarithm function is 1 since the expression (C == 0) is evaluated as 1.

Tonechas
  • 13,398
  • 16
  • 46
  • 80
  • 1
    That was a beautiful solution! (Although I would need a dot before 'log2' to make it an element-wise operation). I ended up doing some ugly nested for loops and an if-statement. And thanks for the edit. – Postermaestro Jan 31 '18 at 20:14
  • 1
    Thanks for spotting the need to use element-wise product rather than matrix product – Tonechas Jan 31 '18 at 20:45