Decision Tree - Can the Entropy of a Node be Zero?

Question

The formula for information gain is give by,

Information Gain = entropy(parent) – [average entropy(children)]

Can the entropy be zero, which means in some case:

entropy(parent) == [average entropy(children)]

Answer 1

"When H(S) = 0, the set S is perfectly classified (i.e. all elements in S are of the same class)." -- https://en.wikipedia.org/wiki/ID3_algorithm

H(S) = entropy ;)

Answer 2

Guiem gave the correct answer, which is that the entropy is zero when all elements of a set belong to the same class. But with regard to your question, there are two additional points worth noting:

First, when implementing a decision tree, if entropy(parent) is zero, there is no reason to compute the Information Gain of children, since the data are already perfectly classified (i.e., you are at a leaf node of the tree).

Second, the case of entropy(parent) == [average entropy(children)] doesn't necessarily only occur when the entropy of parent is zero. It can also occur when parent has nonzero entropy (i.e., the Information Gain from splitting on children is zero), which suggests that splitting on children will not improve classification performance.