Information that is in a message; an event and a random variable is called "information entropy; Shannon entropy; simply entropy". Calculating information and entropy is a useful tool in machine learning and is used as the basis for techniques such as feature selection, building decision trees, and more generally, fitting classification models. The entropy H(X), expressed in bits, of a discrete random variable is defined by H(X) = -∑[p(x)log(p(x))] for all p(x)>0. Compute the entropy of a fair coin: Compute the entropy of a fair dice: