What is Mutual Information (MI)?

The concept of Mutual Information measures the amount of information shared between two random variables, which in the context of unsupervised learning, translates to the similarity of clustering between two approaches. It has the advantages of being symmetric and unaffected by different permutations of labels. Just as with the Rand Index, an adjusted version of the score subtracts the expected mutual information to account for cluster assignments being consistent due to chance. There is also a normalized version that scales the values to a range between 0 and 1, where a value of 0 indicates no mutual information and a value of 1 implies full concordance.