What is a Gaussian Mixture Model (GMM)?

A Gaussian Mixture Model describes an underlying distribution that is composed of multiple individual Gaussian distributions, each having separate mean and variance parameters. If there are k components to the GMM, the probability of an observation belonging to any component is given by pk, where the pk’s sum to 1. GMM’s have an application in clustering, where the number of components of the mixture corresponds to the number of underlying clusters in the data. Each cluster represents a different data generation process, which is a latent variable in the unsupervised context, since the actual clusters are most likely unknown. Since the mean and variance of each component need to be estimated, the EM approach is a way to iterate between assigning observations to their components and then optimizing the parameters of the components, or clusters, found on each pass through the data, until the assignments and parameters are stable.