Machine Learning Resources

What does it mean if observations are iid, and why is this a desirable property?

The abbreviation iid stands for independent, identically distributed, and refers to instances of a random variable that are samples in such a fashion. This means that each observation is sampled from the same underlying probability distribution and is a separate instance that has no dependency on any other observation. This is a desirable quality for simplifying the variance structure (and providing validation of independence necessary for most linear models) and the computation of the likelihood function.

An example of a random sample that is not iid would be drawing from a deck of cards without replacement, as once a card is drawn, it cannot be placed back into the deck. Thus, subsequent draws would have a different probability of coming up based on what is remaining compared to if they were drawn with replacement.

Partner Ad