Machine Learning Resources

What are some desirable properties of estimators?

  • Unbiased: If an estimator is unbiased, its average value is equal to the population parameter. If is an estimator of the parameter , it is unbiased if . An unbiased estimator is thus one that hits the target on average. 
  • Low Variance: As a primary motivation of statistics is to quantify uncertainty in stochastic processes, there is always some variability inherent when estimating something that cannot be known exactly. However, if the variability of an estimator is unduly large, it is difficult to arrive at a precise conclusion about the point estimate obtained from a given random sample. If a separate random sample is taken and a widely different estimate is achieved, the point estimate produced cannot be interpreted with high confidence. 
  • Consistency: A consistent estimator is one that converges to the true value of the parameter with a large enough sample size. This is basically saying that if you take a sample that is almost as large as the entire population, the estimator should be a highly accurate representation of the true parameter. 
  • Invariant to Transformation: For an estimator that has this property, a function of the estimator can be found by simply transforming the estimate produced. For example, if the estimator finds , then can be found by

Partner Ad