Why this part matters
Part II showed why there is usually no universally best estimator. Here we recover precise forms of optimality by specifying the comparison class. Sufficiency and completeness produce minimum-variance unbiased estimators. Fisher information supplies a lower bound on unbiased variance under regularity.
These are related but distinct routes: the information bound need not be attained, and optimality within the unbiased class does not imply optimality among all estimators. Likelihood curvature also connects this fixed-sample theory with the MLE approximation in Part III. The optional shrinkage part will revisit what changes when bias is allowed.
Learning objectives
Use Rao–Blackwell and Lehmann–Scheffé to identify a UMVUE (Course outcome 3).
Compute Fisher information and an unbiased variance bound (Course outcome 2).
Interpret information as local likelihood curvature (Course outcome 1).
Check whether an information bound is attained (Course outcome 3).
Apply the projection and Gauss–Markov interpretations of least squares (Course outcome 4).
Roadmap
| Lecture | Purpose |
| 12. Unbiased estimation and UMVUE | Improve unbiased estimators through conditional averaging and completeness. |
| 13. Fisher information and the Cramér–Rao bound | Relate unbiased variance bounds to likelihood sensitivity and linear models. |
What you need from earlier lectures
Where this leads
Concentration studies finite-sample deviations. Optional shrinkage revisits the limitations of optimality restricted to unbiased estimators. Continue to Part V.
Readings
Use the lecture notes together with the course texts and original notes. Exact textbook sections and proof references are linked in the lectures.
Part checkpoint
Try each question before opening its answer. This self-check reviews the linked material and adds no exam requirements; the lecture priorities and exceptions still apply.
Does conditional averaging need unbiasedness to reduce MSE?
No. The conditional-variance decomposition gives the improvement relative to a constant target. Sufficiency matters because it allows the conditional averaging rule to be chosen without the unknown parameter. Return to the explanation.
What does completeness add to sufficiency?
Completeness forces a zero-mean function of the statistic to vanish almost surely under every model law. This gives uniqueness of an unbiased estimator based on the statistic. Sufficiency and Rao–Blackwell then give minimum variance among all unbiased estimators. Return to the explanation.
Must an unbiased estimator attain the Cramér–Rao bound?
No. Attainment requires the equality condition in the covariance inequality, in addition to the regularity assumptions. The bound may be strict for every unbiased estimator. Return to the explanation.
Why are Gaussian fitted values and residuals independent?
They are linear transformations of a Gaussian vector and are therefore jointly Gaussian. With \(\operatorname{Cov}(\epsilon)=\sigma^2I\), their cross-covariance is \(\sigma^2P(I-P)=0\), because the projection and residual operators are orthogonal. Orthogonality alone would not give independence for a general distribution. Return to the explanation.
You should now be able to carry out the learning objectives above and explain where each result needs its assumptions.