Why this part matters
Statistics begins when the distribution is unknown. We specify a family of models, decide what feature of the distribution matters, and choose a way to measure the cost of an estimation error. A statistic summarizes the data; sufficiency identifies summaries that retain the information in the model.
Bias and variance explain why a deliberately biased estimator can have smaller risk. Moment matching and maximum likelihood then provide two general ways to construct estimators. The following parts study their large-sample behavior and ask when an estimator is optimal.
Learning objectives
Distinguish model, parameter, observation and statistic; check identifiability (Course outcome 1).
Compare procedures through loss and risk (Course outcome 3).
Verify sufficiency using factorization (Course outcome 4).
Analyze bias, variance and regularization (Course outcome 2).
Derive moment or likelihood estimators and check existence and identifiability (Course outcome 2).
Roadmap
| Lecture | Purpose |
| 6. Statistical decision theory and sufficiency | Define models, loss, risk and sufficient statistics. |
| 7. Bias–variance tradeoff and regularization | Use ridge and nearest neighbors to study bias, variance and model complexity. |
| 8. Moments and maximum likelihood | Construct estimators by moment matching and maximum likelihood. |
What you need from earlier lectures
Where this leads
Large-sample theory studies whether these estimator recipes approach their targets and how their errors fluctuate. Continue to Part III.
Readings
Use the lecture notes together with the course texts and original notes. Exact textbook sections and proof references are linked in the lectures.
Part checkpoint
Try each question before opening its answer. This self-check reviews the linked material and adds no exam requirements; the lecture priorities and exceptions still apply.
Can a biased estimator have smaller squared-error risk?
Yes. Risk is squared bias plus variance. Shrinkage can reduce variance by more than the squared bias it introduces; the comparison depends on the true parameter. Return to the explanation.
What does sufficiency preserve?
The conditional law of the full data given the statistic has a version independent of the parameter. Under the dominated-model assumptions, factorization provides a useful criterion. Return to the explanation.
What can fail in a moment-estimation recipe?
The population moments may fail to identify the parameter, empirical moments may fall outside the feasible model range, or the required moments may not exist. Return to the explanation.
Does solving the score equation always find an MLE?
No. It only identifies candidate interior stationary points when differentiability holds. Boundaries, multiple stationary points, nonattainment and global optimization still need attention. Return to the explanation.
You should now be able to carry out the learning objectives above and explain where each result needs its assumptions.