Mathematical Statistics

Part I · Lectures 1–5Foundations: the language of probability

Part overview and checkpoint

How do we describe randomness precisely enough to prove things about it?

Build the language of probability, integration, and conditional expectation used throughout statistical theory.

• Lecture 1 — Mathematical basics for statistics HTMLPDF
• Lecture 2 — Measure spaces, measures, and random variables HTMLPDF
• Lecture 3 — Integration, convergence theorems, and densities HTMLPDF
• Lecture 4 — Expectation, inequalities, and applications HTMLPDF
• Lecture 5 — Independence and statistical applications HTMLPDF

Part II · Lectures 6–8Statistical models and estimators

Part overview and checkpoint

What makes an estimator good, and how do we build one?

Use models, sufficiency, loss and risk to compare estimators, then construct them through moments and likelihood.

• Lecture 6 — Statistical decision theory and sufficiency HTMLPDF
• Lecture 7 — Bias–variance tradeoff and regularization HTMLPDF
• Lecture 8 — Moments and maximum likelihood HTMLPDF

Part III · Lectures 9–11Large-sample theory

Part overview and checkpoint

What happens to estimators as the sample size grows?

Study consistency and Gaussian approximations, and learn when limit theorems justify an approximation.

• Lecture 9 — Convergence, laws of large numbers, and consistency HTMLPDF
• Lecture 10 — Weak convergence and the central limit theorem HTMLPDF
• Lecture 11 — Asymptotic normality and the delta method HTMLPDF

Part IV · Lectures 12–13Optimality

Part overview and checkpoint

How good can an estimator possibly be?

Find the best unbiased estimators and use Fisher information to bound the precision of estimation.

• Lecture 12 — Unbiased estimation and UMVUE HTMLPDF
• Lecture 13 — Fisher information and the Cramér–Rao bound HTMLPDF

Part V · Lectures 14–17High dimensions: tools and phenomena

Part overview and checkpoint

What changes when the number of parameters grows with the sample size?

Use concentration and random matrix tools to understand high-dimensional estimation, PCA, and regression.

• Lecture 14 — Concentration inequalities HTMLPDF
• Lecture 15 — Random vectors in high dimensions HTMLPDF
• Lecture 16 — Norms of random matrices HTMLPDF
• Lecture 17 — High-dimensional statistical phenomena HTMLPDF

Part VI · Lectures 18–20Shrinkage and sparsity* (optional)

Part overview

How do we estimate well when the dimension is large?

Explore how shrinkage, thresholding, and sparse recovery improve estimation in high dimensions.

• Lecture 18 — Stein’s phenomenon and shrinkage estimation* HTMLPDF
• Lecture 19 — Denoising via shrinkage* HTMLPDF
• Lecture 20 — Basis pursuit, compressed sensing, and LASSO* HTMLPDF

Part VII · Coming laterTesting and confidence sets

Part overview

How do we turn estimates into decisions and uncertainty statements?

Hypothesis testing and confidence sets. Lecture notes will be added here; no lectures are assigned to this part yet.