Part I

Foundations: the language of probability

How do we describe randomness precisely enough to prove things about it?

Report an issue

Why this part matters

Every guarantee in this course uses integrals, limits or conditional expectations. This part develops one language for discrete, continuous and mixed data, for infinite sequences of observations, and for densities relative to different measures. The definitions earn their place by letting us justify calculations that otherwise look merely plausible.

The payoff returns throughout the course: exchanging derivatives and integrals justifies score calculations, conditional expectation drives sufficiency and Rao–Blackwell, and elementary tail inequalities lead to laws of large numbers and concentration.

Learning objectives

  1. Formulate a probability model using measurable spaces, random variables and laws (Course outcome 1).

  2. Justify a limit, derivative or integration-order exchange using the appropriate theorem (Course outcome 4).

  3. Apply Markov, Jensen and Hölder after checking their hypotheses (Course outcome 4).

  4. Interpret a density relative to its reference measure (Course outcome 1).

  5. Compute conditional means and explain the squared-error prediction property (Course outcome 2).

Roadmap

LecturePurpose
1. Mathematical basics for statisticsReview the analysis, Gaussian, linear-algebra and optimization tools used throughout the course.
2. Measure spaces, measures, and random variablesDescribe measurable events, measures, random variables and their laws.
3. Integration, convergence theorems, and densitiesConstruct integration and justify limit and order exchanges.
4. Expectation, inequalities, and applicationsExpress expectation as an integral and derive useful inequalities.
5. Independence and statistical applicationsDescribe independence, conditional laws and squared-error prediction.

What you need from Lecture 1

Where this leads

Models and estimators use this probability language; return to these links when a later proof exchanges limits or averages. Continue to Part II.

Readings

Use the lecture notes together with the course texts and original notes. Exact textbook sections and proof references are linked in the lectures.

Part checkpoint

Try each question before opening its answer. This self-check reviews the linked material and adds no exam requirements; the lecture priorities and exceptions still apply.

Does a pointwise limit always pass through an integral?

No. Shrinking spikes can converge pointwise while retaining their integral. Use MCT for nonnegative increasing sequences or a common integrable bound for DCT; pointwise convergence alone is insufficient. Return to the explanation.

Why specify a reference measure when writing a density?

A density is a Radon–Nikodym derivative. A law with atoms and a continuous component can have a density relative to a mixed measure even though it has no density relative to Lebesgue measure alone. Return to the explanation.

Which average minimizes squared prediction error?

For square-integrable \(Y\)YY, among square-integrable functions of the observation \(Z\)ZZ, the conditional mean \(\mathbb E[Y\mid Z]\)𝔼[Y∣Z]\mathbb E[Y\mid Z] minimizes mean squared error. The residual is orthogonal to every square-integrable function of \(Z\)ZZ. Return to the explanation.

What justifies the tail-integral identity?

Write a nonnegative \(Y\)YY as \(\int_0^\infty 1_{\{Y>t\}}dt\)∫0∞1{Y>t}dt\int_0^\infty 1_{\{Y>t\}}dt and apply Tonelli. Both sides may be infinite. Return to the explanation.

You should now be able to carry out the learning objectives above and explain where each result needs its assumptions.