Linear models: Inference

We now understand the least squares estimator \(\boldsymbol{\widehat{\beta}}\) from geometric and algebraic points of view. In Unit 2, we switch to a probabilistic perspective to derive inferential statements for linear models, in the form of hypothesis tests and confidence intervals. In order to facilitate this, we will assume that the error terms are normally distributed:

\[ \boldsymbol{y} = \boldsymbol{X} \boldsymbol{\beta} + \boldsymbol{\epsilon}, \quad \text{where} \ \boldsymbol{\epsilon} \sim N(\boldsymbol{0}, \sigma^2 \boldsymbol{I}_n). \] We first establish some building blocks necessary for linear models inference, primarily related to manipulating the normal distribution (7  Building blocks). Then, we discuss univariate and multivariate hypothesis testing in linear models (8  Hypothesis testing), as well as the power of these hypothesis tests (9  Power). We then move on to the construction of confidence intervals and confidence regions (10  Confidence intervals). We conclude with a discussion of practical considerations (11  Practical considerations) and an R demo (12  R demo).