Continuing with Exercise 6.3 .14, besides estimation there...

Continuing with Exercise $6.3 .14$, besides estimation there is also a nice geometric interpretation for testing. For the model $(6.3 .26)$, consider the hypotheses $$ H_{0}: \theta=\theta_{0} \text { versus } H_{1}: \theta \neq \theta_{0} $$ where $\theta_{0}$ is specified. Given a norm $\|\cdot\|$ on $R^{n}$, denote by $d(\mathbf{X}, V)$ the distance between $\mathbf{X}$ and the subspace $V$; i.e., $d(\mathbf{X}, V)=\|\mathbf{X}-\widehat{\boldsymbol{\mu}}\|$, where $\hat{\boldsymbol{\mu}}$ is defined in equation $(6.3 .27) .$ If $H_{0}$ is true, then $\hat{\mu}$ should be close to $\mu=\theta_{0} 1$ and, hence, $\left\|\mathbf{X}-\theta_{0} \mathbf{1}\right\|$ should be close to $d(\mathbf{X}, V) .$ Denote the difference by $$ R D=\left\|\mathbf{X}-\theta_{0} \mathbf{1}\right\|-\|\mathbf{X}-\widehat{\boldsymbol{\mu}}\| $$ Small values of $R D$ indicate that the null hypothesis is true, while large values indicate $H_{1}$. So our rejection rule when using $R D$ is Reject $H_{0}$ in favor of $H_{1}$ if $R D>c$. (a) If the error pdf is the Laplace, $(6.1 .6)$, show that expression $(6.3 .31)$ is equivalent to the likelihood ratio test when the norm is given by $(6.3 .28)$. (b) If the error pdf is the $N(0,1)$, show that expression (6.3.31) is equivalent to the likelihood ratio test when the norm is given by the square of the $l_{2}$ norm, $(6.3 .29) .$

Continuing with Exercise $6.3 .14$, besides estimation there is also a nice geometric interpretation for testing. For the model $(6.3 .26)$, consider the hypotheses
$$
H_{0}: \theta=\theta_{0} \text { versus } H_{1}: \theta \neq \theta_{0}
$$
where $\theta_{0}$ is specified. Given a norm $\|\cdot\|$ on $R^{n}$, denote by $d(\mathbf{X}, V)$ the distance between $\mathbf{X}$ and the subspace $V$; i.e., $d(\mathbf{X}, V)=\|\mathbf{X}-\widehat{\boldsymbol{\mu}}\|$, where $\hat{\boldsymbol{\mu}}$ is defined in equation $(6.3 .27) .$ If $H_{0}$ is true, then $\hat{\mu}$ should be close to $\mu=\theta_{0} 1$ and, hence, $\left\|\mathbf{X}-\theta_{0} \mathbf{1}\right\|$ should be close to $d(\mathbf{X}, V) .$ Denote the difference by
$$
R D=\left\|\mathbf{X}-\theta_{0} \mathbf{1}\right\|-\|\mathbf{X}-\widehat{\boldsymbol{\mu}}\|
$$
Small values of $R D$ indicate that the null hypothesis is true, while large values indicate $H_{1}$. So our rejection rule when using $R D$ is
Reject $H_{0}$ in favor of $H_{1}$ if $R D>c$.
(a) If the error pdf is the Laplace, $(6.1 .6)$, show that expression $(6.3 .31)$ is equivalent to the likelihood ratio test when the norm is given by $(6.3 .28)$.
(b) If the error pdf is the $N(0,1)$, show that expression (6.3.31) is equivalent to the likelihood ratio test when the norm is given by the square of the $l_{2}$ norm, $(6.3 .29) .$

Key Concepts

Error Distributions and Norms

The distribution of errors in a model affects both estimation and testing procedures. For example, a Laplace distribution’s heavy tails make the L1 norm a natural choice for measuring deviations, aligning with the properties of the Laplace maximum likelihood estimator. Similarly, when errors follow a normal distribution, the L2 norm is preferred because it corresponds to the sum of squared deviations, which is optimal for normal errors. This connection between error distribution and the corresponding norm solidifies the link between statistical inference and the underlying geometric framework.

Projection onto Subspaces

Projection onto subspaces is a geometric concept where a data vector is orthogonally projected onto a subspace that represents a constrained model under the null hypothesis. This projection minimizes the distance between the data and all points in the subspace, effectively yielding the best approximation of the data under the model constraints. The difference between the observed vector and its projection is then used as a measure of deviation from the null hypothesis, thus framing hypothesis testing in terms of distances in vector spaces.

Norms in R^n

Norms are functions that assign lengths or sizes to vectors in R^n and are crucial in measuring distances and errors in statistical modeling. Different norms can capture different characteristics – for instance, the L1 norm sums absolute deviations whereas the L2 norm sums squared deviations. The choice of norm impacts the analysis and interpretation of data; as shown in the problem, the Laplace error distribution is naturally associated with the L1 norm, while the normal error distribution corresponds well with the L2 norm, linking the error model to the geometry of the test statistic.

Likelihood Ratio Test

The likelihood ratio test (LRT) is a fundamental method in statistical hypothesis testing that compares the maximum likelihood under the null hypothesis to the maximum likelihood under the alternative hypothesis. By forming the ratio of these two likelihoods or their logarithms, one can derive test statistics that often have known distributions or asymptotic behaviors, aiding in decision making. In the provided context, the LRT is shown to be equivalent to a test based on a geometric distance measure, linking probability theory with geometric intuition.

Geometric Interpretation of Hypothesis Testing

The geometric interpretation of hypothesis testing involves viewing the statistical model as a geometrical object, where the parameter estimates are points or vectors in a space. Under this perspective, testing a hypothesis can be related to measuring the distance of the observed data vector from a specific subspace defined by the null hypothesis. The closer the data are to this subspace, the more plausible the null hypothesis is, and vice versa. This interpretation provides an intuitive visualization of how model fitting and hypothesis testing work jointly.

Please give Ace some feedback