PROBLEM 2
1. Estimation of diagonal covariances: Let $(X_i)_{i=1,...,n}$ be an i.i.d. sequence of $d$-dimensional
vectors, drawn from a zero-mean distribution with diagonal covariance matrix $\Sigma = D$.
Consider the estimate $\hat{D} = \text{diag}(\hat{\Sigma})$, where $\hat{\Sigma}$ is the usual sample covariance matrix. Suppose
further that each component $X_{ij}$ is sub-Gaussian with parameter at most $\sigma = 1$. Show the
following:
(a) $X_{ij}^2$ is sub-exponential with parameters $(2, 4)$.
(b) $\sum_{i=1}^n X_{ij}^2$ is sub-exponential with parameters $(2\sqrt{n}, 4)$
(c) For each $i = 1, ..., d$, we get
$P(|\hat{D}_{ii} - D_{ii}| \ge t) \le 2e^{-\frac{n}{8}\min\{t, t^2\}}$
2. Suppose that the random vector $X \in \mathbb{R}^n$ has a $N_n(\mu, \Sigma)$ distribution, where $\Sigma$ is positive.
Show the the random variable $Y = (X - \mu)^T \Sigma (X - \mu)$ is sub-exponential.
Note: The question has a typo. It should be $Y = (X - \mu)^T \Sigma^{-1} (X - \mu)$
