Uncorrelated residuals from linear models

In the usual linear model $\text{y}\text{=}\text{Xβ}\text{+}\text{u}$, the error vector u is not observable and the vector r of least squares residuals has a singular covariance matrix that depends on the design matrix X. We approximate u by a vector$\text{r}{}^1\text{=}\text{G}\text{(}\text{JA'y}\text{+}\text{Kz}\text{)}$ of uncorrelated ‘residuals’, where G and $\text{(}\text{J, K}\text{)}$ are orthogonal matrices, $\text{A'X}\text{=}\text{0}$ and $\text{A'A}\text{=}\text{I}$, while z is either 0 or a random vector uncorrelated with u satisfying $\text{E(}\text{z}\text{) = E(}\text{J'u}\text{) =}\text{0}$, $\text{V(}\text{z}\text{) = V(}\text{J'u}\text{) = σ}{}^2\text{I}$. We prove that $\text{r}{}^1\text{-}\text{r}$ is uncorrelated with $\text{r}\text{-}\text{u}$, for any such r¹, extending the results of Neudecker (1969). Building on results of Hildreth (1971) and Tiao and Guttman (1967), we show that the BAUS residual vector $\text{r}{}_{\mathrm{h}}\text{=}\text{r}\text{+}\text{P}{}_{\mathrm{<mtext>1</mtext>}}\text{z}$, where P₁ is an orthonormal basis for X, minimizes each characteristic root of $\text{V(}\text{r}{}^1\text{-}\text{u}\text{)}$, while the vector r_b of Theil's BLUS residuals minimizes each characteristic root of $\text{V(}\text{Jr}{}_{\mathrm{a}}\text{-}\text{r}\text{)}$, cf. Grossman and Styan (1972). We find that $\text{tr V(}\text{r}{}_{\mathrm{h}}\text{-}\text{u}\text{) < tr V(}\text{Jr}{}_{\mathrm{b}}\text{-}\text{u}\text{)}$ if and only if the average of the singular values of $\text{P}\text{′}{}_1\text{K}$ is less than $\text{1}\text{2}$, and give examples to show that BAUS is often better than BLUS in this sense.