Minimum Variance Unbiased Estimator (MVUE)

Powerful ideas in Statistical Inference.

By Vamshi Jandhyala in mathematics

November 12, 2020

Problem

If $T_1$ and $T_2$ are two unbiased estimates of θ such that $Var(T_1) = σ_1^2$, $Var(T_2) = σ_2^2$ and correlation of $T_1$ and $T_2$ is $ρ$, find the best linear combination of $T_1$ and $T_2$ that serves as an unbiased estimate for θ. What is the variance of this estimate?

Solution

Let $aT_1 + bT_2$ be the best linear combination which is a unbiased estimator of $\theta$. We have $\mathbb{E}[aT_1 + bT_2] = a\theta + b \theta = \theta$.

Therefore $a + b = 1$.

The variance of the combination is

$$ Var(aT_1 + bT_2) = a^2 \sigma_1^2 + b^2 \sigma_2^2 + 2ab \rho \sigma_1\sigma_2 $$

Using the method of Lagrange Multipliers to minimize the variance subject to the constraint $a+b=1$, we define the Lagrangian function $\mathcal{L} = Var(aT_1 + bT_2) - \lambda(a+b-1)$.

We get the following equations

$$ \begin{align} \frac{\partial \mathcal{L}}{\partial \lambda} &= a + b -1 = 0\\
\frac{\partial \mathcal{L}}{\partial a} &= 2a\sigma_1^2 + 2b\rho \sigma_1 \sigma_2 - \lambda = 0\\
\frac{\partial \mathcal{L}}{\partial b} &= 2b\sigma_2^2 + 2a\rho \sigma_1 \sigma_2 - \lambda = 0 \end{align} $$

Solving the above equations for $a$ and $b$, we have

$$ \begin{align*} a &= \frac{\sigma_2^2 - \rho \sigma_1 \sigma_2}{\sigma_1^2 + \sigma_2^2 - 2\rho \sigma_1 \sigma_2} \\
b &= \frac{\sigma_1^2 - \rho \sigma_1 \sigma_2}{\sigma_1^2 + \sigma_2^2 - 2\rho \sigma_1 \sigma_2} \end{align*} $$

Therefore the variance of the best linear combination of $T_1$ and $T_2$ which is unbiased is

$$ \frac{\sigma_1^2 \sigma_2^2 (1-\rho^2)}{\sigma_1^2 + \sigma_2^2 - 2\rho \sigma_1 \sigma_2}. $$