第3講統計的推定

クラメール・ラオの下限 (情報量不等式)

$\hat{\theta}$ を $\theta$ の不偏推定量とする．このとき，次が成り立つ．

V [\hat{θ}] \geq \frac{1}{I_{n} (θ)}

$\begin{equation} V[\hat{\theta}] \ge \frac{1}{I_n(\theta)} \end{equation}$

ただし，フィッシャー情報量が正であること，微分と積分の交換ができることが条件である．

不偏推定量 $\hat{\theta}^*$ が

V [{\hat{θ}}^{*}] = \frac{1}{I_{n} (θ)}

$\begin{equation} V[\hat{\theta}^*] = \frac{1}{I_n(\theta)} \end{equation}$

を満たせば，

{\hat{θ}}^{*}

$\hat{\theta}^*$ は一様最小分散不偏推定量 (UMVU) である．ただし，上式を満たさなくても，一様最小分散不偏推定量となる場合もあることに注意すること．

補足1: クラメール - ラオの不等式の証明

l (θ, X) = \log f (x, θ)

$\begin{equation} l(\theta,~\bm{X}) = \log f(\bm{x},~\theta) \end{equation}$

とおく．

\begin{aligned} E [l^{'} (θ, X)] & = E [\frac{d}{d θ} \log f (x, θ)] \\ = \int {\frac{d}{d θ} \log f (x, θ)} f (x, θ) d x \\ = \int {\frac{1}{f (x, θ)} \frac{d}{d θ} f (x, θ)} f (x, θ) d x \\ = \int \frac{d}{d θ} f (x, θ) d x \\ = \frac{d}{d θ} \int f (x, θ) d x \\ = 0 \end{aligned}

$\begin{align} E[l'(\theta,~\bm{X})] &= E\left[\frac{d}{d\theta}\log f(\bm{x},~\theta)\right] \\ &= \int \left\{\frac{d}{d\theta}\log f(\bm{x},~\theta)\right\}f(\bm{x},~\theta)\,dx \\ &= \int \left\{\frac{1}{f(\bm{x},~\theta)}\frac{d}{d\theta} f(\bm{x},~\theta)\right\}f(\bm{x},~\theta)\,dx \\ &= \int \frac{d}{d\theta} f(\bm{x},~\theta)\,dx \\ &= \frac{d}{d\theta} \int f(\bm{x},~\theta)\,dx \\ &= 0 \\ \end{align}$

θ = E [\hat{θ} (X)] = \int \hat{θ} (x) f (x, θ) d x

$\begin{equation} \theta = E[\hat{\theta}(\bm{X})] = \int \hat{\theta}(x)\,f(\bm{x},~\theta)\,d\bm{x} \end{equation}$

両辺を

θ

$\theta$ で微分すると，

\begin{aligned} 1 & = \frac{\partial}{\partial θ} \int \hat{θ} (x) f (x, θ) d x \\ = \int \hat{θ} (x) \frac{\partial}{\partial θ} f (x, θ) d x \\ = \int \hat{θ} (x) \frac{\partial \log f (x, θ)}{\partial θ} f (x, θ) d x \\ = E [\hat{θ} (X) l^{'} (θ, X)] \\ = E [\hat{θ} (X) l^{'} (θ, X)] - θ E [l^{'} (θ, X)] ∵ E [l^{'} (θ, X)] = 0 \\ = E [(\hat{θ} (X) - θ) l^{'} (θ, X)] \\ = C o v [\hat{θ} (X), l^{'} (θ, X)] \end{aligned}

$\begin{align} 1 &= \frac{\partial}{\partial\theta}\int \hat{\theta}(x)\,f(\bm{x},~\theta)\,d\bm{x} \\ &= \int \hat{\theta}(x)\,\frac{\partial}{\partial\theta}f(\bm{x},~\theta)\,d\bm{x} \\ &= \int \hat{\theta}(x)\,\frac{\partial \log f(\bm{x},~\theta)}{\partial\theta}f(\bm{x},~\theta)\,d\bm{x} \\ &= E[\hat{\theta}(\bm{X})\,l'(\theta,~\bm{X})] \\ &= E[\hat{\theta}(\bm{X})\,l'(\theta,~\bm{X})] - \theta\,E[l'(\theta,~\bm{X})]~~~~\because~E[l'(\theta,~\bm{X})] = 0\\ &= E[(\hat{\theta}(\bm{X}) - \theta)\,l'(\theta,~\bm{X})] \\ &= Cov[\hat{\theta}(\bm{X}),~l'(\theta,~\bm{X})] \end{align}$

\begin{aligned} 1 & = C o v [\hat{θ} (X), l^{'} (θ, X)]^{2} \\ \leq V [\hat{θ} (X)] V [l^{'} (θ, X)] ∵ コーシー・シュワルツの不等式 \\ = V [\hat{θ} (X)] I_{n} (θ) \end{aligned}

$\begin{align} 1 &= Cov[\hat{\theta}(\bm{X}),~l'(\theta,~\bm{X})]^2 \\ &\le V[\hat{\theta}(\bm{X})]\,V[l'(\theta,~\bm{X})]~~~~\because~\text{コーシー・シュワルツの不等式} \\ &= V[\hat{\theta}(\bm{X})]\,I_n(\theta) \end{align}$

この両辺を

I_{n}

$I_n$ で割ると，

V [\hat{θ}] \geq \frac{1}{I_{n} (θ)}

$\begin{equation} V[\hat{\theta}] \ge \frac{1}{I_n(\theta)} \end{equation}$

第3講 統計的推定

クラメール・ラオの下限 (情報量不等式)

補足1: クラメール - ラオの不等式の証明

第3講統計的推定