第5講回帰分析

残差分析と決定係数

残差平方和

残差平方和の最小値は，

\begin{aligned} S_{e} & = \sum_{i = 1}^{n} {y_{i} - ({\hat{β}}_{0} + {\hat{β}}_{1} x_{i})}^{2} \\ = \sum_{i = 1}^{n} {y_{i} - \bar{y} - {\hat{β}}_{1} (x_{i} - \bar{x})}^{2} \\ = \sum_{i = 1}^{n} {(y_{i} - \bar{y})^{2} - 2 {\hat{β}}_{1} (y_{i} - \bar{y}) (x_{i} - \bar{x}) + {\hat{β}}_{1}^{2} (x_{i} - \bar{x})^{2}} \\ = \sum_{i = 1}^{n} (y_{i} - \bar{y})^{2} - 2 {\hat{β}}_{1} \sum_{i = 1}^{n} (y_{i} - \bar{y}) (x_{i} - \bar{x}) + {\hat{β}}_{1}^{2} \sum_{i = 1}^{n} (x_{i} - \bar{x})^{2} \\ = \sum_{i = 1}^{n} (y_{i} - \bar{y})^{2} - 2 \frac{S_{x y}}{S_{x x}} \sum_{i = 1}^{n} (y_{i} - \bar{y}) (x_{i} - \bar{x}) + \frac{{S_{x y}}^{2}}{{S_{x x}}^{2}} \sum_{i = 1}^{n} (x_{i} - \bar{x})^{2} \\ = S_{y y} - 2 \frac{S_{x y}}{S_{x x}} S_{x y} + \frac{{S_{x y}}^{2}}{{S_{x x}}^{2}} S_{x x} \\ = S_{y y} - \frac{S_{x y}^{2}}{S_{x x}} \\ = S_{y y} (1 - r_{x y}^{2}) \end{aligned}

$\begin{align} S_{e} &= \sum_{i=1}^{n}\left\{y_i - (\hat{\beta}_0 + \hat{\beta}_1 x_i)\right\}^2 \\ &= \sum_{i=1}^{n}\left\{y_i - \bar{y} - \hat{\beta}_1(x_i - \bar{x})\right\}^2 \\ &= \sum_{i=1}^{n}\left\{(y_i - \bar{y})^2 - 2\hat{\beta}_1(y_i - \bar{y})(x_i - \bar{x}) + \hat{\beta}_1^2(x_i - \bar{x})^2\right\} \\ &= \sum_{i=1}^{n}(y_i - \bar{y})^2 - 2\hat{\beta}_1\sum_{i=1}^{n}(y_i - \bar{y})(x_i - \bar{x}) + \hat{\beta}_1^2\sum_{i=1}^{n}(x_i - \bar{x})^2 \\ &= \sum_{i=1}^{n}(y_i - \bar{y})^2 - 2\frac{S_{xy}}{S_{xx}}\sum_{i=1}^{n}(y_i - \bar{y})(x_i - \bar{x}) + \frac{{S_{xy}}^2}{{S_{xx}}^2}\sum_{i=1}^{n}(x_i - \bar{x})^2 \\ &= S_{yy} - 2\frac{S_{xy}}{S_{xx}}S_{xy} + \frac{{S_{xy}}^2}{{S_{xx}}^2}S_{xx} \\ &= S_{yy} - \frac{S_{xy}^{2}}{S_{xx}} \\ &= S_{yy}(1 - r_{xy}^2) \end{align}$

誤差の母分散の推定

{\hat{σ}}^{2} = \frac{S_{e}}{ϕ_{e}} = \frac{S_{e}}{n - 2}

$\begin{equation} \hat{\sigma}^2 = \frac{S_e}{\phi_e} = \frac{S_e}{n-2} \end{equation}$

回帰平方和

\sum_{i = 1}^{n} (y_{i} - \bar{y})^{2} = \sum_{i = 1}^{n} ({\hat{y}}_{i} - \bar{y})^{2} + \sum_{i = 1}^{n} (y_{i} - {\hat{y}}_{i})^{2}

$\begin{equation} \sum_{i=1}^{n}(y_{i} - \bar{y})^{2} = \sum_{i=1}^{n}(\hat{y}_{i} - \bar{y})^{2} + \sum_{i=1}^{n}(y_{i} - \hat{y}_i)^{2} \end{equation}$

S_{y y} = S_{R} + S_{e}

$\begin{equation} S_{yy} = S_{R} + S_{e} \end{equation}$

$S_{yy}$: 全平方和 (自由度 $\phi_{yy} = n - 1$ )
$S_{R}$: 回帰平方和 (自由度 $\phi_{R} = 1$ )
$S_{e}$: 残差平方和 (自由度 $\phi_{e} = n - 2$ )

よって，

S_{R} = \frac{S_{x y}^{2}}{S_{x x}}

$\begin{equation} S_{R} = \frac{S_{xy}^{2}}{S_{xx}} \end{equation}$

分散分析表

H_{0} : β_{1} = 0

$\begin{equation} H_{0}~:~~\beta_1 = 0 \end{equation}$

H_{1} : β_{1} \neq 0

$\begin{equation} H_{1}~:~~\beta_1 \ne 0 \end{equation}$

要因	平方和	自由度	平均変動	分散比
$R$	$S_{R}$	$\phi_{R} = 1$	$\ds V_{R}=\frac{S_{R}}{\phi_{R}}$	$\ds F=\frac{V_{R}}{V_{e}}$
$e$	$S_{e}$	$\phi_{e} = n-2$	$\ds V_{e}=\frac{S_{e}}{\phi_{e}}$
合計	$S_{yy}$	$\phi_{yy} = n-1$

決定係数

R^{2} \equiv \frac{S_{R}}{S_{y y}} = \frac{S_{x y}^{2}}{S_{x x} S_{y y}} = {(\frac{S_{x y}}{\sqrt{S_{x x}} \sqrt{S_{y y}}})}^{2} = r_{x y}^{2}

$\begin{equation} R^{2} \equiv \frac{S_{R}}{S_{yy}} = \frac{S_{xy}^{2}}{S_{xx}S_{yy}} = \left(\frac{S_{xy}}{\sqrt{S_{xx}}\sqrt{S_{yy}}}\right)^2 = r_{xy}^2 \end{equation}$