Section 14.2

Covariance and Correlation Calculations

Bilinearity and affine invariance, correlations of sums and differences, uncorrelated but dependent variables, Var(XY), covariances of transformed variables.

Toolkit

Covariance is a symmetric bilinear form: $\operatorname{Cov}(aX+bY,cZ+dW)=ac\operatorname{Cov}(X,Z)+ad\operatorname{Cov}(X,W)+bc\operatorname{Cov}(Y,Z)+bd\operatorname{Cov}(Y,W)$ , and $\operatorname{Cov}(X+c,Y)=\operatorname{Cov}(X,Y)$ . Correlation is invariant under $X\mapsto aX+b$ , $Y\mapsto cY+d$ with $a,c>0$ , and flips sign if exactly one scale factor is negative. Key identities:

\operatorname{Var}(X\pm Y)=\sigma_X^2+\sigma_Y^2\pm2\rho\sigma_X\sigma_Y,\qquad \operatorname{Cov}(X+Y,X-Y)=\operatorname{Var}X-\operatorname{Var}Y.

Zero covariance does not imply independence: it only kills the linear association, so nonlinear or symmetric dependence (e.g. under sign-flip symmetries) can leave $\operatorname{Cov}=0$ . For independent $X,Y$ ,

\operatorname{Var}(XY)=\mathbb{E}[X^2]\mathbb{E}[Y^2]-(\mathbb{E}X)^2(\mathbb{E}Y)^2,

which differs from $\operatorname{Var}X\operatorname{Var}Y$ . The regression slope minimizing correlation residual is $k=\operatorname{Cov}(X,Y)/\operatorname{Var}X$ , giving $\operatorname{Cov}(X,Y-kX)=0$ .

Problems