Section 20.2

R-Squared, Data Transformations and Pitfalls

Bounds on R^2 when regressors are combined, the effect of duplicating or rescaling data on R^2, coefficient variances and t-statistics, Simpson's paradox.

Toolkit

$R^2$ measures the fraction of variance in $y$ explained by the fitted regressors; these problems probe how it (and related quantities) behave under manipulations of the data or regressor set.

$R^2 = 1 - \frac{\text{SSE}}{\text{SST}} = \operatorname{Corr}(y,\hat y)^2$ ; for simple regression $R^2=\rho_{xy}^2$ .
Adding regressors cannot decrease $R^2$ : $R^2_{\text{full}}\ge R^2_{\text{reduced}}$ , with equality iff the new regressor is uncorrelated with the residuals of the reduced fit. Via suppression (regressors correlated with each other but with opposing effects on $y$ ), joint $R^2$ can approach $1$ even if each marginal $R^2$ is small.
Duplicating every observation leaves all centred sums of squares and cross-products (hence $\hat\beta$ , $R^2$ , correlations) unchanged, but doubles $X'X$ , halving coefficient variances and inflating $t$ -statistics by $\sqrt2$ .
Rescaling $x\to cx$ , $y\to dy$ leaves $R^2$ and $t$ -statistics unchanged (slope scales by $d/c$ , but relative precision is invariant).
Slope $t$ -statistic: $t=\dfrac{\hat\beta_1}{\mathrm{se}(\hat\beta_1)}=\dfrac{\rho\sqrt{n-2}}{\sqrt{1-\rho^2}}$ (using $n-2$ d.f.), useful for solving for $n$ given a target significance.
Simpson’s paradox: pooled rate $=\sum_i w_i p_i$ with group-specific weights $w_i$ ; a within-group comparison can reverse once weights differ across the groups being compared.

Problems