Problem 20.1 Easy Free 1 variation

Reverse Regression Slope

Two datasets $X$ and $Y$ satisfy $\operatorname{Var}(X) = 10$ and $\operatorname{Var}(Y) = 20$ . Running the ordinary least squares regression $y \sim \alpha_x + \beta_x x$ gives slope $\beta_x = 1$ . Now instead regress $x \sim \alpha_y + \beta_y y$ . Find $\beta_y$ , or state that it cannot be determined from the given information.

Work it out, then compare with the answer.

SolutionShow the worked solution

The OLS slope from regressing a response on a regressor is the sample covariance divided by the regressor’s sample variance:

\beta_x=\frac{\operatorname{Cov}(X,Y)}{\operatorname{Var}(X)},\qquad \beta_y=\frac{\operatorname{Cov}(X,Y)}{\operatorname{Var}(Y)}.

The two slopes have the same numerator, so their ratio is fixed:

\frac{\beta_y}{\beta_x}=\frac{\operatorname{Var}(X)}{\operatorname{Var}(Y)}.

Hence

\beta_y=\beta_x\,\frac{\operatorname{Var}(X)}{\operatorname{Var}(Y)}=1\cdot\frac{10}{20}.

These data are consistent. The covariance is $\operatorname{Cov}(X,Y)=\beta_x\operatorname{Var}(X)=10$ . The correlation then satisfies $\rho^2=\beta_x\beta_y=\tfrac12\le 1$ , so it is a valid value.

\boxed{\beta_y=\tfrac12}

Key idea. The reverse slope is not $1/\beta_x$ . The product of the two slopes is $\beta_x\beta_y=\rho^2$ , so $\beta_y=1/\beta_x$ only when the fit is perfect, that is $|\rho|=1$ .

Variations

The same idea with a twist. Each solution gives only the changed step.

Variation 20.1a

Consider the three data points $(5,5)$ , $(4,0)$ , $(6,10)$ . Regressing $Y$ on $X$ by ordinary least squares gives some slope $\beta$ . Without performing any calculation, determine whether the slope changes when you instead regress $X$ on $Y$ : does it increase, stay the same, or decrease?

Work it out, then compare with the answer.

Solution

The three points lie exactly on the line $Y=5X-20$ . Equivalently, $X=\tfrac15Y+4$ .

When the points lie exactly on a line, both regressions recover that line. So regressing $Y$ on $X$ gives slope $5$ , and regressing $X$ on $Y$ gives slope $\tfrac15$ .

In general, the two slopes are $\operatorname{Cov}/\operatorname{Var}(X)$ and $\operatorname{Cov}/\operatorname{Var}(Y)$ . The covariance here is positive, and the $Y$ -values $0,5,10$ are far more spread out than the $X$ -values $4,5,6$ . The larger denominator in the second regression makes its slope smaller.

\boxed{\text{It decreases (from }5\text{ to }\tfrac15\text{)}}

Spotted an error? Email us