Regression line

The line of best fit

Using the line to predict, with caution

The correlation coefficient r

We can calculate the line of $Y$ on $X$ if $\sigma_x^2>0$, even if the fit is poor. If all the values of $X$ are equal, that formula divides by zero and does not determine a unique gradient. The correlation coefficient also requires $\sigma_y>0$: if one variable is constant, $r$ is not defined, it is not zero.

Correlation is not causation

An $r$ close to $1$ tells you that two variables move together very regularly. It does not tell you why.

It is the same caution you already saw with Simpson's paradox in the previous section: the numbers in a table or a correlation coefficient summarise what happens, but interpreting them as why it happens requires reasoning about the context, not just looking at the figure.

Spreadsheet investigation: practice and results

Question: is the number of days of practice in the previous week related to the number of correct answers in the same 20-question activity? Define what counts as a day of practice and use the same activity and conditions for everyone. In two classes of 20 people, draw five names from each by lot. Ask for voluntary, anonymous responses, and record the non-responses: they are not zeros. Random selection helps, but ten participants and a single activity do not guarantee that they represent the whole school. Do not attribute differences to innate ability or gender.

Before collecting data, rehearse with these five fictitious pairs, not with real students' results:

1. Write the headings in A1:B1 and the pairs in A2:B6. Keep each pair in its row when sorting. Check that days are between 0 and 7 and correct answers between 0 and 20; review impossible values with the person who recorded them.

2. In E1 write Gradient and in F1 =SLOPE(B2:B6,A2:A6). In E2 write Intercept and in F2 =INTERCEPT(B2:B6,A2:A6). You should obtain $0{,}9$ and $11{,}3$. The Y values come first in these formulae.

3. In E3 write Correlation and in F3 =CORREL(A2:A6,B2:B6): it should give $0{,}9$. With =AVERAGE(A2:A6) and =AVERAGE(B2:B6) check the means $3$ and $14$.

4. Select A1:B6 and insert an XY scatter chart, with days on the horizontal axis and correct answers on the vertical axis. Add a linear trendline and display its equation. Check $y=0{,}9x+11{,}3$; do not join the points as if they were a path over time. Look for curvature or isolated points before accepting a linear fit.

5. In C1 write Prediction and in C2 =$F$1*A2+$F$2; copy down to C6. In D1 write Residual and in D2 =B2-C2; copy down to D6. The predictions are $12{,}2;13{,}1;14;14{,}9;15{,}8$ and the residuals $-0{,}2;-0{,}1;1;-0{,}9;0{,}2$. Having residuals is normal: the line does not pass through all the data points.

6. For four days, the prediction is $14{,}9$ correct answers; it is an estimate, not a promise or a mark. Do not extrapolate to ten days in a week. If a column is constant, explain why the correlation cannot be calculated.

The names of functions and the separators may change depending on the language of the application; look for the equivalent functions for gradient, regression intercept and Pearson correlation.

Conclusion: in the fictitious data there is a positive linear association, but it has not been proved that practising more is the only cause. Prior knowledge, help or the type of practice may all have an influence. In a real investigation, present the question, selection, non-responses, chart, fit and limitations; do not automatically generalise to the whole school. Keep the data free of personal identifiers.

Modifying an algorithm and checking it

The procedure above is an algorithm: validate pairs → plot → check variation and shape → fit → calculate residuals → interpret. If someone sorts A and B separately, they destroy the pairs: go back to the original table to correct it.

Design instructions to add another pair and extend all the ranges, including the chart. Test them with the fictitious pair $(3,14)$: it lies on the line and keeps the gradient and intercept unchanged. Then change the unit of X by multiplying each value by 7: the new gradient should be $0{,}9/7$, but the intercept, the corresponding predictions and the correlation do not change. Explain which step you need to modify and why. Finally, create a warning that stops the fit if all the values of X are equal.

Working together and learning from mistakes

Take turns collecting data, checking and explaining. Each person must run at least one formula and justify one conclusion; share out tasks without gender stereotypes. When results differ, compare data and ranges before deciding. If you get stuck, identify whether the difficulty lies in the data, the formula or its interpretation, and try with three pairs first. Ask for help by explaining what you have tried. Finish with one sentence about the mistake you found and the change of strategy that helped.

Back to the start: To estimate weight from height, we calculate the regression line of $Y$ (weight) on $X$ (height) using the means, the covariance and the variance of height; by substituting the height into the equation we obtain the expected weight, provided that the correlation coefficient is high and we do not extrapolate outside the measured range.

Summary