Nonlinear Regression and Causation
During a hot spell, an ice cream shop in Guadalajara sells more popsicles and, at the same time, there are more accidents at swimming pools. Does that mean popsicles cause accidents? Also, not every cloud of points follows a line: sometimes it rises and then falls, sometimes it speeds up. Here you will learn to choose a model, to turn a curve into a line using logarithms, and to avoid drawing false conclusions from a correlation.
When a Line Is Not the Right Model
The three scatter plots in the diagram correspond to three different studies. For each of them, indicate which type of model seems most suitable — linear ($y = a + bx$),…
The three scatter plots in the diagram correspond to three different studies. For each of them, indicate which type of model seems most suitable — linear ($y = a + bx$), quadratic ($y = a + bx + cx^2$) or exponential ($y = a \times b^x$) — and briefly justify your choice.
Coefficient of Determination and Choosing a Model
For any model (including the quadratic) we define $R^2=1-\dfrac{\text{SSE}}{\text{SST}}$, where SSE is the sum of the squared residuals (observed minus predicted) and SST is the sum of the squares of $y_i-\bar y$. The closer to $1$, the better the model explains those data.
In a linear least squares fit, a correlation coefficient $r = -0.9$ has been obtained. Calculate the coefficient of determination $r^2$ and interpret its meaning: what…
In a linear least squares fit, a correlation coefficient $r = -0.9$ has been obtained. Calculate the coefficient of determination $r^2$ and interpret its meaning: what percentage of the variation in the variable $y$ is explained by the variation in $x$?
In two statistical studies, the correlation coefficients $r_1 = 0.8$ and $r_2 = -0.95$ were obtained. Which of the two linear fits allows more reliable predictions to be…
In two statistical studies, the correlation coefficients $r_1 = 0.8$ and $r_2 = -0.95$ were obtained. Which of the two linear fits allows more reliable predictions to be made? Explain why the sign of the coefficient does not affect reliability.
For the same data, two models have been tested: a linear fit with $r = 0.84$ and a quadratic fit with coefficient of determination $r^2 = 0.93$. a) Calculate the…
For the same data, two models have been tested: a linear fit with $r = 0.84$ and a quadratic fit with coefficient of determination $r^2 = 0.93$.
- a) Calculate the coefficient of determination of the linear model.
- b) What percentage of the variation in $y$ does each model explain?
- c) Which one would you choose to make predictions? Justify your answer.
After calculating a correlation coefficient $r = 0.4$, a student writes in their report: 'Since $r = 0.4$, $40\,\%$ of the variation in $y$ is due to the variation in…
After calculating a correlation coefficient $r = 0.4$, a student writes in their report: 'Since $r = 0.4$, $40\,\%$ of the variation in $y$ is due to the variation in $x$.' Identify the conceptual error, correct it by calculating the correct percentage, and assess whether the linear fit is suitable for making predictions with this data.
In a bivariate distribution, the regression line of $y$ on $x$ has a gradient of $m_{yx} = 1.2$ and the regression line of $x$ on $y$ has a gradient of $m_{xy} = 0.75$.…
In a bivariate distribution, the regression line of $y$ on $x$ has a gradient of $m_{yx} = 1.2$ and the regression line of $x$ on $y$ has a gradient of $m_{xy} = 0.75$.
- a) Using the relationship $m_{yx} \times m_{xy} = r^2$, calculate the coefficient of determination.
- b) Deduce the value of the correlation coefficient $r$, justifying its sign.
- c) Interpret the value of $r^2$ as a percentage of explained variation and assess the reliability of predictions made using these lines.
Linearizing an Exponential Model with Logarithms
Why does it work? In an exponential, $y$ is multiplied by the same amount $b$ each time $x$ increases by $1$; logarithms turn multiplying into adding, and adding the same amount each time gives a line.
If what you want is the growth rate, look at $b$: if $b=1.25$, $y$ grows $25\%$ in each period, because $b=1+\text{rate}$.
The number of users of an application grows approximately exponentially: $y = a \times b^x$, where $x$ is the number of months elapsed. Taking natural logarithms,…
The number of users of an application grows approximately exponentially: $y = a \times b^x$, where $x$ is the number of months elapsed. Taking natural logarithms, $\ln y = \ln a + x \times \ln b$, so that the pairs $(x, \ln y)$ fit a straight line. The linearised data are:
| $x$ (months) | $0$ | $1$ | $2$ | $3$ |
|---|---|---|---|---|
| $\ln y$ | $1.0$ | $1.7$ | $2.4$ | $3.1$ |
- a) Verify that the four linearised points lie exactly on a straight line and find its equation.
- b) Deduce the values of $a$ and $b$ (express $a$ and $b$ as powers of $e$ and approximate them to two decimal places).
- c) Estimate the number of users after $6$ months.
The area of a lake covered by an invasive plant has been measured over $5$ months: | $x$: month | $1$ | $2$ | $3$ | $4$ | $5$ | |---|---|---|---|---|---| | $y$: area…
The area of a lake covered by an invasive plant has been measured over $5$ months:
| $x$: month | $1$ | $2$ | $3$ | $4$ | $5$ |
|---|---|---|---|---|---|
| $y$: area ($\text{m}^2$) | $3.5$ | $7.1$ | $14.0$ | $28.2$ | $56.5$ |
- a) Plot the pairs $(x, \ln y)$, verify that they align reasonably well, and fit a straight line to the linearised data (take $\ln y$ to two decimal places).
- b) Write down the corresponding exponential model $y = a \times b^x$ and calculate the monthly growth rate as a percentage.
- c) Predict the area covered in month $8$ and, given that the lake has $1,500\ \text{m}^2$, comment on the problem presented by extrapolating this model in the long term.
Linearizing a Power Model
In a Physics laboratory, the period $T$ of a pendulum is measured as a function of its length $L$, and a power relationship $T = a \times L^b$ is suspected. Taking…
In a Physics laboratory, the period $T$ of a pendulum is measured as a function of its length $L$, and a power relationship $T = a \times L^b$ is suspected. Taking logarithms, $\ln T = \ln a + b \times \ln L$. The linearised data are:
| $\ln L$ | $-1.386$ | $-0.693$ | $0$ | $0.693$ |
|---|---|---|---|---|
| $\ln T$ | $0$ | $0.347$ | $0.693$ | $1.040$ |
- a) Fit a straight line to the linearised data and write down its equation.
- b) Deduce the values of $a$ and $b$.
- c) Compare the value obtained for $b$ with that predicted by Physics, $b = \frac{1}{2}$, and comment on the result.
Quadratic Fit by Least Squares
Why? When we require the total error to be a minimum, the derivative of that error with respect to each parameter ($a$, $b$, and $c$) must be zero; each one gives one of these equations.
It is suspected that the variable $y$ has a quadratic dependence on $x$. Using the five data items in the table | $x$ | $0$ | $1$ | $2$ | $3$ | $4$ |…
It is suspected that the variable $y$ has a quadratic dependence on $x$. Using the five data items in the table
| $x$ | $0$ | $1$ | $2$ | $3$ | $4$ |
|---|---|---|---|---|---|
| $y$ | $5$ | $3$ | $3$ | $5$ | $9$ |
the sums $\sum x_i = 10$, $\sum x_i^2 = 30$, $\sum x_i^3 = 100$, $\sum x_i^4 = 354$, $\sum y_i = 25$, $\sum x_i y_i = 60$ and $\sum x_i^2 y_i = 204$ have been calculated.
- a) Set up the system of three equations that the coefficients of the regression parabola $y = a x^2 + bx + c$ must satisfy.
- b) Solve it and write down the equation of the quadratic fit.
- c) Estimate the value of $y$ for $x = 5$ and comment on the reliability of this prediction.
The consumption of an electric car, in $\text{kWh}$ per $100\ \text{km}$, has been measured at different speeds and the data are shown in the figure: | $x$: speed…
The consumption of an electric car, in $\text{kWh}$ per $100\ \text{km}$, has been measured at different speeds and the data are shown in the figure:
| $x$: speed ($\text{km/h}$) | $40$ | $60$ | $80$ | $100$ | $120$ |
|---|---|---|---|---|---|
| $y$: consumption ($\text{kWh}/100\ \text{km}$) | $13.1$ | $12.4$ | $12.5$ | $13.4$ | $15.1$ |
- a) Mentally represent (or sketch) the scatter plot and explain why a quadratic model is more suitable than a linear one.
- b) Calculate the necessary sums $\sum x_i$, $\sum x_i^2$, $\sum x_i^3$, $\sum x_i^4$, $\sum y_i$, $\sum x_i y_i$, $\sum x_i^2 y_i$ and set up the system for the quadratic fit.
- c) Solve the system and write down the equation of the regression parabola (round the coefficients to four decimal places).
- d) Find the speed for minimum consumption according to the model and predict the consumption at $140\ \text{km/h}$, assessing the reliability of this prediction.
You can do it on paper and send a photo with your answer.
Interpolating, Extrapolating, and the Natural Bound
Interpolation is fairly reliable if $r^2$ is high. Extrapolation is risky because you don't know whether the relationship continues the same beyond what was observed. Also, the line is most precise near $\bar x$ and becomes less reliable toward the ends.
For the line use $m=\dfrac{s_{xy}}{s_x^2}$ and $b=\bar y-m\bar x$.
Using data from a bivariate distribution where $x$ varies between $10$ and $30$, with $\bar{x} = 20$ and $r = 0.97$, a regression line has been obtained. Two students…
Using data from a bivariate distribution where $x$ varies between $10$ and $30$, with $\bar{x} = 20$ and $r = 0.97$, a regression line has been obtained. Two students use it to estimate $y$: one for $x = 22$ and another for $x = 55$.
- a) Which estimate is more reliable? Give reasons for your answer using the position of each value relative to the range of the data and to $\bar{x}$.
- b) What is the name of the type of prediction made outside the range of the data, and why is it risky?
The diagram shows the percentage of Spanish households with a fiber-optic connection as a function of the years elapsed since 2015, according to INE data. a) Fit a…
The diagram shows the percentage of Spanish households with a fiber-optic connection as a function of the years elapsed since 2015, according to INE data.
- a) Fit a regression line to the first five data items using the means $\bar{x} = 4$, $\bar{y} = 64.6$ and the values $s_{xy} = 55.2$, $s_x^2 = 8$.
- b) Predict the percentage for $x = 12$ (year 2027) using this line.
- c) By observing the shape of the full scatter plot, criticise the validity of this long-term prediction: what natural bound does the variable have, and what type of model would be more sensible?
Correlation Is Not Causation
Let's return to the beginning: popsicle sales and pool accidents have a high correlation.
State whether the following statement is true or false, giving reasons: 'The correlation coefficient between ice cream sales and the number of drownings in swimming…
State whether the following statement is true or false, giving reasons: 'The correlation coefficient between ice cream sales and the number of drownings in swimming pools is high and positive; therefore, selling more ice cream causes more drownings.' If it is false, explain the misinterpretation error and name a variable that could explain both at the same time.
In a city, a correlation $r = 0.92$ is observed between the number of fire engines that attend a fire and the financial damage it causes. A politician states: 'To reduce…
In a city, a correlation $r = 0.92$ is observed between the number of fire engines that attend a fire and the financial damage it causes. A politician states: 'To reduce fire damage, we should send fewer fire engines.' Analyse the fallacy in their reasoning, identify the variable that explains the correlation, and formulate a correct conclusion from the same data.
In several districts of a European region, a correlation $r = 0.85$ has been observed between the number of storks recorded and the number of births registered. a)…
In several districts of a European region, a correlation $r = 0.85$ has been observed between the number of storks recorded and the number of births registered.
- a) Explain why it would be absurd to conclude that storks 'bring' babies, identifying the type of correlation shown in this example.
- b) Propose a plausible confounding variable that explains the correlation, and explain how it acts on both variables.
- c) Briefly outline what data you would collect and what calculation you would perform to unmask the spurious relationship.
Back to the beginning: popsicles don't cause accidents; the heat increases both. A high $r$ measures association, not cause.