Regression Line
Ximena studies 2 hours for the Math exam and wonders: "what grade can I expect?". If we know how study hours and grades are related for many classmates, we can predict. The tool is the regression line. At the end of the page we will answer Ximena's question.
Covariance and correlation: what you need
A bivariate distribution has pairs $(x_i, y_i)$. Its center of gravity is the point of the means, $(\bar x, \bar y)$.
Observe the following scatter diagrams representing the relationship between two statistical variables. Match, giving reasons, each of the values of Pearson's…
Observe the following scatter diagrams representing the relationship between two statistical variables. Match, giving reasons, each of the values of Pearson's correlation coefficient, $r$, with its corresponding graph:
Values of $r$: $0.95$; $-0.85$; $0.05$.
Indicate, for each of the following pairs of variables, whether you expect them to have positive, negative, or zero correlation: 1. The number of kilometers driven by a…
Indicate, for each of the following pairs of variables, whether you expect them to have positive, negative, or zero correlation:
-
The number of kilometers driven by a car and the fuel remaining in its tank.
-
A person's height and their grade on the math exam.
-
A family's monthly income and their expenditure on leisure and culture.
-
The daily training time of an athlete and their personal best time in a race.
Given the following table of values for a bivariate variable: | $X$ | $2$ | $4$ | $6$ | $8$ | | :--- | :---: | :---: | :---: | :---: | | $Y$ | $10$ | $18$ | $22$ | $30$…
Given the following table of values for a bivariate variable:
| $X$ | $2$ | $4$ | $6$ | $8$ |
|---|---|---|---|---|
| $Y$ | $10$ | $18$ | $22$ | $30$ |
- a) Calculate the center of gravity $(\bar{x}, \bar{y})$.
- b) Plot the points in a small hand-drawn scatter diagram and deduce the sign of the covariance $\sigma_{xy}$ without carrying out the full calculation.
The regression line of $Y$ on $X$
It is the point-slope equation you already know: the point is $(\bar x,\bar y)$ and the slope is $m=\sigma_{xy}/\sigma_x^2$.
Answer to the last step: $y=0.8(3.5)+4.6=7.4$.
A study on study habits in 12th grade analyzes the relationship between the number of weekly hours spent reading, $X$, and the grade obtained on the Spanish exam, $Y$.…
A study on study habits in 12th grade analyzes the relationship between the number of weekly hours spent reading, $X$, and the grade obtained on the Spanish exam, $Y$. The regression line of $Y$ on $X$ is:
$y = 0.65x + 3.2$
- a) Justify whether the correlation between both variables is positive or negative.
- b) If a student spends $8$ weekly hours reading, what grade is estimated for them on the exam?
In a bivariate distribution, it is known that the mean of variable $X$ is $\bar{x} = 12$ and the mean of variable $Y$ is $\bar{y} = 15$. a) What are the coordinates of…
In a bivariate distribution, it is known that the mean of variable $X$ is $\bar{x} = 12$ and the mean of variable $Y$ is $\bar{y} = 15$.
- a) What are the coordinates of the center of gravity of this distribution?
- b) If the regression line of $Y$ on $X$ is $y = 0.8x + a$, calculate the value of the y-intercept $a$ knowing that the line must pass through the center of gravity.
A group of 1st Bachillerato students has conducted a study on the relationship between the daily time (in hours) spent using a popular short-video social network ($X$)…
A group of 1st Bachillerato students has conducted a study on the relationship between the daily time (in hours) spent using a popular short-video social network ($X$) and the battery percentage remaining on their mobile at the end of the day ($Y$). The data obtained from 6 classmates are as follows:
| Hours of use ($X$) | 1.5 | 2 | 3 | 3.5 | 4 | 5 |
|---|---|---|---|---|---|---|
| % Battery ($Y$) | 75 | 68 | 50 | 45 | 32 | 20 |
a) Calculate the center of gravity of the distribution $(\bar{x}, \bar{y})$.
b) Find the covariance $\sigma_{xy}$ and the linear correlation coefficient $r$. Interpret the result obtained regarding the sign and the strength of the relationship.
c) Determine the equation of the regression line of $Y$ on $X$.
d) If a student uses the social network for 2.5 hours, what battery percentage is estimated to remain? Is this prediction reliable? Justify your answer based on the coefficient of determination $r^2$.
Slope from $r$ and the standard deviations
Sometimes the problem doesn't give the data, but summaries. Since $\sigma_{xy}=r\,\sigma_x\sigma_y$, the slope can be written in several ways.
Answer to the last step: $y=4.5(5)+12=34.5$.
A courier company analyzed the relationship between the distance covered (in km) by its bicycle delivery riders ($X$) and the number of calories burned according to…
A courier company analyzed the relationship between the distance covered (in km) by its bicycle delivery riders ($X$) and the number of calories burned according to their smart devices ($Y$). After calculating the statistical parameters, the following values were obtained:
$\bar{x} = 15 \text{ km}$; $\bar{y} = 480 \text{ kcal}$
$\sigma_x = 4.2$; $\sigma_y = 120$
$\sigma_{xy} = 453.6$
a) Calculate the linear correlation coefficient and state what type of dependency exists between the variables.
b) Find the regression line of calories burned ($Y$) as a function of the distance covered ($X$).
c) How many calories is a delivery rider who covers 22 km estimated to burn?
d) If a delivery rider wishes to burn approximately 600 kcal, how many kilometers should they cover according to the regression line of $X$ on $Y$?
A pet adoption center wants to predict the time it takes for a dog to be adopted ($Y$, in days) based on its age ($X$, in years). They have collected data from 10 dogs…
A pet adoption center wants to predict the time it takes for a dog to be adopted ($Y$, in days) based on its age ($X$, in years). They have collected data from 10 dogs and obtained a correlation of $r = 0.85$. The mean age is 4 years and the mean number of days until adoption is 25 days. The standard deviation of age is 2 years and that of the days is 10 days.
a) Find the equation of the regression line.
b) Estimate how long it will take for a 0.5-year-old puppy to be adopted.
c) Estimate how long it will take for a 12-year-old senior dog.
d) Of the two previous estimates, indicate which is more reliable and why, taking into account the range of the original data (ages between 1 and 8 years).
The line of $X$ on $Y$ and how the two lines are related
The two lines intersect at the center of gravity. If $m$ is the slope of $Y$ on $X$ and $m'$ the coefficient of $y$ in the one for $X$ on $Y$ ($x=m'y+n'$):
Answer to the last step: $x=3+1.23(9-7)=5.46$ hours (slight extrapolation: the data go up to $5$ hours).
In a study on sleep habits and academic performance, the following regression lines were obtained for the variables $X$ (daily hours of sleep) and $Y$ (average term…
In a study on sleep habits and academic performance, the following regression lines were obtained for the variables $X$ (daily hours of sleep) and $Y$ (average term grade):
Regression line of $Y$ on $X$: $y = 0.65x + 3.2$
Regression line of $X$ on $Y$: $x = 1.15y - 0.45$
a) Knowing that both lines intersect at the center of gravity of the distribution, calculate the mean hours of sleep ($\bar{x}$) and the students' average grade ($\bar{y}$).
b) Calculate the linear correlation coefficient $r$ from the slopes of both lines.
c) State with reasoning whether the correlation is positive or negative and whether it is strong or weak.
In a study on the performance of a new cryptocurrency, the variables $X$ (advertising expenditure in thousands of pesos) and $Y$ (daily transaction volume in millions of…
In a study on the performance of a new cryptocurrency, the variables $X$ (advertising expenditure in thousands of pesos) and $Y$ (daily transaction volume in millions of pesos) have been analyzed. After processing the data, the equations of the two regression lines have been obtained:
-
Regression line of $Y$ on $X$: $y = 0.85x + 1.24$
-
Regression line of $X$ on $Y$: $x = 0.92y - 0.15$
Determine with reasoning:
-
The center of gravity of the distribution $(\bar{x}, \bar{y})$.
-
Pearson's linear correlation coefficient ($r$).
-
If an advertising expenditure of $15\,000$ pesos is intended, what would be the expected transaction volume? Justify the reliability of this prediction based on the coefficient of determination.
The grades of 50 students in Economics ($X$) and Geography ($Y$) have a mean of 6.5 in both subjects. The variance of Economics is 4 and that of Geography is 2.25. It is…
The grades of 50 students in Economics ($X$) and Geography ($Y$) have a mean of 6.5 in both subjects. The variance of Economics is 4 and that of Geography is 2.25. It is known that the correlation coefficient is $r = 0.8$.
-
If a student scored 4 in Economics, estimate their grade in Geography.
-
If another student scored 9 in Geography, estimate their grade in Economics using the relevant regression line.
-
Compare both estimates and explain why the same line cannot be used for both cases if we want to minimize the mean squared error.
Interpreting the slope and reliability
The slope $m$ tells you how much $y$ changes (on average) for each unit that $x$ increases, in the units of the problem. The y-intercept $n$ is the estimated value when $x=0$, which often makes no real-world sense.
The reliability of the predictions depends on $|r|$: the closer to $1$, the closer the points hug the line and the better the estimates. If $|r|$ is close to $0$, the line predicts little.
A nutritionist analyses the relationship between the weight of a group of teenagers ($X$, in kg) and their height ($Y$, in cm). The collected data have a correlation…
A nutritionist analyses the relationship between the weight of a group of teenagers ($X$, in kg) and their height ($Y$, in cm). The collected data have a correlation coefficient of $r = 0.92$ and the regression line is $y = 1.4x + 85$.
Explain the meaning of the gradient ($1.4$) in the context of the problem and assess the reliability of the predictions made using this line.
A courier company knows that the cost of a shipment, $Y$ (in pesos), depends on the weight of the package, $X$ (in kg), according to the regression line $y = 50x + 100$.…
A courier company knows that the cost of a shipment, $Y$ (in pesos), depends on the weight of the package, $X$ (in kg), according to the regression line $y = 50x + 100$.
If the correlation coefficient is $r = 0.98$, which of the following statements is correct?
-
For every kilogram increase in weight, the cost increases by exactly $50$ pesos.
-
Predictions made for packages of a similar weight to those studied will be very reliable.
-
If a package weighs $0$ kg, the cost will necessarily be $0$ pesos.
-
The relationship between weight and cost is inverse.
A survey has been conducted among a group of teenagers on the number of followers they have on a platform ($X$, in hundreds) and the average number of likes they receive…
A survey has been conducted among a group of teenagers on the number of followers they have on a platform ($X$, in hundreds) and the average number of likes they receive per post ($Y$). The data are summarised in the following table:
| Followers ($X$) | 2 | 4 | 6 | 8 | 10 |
|---|---|---|---|---|---|
| Likes ($Y$) | 45 | 92 | 135 | 180 | 220 |
a) Plot the scatter diagram and sketch the regression line freehand.
b) Calculate the coefficients of the regression line $y = mx + n$.
c) What is the physical meaning of the gradient $m$ in this context?
d) Calculate the coefficient of determination $r^2$ and explain what percentage of the variation in likes is explained by the number of followers.
The residual: error of a prediction
Answer to the last step: $\hat y=6.2$ and $e=6-6.2=-0.2$.
An athletics coach analyses the relationship between the height of his athletes in cm ($X$) and their stride length in cm ($Y$). The data for 5 athletes are: (160, 110),…
An athletics coach analyses the relationship between the height of his athletes in cm ($X$) and their stride length in cm ($Y$). The data for 5 athletes are: (160, 110), (170, 120), (175, 125), (180, 132) and (190, 145).
a) Obtain the regression line of $Y$ on $X$.
b) What stride length would be expected for an athlete who is 185 cm tall?
c) Calculate the prediction error (residual) for the athlete who is 180 cm tall. The error is defined as the difference between the actual value and the value estimated by the line: $e = y_i - \hat{y}_i$.
d) If the correlation coefficient were $r = 0.98$, what could you state about the reliability of the coach's predictions?
Predicting with caution: interpolating and extrapolating
Before predicting, ask yourself three questions: is the value of $x$ within the range of the data? Is $|r|$ high? Does the result make sense in real life?
Also, statistical relationship is not functional relationship: in a function each $x$ gives exactly one $y$ and all points lie on the graph; in a statistical relationship the points scatter around the line. And two variables being correlated does not prove that one causes the other.
The charging time of a cell phone battery ($X$, in minutes) and the charge percentage reached ($Y$, in %) have been measured. The results are shown in the following…
The charging time of a cell phone battery ($X$, in minutes) and the charge percentage reached ($Y$, in %) have been measured. The results are shown in the following graph:
If the equation of the orange line is $y = 0.5x + 10$:
- a) Estimate the charge percentage after $60$ minutes.
- b) Is it a deterministic (functional) or statistical relationship? Explain your answer by observing the position of the points relative to the line.
Observe the following scatter diagram relating R&D investment ($X$) and annual net profit ($Y$) of several technology companies: Knowing that the regression line is…
Observe the following scatter diagram relating R&D investment ($X$) and annual net profit ($Y$) of several technology companies:
Knowing that the regression line is $y = 2.5x + 10$ and that the correlation coefficient is $r = 0.92$:
a) State, giving reasons, whether there is a strong relationship between investment and profits.
b) If a company invests nothing in R&D ($x = 0$), according to the model, what would its estimated profit be? Does this value make sense in reality?
c) Explain what would happen to the gradient of the line if the correlation coefficient were negative.
d) If we multiply all the profit values by 2, how would the value of the correlation coefficient $r$ change? Justify your answer.
Outliers: how they move the line
An outlier is a point that lies far from the rest of the cloud. A single point like that can pull the line toward it, because the line minimizes the sum of the squared residuals and a very large residual weighs enormously.
An outlier has three typical effects:
-
It changes the slope and the y-intercept: the line tilts toward the point.
-
It changes $|r|$: a point that breaks the trend reduces it; if you remove it, $|r|$ increases.
-
It is the one with the largest residual.
Given the following scatter plot corresponding to the variables $X$ (age of a car in years) and $Y$ (market value in thousands of pesos), the regression line of $Y$ on…
Given the following scatter plot corresponding to the variables $X$ (age of a car in years) and $Y$ (market value in thousands of pesos), the regression line of $Y$ on $X$ has been drawn. It is observed that an outlier exists at the coordinates $(1, 40)$.
-
If we remove this point from the dataset, what would you expect to happen to the absolute value of the correlation coefficient? And to the slope of the regression line?
-
If the original line is $y = -30x + 400$, calculate the estimated value for a 10-year-old car and discuss the suitability of using this model for cars over 15 years old.
Changes of units and scale
If $Y'=aY+b$ (for example, from pesos to thousands of pesos, or adding a tax), the new line is obtained by transforming the previous one, and $|r|$ does not change.
If an exercise statement gives a different tax percentage (or says you have to remove it, that is, divide by the factor), use the factor indicated there: the method is exactly the same.
A study has been carried out on the relationship between the price of a monthly subscription to a music platform ($X$, in pesos) and the number of hours users listen to…
A study has been carried out on the relationship between the price of a monthly subscription to a music platform ($X$, in pesos) and the number of hours users listen to music per month ($Y$). It is known that:
-
The variance of $X$ is $\sigma_x^2 = 4$.
-
The variance of $Y$ is $\sigma_y^2 = 25$.
-
The regression line of $Y$ on $X$ is $y = -1.2x + 42$.
a) Determine the covariance $\sigma_{xy}$ between both variables.
b) Calculate the linear correlation coefficient $r$.
c) How would the correlation coefficient change if we decided to express listening time in minutes instead of hours? Justify your answer.
A marketing team studies the relationship between the likes obtained on an Instagram post ($X$) and the sales generated for a product ($Y$). They have observed that the…
A marketing team studies the relationship between the likes obtained on an Instagram post ($X$) and the sales generated for a product ($Y$). They have observed that the regression line of $Y$ on $X$ is $y = 0.04x + 150$. However, they realize that the sales data were recorded without tax (21%).
If we define a new variable $Y' = 1.21Y$ to include the tax:
-
What will be the new equation of the regression line of $Y'$ on $X$?
-
How will the original Pearson correlation coefficient be affected? Justify your answer based on the properties of linear transformations in statistics.
Back to the beginning: with the line $y=0.8x+4.6$ from Ximena's classmates, studying $2$ hours gives an estimated grade of $0.8(2)+4.6=6.2$. It is an interpolation (the data run from $1$ to $5$ hours) and with $r\approx0.99$ it is a reliable prediction, although not exact.