Bivariate Data, Correlation, and Regression Lines
Coach Rivera notices that players who practice free throws longer tend to make more of them in games. Can she use that pattern to predict how many free throws (out of 20) a player will make after 35 minutes of practice? And how much should she trust that prediction? To answer, you need tools that look at two variables at once.
Functional vs. statistical relationships
When you measure two variables $x$ and $y$ on the same individuals, you get bivariate data: a list of pairs $(x_1, y_1), (x_2, y_2), \dots, (x_n, y_n)$. We call $x$ the explanatory variable and $y$ the response variable.
Why does the difference matter? With a functional relationship you can compute $y$ exactly. With a statistical one you can only estimate $y$, and the rest of this page is about how to do that well and how to measure how good the estimate is.
In the study of relationships between two variables, it is essential to distinguish whether there is an exact dependence or a statistical relationship. State in which of…
In the study of relationships between two variables, it is essential to distinguish whether there is an exact dependence or a statistical relationship. State in which of the following cases the relationship is functional (an exact formula exists) and in which it is statistical (there is a trend but with variability):
-
a) The number of gallons of gas refueled at a gas station and the total cost to pay.
-
b) The grade obtained in a Math test and the number of hours the student has spent studying that week.
-
c) The time an energy-saving light bulb remains switched on and the energy consumption recorded by the meter.
-
d) A person's height and their shoe size.
-
e) The maximum daily temperature in a city and the number of ice creams sold at a kiosk in that same city.
Scatter plots: direction, strength, form
A scatter plot draws each pair $(x_i, y_i)$ as a point. Before computing anything, look at the picture and describe four things:
-
Direction: positive (as $x$ increases, $y$ tends to increase), negative (as $x$ increases, $y$ tends to decrease), or none.
-
Strength: how tightly the points cluster around a pattern.
-
Form: does the cloud follow a line (linear) or a curve?
-
Outliers: points far from the rest.
A survey has been carried out among 10 11th grade students on the number of weekly hours they spend doing sports ($X$) and their resting heart rate ($Y$). The results…
A survey has been carried out among 10 11th grade students on the number of weekly hours they spend doing sports ($X$) and their resting heart rate ($Y$). The results have been represented in the following scatter plot:
Given the following pairs of variables, state whether you expect to find a positive, negative or zero correlation: a) The number of miles driven by a car and the amount…
Given the following pairs of variables, state whether you expect to find a positive, negative or zero correlation:
-
a) The number of miles driven by a car and the amount of fuel left in the tank.
-
b) The high school GPA and the number of hours a student spends playing video games.
-
c) A person's age and their ID number.
-
d) A person's monthly income and the amount of money they save per month.
-
e) The average speed of an athlete in a race and the time it takes them to complete it.
A social media analyst has collected data on the number of short videos published by a group of content creators ($X$) and the number of new followers gained in a week…
A social media analyst has collected data on the number of short videos published by a group of content creators ($X$) and the number of new followers gained in a week ($Y$). The resulting scatter plot is shown below:
From the graph:
-
State whether there is a correlation between both variables and, if so, determine its sign and approximate strength.
-
Is it a functional or statistical relationship? Justify your answer.
-
Reasonably estimate which of the following values could correspond to Pearson's linear correlation coefficient $r$: $-0,92$; $0,15$; $0,94$ or $1$.
Two-way tables: joint, marginal, and conditional
Sometimes both variables are categories (grade level and device, band and sports). Then we organize the counts in a two-way table (some problems call it a contingency table or joint frequency table).
That last check gives $10$, so it matches too. In a $2 \times 2$ table, once one cell matches, all the others do.
A group of 20 teenagers has been classified according to their reading habits (Frequent reader / Non-reader) and their preference for book format (Paper / Digital). The…
A group of 20 teenagers has been classified according to their reading habits (Frequent reader / Non-reader) and their preference for book format (Paper / Digital). The data obtained is:
-
8 are frequent readers and prefer the paper format.
-
4 are frequent readers and prefer the digital format.
-
2 are non-readers but prefer the paper format.
-
6 are non-readers and prefer the digital format.
Organize this information into a joint frequency table (contingency table), including the marginal totals in the rows and columns.
From the following contingency table showing the main type of device used for studying and the student's grade level (10th Grade or 11th Grade), answer the questions…
From the following contingency table showing the main type of device used for studying and the student's grade level (10th Grade or 11th Grade), answer the questions posed:
| Grade Level / Device | Tablet | Laptop | Cell phone | Total |
|---|---|---|---|---|
| 10th Grade | 12 | 15 | 8 | 35 |
| 11th Grade | 10 | 25 | 5 | 40 |
| Total | 22 | 40 | 13 | 75 |
-
a) What is the joint absolute frequency of the students who are in 11th grade and mainly use a laptop?
-
b) Determine the marginal frequency of the students who use a tablet.
-
c) How many students in total have participated in the study?
-
d) What percentage of the total students are from 10th grade?
Mean point and covariance
For numerical data, the first summary is the mean point $(\bar x, \bar y)$, the point made of the two means. The scatter plot balances around it. (Some exercises call it the center of gravity of the distribution; it is the same point.)
Why does the sign work? Draw the lines $x = \bar x$ and $y = \bar y$. Points in the upper right and lower left have $(x_i - \bar x)(y_i - \bar y) > 0$. Points in the other two regions give negative products. Whichever kind wins decides the sign.
You should get $\sigma_x^2 = 100$, so $\sigma_x = 10$, and $\sigma_y^2 = \dfrac{49 + 9 + 0 + 1 + 81}{5} = 28$, so $\sigma_y = \sqrt{28} \approx 5.29$. We will keep using these numbers.
A note on formulas: here we divide by $n$, as the exercises do. Calculators and spreadsheets may divide by $n - 1$ for sample statistics, but $r$ and the regression line come out the same either way, because the factor cancels.
A courier company is studying the relationship between the weight of parcels in pounds ($X$) and the shipping cost in dollars ($Y$). 5 shipments have been analysed with…
A courier company is studying the relationship between the weight of parcels in pounds ($X$) and the shipping cost in dollars ($Y$). 5 shipments have been analysed with the following results:
| Weight $X$ (lb) | 1 | 2 | 3 | 5 | 9 |
|---|---|---|---|---|---|
| Cost $Y$ ($\$$) | 5 | 7 | 8 | 12 | 18 |
Calculate:
-
The center of gravity of the distribution $(\bar{x}, \bar{y})$.
-
The marginal standard deviations $\sigma_x$ and $\sigma_y$.
Given the following joint frequency table for the variables $X$ (number of children) and $Y$ (number of rooms in the dwelling): | X \ Y | 1 | 2 | 3 |…
Given the following joint frequency table for the variables $X$ (number of children) and $Y$ (number of rooms in the dwelling):
| X \ Y | 1 | 2 | 3 |
|---|---|---|---|
| 0 | 5 | 2 | 0 |
| 1 | 1 | 6 | 3 |
| 2 | 0 | 2 | 6 |
- a) Calculate the marginal distributions of $X$ and $Y$.
- b) Determine the mean and variance of the variable $Y$ conditional on $X = 1$.
- c) Calculate the covariance $S_{xy}$ and determine whether the variables are independent or show some type of association.
The correlation coefficient $r$
Why divide by $\sigma_x\sigma_y$? It cancels the units, so $r$ can be compared across problems. That is exactly what the covariance could not do.
You get $r \approx -0.56$: a moderate negative linear relationship. More hours on social media goes with somewhat lower quiz scores, but it is far from exact.
Back to Coach Rivera's data: $r = \dfrac{52}{10 \cdot \sqrt{28}} \approx 0.983$, a very strong positive linear relationship.
Match each of the following values of Pearson's linear correlation coefficient, $r$, with the graph that best represents it: 1) $r = 0,95$ 2) $r = -0,85$ 3) $r = 0,05$
Match each of the following values of Pearson's linear correlation coefficient, $r$, with the graph that best represents it:
-
1) $r = 0,95$
-
2) $r = -0,85$
-
3) $r = 0,05$
A study has been carried out on the relationship between the time an 11th grade student spends playing video games daily in hours ($X$) and their reaction time measured…
A study has been carried out on the relationship between the time an 11th grade student spends playing video games daily in hours ($X$) and their reaction time measured in milliseconds in a specific test ($Y$). The parameters obtained are:
$\bar{x} = 2,5$; $\bar{y} = 310$; $\sigma_x = 1,2$; $\sigma_y = 45$ and the covariance $\sigma_{xy} = -48,6$.
-
Calculate Pearson's linear correlation coefficient.
-
Interpret the result obtained based on the context of the problem. What happens to the reaction time as the hours of gaming increase?
A basketball club wants to predict the number of points scored by its players ($Y$) as a function of the minutes played ($X$). After analyzing the players, the following…
A basketball club wants to predict the number of points scored by its players ($Y$) as a function of the minutes played ($X$). After analyzing the players, the following data is obtained:
$\sum x_i = 240$
$\sum y_i = 120$
$\sum x_i^2 = 6500$
$\sum y_i^2 = 1800$
$\sum x_i y_i = 3200$
Number of players analyzed: $n = 10$.
-
Calculate Pearson's linear correlation coefficient.
-
If a player plays for $25$ minutes, estimate the points they will score using the corresponding regression line.
The regression line of $y$ on $x$
When the cloud is roughly linear, we want the one line that best predicts $y$ from $x$. The least-squares regression line is the line that makes the sum of the squared vertical gaps $\sum (y_i - \hat y_i)^2$ as small as possible, where $\hat y$ ("y-hat") is the value predicted by the line.
Because the line passes through $(\bar x, \bar y)$, you can always write $\bar y = m\bar x + b$. Use this to find a missing mean, or to find $b$ after you know $m$.
Slide the slope below. Whatever the slope, the line pivots around the mean point $(25, 11)$ from Coach Rivera's data. The least-squares slope is $0.52$.
Plugging in gives $\hat y = 0.52(35) - 2 = 16.2$, so about $16$ free throws out of 20. That answers the first half of the hook. We will discuss the second half (can she trust it?) soon.
A bivariate variable $(X, Y)$ is given, where $X$ represents the outside temperature in $^\circ\text{C}$ and $Y$ the electrical consumption of a shopping mall in $kW$.…
A bivariate variable $(X, Y)$ is given, where $X$ represents the outside temperature in $^\circ\text{C}$ and $Y$ the electrical consumption of a shopping mall in $kW$. We know that the mean point of the distribution is the point $(\bar{x}, \bar{y}) = (22, 450)$.
If the regression line of $Y$ on $X$ has a slope of $b = 15$:
-
a) Write the equation of the regression line of $Y$ on $X$ using the point-slope form: $y - \bar{y} = b(x - \bar{x})$.
-
b) What would be the expected consumption for a temperature of $30^\circ\text{C}$?
The regression line relating the daily usage time of a social network in minutes ($X$) to the battery level consumed in percentage ($Y$) has been obtained. The equation…
The regression line relating the daily usage time of a social network in minutes ($X$) to the battery level consumed in percentage ($Y$) has been obtained. The equation of the line is:
$y = 0,15x + 2,5$
-
a) Estimate the percentage of battery consumed if a user uses the social network for $120$ minutes.
-
b) If the linear correlation coefficient is $r = 0,92$, do you think the previous estimate is reliable? Justify your answer.
The following table shows the grades obtained by 6 students in a Math test ($X$) and in an Economics test ($Y$): | Math ($X$) | 4 | 6 | 5 | 8 | 9 | 7 | | :--- | :---: |…
The following table shows the grades obtained by 6 students in a Math test ($X$) and in an Economics test ($Y$):
| Math ($X$) | 4 | 6 | 5 | 8 | 9 | 7 |
|---|---|---|---|---|---|---|
| Economics ($Y$) | 5 | 5 | 6 | 7 | 8 | 8 |
-
Find the covariance $\sigma_{xy}$.
-
Obtain the equation of the regression line of Economics ($Y$) on Math ($X$).
-
What grade would be expected in Economics for a student who has scored a $10$ in Math?
Interpreting and using the line: interpolation vs. extrapolation
A regression line is a model, so always say what its numbers mean in context.
-
Slope $m$: for each extra unit of $x$, the predicted $y$ changes by $m$ units. For Coach Rivera's line, each extra minute of practice goes with about $0.52$ more free throws made.
-
Intercept $b$: the predicted $y$ when $x = 0$. It only means something if $x = 0$ is a realistic value for the data.
-
Direct vs. inverse: positive slope means a direct relationship, negative slope an inverse one.
-
Functional dependence: all points lie exactly on the line only if $|r| = 1$.
At $100\,°\text{F}$ you get $118$ smoothies. That is an extrapolation, so treat it with caution: sales might level off because the shop runs out of cups or customers go to the pool instead.
A streaming* platform analyzes the relationship between the mean outside temperature in degrees Fahrenheit ($X$) and the number of hours its users spend watching shows…
A streaming platform analyzes the relationship between the mean outside temperature in degrees Fahrenheit ($X$) and the number of hours its users spend watching shows on a weekend ($Y$). The regression line of $Y$ on $X$ obtained is:
$y = -0,45x + 18,2$
-
If the mean temperature on a weekend is $8 ^\circ C$, how many hours is a user estimated to spend watching shows?
-
If the coefficient of determination is $R^2 = 0,81$, how reliable is the previous prediction? Explain the meaning of this value.
It is investigated whether the price of a second-hand cell phone model ($Y$, in dollars) depends on its age ($X$, in months). The data from 100 phones show that:…
It is investigated whether the price of a second-hand cell phone model ($Y$, in dollars) depends on its age ($X$, in months). The data from 100 phones show that:
$\bar{x} = 18$ months; $\$\bar{y} = 450$; $\sigma_x = 6$; $\sigma_y = 80$ and $r = -0,88$.
-
Calculate the covariance $\sigma_{xy}$.
-
Find the regression line of $Y$ on $X$.
-
What is the physical meaning of the slope of the line in this context?
-
Estimate the price of a phone that is a year and a half old. Is this estimate very reliable? Why?
In a study on healthy habits, the distance cycled per week in miles ($X$) and the expenditure on bicycle maintenance in dollars ($Y$) are measured. It is known that the…
In a study on healthy habits, the distance cycled per week in miles ($X$) and the expenditure on bicycle maintenance in dollars ($Y$) are measured. It is known that the regression line of $Y$ on $X$ is $y = 0,12x + 15$ and that the correlation coefficient is $r = 0,85$.
-
Justify whether the following statements are true or false:
-
a) There is a functional dependence between the miles cycled and the expenditure.
-
b) The relationship between the variables is direct.
-
c) If a cyclist does not cycle any miles ($x=0$), the model estimates an expenditure of $\$15$.
-
-
If the mean annual expenditure is $\$\bar{y} = 33$, what is the mean number of miles cycled weekly $\bar{x}$?
The two regression lines
There are actually two regression lines. One predicts $y$ from $x$. The other predicts $x$ from $y$.
Why is the product $r^2$? Multiply the two slopes: the covariance gets squared and the variances end up in the denominator, giving $\dfrac{\sigma_{xy}^2}{\sigma_x^2\sigma_y^2} = r^2$. The lines coincide only when $|r| = 1$ (then $m_1m_2 = 1$). The weaker the relationship, the further apart the lines spread, and the smaller the product.
In a study on the performance of a new solar panel, the two regression lines relating the captured light intensity ($X$, in hundreds of lux) and the electrical energy…
In a study on the performance of a new solar panel, the two regression lines relating the captured light intensity ($X$, in hundreds of lux) and the electrical energy generated ($Y$, in watts) have been determined. The equations obtained are:
Regression line of $Y$ on $X$: $y = 0,84x + 12,2$
Regression line of $X$ on $Y$: $x = 0,95y - 7,4$
Calculate, justifying each step:
- a) The center of gravity of the distribution $(\bar{x}, \bar{y})$.
- b) Pearson's linear correlation coefficient ($r$) and interpret its sign and strength.
- c) What percentage of the variance of the generated energy is explained by the light intensity through the linear model?
The regression lines of a bivariate distribution are $4x - 5y + 12 = 0$ and $9x - 10y + 15 = 0$. a) Identify which of them is the regression line of $Y$ on $X$ and which…
The regression lines of a bivariate distribution are $4x - 5y + 12 = 0$ and $9x - 10y + 15 = 0$.
- a) Identify which of them is the regression line of $Y$ on $X$ and which is the regression line of $X$ on $Y$. Justify your answer using the fact that the product of their slopes must be equal to $r^2$ and that $0 \leq r^2 \leq 1$.
- b) Calculate the linear correlation coefficient $r$.
- c) Determine the mean values $\bar{x}$ and $\bar{y}$.
A team of data analysts from a well-known microblogging network is studying the relationship between the number of words in a post ($X$) and the number of "likes"…
A team of data analysts from a well-known microblogging network is studying the relationship between the number of words in a post ($X$) and the number of "likes" received ($Y$). After analyzing a random sample, the following parameters are known:
$\bar{x} = 45$ words
$\sigma_x = 12$
$S_{xy} = 156$
Regression line of $Y$ on $X$: $y = 1,083x + 11,25$
Knowing that the coefficient of determination is $R^2 = 0,64$, determine:
- a) The standard deviation of the variable $Y$.
- b) The equation of the regression line of $X$ on $Y$.
- c) If a post has a number of "likes" that is twice the mean $\bar{y}$, what would be the prediction for its number of words?
How good is the line? $r^2$ and residuals
A residual is the vertical gap between a data point and the line: $y_i - \hat y_i$. Small residuals mean a good fit.
Why is it so? The residuals have mean $0$, and their variance is the part of $\sigma_y^2$ the line fails to explain, namely $(1 - r^2)\sigma_y^2$. If $r = \pm1$ there is no leftover and $\sigma_{res} = 0$.
Predictions are typically off by about one free throw.
A sociological study relates the digitalization index of a household ($X$, scale 1-10) to the annual savings ($Y$, in thousands of dollars). It is known that the…
A sociological study relates the digitalization index of a household ($X$, scale 1-10) to the annual savings ($Y$, in thousands of dollars). It is known that the regression line of $Y$ on $X$ is $y = -1,2x + 15$ and that the coefficient of determination is $R^2 = 0,81$.
- a) If the variance of the digitalization indices is $\sigma_x^2 = 4$, what is the variance of the savings variable $\sigma_y^2$?
- b) If a household has a digitalization index of 8, estimate its annual savings and calculate the standard error of the estimate (residual standard deviation), defined as $\sigma_{res} = \sigma_y \sqrt{1 - R^2}$.
The relationship between the altitude of a town ($X$, in feet) and the atmospheric pressure ($Y$, in mb) is investigated. The data from 5 towns show an almost perfect…
The relationship between the altitude of a town ($X$, in feet) and the atmospheric pressure ($Y$, in mb) is investigated. The data from 5 towns show an almost perfect correlation ($r = -0,99$). The standard deviations are $\sigma_x = 400$ and $\sigma_y = 45$.
If the mean pressure is $950$ mb at a mean altitude of $600$ feet:
- a) Find the equation of the regression line to predict the pressure as a function of the altitude.
- b) At what altitude would a pressure of $1013$ mb be expected? Evaluate the reliability of this answer considering whether it is an interpolation or an extrapolation.
- c) If it were discovered that the altimeters had a systematic error and all actual altitudes are $50$ feet higher than recorded, how would this affect the slope of the regression line?
Changing units and the effect of outliers
What happens to $\sigma_{xy}$ and $r$ if you convert units, or add a new point?
Why? Adding $b$ shifts every value and the mean by $b$, so deviations do not change: $x_i' - \bar x' = a(x_i - \bar x)$. Multiplying by $a$ stretches all deviations by $a$. In $r$, the factors $|a|$ and $|c|$ cancel out of the quotient, and only the sign survives.
The answer is $r' = -0.8$ (here $a = -1$ so $ac < 0$). The covariance changes with the units, but $|r|$ does not. That is why $r$ is the right measure of strength, and why a predictive model built on the new units is just as reliable as before.
The relationship between the weekly study time ($X$, in hours) and the average grade ($Y$) of a group of students is analyzed. It is known that the covariance is…
The relationship between the weekly study time ($X$, in hours) and the average grade ($Y$) of a group of students is analyzed. It is known that the covariance is $S_{xy} = 4,5$ and that the correlation coefficient is $r = 0,75$. If all students are rewarded with 1 extra point on their average grade and, furthermore, they decide to study twice as many hours the following week:
- a) Prove mathematically how the new covariance $S'_{xy}$ and the new correlation coefficient $r'$ vary.
- b) Would the reliability of the linear model's predictions change after these changes? Justify your answer based on the properties of bivariate parameters.
In an e-sports competition, the age of the players ($X$) and their reaction time to a visual stimulus ($Y$, in milliseconds) are measured. The scatter graph suggests an…
In an e-sports competition, the age of the players ($X$) and their reaction time to a visual stimulus ($Y$, in milliseconds) are measured. The scatter graph suggests an inverse linear relationship. The following simplified information is available:
$\sum x_i = 200$, $\sum y_i = 2500$, $\sum x_i^2 = 4150$, $\sum y_i^2 = 628000$, $\sum x_i y_i = 49200$ (for $n = 10$ players).
- a) Calculate the linear correlation coefficient.
- b) A 32-year-old player (outside the range of the original sample, which was 18 to 25 years) enters the study. If their reaction time were 280 ms, how would this new point affect the gradient of the regression line of $Y$ on $X$? Justify your answer without needing to recalculate everything.