Regression Line of y on x
Ximena collects data from her high school classmates: hours of study per week and score on the admission exam. The scatter plot rises in a more or less straight line. How can she draw the line that best summarizes that cloud of points to predict the score of someone who studies 9 hours? And, if she wanted to predict the hours from the score, would she use the same line? We solve this in this section.
The Regression Line of Y on X
If the scatter plot has a linear trend, we look for the line that stays "as close as possible" to all the points. For each data point, the residual is the vertical distance between the observed point and the line. The best line is the one that makes the sum of the squared residuals as small as possible (the least squares method).
It is the usual point-slope equation, $y-y_1=m(x-x_1)$, with the point $(\bar x,\bar y)$ and the slope $\sigma_{xy}/\sigma_x^2$. The slope has the covariance on top (it tells whether the cloud rises or falls) and the variance of $X$ on the bottom (it tells how much $X$ varies).
If you are given the correlation instead of the covariance
Since $r=\dfrac{\sigma_{xy}}{\sigma_x\sigma_y}$, we have $\sigma_{xy}=r\,\sigma_x\sigma_y$. Substituting into the slope:
$$m_{yx}=\frac{r\,\sigma_x\sigma_y}{\sigma_x^2}=r\,\frac{\sigma_y}{\sigma_x}$$If you are given the sums
When the problem gives $N$, $\sum x_i$, $\sum y_i$, $\sum x_i^2$ and $\sum x_iy_i$, you first find the means, the variance and the covariance:
$$\bar x=\frac{\sum x_i}{N},\quad \sigma_x^2=\frac{\sum x_i^2}{N}-\bar x^2,\quad \sigma_{xy}=\frac{\sum x_iy_i}{N}-\bar x\,\bar y$$Given the statistical parameters of a bivariate distribution: mean of $X$ equal to $4$, mean of $Y$ equal to $6$, variance of $X$ equal to $1.5$, and covariance equal to…
Given the statistical parameters of a bivariate distribution: mean of $X$ equal to $4$, mean of $Y$ equal to $6$, variance of $X$ equal to $1.5$, and covariance equal to $3$. Determine the equation of the regression line of $Y$ on $X$.
In a bivariate distribution, the standard deviations are known to be $\sigma_x = 2$ and $\sigma_y = 5$, and the linear correlation coefficient is $r = -0.8$. Calculate…
In a bivariate distribution, the standard deviations are known to be $\sigma_x = 2$ and $\sigma_y = 5$, and the linear correlation coefficient is $r = -0.8$. Calculate the equation of the regression line of $Y$ on $X$ knowing that the mean point of the distribution is $(10, 20)$.
A bivariate distribution has as its data matrix the following summations: $\sum x_i = 20$, $\sum y_i = 40$, $\sum x_i^2 = 120$, $\sum x_i y_i = 180$, and the total…
A bivariate distribution has as its data matrix the following summations: $\sum x_i = 20$, $\sum y_i = 40$, $\sum x_i^2 = 120$, $\sum x_i y_i = 180$, and the total number of observations is $N = 10$. Calculate the covariance and the variance of $X$, and write the equation of the regression line of $Y$ on $X$.
Finding the Line from a Table
If you are given the pairs of values, find $\bar x$, $\bar y$, $\sigma_x^2$ and $\sigma_{xy}$ and apply the formula. You can shorten it with $\sigma_{xy}=\dfrac{\sum (x_i-\bar x)(y_i-\bar y)}{N}$.
When a value in the table is missing
A useful trick: since $\sum(x_i-\bar x)=0$, the slope is also
$$m_{yx}=\frac{\sum (x_i-\bar x)\,y_i}{\sum (x_i-\bar x)^2}$$This way the unknown value of $Y$ appears in only one sum.
Given the following table with three pairs of values of a bivariate distribution: $X$: $1 \quad 2 \quad 3$ $Y$: $2 \quad 4 \quad 5$ Calculate the gradient of the…
Given the following table with three pairs of values of a bivariate distribution:
-
$X$: $1 \quad 2 \quad 3$
-
$Y$: $2 \quad 4 \quad 5$
Calculate the gradient of the regression line of $Y$ on $X$.
A digital marketing company has collected data on investment in social media campaigns ($X$, in thousands of pesos) and sales revenue generated ($Y$, in tens of…
A digital marketing company has collected data on investment in social media campaigns ($X$, in thousands of pesos) and sales revenue generated ($Y$, in tens of thousands of pesos) over $5$ consecutive months:
-
$X$: $50 \quad 60 \quad 70 \quad 80 \quad 90$
-
$Y$: $240 \quad 280 \quad 300 \quad 360 \quad 380$
Calculate the regression line of revenue as a function of investment and estimate the expected revenue if $100\,000$ pesos is invested.
A set of bivariate data $(x_i, y_i)$ is available, where the variable $X$ takes the values $1, 2, 3, 4$ and the variable $Y$ takes the values $2, k, 5, 8$. Given that…
A set of bivariate data $(x_i, y_i)$ is available, where the variable $X$ takes the values $1, 2, 3, 4$ and the variable $Y$ takes the values $2, k, 5, 8$. Given that the gradient of the regression line of $Y$ on $X$ is $m_{yx} = 1.9$, set up an equation and determine the exact value of the parameter $k$.
Interpreting the Line and Estimating with It
In $y=mx+b$:
-
The slope $m$ tells how much $Y$ changes, on average, for each unit that $X$ increases. If $m<0$, $Y$ goes down as $X$ grows.
-
The y-intercept $b$ is the estimated value of $Y$ when $X=0$. It only makes sense if $X=0$ is reasonable in the problem.
-
To estimate, substitute the value of $X$ into the line.
If you are given the line and only one mean, the center of gravity gives you the other: since the line passes through $(\bar x,\bar y)$, we have $\bar y=m\bar x+b$.
Reading the line from a graph
If the line is drawn and two points are marked on it, $(x_1,y_1)$ and $(x_2,y_2)$, the slope is $m=\dfrac{y_2-y_1}{x_2-x_1}$ and then you use point-slope. The y-intercept is where the line crosses the $Y$ axis.
In a study on the relationship between the amount of fertilizer applied ($X$, in kg/ha) and the yield obtained ($Y$, in metric tons/ha), the regression line of $Y$ on…
In a study on the relationship between the amount of fertilizer applied ($X$, in kg/ha) and the yield obtained ($Y$, in metric tons/ha), the regression line of $Y$ on $X$ was calculated to be $y = 0.4x + 2.1$. Estimate the yield that will be obtained if $15$ kg/ha of fertilizer is applied.
The regression line of $Y$ on $X$ of a bivariate distribution is $y = 1.2x - 0.5$. If we know that the marginal mean of the variable $X$ is $\bar{x} = 3$, what is the…
The regression line of $Y$ on $X$ of a bivariate distribution is $y = 1.2x - 0.5$. If we know that the marginal mean of the variable $X$ is $\bar{x} = 3$, what is the exact value of the marginal mean $\bar{y}$?
The variables $X$ and $Y$ represent, respectively, the number of weekly training hours and the time in seconds to run $100$ meters for a group of athletes. If the…
The variables $X$ and $Y$ represent, respectively, the number of weekly training hours and the time in seconds to run $100$ meters for a group of athletes. If the regression line of $Y$ on $X$ is $y = -0.5x + 14$, interpret the physical meaning of the slope of this line in the context of the problem.
The following scatter plot shows the relationship between the years of experience of a group of technicians and their monthly salary in hundreds of pesos, together with…
The following scatter plot shows the relationship between the years of experience of a group of technicians and their monthly salary in hundreds of pesos, together with the regression line of $Y$ on $X$. From the graph, visually estimate the expected salary for a technician with $6$ years of experience and determine the y-intercept of the line.
The following scatter plot shows the relationship between the depth of a mineral seam ($X$, in meters) and the extraction cost per metric ton ($Y$, in thousands of…
The following scatter plot shows the relationship between the depth of a mineral seam ($X$, in meters) and the extraction cost per metric ton ($Y$, in thousands of pesos). The line plotted is the regression line of $Y$ on $X$. From the graph, obtain the coordinates of the two marked points on the line, calculate its exact equation, and estimate the extraction cost for a depth of $120$ meters.
Two Lines and Their Relation to r
There is also the line of $X$ on $Y$, which minimizes the errors when predicting $X$:
$$x-\bar x = \frac{\sigma_{xy}}{\sigma_y^2}(y-\bar y)$$Its slope, measured as $\Delta x/\Delta y$, is $m_{xy}=\sigma_{xy}/\sigma_y^2$.
The reason: both lines are built with the same numerator $\sigma_{xy}$, and both pass through the center because they are built from deviations from the means.
The regression lines of a bivariate distribution are given by the equations $y = 0.5x + 4$ and $x = 1.6y - 2$. Determine the coordinates of the mean point of the…
The regression lines of a bivariate distribution are given by the equations $y = 0.5x + 4$ and $x = 1.6y - 2$. Determine the coordinates of the mean point of the distribution and the linear correlation coefficient $r$, giving reasons for its sign.
The relationship between the amount of impurities in a metal alloy ($X$, in $\%$) and its tensile strength ($Y$, in MPa) has been studied. After analysing $20$ samples,…
The relationship between the amount of impurities in a metal alloy ($X$, in $\%$) and its tensile strength ($Y$, in MPa) has been studied. After analysing $20$ samples, $\bar{x} = 2.1$, $\bar{y} = 450$, and a regression line of $Y$ on $X$ given by $y = -15x + 481.5$ were obtained. Calculate the covariance of the distribution if the variance of the impurities is known to be $\sigma_x^2 = 0.8$.
The following graph shows a scatter plot relating the speed of a vehicle ($X$, in km/h) to the braking distance ($Y$, in meters). The two regression lines have been…
The following graph shows a scatter plot relating the speed of a vehicle ($X$, in km/h) to the braking distance ($Y$, in meters). The two regression lines have been drawn: Line A (in blue) and Line B (in green). Identify with reasoning, based on the properties of the slopes, which of the two lines is the regression line of $Y$ on $X$ and find its equation, knowing that both intersect at the point $(80, 40)$ and that the blue line passes through $(100, 56)$.
If |r| = 1, the Two Lines Coincide
The slope of $Y$ on $X$ is $m_{yx}=\dfrac{\sigma_{xy}}{\sigma_x^2}$. To compare, we write the line of $X$ on $Y$ in the form $y=mx+n$: its slope in the plane is $\dfrac{1}{m_{xy}}=\dfrac{\sigma_y^2}{\sigma_{xy}}$.
Standardized variables
If you define $Z_x=\dfrac{X-\bar x}{\sigma_x}$ and $Z_y=\dfrac{Y-\bar y}{\sigma_y}$, both have mean $0$ and standard deviation $1$. Then $\sigma_{z_xz_y}=\dfrac{\sigma_{xy}}{\sigma_x\sigma_y}=r$ and $\sigma_{z_x}^2=1$. The line of $Z_y$ on $Z_x$ is
$$z_y-0=\frac{r}{1}(z_x-0)\ \Rightarrow\ z_y=r\,z_x$$It passes through the origin and its slope is exactly $r$.
Prove analytically that if the variables $X$ and $Y$ have a perfect linear correlation ($r = 1$ or $r = -1$), then the regression line of $Y$ on $X$ coincides exactly…
Prove analytically that if the variables $X$ and $Y$ have a perfect linear correlation ($r = 1$ or $r = -1$), then the regression line of $Y$ on $X$ coincides exactly with the regression line of $X$ on $Y$. Briefly explain what geometric implication this has for the angle formed by both lines in the plane.
In a pharmaceutical study, the concentration of an active ingredient in blood ($Y$, in mg/L) is modelled as a function of the administered dose ($X$, in mg). If we…
In a pharmaceutical study, the concentration of an active ingredient in blood ($Y$, in mg/L) is modelled as a function of the administered dose ($X$, in mg). If we change variables by standardising both variables, that is, we define $Z_x = \frac{X - \bar{x}}{\sigma_x}$ and $Z_y = \frac{Y - \bar{y}}{\sigma_y}$, prove algebraically that the regression line of $Z_y$ on $Z_x$ passes through the origin and that its gradient is exactly the linear correlation coefficient $r$.
How Good Is the Line: R² and Residuals
The coefficient of determination is $R^2=r^2$. It indicates the fraction of the variability of $Y$ that the line explains; the rest, $1-r^2$, remains unexplained. With $N$ data points, the sum of the squared residuals is
$$\sum (y_i-\hat y_i)^2=N\,\sigma_y^2\,(1-r^2)$$And if you know the slope, $m_{yx}=r\dfrac{\sigma_y}{\sigma_x}$ or, equivalently, $m_{yx}^2=r^2\dfrac{\sigma_y^2}{\sigma_x^2}$. With $r^2$ and the variances you can solve for $|m_{yx}|$. Also, if you know $m_{yx}$ and $\sigma_x^2$, then $\sigma_{xy}=m_{yx}\sigma_x^2$.
At a meteorological center, atmospheric pressure ($X$, in hPa) is related to the probability of precipitation ($Y$, in $\%$). Knowing that the mean point of the…
At a meteorological center, atmospheric pressure ($X$, in hPa) is related to the probability of precipitation ($Y$, in $\%$). Knowing that the mean point of the distribution is the point $(1013, 40)$, the standard deviation of the pressure is $5$ hPa, that of the probability is $20\%$, and the linear correlation coefficient is $-0.85$, find the equation of the regression line that allows the probability of rain to be estimated given the atmospheric pressure.
In a macroeconomic analysis, the inflation rate ($X$, in $\%$) is related to the unemployment rate ($Y$, in $\%$). The regression line of $Y$ on $X$ is $y = -0.6x + 8$.…
In a macroeconomic analysis, the inflation rate ($X$, in $\%$) is related to the unemployment rate ($Y$, in $\%$). The regression line of $Y$ on $X$ is $y = -0.6x + 8$. If the marginal variance of inflation is $\sigma_x^2 = 4$ and the marginal variance of unemployment is $\sigma_y^2 = 2.25$, determine the linear correlation coefficient $r$ and the percentage of the variability of unemployment that is explained by the linear regression model.
In a study on the performance of solar panels, the variable $X$ represents solar irradiance ($W/m^2$) and the variable $Y$ the electric power generated ($W$). It is…
In a study on the performance of solar panels, the variable $X$ represents solar irradiance ($W/m^2$) and the variable $Y$ the electric power generated ($W$). It is known that the regression line of $Y$ on $X$ is given by $y = 0.18x + 5$, and the regression line of $X$ on $Y$ by $x = 5y - 10$. Calculate the marginal means $\bar{x}$ and $\bar{y}$, and find the exact percentage of the variability of the electric power that is explained by the solar irradiance.
A civil engineer is studying the relationship between the load supported by a beam ($X$, in metric tons) and its maximum deformation ($Y$, in millimeters). Some data…
A civil engineer is studying the relationship between the load supported by a beam ($X$, in metric tons) and its maximum deformation ($Y$, in millimeters). Some data from the original report has been lost, but it is known that the mean load is $\bar{x} = 15$, the variance of the load is $\sigma_x^2 = 16$, the covariance is $\sigma_{xy} = 18$, the correlation coefficient is $r = 0.9$ and the y-intercept of the regression line of $Y$ on $X$ is $b = 2$. Calculate the mean deformation $\bar{y}$, the standard deviation $\sigma_y$ and the equation of the regression line of $Y$ on $X$.
A bivariate distribution of $100$ pairs of values has been studied. It is known that the sum of squared residuals (the sum of squared differences between the actual…
A bivariate distribution of $100$ pairs of values has been studied. It is known that the sum of squared residuals (the sum of squared differences between the actual observed values $y_i$ and those estimated by the line $\hat{y}_i$) is equal to $150$. If the marginal variance of the variable $Y$ is $\sigma_y^2 = 4$, find the coefficient of determination $R^2$ using the relationship $\sum (y_i - \hat{y}_i)^2 = N \times \sigma_y^2 (1 - r^2)$, and calculate the absolute value of the gradient of the regression line of $Y$ on $X$ knowing that the marginal variance of $X$ is $\sigma_x^2 = 9$.
Changes of Unit or Origin
If you transform the data with $X'=X+a$ and $Y'=cY$, the scatter plot is shifted and stretched, and the line changes with it. The safest way is to substitute.
Adding a constant to $X$ does not change the slope; multiplying $Y$ by $c$ multiplies the slope by $c$.
The regression line of $Y$ on $X$ of a bivariate distribution is $y = 3x - 2$. If $4$ units are added to all observed values of the variable $X$ and all values of the…
The regression line of $Y$ on $X$ of a bivariate distribution is $y = 3x - 2$. If $4$ units are added to all observed values of the variable $X$ and all values of the variable $Y$ are multiplied by $2$, what will be the new equation of the regression line of $Y$ on $X$?
Predicting with Caution: Interpolation and Extrapolation
Two rules before trusting an estimate:
-
Interpolating is estimating with a value of $X$ inside the range of the data. It is usually reasonable.
-
Extrapolating is estimating outside the range. Nothing guarantees that the trend will remain linear, and it can give absurd results.
In addition, reliability depends on $|r|$ (or $R^2$): the closer to $1$, the better the prediction.
In a thermodynamics experiment, the temperature of a liquid ($Y$, in ºC) is measured as a function of elapsed time ($X$, in minutes). The regression line obtained is…
In a thermodynamics experiment, the temperature of a liquid ($Y$, in ºC) is measured as a function of elapsed time ($X$, in minutes). The regression line obtained is $y = -1.5x + 85$. Knowing that the coefficient of determination is $R^2 = 0.95$ and that the total duration of the experiment was $30$ minutes, is it reliable to estimate the temperature at $45$ minutes using this line? Justify your answer based on the concepts of interpolation and extrapolation.
In a biomechanics study, the relationship between femur length ($X$, in cm) and height of an individual ($Y$, in cm) is analysed. The following statistical summaries are…
In a biomechanics study, the relationship between femur length ($X$, in cm) and height of an individual ($Y$, in cm) is analysed. The following statistical summaries are available for a sample of $50$ people:
$\bar{x} = 45$, $\bar{y} = 170$, $\sigma_x = 3.2$, $\sigma_y = 8.5$, $r = 0.92$.
Find the regression line of $Y$ on $X$ and estimate the height of a person whose femur measures $48$ cm. Briefly justify the reliability of this estimate.
Back to the beginning: Ximena finds $\bar x$, $\bar y$, $\sigma_x^2$ and $\sigma_{xy}$, writes the line of score on hours, and substitutes $x=9$ only if 9 is within the range of her data. To predict the hours from the score she needs the line of $X$ on $Y$, which is a different one.