Simple linear regression is one of the most commonly used statistical analysis tools in the field. In order to understand the ideas that make up simple linear regression, try to solve the following problem by reviewing what you already know. Read through this guide if you’re unsure about or are encountering this concept for the first time.
Problem 4
There are two variables that need to be studied: weight loss and days spent exercising one month. You are given a data set in which individuals have been asked the number of days they exercise for more than half an hour in one month. What kind of regression model can you use here? What are the results of this regression given the data set below. Interpret the model’s estimators.
| Exercise Days | Weight Loss (in kg) |
| 0 | 4 |
| 4 | 1 |
| 8 | 1.5 |
| 12 | 2 |
| 16 | 4 |
| 20 | 5 |
| 24 | 2 |
What is SLR-OLS?
Simple linear regression, or SLR, is a regression analysis that involves only one explanatory and one response variable. The definitions of these two variables can be found below.
| Explanatory or Independent Variable | The variable manipulated by the investigator, the one that will explain the variance in the dependent variable | The explanatory variable of time spent studying on an exam |
| Response or Dependent Variable | The variable that responds to the manipulation of the independent variable, the one that responds to changes in the independent variable | With the explanatory variable of time spent studying, the response variable of the exam score |
SLR analysis yields a regression model between two variables that can be used to make predictions, or estimations, about observations inside and outside the range of the data used to make that regression model.
OLS Explained
One important component of conducting a simple linear regression analysis is ordinary least squares, otherwise known as OLS. OLS is a method for estimating the unknown parameters of a linear regression model. In every regression analysis, there are independent and dependent variables that are being studied from a dataset. In the majority of cases, this dataset is a list of observations for the dependent and independent variables from individuals that make up a sample. This sample is taken from the population because it is extremely rare to have data from the entire population. Recall that the population is made up of all the individuals, objects or ideas that you want to study. The measurements of a population are called parameters, while the measurements of a sample are called statistics. When it comes to linear regression, there are many different ways we can try to estimate these population parameters with our sample statistics. The reason why OLS is so vital to linear regression is because it minimizes the difference between the observations in the dataset and the predicted responses of the linear regression model. The formula for the regression line, estimators and residuals of OLS can be found in the table below. 
| Element | Formula | Description |
| Slope |  | Slope of the regression model |
| y-intercept |  | The y-intercept of the regression model |
| Model |  | The regression model |
| Residual |  | Difference between observed y and predicted y |
Meaning of OLS
As you can see, the linear regression line includes residuals, which are the estimates of the error. The error is the same concept as the residuals, except for the actual population, making the residual a statistic, or estimate of the true population parameter. Rearranging the formula, we can see that the residual is simply [ y_{i} = alpha + beta x_{i} + u_{i} ; rightarrow ; u_{i} = y_{i} - alpha - beta x_{i} ] 
Interpretation of Regression Line
To interpret the linear regression line, you have to understand what the estimators within the formula mean. The slope of the regression model is called the regression coefficient while the y-intercept is simply the y-intercept. Take a look at the table below for an interpretation of each estimator.
| Estimator | Value | Interpretation |
| Slope | Weight = 0 + 100(Height) | The weight here is in kg while the height here is in meters. This means that an increase in one meter will result in an increase of 100 kg to height. |
| y-intercept | Weight = 0 + 100(Height) | Recall that the y-intercept is the point on the y-axis for which the x value is zero. This means that, in this model, if the height were zero, the weight would be 0 kg. This makes sense, as you can’t weigh something when you don’t have any height. |
| Slope | Sales = 40 - 5(Price) | Here, the price is in pounds and the sales are per unit. Because the slope is negative, this means that for an increase of 1 pound in the price there would be a decrease of 5 units sold. |
| y-intercept | Sales = 40 - 5(Price) | This is an example of a y-intercept that is nonsensical. While you can interpret it as being the sales for when the price is 0, it wouldn’t make any sense for the price to be 0. This is why, before interpreting the y-intercept, think about the normal range for the x variable. |
OLS by Hand
| Observations | Exercise Days | Weight Loss |   |   |  |  |  |
| 1 | 4 | 4 | -12.0 | 1.2 | -14.6 | 144.0 | 1.5 |
| 2 | 8 | 1 | -8.0 | -1.8 | 14.3 | 64.0 | 3.2 |
| 3 | 12 | 1.5 | -4.0 | -1.3 | 5.1 | 16.0 | 1.7 |
| 4 | 16 | 2 | 0.0 | -0.8 | 0.0 | 0.0 | 0.6 |
| 5 | 20 | 4 | 4.0 | 1.2 | 4.9 | 16.0 | 1.5 |
| 6 | 24 | 5 | 8.0 | 2.2 | 17.7 | 64.0 | 4.9 |
| 7 | 28 | 2 | 12.0 | -0.8 | -9.4 | 144.0 | 0.6 |
| Mean | 16.0 | 2.8 | Total | 18.0 | 448.0 | 13.9 |
Summarise with AI:
Did you like this article? Rate it!
Loading...
