When is simple linear regression used

In statistics, a simple linear regression model uses a single variable to predict the result of the other variable. You may confuse it with linear regression but let us make it clear through this blog that they both are different. This blog carries the basics concept of simple linear regression such as the definition of linear regression, types of linear regression, the definition of simple linear regression, its assumptions, how to perform linear regression, limits of linear regression, examples of simple linear regression, and many more.

In layman terms, we can define linear regression as it is used for learning the linear relationship between the target and one or more forecasters , and it is probably one of the most popular and well inferential algorithms in statistics. Linear regression endeavours to demonstrate the connection between two variables by fitting a linear equation to observed information. One variable is viewed as an explanatory variable, and the other is viewed as a dependent variable.

For instance, a modeller should relate loads of people to their heights utilizing a linear regression model. Thus, it is a crucial and generally used type of foreseeing examination. Normally, linear regression is divided into two types: Multiple linear regression and Simple linear regression. So, for better clearance, we will discuss these types in detail.

In this type of linear regression, we always attempt to discover the relationship between two or more independent variables or inputs and the corresponding dependent variable or output and the independent variables can be either continuous or categorical.

This linear regression analysis is very helpful in several ways like it helps in foreseeing trends, future values, and moreover predict the impacts of changes. In simple linear regression, we aim to reveal the relationship between a single independent variable or you can say input, and a corresponding dependent variable or output. We can also discuss this in the form of a graph and here is a sample simple linear regression model graph.

Thus, in this whole blog, you will get to learn so many new things about simple linear regression in detail. Simple Linear Regression graph, Image Source. It can be described as a method of statistical analysis that can be used to study the relationship between two quantitative variables. Read also: Statistical Data Analysis.

Primarily, there are two things which can be found out by using the method of simple linear regression:. Strength of the relationship between the given duo of variables. For example, the relationship between global warming and the melting of glaciers. How much the value of the dependent variable is at a given value of the independent variable. For example, the amount of melting of a glacier at a certain level of global warming or temperature. Regression models are used for the elaborated explanation of the relationship between two given variables.

There are certain types of regression models like logistic regression models , nonlinear regression models, and linear regression models. The linear regression model fits a straight line into the summarized data to establish the relationship between two variables. Also read: What is Statistics? Types, Variance, and Bayesian Statistics. To conduct a simple linear regression, one has to make certain assumptions about the data. This is because it is a parametric test. The assumptions used while performing a simple linear regression are as follows:.

We are often interested in understanding the relationship among several variables. Scatterplots and scatterplot matrices can be used to explore potential relationships between pairs of variables. For example, if the relationship is curvilinear, the correlation might be near zero.

You can use regression to develop a more formal understanding of relationships between variables. In regression, and in statistical modeling in general, we want to model the relationship between an output variable, or a response, and one or more input variables, or factors.

Depending on the context, output variables might also be referred to as dependent variables, outcomes , or simply Y variables, and input variables might be referred to as explanatory variables , effects , predictors or X variables.

We can use regression, and the results of regression modeling, to determine which variables have an effect on the response or help explain the response. This is known as explanatory modeling. We can also use regression to predict the values of a response variable based on the values of the important predictors.

This is generally referred to as predictive modeling. Or, we can use regression models for optimization, to determine settings of factors to optimize a response. Our optimization goal might be to find settings that lead to a maximum response or to a minimum response. Or the goal might be to hit a target within an acceptable window. We might also use the knowledge gained through regression modeling to design an experiment that will refine our process knowledge and drive further improvement.

We have 50 parts with various inside diameters, outside diameters, and widths. Parts are cleaned using one of three container types. Cleanliness is a measure of the particulates on the parts. This is measured before and after running the parts through the cleaning process. The response of interest is Removal. This is the difference between pre-cleaning and post-cleaning measures.

The relationship we develop linking the predictors to the response is a statistical model or, more specifically, a regression model.

The term regression describes a general collection of techniques used in modeling a response as a function of predictors.

The only regression models that we'll consider in this discussion are linear models. Because visual examinations are largely subjective, we need a more precise and objective measure to define the correlation between the two variables. To quantify the strength and direction of the relationship between two variables, we use the linear correlation coefficient:.

The sample size is n. This statistic numerically describes how strong the straight-line or linear relationship is between the two variables and the direction, positive or negative. Correlation is not causation!!! Just because two variables are correlated does not mean that one variable causes another variable to change.

Examine these next two scatterplots. Plot 1 shows little linear relationship between x and y variables. Plot 2 shows a strong non-linear relationship. Ignoring the scatterplot could result in a serious mistake when describing the relationship between two variables. When you investigate the relationship between two variables, always begin with a scatterplot.

This graph allows you to look for patterns both linear and non-linear. Once you have established that a linear relationship exists, you can take the next step in model building. Once we have identified two variables that are correlated, we would like to model this relationship.

We want to use one variable as a predictor or explanatory variable to explain the other variable, the response or dependent variable. In order to do this, we need a good relationship between our two variables.

The model can then be used to predict changes in our response variable. A strong relationship between the predictor variable and the response variable leads to a good model. A simple linear regression model is a mathematical equation that allows us to predict a response for a given predictor value. The slope describes the change in y for each one unit change in x. A hydrologist creates a model to predict the volume flow for a stream at a bridge crossing with a predictor variable of daily rainfall in inches.

The y-intercept of 1. The slope tells us that if it rained one inch that day the flow in the stream would increase by an additional 29 gal. If it rained 2 inches that day, the flow would increase by an additional 58 gal.

The Least-Squares Regression Line shortcut equations. An alternate computational equation for slope is:. This simple model is the line of best fit for our sample data. The regression line does not go through every point; instead it balances the difference between all data points and the straight-line model. The difference between the observed data value and the predicted value the value on the straight line is the error or residual.

The criterion to determine the line that best describes the relation between two variables is based on the residuals. For example, if you wanted to predict the chest girth of a black bear given its weight, you could use the following model.

But a measured bear chest girth observed value for a bear that weighed lb. A negative residual indicates that the model is over-predicting. A positive residual indicates that the model is under-predicting.

In this instance, the model over-predicted the chest girth of a bear that actually weighed lb. This random error residual takes into account all unpredictable and unknown factors that are not included in the model. An ordinary least squares regression line minimizes the sum of the squared errors between the observed and predicted values to create a best fitting line.

The differences between the observed and predicted values are squared to deal with the positive and negative differences. After we fit our regression line compute b 0 and b 1 , we usually wish to know how well the model fits our data. To determine this, we need to think back to the idea of analysis of variance. In ANOVA, we partitioned the variation using sums of squares so we could identify a treatment effect opposed to random variation that occurred in our data.

The idea is the same for regression. We want to partition the total variability into two parts: the variation due to the regression and the variation due to random error. And we are again going to compute sums of squares to help us do this. Suppose the total variability in the sample measurements about the sample mean is denoted by , called the sums of squares of total variability about the mean SST.

The squared difference between the predicted value and the sample mean is denoted by , called the sums of squares due to regression SSR.

The SSR represents the variability explained by the regression line. Finally, the variability which cannot be explained by the regression line is called the sums of squares due to error SSE and is denoted by. SSE is actually the squared residual. The sums of squares and mean sums of squares just like ANOVA are typically presented in the regression analysis of variance table.

The ratio of the mean sums of squares for the regression MSR and mean sums of squares for error MSE form an F-test statistic used to test the regression model. The larger the explained variation, the better the model is at prediction. The larger the unexplained variation, the worse the model is at prediction. A quantitative measure of the explanatory power of a model is R 2 , the Coefficient of Determination:.

The Coefficient of Determination measures the percent variation in the response variable y that is explained by the model. The Coefficient of Determination and the linear correlation coefficient are related mathematically.

Even though you have determined, using a scatterplot, correlation coefficient and R 2 , that x is useful in predicting the value of y , the results of a regression analysis are valid only when the data satisfy the necessary regression assumptions. We can use residual plots to check for a constant variance, as well as to make sure that the linear model is in fact adequate. The center horizontal axis is set at zero. One property of the residuals is that they sum to zero and have a mean of zero.

A residual plot should be free of any patterns and the residuals should appear as a random scatter of points about zero. A residual plot with no appearance of any patterns indicates that the model assumptions are satisfied for these data. The residuals tend to fan out or fan in as error variance increases or decreases.

The model may need higher-order terms of x , or a non-linear model may be needed to better describe the relationship between y and x. Transformations on x or y may also be considered. A normal probability plot allows us to check that the errors are normally distributed. It plots the residuals against the expected value of the residual as if it had come from a normal distribution. Recall that when the residuals are normally distributed, they will follow a straight-line pattern, sloping upward. The most serious violations of normality usually appear in the tails of the distribution because this is where the normal distribution differs most from other types of distributions with a similar mean and spread.

Curvature in either or both ends of a normal probability plot is indicative of nonnormality. Our regression model is based on a sample of n bivariate observations drawn from a larger population of measurements.

We use the means and standard deviations of our sample data to compute the slope b 1 and y-intercept b 0 in order to create an ordinary least-squares regression line. But we want to describe the relationship between y and x in the population, not just within our sample data. We want to construct a population model.

Now we will think of the least-squares line computed from a sample as an estimate of the true regression line for the population. In our population, there could be many different responses for a value of x. In simple linear regression, the model assumes that for each value of x the observed values of the response variable y are normally distributed with a mean that depends on x.

We also assume that these means all lie on a straight line when plotted against x a line of means. In other words, the noise is the variation in y due to other causes that prevent the observed x, y from forming a perfectly straight line.