Machine Learning Resources

What is Linear Regression?

Linear regression is a statistical technique that relates the mean, or expected value, of a continuous response variable through a weighted combination of one or more independent predictor variables. If there is only one predictor, the model is referred to as simple linear regression. One of the advantages regression provides, especially to subject matter researchers, is a clearly defined equation that precisely quantifies how a change in any of the input variables affects the outcome. 

In the case of simple linear regression the model takes the following theoretical form:

[latex]Y_{i} = {\beta}_{0} + {\beta}_{1}X_{1} + {\epsilon}_{i} [/latex]

[latex]where, \\
\qquad i = 1, .., n \\
\qquad {\epsilon}_{i} = \sim {N(0, {\sigma}^2)} [/latex]

[latex] y_{i} [/latex] refers to the [latex]i^{th}[/latex] observation of the target variable, 
?0 is is an intercept term, or the value of the target when the predator variable is set at 0, 
?1  is the coefficient of the predictor, or the average magnitude of change seen in the response for a 1-unit change in the value of the predictor,
Xi  is the ith value of the predictor variable,
And εi is the residual, or the unexplained effect of everything else not accounted for in the behavior of the response through the predictor X.

Recommended Lecture Series on Regression

Want to learn Regression methods in detail? If so, we recommend the following two lecture series on Regression methods. They are thorough with clear explanations (no affiliations):

1. Regression Series by Prof. Justin Zeltzer from University of Sydney:

2. Regression Modeling for Health Research from Prof. Mike Marin at University of British Columbia:

Partner Ad