– What is Logistic Regression?
– What is the Loss function for Logistic Regression?
– What are the major assumptions of logistic regression?
Logistic regression is a classification algorithm used to predict the probability of a binary or categorical outcome based on one or more independent variables. The algorithm learns a linear relationship from the input dataset and applies a non-linear transformation through the use of the Sigmoid function. The advantages and disadvantages of logistic regression are presented below.
Advantages of Logistic Regression:
|1||Simple and easy to understand||Logistic Regression is a simple and easy-to-understand algorithm, making it suitable for beginners in machine learning|
|2||Scalable and Fast||It can be trained quickly, even on large datasets, and can handle a high number of features|
|3||Interpretable||Logistic regression provides interpretable results in the form of coefficients that represent the contribution of each feature to the outcome|
|4||Flexible||It can handle both categorical and continuous variables as input features and can be extended to model more complex relationships such as interactions between variables|
|5||Robust||Logistic regression is a robust algorithm that can work well even when the assumptions of the model are not perfectly met|
|6||Probabilities well-calibrated||Predicted probabilities are usually well calibrated, which is often not the case in some machine learning algorithms. This means that, for example, it is accurate to translate a predicted probability of 0.10 to a 10% probability of success for that observation|
Disadvantages of Logistic Regression:
|1||Linearity assumption||Logistic regression assumes a linear relationship between the input features and the output variable, which may not always be the case in real-world problems. Since it is a linear model, any non-linear relationships needs to be explicitly defined by transforming the original data and adding additional terms to the model|
|2||Sensitive to outliers||It can be sensitive to outliers, which can affect the coefficients and predictions|
|3||Overfitting||Logistic regression can be prone to overfitting if the number of independent variables is too large relative to the sample size|
|4||Binary outcome||Logistic regression is only suitable for analyzing binary outcomes and may not be appropriate for other types of outcomes|
The following video from Data Unite walks through the advantages and disadvantages of Logistic Regression (Initial 2:20m of the video focuses on Advantages/Disadvantages) :