# Logistic Regression

### What is Logistic Regression?

Logistic regression is a supervised learning algorithm that is used to predict the probability of categorical dependent variable.

### Among the common classification algorithms, which all produce probability estimates in addition to class labels?

In some classification contexts, it might be more of interest to obtain predicted probabilities of class membership rather than simply the labels themselves.

### What is the error / loss function in logistic regression?

Logistic regression uses a logistic loss function, where the cost for a single observation is represented by:

Advantages: Simple and easy to understand, efficient, interpretable. Disadvantages: Linearity assumption, sensitive to outliers, overfitting

### What is the equivalent of the overall F test in logistic regression?

The equivalent of the overall F-Test in logistic regression is Deviance.

### Why are coefficients estimated through Maximum Likelihood (MLE) instead of Least Squares?

The least squares cost function is non-convex in a binary classification setting, meaning the algorithm could get stuck in a local rather than global minimum and thus fail to optimize the loss.

### How are the coefficients in a logistic expression interpreted?

Each ? is interpreted as the change in log odds of a success for a 1-unit increase in the corresponding predictor, holding all other variables constant.

### What is the relationship between the log odds ratio and probability?

Logistic regression relates the log odds to the weighted combination of predictors and coefficients

### Why are the log odds used in the link function instead of just the regular odds ratio?

The odds, or the ratio of the probability of success to that of failure, can only take on positive values.