How can underfitting be mitigated?

  • Improve feature selection process to capture as many informative predictors as possible. If the features being used are not able to produce separation against the target variable, the model will not pick up much signal. 
  • Choose an algorithm appropriate for the data set at hand (i.e. linear regression might not be the best algorithm for a complex, high-dimensional data set)
  • Specify an appropriate form of the relationship between the predictors and target, i.e. include higher order terms in a regression if there is a complex relationship
  • Decrease regularization if it is causing too much shrinkage