The website is in Maintenance mode. We are in the process of adding more features.
Any new bookmarks, comments, or user profiles made during this time will not be saved.

Machine Learning Resources

What hyper-parameters are typically tuned in SVM?

Bookmark this question
  • C (regularization parameter): The C parameter influences the shape of the decision boundary and degree to which misclassifications are allowed in the determination of the soft margin hyperplane. It is inversely proportional to the amount of regularization, meaning larger values for C result in less regularization and thus more complex, jagged decision boundaries. On the other hand, smaller values for C imply increased regularization and thus smoother decision boundaries that allow for more misclassifications in the training data.
  • Kernel Function: As discussed, the kernel function must be specified when training a SVM. If the decision boundary is believed to be linear, the linear kernel is the best and most efficient choice. In more complex situations, the RBF is usually the recommended choice, and it requires the gamma parameter to be provided and tuned.
  • Gamma (RBF kernel only): The gamma parameter controls the amount of influence an individual observation carries in determining the similarity between observations. Smaller gamma values imply that observations further apart can be considered similar, which results in smoother decision boundaries. If gamma is larger, points must be close together to be considered similar, thus resulting in less smooth decision boundaries. Gamma and C should be tuned together using cross validation, as there is often interaction present between the hyper-parameters that requires a grid search to determine the optimal combination of settings. 

Leave your Comments and Suggestions below:

Please Login or Sign Up to leave a comment

Partner Ad  

Find out all the ways
that you can

Explore Questions by Topics

Partner Ad

Learn Data Science with Travis - your AI-powered tutor |