What does Gradient in Gradient Boosted Trees refer to?
The “gradient” in Gradient Boosting Machine is a reference to the concept of gradient descent.
Classification & Regression Trees (CART)
What is XGBoost, and how does it improve upon standard GBM?
XGBoost is a modern implementation of Gradient Boosting Machine that works largely the same as the standard GBM.
What is the difference between Adaboost vs Gradient boost?
Adaboost (Adaptive Boosting) is a simple boosting technique that is a predecessor to modern algorithms like GBM and its offshoots.
Distinguish between a Weak learner vs Strong Learner
A weak learner refers to a prediction mechanism that produces results that are only slightly more predictive than those resulting from a random chance model.
What are the options for reporting feature importance from a decision-tree based model?
For any decision-tree based method, feature importance can be measured in a couple of ways.
What are the best ways to safeguard against overfitting a GBM?
Tuning the combination of number of trees and learning rate is a good way to ensure you are creating a model with appropriate complexity.
In what situations is it more advisable to use GBM instead of Random Forest, and vice versa?
On structured datasets, a well-tuned GBM more often outperforms a Random Forest.
How is Gradient Boosting different from Random Forest?
In Random Forest, decision trees are constructed independently and the results are aggregated through either averaging or majority vote after all trees are created.
What are the advantages and disadvantages of a GBM model?
Advantages: High accuracy
Disadvantages: Requires some computing power and time spent in parameter tuning
What are the key hyper-parameters for a GBM model?
Key hyper-parameters for a GBM are: Number of trees, learning rate, and maximum depth