### What is a Decision Tree?

Decision Tree is one of the predicting modeling techniques that can be used for both Regression and Classification problems.

- Machine Learning 101 (30)
- Statistics 101 (38)
- Supervised Learning (114)
- Regression (42)
- Classification (46)
- Logistic Regression (10)
- Support Vector Machine (10)
- Naive Bayes (4)
- Discriminant Analysis (5)
- Classification Evaluations (9)

- Classification & Regression Trees (CART) (23)

- Unsupervised Learning (55)
- Clustering (28)
- Distance Measures (9)
- Dimensionality Reduction (9)

- Deep Learning (23)
- Data Preparation (34)
- General (5)
- Standardization (6)
- Missing data (7)
- Textual Data (16)

Decision Tree is one of the predicting modeling techniques that can be used for both Regression and Classification problems.

Gini Score and Entropy are both measures that quantify the impurity of a node in a decision tree

The continuous variable is first sorted in ascending order of its values, and the midpoint between each pair of adjacent observations is calculated.

Pruning refers to the process of simplifying a decision tree after it has already been created by removing leaf nodes that result in the smallest information gain.

CART, or Classification and Regression Trees, simply refers to the standard algorithm for creating a decision tree.

Advantages: Simple and highly intuitive interpretation

Disadvantages: Prone to overfitting

Bagging, or “Bootstrap Aggregation”, refers to an ensemble design structure in which each instance of the ensemble, such as an individual decision tree in a Random Forest, is created on a different subset of the original dataset.

Random Forest is a supervised machine learning algorithm, which can be used for solving both regression and classification problems, including both binary and multi-class classification.

There are three key parameters that can be tweaked: (a) Number of trees, (b) Number of Features and, (c) Sub sample size

Advantages: Reduces prediction variance compared to single decision tree

Find out all the ways

that you can