SVM is an algorithm typically used for classification (though it can also be extended to regression) that seeks to determine a decision boundary by maximizing the distance between points of different classes. In one-dimensional space, the boundary is formed by a point; in 2D space, a line; and in higher dimensional space, a hyperplane. Once a classification boundary is determined, future observations that reside on one side of the boundary are classified in the first class, and points on the other side are assigned to the other class.
The distance between the line (or hyperplane) and the observations closest to it is referred to as the maximum margin, and in the case where observations are linearly separable, the maximum margin defines the optimal decision boundary. The points closest to the hyperplane are called support vectors, as they are the observations that ultimately support, or define, the hyperplane. If they were shifted in the higher dimensional space by a small amount, the hyperplane would adjust accordingly.
