How are Categorical Features represented? (Explain both one-hot and ordinal encoding)

Categorical features are features that can take a limited, and usually a fixed number of possible values. For example: colors in a rainbow, gender, size of a t-shirt etc. There are two types of categorical features:

(a) Nominal features

(b) Ordinal features

Nominal features correspond to features where there is no inherent ordering between different values. For example: gender, color etc. These features can be represented using one hot encoding, also known as one-of-k encoding. Under this encoding, each category is represented by a dummy variable that can be either a 0 or 1, denoting the absence or presence of the category. Here, “one-hot” refers to the fact that only one of these dummy variables can be 1 at a given time. For example: Gender (Male/Female/Other) can be represented using 3 dummy variables (<1,0,0> (Male), <0,1,0> (Female), <0,0,1> (Other)).  

The primary disadvantage of using nominal features is the explosion of the feature set, especially if the number of unique possible values is large.

Ordinal features are categorical features that have an inherent ordering to them. For example: the size of a t-shirt. Such features are represented using integers, where the natural ordering of the integers are used to represent the underlying ordinal relationship. For example, we can map Size (Small/Medium/Large) to (1/2/3). 

The disadvantage of using Ordinal features is that the difference between two representative integers might not be a true representation of the ordinal difference between the raw features.