How are model hyper-parameters generally selected?

Model hyper-parameters are usually chosen through a grid search procedure, in which a model is trained separately on every combination of hyper-parameters in the grid, and an error metric is stored after the model is evaluated on each setting combination. The setting that results in the lowest cross-validation error is usually considered the optimal combination. It is important to search enough combinations of the grid of hyperparameters to be able to see which settings are resulting in both under and overfitting, and that the optimal setting for each hyperparameter can be found within the range searched.