A closed form solution occurs when the optimization objective function can be derived by hand and written in an explicit form consisting of a finite number of mathematical operations.
Coordinate descent does not require knowledge of or computation of the derivative of the objective function; rather, it only considers the coordinates of the function itself.
Batch Gradient Descent, Stochastic Gradient Descent, Mini Batch Gradient Descent
Gradient descent is an iterative optimization algorithm
The curse of dimensionality refers to the potential dangers associated with modeling from a dataset that has a large number of features.
Data Sparsity occurs in high-dimensional datasets when the full extent of the possible feature space is not provided to the training data.
Data leakage occurs when information outside the scope of the training data is used in the model building process.
Structured Data is data that has a clear and pre-defined schema. Unstructured Data encompasses the wide spectrum of data that does not fall within the structured category.
In a supervised learning problem, there is an explicit, pre-defined target variable which has known labels.
The different subtypes of cross validation are: k-fold cross validation, leave one out cross validation, validation data set