DL Training and Optimization Quiz (Medium)

What is the difference between early stopping and learning rate scheduling as techniques for regularization during training?

When applying weight regularization to a neural network, which term is added to the loss function to encourage small weights?

What is the difference between stochastic gradient descent (SGD) and mini-batch gradient descent?

When is it beneficial to use mini-batch training rather than full-batch training?

During training, what is the purpose of batch normalization layers in a neural network?

In the context of optimization algorithms, what is the “momentum” term?

Which optimization algorithm is known for its ability to escape local minima by using a momentum term and adaptive learning rates?

Which optimization algorithm adapts the learning rate for each parameter individually and has separate learning rates for different parameters?

What is the purpose of gradient clipping during deep learning model training?

What is the purpose of a learning rate schedule, such as a step decay or a learning rate annealing strategy, in training deep learning models?