Learning Rate

How big a step gradient descent takes on each update. Too high and training diverges or oscillates; too low and training is stable but painfully slow — usually the first hyperparameter worth tuning.