Learning rate and stopping criterion

[matsen]

Training can look like this:

Is that too big of a learning rate for the green model?

Also, I would have thought that ReduceLROnPlateau would have stopped things earlier, but perhaps the upper loss is keeping things going? Should we change the threshold? (I think we'd want to increase the threshold, but it's a sleepless night...)

Link to our use of ReduceLROnPlateau