[Feat]: Cosine with hard restarts Tunable floor

[ppbrown]

Describe your use-case.

How about adding a "..." menu option for CosHard, that allows the user to set a floor, instead of going to 0?
I'm not sure what type of value would be best.. maybe just a specific LR value, where if it hits THAT value, reset to full, instead of dropping all the way to virtual-0 before the reset?

What would you like to see as a solution?

LR floor value option

Have you considered alternatives? List them here.

?

[Zokreb]

Hi community,
I would second this idea :)

[Calamdor]

It is possible to do with the custom learning rate scheduler.
https://github.com/Nerogar/OneTrainer/wiki/Custom-Scheduler#cosine-annealing

[ppbrown]

Thanks for the tip. But... if I wanted to mess around with things like manually filling in long.code.magic.function.names like that, I wouldnt be using OneTrainer :-/