You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
hi, i am using the DINOv2 codebase to train a ViT on a custom dataset of ~100M images.
i would like to do only one pass over this dataset, as it is much more redundant than natural images (hence no clear benefit of seing the same images multiple times).
in the configuration file, there are a few epoch-defined parameters:
warmup_epochs
warmup_teacher_temp_epochs
freeze_last_layer_epochs
when epoch = 100, warmup_epochs = 10, meaning we warm up for 10% of training iterations.
when epoch = 1, my idea is to linearly scale down the other epoch-defined parameter and use percentage values instead of fixed integers.
to give a concrete example, i would use warmup_epochs = 0.1 which would always warm up for 10% training iterations
the question is: shall these 3 epoch-defined parameters be linearly scaled down or not?
thanks for the help!
The text was updated successfully, but these errors were encountered:
hi, i am using the DINOv2 codebase to train a ViT on a custom dataset of ~100M images.
i would like to do only one pass over this dataset, as it is much more redundant than natural images (hence no clear benefit of seing the same images multiple times).
in the configuration file, there are a few epoch-defined parameters:
warmup_epochs
warmup_teacher_temp_epochs
freeze_last_layer_epochs
when
epoch = 100
,warmup_epochs = 10
, meaning we warm up for 10% of training iterations.when
epoch = 1
, my idea is to linearly scale down the other epoch-defined parameter and use percentage values instead of fixed integers.to give a concrete example, i would use
warmup_epochs = 0.1
which would always warm up for 10% training iterationsthe question is: shall these 3 epoch-defined parameters be linearly scaled down or not?
thanks for the help!
The text was updated successfully, but these errors were encountered: