Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don't run on_validation_epoch_end() during sanity checking if val-accuracy-interval is greater than max epochs. #859

Open
bw4sz opened this issue Dec 17, 2024 · 0 comments
Labels
API This tag is used for small improvements to the readability and usability of the python API. good first issue Good for newcomers

Comments

@bw4sz
Copy link
Collaborator

bw4sz commented Dec 17, 2024

In https://deepforest.readthedocs.io/en/v1.4.1/user_guide/09_configuration_file.html#val-accuracy-interval, we tell users that to turn off the validation callback, set the interval to greater than the number of epochs.

if self.current_epoch % self.config["validation"]["val_accuracy_interval"] == 0:

But that's not quite true, because during sanity checking the self.current_epoch = 0, this means it still checks the line above, there is no reason and is probably dangerous, to check the validation dataloader if we don't intend on ever calling it during training.

More on sanity checking

https://lightning.ai/forums/t/validation-sanity-check/174

https://pytorch-lightning.readthedocs.io/en/0.9.0/api/pytorch_lightning.trainer.evaluation_loop.html

  1. We need to test and document and check the current behavior.
  2. Update docs with what is actually happening.

This issue should wait on the refactor of #858

@bw4sz bw4sz added good first issue Good for newcomers API This tag is used for small improvements to the readability and usability of the python API. labels Dec 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
API This tag is used for small improvements to the readability and usability of the python API. good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

1 participant