Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Baseline ++ validation wrongly reported in paper? #77

Open
DianeBouchacourt opened this issue Nov 9, 2022 · 0 comments
Open

Baseline ++ validation wrongly reported in paper? #77

DianeBouchacourt opened this issue Nov 9, 2022 · 0 comments

Comments

@DianeBouchacourt
Copy link

In the paper, it seems you perform validation on novel classes for Baseline and Baseline++ too, as per the sentence
" To reduce the risk of over-fitting, we use the validation set to select the epoch or episode
with the best accuracy for all methods, including baseline and baseline++"

But looking at the code, it seems you actually do not validation and set acc=-1 for Baseline (see

if acc > max_acc : #for baseline and baseline++, we don't use validation in default and we let acc = -1, but we allow options to validate with DB index
and
#Emperically, this only works for CUB dataset but not for miniImagenet dataset
)

Could you explain?

@DianeBouchacourt DianeBouchacourt changed the title Baseline ++ validation wrong? Baseline ++ validation wrongly reported in paper? Nov 9, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant