Use custom Evaluation Hook instead of EvalHook

[mdcaio]

Hi everyone,

I have implemented a custom evaluation Hook, let's call it MyEvalHook, which inherits from the standard mmcv.runner.EvalHook. What should I do to use my evaluation hook during training?

I have tried inserting it as a custom hook in the config file

custom_hooks = [
    dict(type="MyEvalHook", interval=1, metric="bbox"),
]

But it breaks because it doesn't get in input the data loader.

I have also tried to insert it in the config as

evaluation = dict(
    type="TeamInvariantEvalHook", interval=1, metric="bbox"]
)

but it also breaks because it doesn't know what to do with the type kwarg.

Obviously the file of MyEvalHookis properly imported in the custom_import section of the config file.

Any way to use MyEvalHook?
Many thanks for the outstanding work.

[baibaidj]

I believe you have to modify the code in mmdet/apis/train.py to get what you want in this line

mmdetection/mmdet/apis/train.py

Line 193 in ff9bc39

eval_hook = DistEvalHook if distributed else EvalHook

Change this snippet
eval_hook = DistEvalHook if distributed else EvalHook runner.register_hook( eval_hook(val_dataloader, **eval_cfg), priority='LOW')
to

eval_hook = MyDistEvalHook if distributed else MyEvalHook runner.register_hook( eval_hook(val_dataloader, **eval_cfg), priority='LOW')
So you may also have to implement MyDistEvalHook for distributed training.
You can refer to DistEvalHook to implement your custom counterpart, like you did with MyEvalHook.
Best.

[mdcaio]

Fantastic, it works, thank you very much!

Of course, it would be amazing if custom evaluation hooks were treated like other custom hooks, namely, that they could be chosen in the config file.

Many thanks again,
Best,
Marce