Difference between dice scores in postprocessing.json

[ironb25]

hello,

I got vastly different dice scores for the 3 variables(dc_per_class_pp_all, dc_per_class_pp_per_class, dc_per_class_raw) in the postprocessing.json.

{
    "dc_per_class_pp_all": {
        "1": 0.28841701194610914,
        "2": 0.1697437962574563,
        "3": 0.35948208655626934,
        "4": 0.8074797873562303
    },
    "dc_per_class_pp_per_class": {
        "1": 0.21455320169633735,
        "2": 0.3946236032306159,
        "3": 0.3563195918778074,
        "4": 0.797078728036078
    },
    "dc_per_class_raw": {
        "1": 0.28841701194610914,
        "2": 0.6572186524583153,
        "3": 0.35941580761019903,
        "4": 0.8074797873562303
    },

I looked at the script(https://github.com/MIC-DKFZ/nnUNet/blob/a7d1d875e8fc3f4e93ca7b51b1ba206711844d92/nnunet/postprocessing/connected_components.py) and found in comments their explanation but its unclear to me what these 3 variables mean and which one I should trust/report for dice scores especially for Class 2?

thanks
Rohan

[FabianIsensee]

Hi Rohan, you don't need to worry about that. dc_per_class_raw is the dice scores without any postprocessing. dc_per_class_pp_all is what the scores would be if everything except the largest foreground region was removed (in your case it gets a lot worse for class 2 so this step is not done) and finally it uses the outcome of dc_per_class_pp_all (in your case it doesnt use that but the raw data) and tried to remove all but the largest connected component for each individual foreground class (dc_per_class_pp_per_class)

[ironb25]

Thanks Fabian! So is it okay to consider raw dice scores for reporting and prediction.?

[FabianIsensee]

If you are looking to compare a new segmentation method vs nnU-Net and the new method does not use this kind of postprocessing then yes it is okay to report the raw scores :-)