Mapping of classfication layers to classes

[Maenansyn]

When using most pretrained models, e.g. Faster R-CNN, the classification layer consists of 81 neurons (80 classes and probably one None-neuron or maybe "Other Object not covered by other classes"). When I use the model of Faster R-CNN aimed only for detection of persons, the classes are reduced to 1 (therefore 2 neurons).

When I use a 80-class pretrained model and want to reduce the number of classes e.g. to person, bicycle, car (category-ids 1, 2 and 3 according to the COCO labels) this means that I have to reduce the output neurons from 81 to 4. Besides the declaration in the config this means to adapt the weighs and biases loaded from the pretrained checkpoint and its statedict which are the following:

ckpt['state_dict']['roi_head.bbox_head.fc_cls.weight']
ckpt['state_dict']['roi_head.bbox_head.fc_cls.bias']
ckpt['state_dict']['roi_head.bbox_head.fc_reg.weight']
ckpt['state_dict']['roi_head.bbox_head.fc_reg.bias']

My interpretation of the structure is that index 0 in the list is the "None-neuron", 1 is person, 2 bicycle, 3 car. Therefore I just reduce it that way:

ckpt['state_dict']['roi_head.bbox_head.fc_cls.weight'] = ckpt['state_dict']['roi_head.bbox_head.fc_cls.weight'][:4]
ckpt['state_dict']['roi_head.bbox_head.fc_cls.bias'] = ckpt['state_dict']['roi_head.bbox_head.fc_cls.bias'][:4]
ckpt['state_dict']['roi_head.bbox_head.fc_reg.weight'] = ckpt['state_dict']['roi_head.bbox_head.fc_reg.weight'][:4]
ckpt['state_dict']['roi_head.bbox_head.fc_reg.bias'] = ckpt['state_dict']['roi_head.bbox_head.fc_reg.bias'][:4]

Is that assumption correct or how are class labels mapped from the output neurons? At least, the model created by config with three classes from scratch and model loaded from checkpoint and adapted this way match when calling load_checkpoint().

[zachcoleman]

I believe its the last index actually based on this code:

mmdetection/mmdet/models/roi_heads/bbox_heads/bbox_head.py

Lines 286 to 287 in 9d3e162

	bg_class_ind = self.num_classes
	# 0~self.num_classes-1 are FG, self.num_classes is BG

[Maenansyn]

Ah, okay. Yes, looks like that way. Thank you!