Custom Weight Initialization Effect

[glenn-jocher]

I noticed you use code for custom weight initialization:

ghostnet.pytorch/ghostnet.py

Lines 162 to 169 in 83274b7

	def _initialize_weights(self):
	for m in self.modules():
	if isinstance(m, nn.Conv2d):
	nn.init.kaiming_normal_(m.weight, mode='fan_out', nonlinearity='relu')
	elif isinstance(m, nn.BatchNorm2d):
	m.weight.data.fill_(1)
	m.bias.data.zero_()

I've not seen this before anywhere. Is there a reason behind this specific strategy? Do you know the effect this has on the training, and have you compared this with the pytorch default weight initialization? Thank you!

[d-li14]

The same as iamhankai/ghostnet.pytorch#24