Struggling in Evaluation on MegaFace!

[kalyo-zjl]

Hi, @happynear

Thanks for sharing the detection and alignment procedure for MegaFace dataset.

I have followed your steps to preprocess both FaceScrub and MegaFace roughly. ( I don't use the cpp MTCNN but matlab MTCNN, however, the logic should be similar as yours) I can only get a result of Rank 1: 0.471718 for 1000000 distractors (with released model sphereface-20).

I use this command to get the result without modifying run_experiment.py:
python run_experiment.py -p ../templatelists/facescrub_uncropped_features_list.json ~/mat-MegaFace-feature/FlickrFinal2 ~/mat-FaceScrub-feature _cnn.bin ../../MegaFace/Mat-result -s 1000000

Is there anything wrong with my test command? I notice that there is a default model jb_identity.bin for scoring, and why should we use it for scoring since we may use a normalized l2 as the metric.

If it is correct with the test command, I may wonder, in your logic, what the face score is if using Matlab MTCNN? And I found that there are lots of images whose detection bbox is accurate, however, the alignment points are really really bad.

[happynear]

MegaFace uses Euclidean distance as the metric, so the features should be normalized manually. I just uploaded an example of feature extraction prototxt.
https://github.com/happynear/FaceVerification/blob/master/dataset/Megaface/face_deploy_20.prototxt

Face score is the probability produced by the third network's softmax layer. In the default settings, MTCNN filters out those scores below 0.7. That's too harsh. I use 0.3 to get more faces in this step.

If the IoU is below 30%, use the last two networks of MTCNN to force get the face score and 

5 key points from the cropped image based on the given bounding box.

The 3 points are usually accurate. But as far as I can see, all the 5 points are wrong. That's why I use 3 points to pre-align the faces.

[kalyo-zjl]

Thank you for your reply!!!

The problem lies in that I didn't normalize the feature before sending it into the test command. Now, I can get a result of 69.5796%.

The remaining difference might be that I use so many 5 alignment points which are not accurate enough to align the face. I will continue to figure it out.

THANK YOU AGAIN!

[zhly0]

@kalyo-zjl did you use cosin distance instead of euclidean distance?

[YangBowenn]

@kalyo-zjl Could you tell me how to normalize the feature, I get rank1：49% for 1M megaface.