Low PSNR in our own dataset

[2hiTee]

Excellent work! I have a question that when I training on my own dataset, which is about 6000 frames, I got a low PNSR result, do you have any suggestions on this? Thanks a lot!

[zjumsj]

Could you provide more information? Otherwise, it is difficult to make a judgment. The differences between datasets can be significant; for example, in the results reported in our paper, the PSNR varies greatly (with a minimum of 26.61 dB and a maximum close to 40 dB). Have you tried any other methods, or is it simply that our method performs poorly?

[2hiTee]

Could you provide more information? Otherwise, it is difficult to make a judgment. The differences between datasets can be significant; for example, in the results reported in our paper, the PSNR varies greatly (with a minimum of 26.61 dB and a maximum close to 40 dB). Have you tried any other methods, or is it simply that our method performs poorly?

Hi, thanks for your reply! Yes, I have tried other methods, the PSNR is significantly higher than this method. So I am wondering whether it is because the camera poses obtained using metrical-tracker are inaccurate. Could you tell me which parameter in your code represents the camera pose? To be specific, my dataset contains 6000 frames and my final results are about 27 PSNR. Thanks again for your help! And thanks for your wonderful work!

[zjumsj]

You can find the camera pose in the getData function within FLAME\dataset.py. The camera information is read from checkpoint/%05d.frame.

[wzr1201]

have u solved the problem, I meet same question

[2hiTee]

You can find the camera pose in the getData function within FLAME\dataset.py. The camera information is read from checkpoint/%05d.frame.

Thanks for your reply, I think this problem maybe due the inaccurate tracking of metrical-tracker, not a problem of GaussianBlendshapes. Thanks for your work!

[2hiTee]

have u solved the problem, I meet same question

Hi, sorry to tell you that I am still trying to fix it, and I believe the low PSNR results are due the inaccurate camera poses obatained from metrical-tracker, and I am trying to replace them, you can also try to replace the poses with accurate poses obtained from other methods. If you have solved it, please reply to this message. Thank you.

[wzr1201]

have u solved the problem, I meet same question

Hi, sorry to tell you that I am still trying to fix it, and I believe the low PSNR results are due the inaccurate camera poses obatained from metrical-tracker, and I am trying to replace them, you can also try to replace the poses with accurate poses obtained from other methods. If you have solved it, please reply to this message. Thank you.

I solved the low psnr problem，the reason in my case is that the training image must have alpha channel, but in my first try, only rgb channel exists.

I also found the inaccurate camera poses problem, because the output image sequence of metrical-tracker exhibits jitter,
but when I follow the preprocess instructions of SimonGiebenhain/MonoNPHM, the jitter problem disappeared. I do not know the difference between them