Noisy feature vs Clean feature?

[Sunnet314159]

Hi!
I have read your paper and know that you want to obtain the noise-free features. But the feature map from DM's UNet you used to train a copy UNet are with noise. I think the feature obtained finally can't represent the clean feature, but the noisy feature. Could you please help me solve my doubts?

[kliyer-ai]

We train CleanDIFT with clean input images (no noise). As such, the CleanDIFT features cannot contain any noise because we don't inject it anywhere. While we indeed align our features with the noisy features of the standard DM UNet, our model won't learn to artificially produce noisy features because it cannot predict the noise in the standard DM features since they are random. The best it can do is to produce maximally informative features because those will align best with the original DM. Additionally, the projection heads can learn to disregard information that is not necessary for the current timestep of the DM.