High codebook utilization of the mimi model

[ciqiangchen]

Due diligence

• I have done my due diligence in trying to find the answer myself.

Topic

The PyTorch implementation

Question

When I attempted to reproduce the MIMI codec, I found that my codebook utilization was very low in all layers except for the first distillation layer. Meanwhile, I noticed that the codebook utilization of all RVQ layers in the official checkpoint was very high.

Is there any technique to improve the utilization of these codebooks, especially for the RVQ layers other than the distillation layer? I’m looking forward to your reply. Thank you!