How can I explicitly profile the memory used for each part of the model?

[dlin511]

Hi,

Does lmdeploy/turbomind engine support any way to profile the memory usage of each part of the model?

For example, can I see how much memory the model is occupying vs the activation states vs the KV cache?

[lvhan028]

Sorry, it doesn't