persistent deployment (qwen2-7b-instruct) model responds abnormally

[chengjixin]

For Chinese and English , the models difference is very large, English returned normal, Chinese returned abnormal

[loadams]

Hi @chengjixin - we would need more info to repro/debug this. What DeepSpeed/MII versions, repro scripts, the error messages or where things appear to be going wrong.