Question about model parallel in minillm/dpkd

[DiaQusNet]

Hi, I noticed that the implementation of minillm and dpkd requires installing your customized version of the transformers. After reviewing the code, I found that your modifications mainly involve model parallelism. So, can I use the official transformers and disable model parallelism? Will this allow me to run your full suite of code (including training and evaluation)?

[t1101675]

We also modified the transformers library to implement teacher-mixed-sampling (mixing the probabilities of the teacher and student models for decoding). The modified lines are wrapped with

# ### MiniLLM BEGIN ###
... SOME NEW CODES ...
# ### MiniLLM END ###

This feature is required for MiniLLM training.