description Distributed training framework for large models. Whale: Efficient Giant Model Training over Heterogeneous GPUs