Skip to content

[Question] Why Tensor parallel communication/GEMM overlap can happen only when sequence parallelism is enabled? #4603

[Question] Why Tensor parallel communication/GEMM overlap can happen only when sequence parallelism is enabled?

[Question] Why Tensor parallel communication/GEMM overlap can happen only when sequence parallelism is enabled? #4603