Inquiry Regarding Benchmark Testing Environment and Parameters

[yhcho87]

❓ Question

I am writing to request some clarification regarding the benchmark tests shared on GitHub at https://github.com/mosaicml/llm-foundry/blob/main/scripts/train/benchmarking/README.md.

Specifically, I have the following questions:

1. Could you please specify the environment in which these benchmark tests were conducted? (e.g., On-premises, OCI, AWS, Azure, etc.)
2. Regarding the H100 80G BF16 benchmark tests, what criteria were used to determine the input sequence length, MicroBatchSize, GradAccum, and GlobalBatchSize parameters?
3. For the same H100 80G BF16 tests, could you explain the rationale behind setting the Activation Checkpointing option to True?

I would greatly appreciate any insights you could provide on these matters. Thank you in advance for your time and assistance.

Additional context