Question about chunked prefill

[SimonSongg]

Hi,
When I read through the LONG-CONTEXT BENCHMARKS section of your paper, I found there will be no latency improvement when using chunk size = context length. I can understand this result as this seems like just full attention, without KV cache pruned.

I found you mentioned in the appendix of the paper that block-sparse-attention was used for training. I am wondering if it could also be used in inference for pre-filling the full context, if memory is enough. As it calculate the full context with some middle attention calculation skipped.

Thanks!