HFMA2 and LDS in cutlass efficient fp16 tensorcore kernel?

[LeiWang1999]

Hi there, I'm currently profiling an efficient cutlass kernel cutlass_tensorop_h1688gemm_256x128_32x2_nt_align8 on a gemm problem with size of M=N=K=16384, what an interesting found is that cutlass use some lds/sds instructions, which even occurs some conflicts,

and I observed some SIMT instructs like HFMA2 or HMUL2 as well,

Anyone could kindly discuss such an interesing phenomenon? why we need simt and shared load/write instructions in fp16 tc kernel(i8 tc only have ld/st matrix instructions.

[hwu36]

The epilogue needs to use lds/sts and hfma2 to do alpha and beta scaling. if you want to use nsight to check bank conflicts, you need to change your problem size to launch just one threadblock.

[Peter9606]

if you want to use nsight to check bank conflicts, you need to change your problem size to launch just one threadblock.

I'm not quite familiar with Nsight, but why is that? Is it a limitation of Nsight?

[hwu36]

It is not a limitation, but more like a mismatch of what you need. When there is more than one threadblock, nsight may collect some extra data that are not needed to determine bank conflict.