[2.8] 3xTF32: FP32 accuracy with 2x Performance

[hwu36]

In today's GTC talk, we announced 3xTF32 as a new feature to be released with upcoming CUTLASS 2.8. Using Ampere tensor cores to emulate FP32 operations, 3xTF32 matches the accuracy of FP32 instruction with at least 2x throughput.

[thakkarV]

Congrats to everyone involved! This work is super cool

[manishucsd]

@hwu36 , Did we have an explanation of why 3xTF32 is more precise than F32?

I remember and understand why 3xTF32 is more accurate than TF32, but cannot seem to remember if we had an explanation of 3xTF32 being more accurate than F32 in slide 39 here