Example of bitblas/ladder with dtypes like int3, int5, int6, int7

[yaoyaoding]

Thanks for the great work!

I am curious whether bitblas/ladder has support for the quantized data types like int3, int5, int6 (or uint3, uint5, uint6, float6_e3m2, etc)?

[LeiWang1999]

hi @yaoyaoding , thanks for your attention! currently we do not have implementations of these bits which is not a pow of 2 (which may lead to some empty slot when compress data, for example, only 2 int3 can be stored at a int8 storage value which lead to only 6/8 utilization).

But the implementation is simple, if you really request these datatypes, I'll be pleasure to add them :)

[yaoyaoding]

hi @yaoyaoding , thanks for your attention! currently we do not have implementations of these bits which is not a pow of 2 (which may lead to some empty slot when compress data, for example, only 2 int3 can be stored at a int8 storage value which lead to only 6/8 utilization).

But the implementation is simple, if you really request these datatypes, I'll be pleasure to add them :)

Thanks for your timely reply, @LeiWang1999 !

Will the slot appear in global memory, or only appear in the shared memory and register?

[LeiWang1999]

@yaoyaoding It depends, bitblas has about three different templates for dequantization (in global, in shared, in register), we can select one to deploy this computation :)

[yaoyaoding]

@LeiWang1999 Got it, thanks for the explanation!