Proposal: Integrate Hugging Face Model Quantization Support

[SMAntony]

Hi Team,

We have been working with Vanna for the past few days and have extensively used Hugging Face models and transformers over the past year. We appreciate the support provided for Hugging Face models in issue #391 .

We have locally implemented and tested Hugging Face quantization support with a few modifications to the existing code. Given the positive results, we are considering opening a pull request to contribute these enhancements to the main repository.

Would you be interested in merging the changes to support Hugging Face model quantization?

[zainhoda]

Yeah absolutely! Please feel free to submit a PR!