Add a client for Gazelle

[handrew]

Created an abstraction over some of the code in the example notebooks (infer, infer-quantized) so that the user can simply:

from gazelle import GazelleClient
client = GazelleClient(quantization="8-bit")
resp = client.infer(audio, prompt="What does the following audio say? \n <|audio|>")
print(resp)

Because of limited resources, I have not yet been able to extensively test it. It seems to work regularly with quantization, but I occasionally get some errors w/ the conv layer without quantization.

Would be happy to take feedback!

[stillmatic]

LGTM! want to also add an example NB or link to colab?

[handrew]

LGTM! want to also add an example NB or link to colab?

Sounds good—where should I? In the readme?

If you want it under your own Google account btw, you can copy it from here. https://colab.research.google.com/drive/1T45iMBWKuhdrXI9gvnR9h6l2IgZovXj7#scrollTo=wBdQsjgKdHgd