MPT 30B inference on Mac M1

[RonanKMcGovern]

Is it realistic to try and get inference running on Mac M1 with similar results quality as on a GPU?

I find 7B and 13B models are not good enough to get working well with functions. Also, I like that MPT has extendable context compared to Falcon and llama.

If I were to try and get MPT 30B running, can I bootstrap using work from llama? Thanks

[ggerganov]

This is now supported in llama.cpp : ggerganov/llama.cpp#3417