Add support for the PaLM API from Google

[lukecav]

Is your enhancement related to a problem? Please describe.

Google is rolling out access to the PaLM API.

https://developers.generativeai.google/
https://developers.generativeai.google/products/palm
https://developers.generativeai.google/api/rest/generativelanguage

Designs

No response

Describe alternatives you've considered

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[jeffpaul]

Thanks for sharing @lukecav! This looks to currently be gated access, so I'm putting this in our Future Release category until this becomes more broadly available. However, we very much want to look at adding service providers across our existing features to allow folks to leverage their provider of choice so thanks again for sharing this!

[jeffpaul]

@lukecav as we continue to plan our roadmap, I'm curious if you had specific use cases and features for the PaLM API beyond just adding support for it as a Service Provider within ClassifAI?

[ishaan-jaff]

Hi @lukecav @jeffpaul - I believe we can make this easier
I’m the maintainer of LiteLLM - we allow you to deploy a LLM proxy to call 100+ LLMs in 1 format - PaLM, Bedrock, OpenAI, Anthropic etc https://github.com/BerriAI/litellm/tree/main/openai-proxy.

If this looks useful (we're used in production)- please let me know how we can help.

Usage

PaLM request

curl http://0.0.0.0:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
     "model": "palm/chat-bison",
     "messages": [{"role": "user", "content": "Say this is a test!"}],
     "temperature": 0.7
   }'

gpt-3.5-turbo request

curl http://0.0.0.0:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
     "model": "gpt-3.5-turbo",
     "messages": [{"role": "user", "content": "Say this is a test!"}],
     "temperature": 0.7
   }'

claude-2 request

curl http://0.0.0.0:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
     "model": "claude-2",
     "messages": [{"role": "user", "content": "Say this is a test!"}],
     "temperature": 0.7
   }'

[jeffpaul]

@ishaan-jaff that's super helpful, thanks for sharing! Besides adding a service provider like Google's PaLM, are there any specific ML features you think any of the service providers abstracted by litellm are useful in a WordPress/CMS context?