Releases: BerriAI/litellm
Releases · BerriAI/litellm
v1.61.3-nightly
What's Changed
- fix(utils.py): fix vertex ai optional param handling by @krrishdholakia in #8477
- Add 'prediction' param for Azure + Add
gemini-2.0-pro-exp-02-05vertex ai model to cost map + Newbedrock/deepseek_r1/*route by @krrishdholakia in #8525 - (UI) - Refactor View Key Table by @ishaan-jaff in #8526
Full Changelog: v1.61.2-nightly...v1.61.3-nightly
Contributors
krrishdholakia and ishaan-jaff
Assets 2
v1.61.3
What's Changed
- Improved wildcard route handling on
/modelsand/model_group/infoby @krrishdholakia in #8473 - (Bug fix) - Using
include_usagefor /completions requests + unit testing by @ishaan-jaff in #8484 - add sonar pricings by @themrzmaster in #8476
- (bug fix)
PerplexityChatConfig- track correct OpenAI compatible params by @ishaan-jaff in #8496 - (fix #2) don't block proxy startup if license check fails & using prometheus by @ishaan-jaff in #8492
- ci(config.yml): mark daily docker builds with
-nightlyby @krrishdholakia in #8499 - (Redis Cluster) - Fixes for using redis cluster + pipeline by @ishaan-jaff in #8442
- Litellm UI stable version 02 12 2025 by @krrishdholakia in #8497
- fix: fix test by @krrishdholakia in #8501
- enables no auth for SMTP by @krrishdholakia in #8494
- UI Fixes p2 by @krrishdholakia in #8502
- add phoenix docs for observability integration by @exiao in #8522
- Added custom_attributes to additional_keys which can be sent to athina by @vivek-athina in #8518
- (UI) fix log details page by @ishaan-jaff in #8524
- Add UI Support for Admins to Call /cache/ping and View Cache Analytics (#8475) by @tahaali-dev in #8519
- LiteLLM Improvements (02/13/2025) p1 by @krrishdholakia in #8523
- fix(utils.py): fix vertex ai optional param handling by @krrishdholakia in #8477
- Add 'prediction' param for Azure + Add
gemini-2.0-pro-exp-02-05vertex ai model to cost map + Newbedrock/deepseek_r1/*route by @krrishdholakia in #8525 - (UI) - Refactor View Key Table by @ishaan-jaff in #8526
Full Changelog: v1.61.1...v1.61.3
## Docker Run LiteLLM Proxy
```
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.61.3
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
## Docker Run LiteLLM Proxy
```
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.61.3
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Failed ❌ | 110.0 | 127.51554087063036 | 6.408067444109619 | 6.408067444109619 | 1917 | 1917 | 94.95955199997752 | 2825.282969 |
| Aggregated | Failed ❌ | 110.0 | 127.51554087063036 | 6.408067444109619 | 6.408067444109619 | 1917 | 1917 | 94.95955199997752 | 2825.282969 |
Contributors
themrzmaster, exiao, and 4 other contributors
Assets 4
v1.61.2-nightly
What's Changed
- Improved wildcard route handling on
/modelsand/model_group/infoby @krrishdholakia in #8473 - (Bug fix) - Using
include_usagefor /completions requests + unit testing by @ishaan-jaff in #8484 - add sonar pricings by @themrzmaster in #8476
- (bug fix)
PerplexityChatConfig- track correct OpenAI compatible params by @ishaan-jaff in #8496 - (fix #2) don't block proxy startup if license check fails & using prometheus by @ishaan-jaff in #8492
- ci(config.yml): mark daily docker builds with
-nightlyby @krrishdholakia in #8499 - (Redis Cluster) - Fixes for using redis cluster + pipeline by @ishaan-jaff in #8442
- Litellm UI stable version 02 12 2025 by @krrishdholakia in #8497
- fix: fix test by @krrishdholakia in #8501
- enables no auth for SMTP by @krrishdholakia in #8494
- UI Fixes p2 by @krrishdholakia in #8502
- add phoenix docs for observability integration by @exiao in #8522
- Added custom_attributes to additional_keys which can be sent to athina by @vivek-athina in #8518
- (UI) fix log details page by @ishaan-jaff in #8524
- Add UI Support for Admins to Call /cache/ping and View Cache Analytics (#8475) by @tahaali-dev in #8519
- LiteLLM Improvements (02/13/2025) p1 by @krrishdholakia in #8523
Full Changelog: v1.61.1...v1.61.2-nightly
## Docker Run LiteLLM Proxy
```
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.61.2-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Failed ❌ | 180.0 | 216.33586769555853 | 6.245273580245063 | 6.245273580245063 | 1869 | 1869 | 145.7912179999994 | 3665.8740830000056 |
| Aggregated | Failed ❌ | 180.0 | 216.33586769555853 | 6.245273580245063 | 6.245273580245063 | 1869 | 1869 | 145.7912179999994 | 3665.8740830000056 |
Contributors
themrzmaster, exiao, and 4 other contributors
Assets 4
v1.61.1.dev5
Full Changelog: v1.61.1.dev1...v1.61.1.dev5
## Docker Run LiteLLM Proxy
```
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.61.1.dev5
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
## Docker Run LiteLLM Proxy
```
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.61.1.dev5
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Failed ❌ | 160.0 | 181.8912512839885 | 6.370642047299698 | 6.370642047299698 | 1905 | 1905 | 129.7774050000271 | 3442.713688000026 |
| Aggregated | Failed ❌ | 160.0 | 181.8912512839885 | 6.370642047299698 | 6.370642047299698 | 1905 | 1905 | 129.7774050000271 | 3442.713688000026 |
v1.61.1.dev1
What's Changed
- Improved wildcard route handling on
/modelsand/model_group/infoby @krrishdholakia in #8473 - (Bug fix) - Using
include_usagefor /completions requests + unit testing by @ishaan-jaff in #8484 - add sonar pricings by @themrzmaster in #8476
- (bug fix)
PerplexityChatConfig- track correct OpenAI compatible params by @ishaan-jaff in #8496 - (fix #2) don't block proxy startup if license check fails & using prometheus by @ishaan-jaff in #8492
- ci(config.yml): mark daily docker builds with
-nightlyby @krrishdholakia in #8499 - (Redis Cluster) - Fixes for using redis cluster + pipeline by @ishaan-jaff in #8442
- Litellm UI stable version 02 12 2025 by @krrishdholakia in #8497
- fix: fix test by @krrishdholakia in #8501
Full Changelog: v1.61.1...v1.61.1.dev1
## Docker Run LiteLLM Proxy
```
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.61.1.dev1
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Failed ❌ | 180.0 | 213.07786790233536 | 6.297898153114872 | 6.297898153114872 | 1884 | 1884 | 146.15093399999068 | 4776.909474999997 |
| Aggregated | Failed ❌ | 180.0 | 213.07786790233536 | 6.297898153114872 | 6.297898153114872 | 1884 | 1884 | 146.15093399999068 | 4776.909474999997 |
Contributors
themrzmaster, krrishdholakia, and ishaan-jaff
Assets 4
v1.61.1.dev2
Full Changelog: v1.61.1...v1.61.1.dev2
v1.61.1
What's Changed
- Show Guardrails on UI by @krrishdholakia in #8447
- Log applied guardrails on LLM API call by @krrishdholakia in #8452
- Ui Fixes Teams Setting #8347 by @tahaali-dev in #8353
- (UI) allow adding model aliases for teams by @ishaan-jaff in #8471
- (round 4 fixes) - Team model alias setting by @ishaan-jaff in #8474
Full Changelog: v1.61.0...v1.61.1
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.61.1
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 160.0 | 180.272351294557 | 6.268555221678184 | 0.0 | 1874 | 0 | 118.979319999994 | 3618.562145999988 |
| Aggregated | Passed ✅ | 160.0 | 180.272351294557 | 6.268555221678184 | 0.0 | 1874 | 0 | 118.979319999994 | 3618.562145999988 |
Contributors
krrishdholakia, ishaan-jaff, and tahaali-dev
Assets 4
v1.61.0.dev1
What's Changed
- (Feat) - Allow calling Nova models on
/bedrock/invoke/by @ishaan-jaff in #8397 - Run litellm in dev mode by @CakeCrusher in #8404
- (Bug Fix) - Bedrock completions with aws_region_name by @ishaan-jaff in #8384
- added gemini 2.0 models to docs by @mubashir1osmani in #8412
- Added filter in Teams and fixed spacing & height issues in Teams tabs (#6192) by @tahaali-dev in #8357
- Revert "Added filter in Teams and fixed spacing & height issues in Teams tabs (#6192)" by @krrishdholakia in #8416
- Allow editing model api key + provider on UI by @krrishdholakia in #8406
- Allow org admin to create teams on UI by @krrishdholakia in #8407
New Contributors
- @CakeCrusher made their first contribution in #8404
Full Changelog: v1.60.8...v1.61.0.dev1
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.61.0.dev1
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 210.0 | 239.66008324484403 | 6.157355937828466 | 0.0033427556665735435 | 1842 | 1 | 171.46882700001242 | 4024.462443999994 |
| Aggregated | Passed ✅ | 210.0 | 239.66008324484403 | 6.157355937828466 | 0.0033427556665735435 | 1842 | 1 | 171.46882700001242 | 4024.462443999994 |
Contributors
krrishdholakia, ishaan-jaff, and 3 other contributors
Assets 4
v1.61.0
What's Changed
- (Feat) - Allow calling Nova models on
/bedrock/invoke/by @ishaan-jaff in #8397 - Run litellm in dev mode by @CakeCrusher in #8404
- (Bug Fix) - Bedrock completions with aws_region_name by @ishaan-jaff in #8384
- added gemini 2.0 models to docs by @mubashir1osmani in #8412
- Added filter in Teams and fixed spacing & height issues in Teams tabs (#6192) by @tahaali-dev in #8357
- Revert "Added filter in Teams and fixed spacing & height issues in Teams tabs (#6192)" by @krrishdholakia in #8416
- Allow editing model api key + provider on UI by @krrishdholakia in #8406
- Allow org admin to create teams on UI by @krrishdholakia in #8407
- Azure api version check - fix str compare to convert to int by @krrishdholakia in #8438
- Fix callback add when user_config passed + support passing openai org client-side by @krrishdholakia in #8443
- Org UI Improvements by @krrishdholakia in #8436
- (e2e testing) - add tests for using litellm
/team/updates in multi-instance deployments with Redis by @ishaan-jaff in #8440 - (Feat) - Allow viewing Request/Response Logs stored in GCS Bucket by @ishaan-jaff in #8449
New Contributors
- @CakeCrusher made their first contribution in #8404
Full Changelog: v1.60.8...v1.61.0
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.61.0
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 180.0 | 213.86169773089247 | 6.297834462789351 | 0.003342799608699231 | 1884 | 1 | 81.07622899996159 | 4173.802059999957 |
| Aggregated | Passed ✅ | 180.0 | 213.86169773089247 | 6.297834462789351 | 0.003342799608699231 | 1884 | 1 | 81.07622899996159 | 4173.802059999957 |
Contributors
krrishdholakia, ishaan-jaff, and 3 other contributors
Assets 4
v1.60.8
What's Changed
- UI Updates by @krrishdholakia in #8345
- OIDC Scope based model access by @krrishdholakia in #8343
- Fix azure max retries error by @krrishdholakia in #8340
- Update deepseek API prices for 2025-02-08 by @Winston-503 in #8363
- fix(nvidia_nim/embed.py): add 'dimensions' support by @krrishdholakia in #8302
- fix: dictionary changed size during iteration error (#8327) by @krrishdholakia in #8341
- fix: add azure/o1-2024-12-17 to model_prices_and_context_window.json by @byrongrogan in #8371
- (Security fix) Mask redis pwd on
/cache/ping+ add timeout value and elapsed time on azure + http calls by @krrishdholakia in #8377 - Handle azure deepseek reasoning response (#8288) by @krrishdholakia in #8366
- Anthropic Citations API Support by @krrishdholakia in #8382
- (Feat) - Add
/bedrock/invokesupport for all Anthropic models by @ishaan-jaff in #8383 - O3 mini native streaming support by @krrishdholakia in #8387
Full Changelog: v1.60.6...v1.60.8
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.60.8
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 170.0 | 189.56173781509457 | 6.206468643400922 | 0.0 | 1855 | 0 | 149.30551800000558 | 3488.08786699999 |
| Aggregated | Passed ✅ | 170.0 | 189.56173781509457 | 6.206468643400922 | 0.0 | 1855 | 0 | 149.30551800000558 | 3488.08786699999 |
Contributors
krrishdholakia, ishaan-jaff, and 2 other contributors