v1.55.0.dev1

Full Changelog: v1.55.0...v1.55.0.dev1

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.55.0.dev1

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	220.0	243.6248328955295	6.208881714875033	0.0	1857	0	195.87834699996165	1936.7717839999727
Aggregated	Passed ✅	220.0	243.6248328955295	6.208881714875033	0.0	1857	0	195.87834699996165	1936.7717839999727

v1.55.0

What's Changed

• Litellm code qa common config by @krrishdholakia in #7113
• (Refactor) Code Quality improvement - use Common base handler for Cohere by @ishaan-jaff in #7117
• (Refactor) Code Quality improvement - Use Common base handler for clarifai/ by @ishaan-jaff in #7125
• (Refactor) Code Quality improvement - Use Common base handler for cloudflare/ provider by @ishaan-jaff in #7127
• (Refactor) Code Quality improvement - Use Common base handler for Cohere /generate API by @ishaan-jaff in #7122
• (Refactor) Code Quality improvement - Use Common base handler for anthropic_text/ by @ishaan-jaff in #7143
• docs: document code quality by @krrishdholakia in #7149
• (Refactor) Code Quality improvement - stop redefining LiteLLMBase by @ishaan-jaff in #7147
• LiteLLM Common Base LLM Config (pt.2) by @krrishdholakia in #7146
• LiteLLM Common Base LLM Config (pt.3): Move all OAI compatible providers to base llm config by @krrishdholakia in #7148
• refactor(sagemaker/): separate chat + completion routes + make them b… by @krrishdholakia in #7151
• rename llms/OpenAI/ -> llms/openai/ by @ishaan-jaff in #7154
• Code Quality improvement - remove symlink to requirements.txt from within litellm by @ishaan-jaff in #7155
• LiteLLM Common Base LLM Config (pt.4): Move Ollama to Base LLM Config by @krrishdholakia in #7157
• Code Quality Improvement - remove file_apis, fine_tuning_apis from /llms by @ishaan-jaff in #7156
• Revert "LiteLLM Common Base LLM Config (pt.4): Move Ollama to Base LLM Config" by @krrishdholakia in #7160
• Litellm ollama refactor by @krrishdholakia in #7162
• Litellm vllm refactor by @krrishdholakia in #7158
• Litellm merge pr by @krrishdholakia in #7161
• Code Quality Improvement - remove tokenizers/ from /llms by @ishaan-jaff in #7163
• build(deps): bump nanoid from 3.3.7 to 3.3.8 in /docs/my-website by @dependabot in #7159
• (Refactor) Code Quality improvement - remove /prompt_templates/ , base_aws_llm.py from /llms folder by @ishaan-jaff in #7164
• Code Quality Improvement - use vertex_ai/ as folder name for vertexAI by @ishaan-jaff in #7166
• Code Quality Improvement - move aleph_alpha to deprecated_providers by @ishaan-jaff in #7168
• (Refactor) Code Quality improvement - rename text_completion_codestral.py -> codestral/completion/ by @ishaan-jaff in #7172
• (Code Quality) - Add test to enforce all folders in /llms are a litellm provider by @ishaan-jaff in #7175
• fix(get_supported_openai_params.py): cleanup by @krrishdholakia in #7176
• fix(acompletion): support fallbacks on acompletion by @krrishdholakia in #7184

Full Changelog: v1.54.1...v1.55.0

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.55.0

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	250.0	286.19507948581224	5.886697197840291	0.0033409178194326278	1762	1	211.68456200001629	3578.4067740000296
Aggregated	Passed ✅	250.0	286.19507948581224	5.886697197840291	0.0033409178194326278	1762	1	211.68456200001629	3578.4067740000296

v1.54.1

What's Changed

• refactor - use consistent file naming convention AI21/ -> ai21 by @ishaan-jaff in #7090
• refactor - use consistent file naming convention AzureOpenAI/ -> azure by @ishaan-jaff in #7092
• Litellm dev 12 07 2024 by @krrishdholakia in #7086

Full Changelog: v1.54.0...v1.54.1

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.54.1

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Failed ❌	280.0	340.7890831504466	5.986291177372485	0.0	1788	0	236.28402200000664	4047.592437999981
Aggregated	Failed ❌	280.0	340.7890831504466	5.986291177372485	0.0	1788	0	236.28402200000664	4047.592437999981

v1.54.0

What's Changed

• (feat) Track custom_llm_provider in LiteLLMSpendLogs by @ishaan-jaff in #7081
• Add MLflow to the side bar by @B-Step62 in #7031
• (bug fix) SpendLogs update DB catch all possible DB errors for retrying by @ishaan-jaff in #7082
• (Feat) Add StructuredOutputs support for Fireworks.AI by @ishaan-jaff in #7085
• added deepinfra/Meta-Llama-3.1-405B-Instruct to the Model json by @AliSayyah in #7084
• (feat) Add created_at and updated_at for LiteLLM_UserTable by @ishaan-jaff in #7089

New Contributors

• @AliSayyah made their first contribution in #7084

Full Changelog: v1.53.9...v1.54.0

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.54.0

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	200.0	220.2003271503722	6.29832230581454	0.0	1882	0	179.34225999999853	1827.969679000006
Aggregated	Passed ✅	200.0	220.2003271503722	6.29832230581454	0.0	1882	0	179.34225999999853	1827.969679000006

v1.53.9

What's Changed

• LiteLLM Minor Fixes & Improvements (12/06/2024) by @krrishdholakia in #7067
• feat(langfuse/): support langfuse prompt management by @krrishdholakia in #7073

Full Changelog: v1.53.8...v1.53.9

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.53.9

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	230.0	262.5035242475303	6.0929671578673235	0.0	1822	0	209.27508400001216	2657.453161000035
Aggregated	Passed ✅	230.0	262.5035242475303	6.0929671578673235	0.0	1822	0	209.27508400001216	2657.453161000035

v1.53.8

What's Changed

• (UI) Fix viewing home page keys on a new DB by @ishaan-jaff in #7042
• build(model_prices_and_context_window.json): add bedrock region model… by @krrishdholakia in #7044
• Update SearchBar by @yujonglee in #6982
• (fix) litellm router.aspeech by @ishaan-jaff in #6962
• (UI) perf improvement - cache internal user tab results by @ishaan-jaff in #7054
• (fix) adding public routes when using custom header by @ishaan-jaff in #7045
• LiteLLM Minor Fixes & Improvements (12/05/2024) by @krrishdholakia in #7051
• feat: Add gemini-exp-1206 model configuration with 2M input tokens by @paulmaunders in #7064
• Correct Vertex Embedding Model Data/Prices by @emerzon in #7069
• litellm not honoring OPENAI_ORGANIZATION env var by @ishaan-jaff in #7066
• Provider Budget Routing - Get Budget, Spend Details by @ishaan-jaff in #7063
• Feat - add groq/llama3.3 models by @ishaan-jaff in #7076
• (feat) Allow enabling logging message / response for specific virtual keys by @ishaan-jaff in #7071

New Contributors

• @paulmaunders made their first contribution in #7064

Full Changelog: v1.53.7...v1.53.8

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.53.8

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	220.0	252.68788116416482	6.227999496173144	0.0	1864	0	198.31458400000201	2829.406032999941
Aggregated	Passed ✅	220.0	252.68788116416482	6.227999496173144	0.0	1864	0	198.31458400000201	2829.406032999941

v1.53.7.dev4

What's Changed

• (UI) Fix viewing home page keys on a new DB by @ishaan-jaff in #7042

Full Changelog: v1.53.7...v1.53.7.dev4

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.53.7.dev4

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Failed ❌	260.0	301.42724034818906	5.997929238673837	0.0	1795	0	228.56110200001467	3614.213866
Aggregated	Failed ❌	260.0	301.42724034818906	5.997929238673837	0.0	1795	0	228.56110200001467	3614.213866

v1.53.7.dev2

Full Changelog: v1.53.7...v1.53.7.dev2

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.53.7.dev2

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Failed ❌	270.0	309.7098386657411	6.030439383998993	0.0	1804	0	234.0091040000516	3472.105875000011
Aggregated	Failed ❌	270.0	309.7098386657411	6.030439383998993	0.0	1804	0	234.0091040000516	3472.105875000011

v1.53.7-stable

What's Changed

• LiteLLM Minor Fixes & Improvements (12/04/2024) by @krrishdholakia in #7037

Full Changelog: v1.53.6...v1.53.7-stable

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.53.7-stable

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	250.0	287.825675572594	6.147983179332712	0.0	1839	0	225.9885929999541	1840.4691450000428
Aggregated	Passed ✅	250.0	287.825675572594	6.147983179332712	0.0	1839	0	225.9885929999541	1840.4691450000428

v1.53.7

What's Changed

• LiteLLM Minor Fixes & Improvements (12/05/2024) by @krrishdholakia in #7037

Full Changelog: v1.53.6...v1.53.7

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.53.7

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	250.0	285.5731596761653	6.103319742596985	0.0	1825	0	229.3374330000688	1651.5534569999772
Aggregated	Passed ✅	250.0	285.5731596761653	6.103319742596985	0.0	1825	0	229.3374330000688	1651.5534569999772