LiteLLM Minor Fixes and Improvements (11/09/2024)

[krrishdholakia]

Title

• fix(caching.py): set ttl for async_increment cache
• fix(router.py): allow setting retry policy via config.yaml s/o @arvinxx @eladsegal
• fix(router.py): don't cooldown single deployments (prevents cooldown errors when just 1 model set for a 'model_name') s/o @dkondoetsy
• fix(litellm_pre_call_utils.py): fix dynamic key logging when team id is set s/o @oz-elhassid
• fix(secret_managers/main.py): load environment variables correctly s/o @oz-elhassid
• feat(spend_tracking_utils.py): support logging additional usage params - e.g. prompt caching values for deepseek s/o @arvinxx
• feat(user_api_key_auth.py): support setting allowed email domains on jwt tokens s/o @andrewbolster

Relevant issues

Fixes #5609

Type

🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test

Changes

[REQUIRED] Testing - Attach a screenshot of any new tests passing locall

If UI changes, send a screenshot/GIF of working UI fixes

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
litellm	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Sep 12, 2024 5:36am

[dkondoetsy]

Thank you, Krrish!

[dkondoetsy]

Will this patch resolve this error?

litellm.proxy.proxy_server.chat_completion(): Exception occured - No deployments available for selected model, Try again in 60 seconds. Passed model=***. pre-call-checks=False, cooldown_list=['c9d48646f355a0ca27eecefe772512a8f609690dfbe31d4c60f5b591ebe1cd2b']

[krrishdholakia]

Will this patch resolve this error?

yes, precisely