-
-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Feature]: Allow configurable_clientside_parameters
for switching b/w finetuning models easily
#5843
Labels
enhancement
New feature or request
Comments
We should probably be specific about what parameters we mean, updating to be |
Updated to |
krrishdholakia
added a commit
that referenced
this issue
Sep 23, 2024
krrishdholakia
added a commit
that referenced
this issue
Sep 24, 2024
* feat(auth_utils.py): enable admin to allow client-side credentials to be passed Makes it easier for devs to experiment with finetuned fireworks ai models * feat(router.py): allow setting configurable_clientside_auth_params for a model Closes #5843 * build(model_prices_and_context_window.json): fix anthropic claude-3-5-sonnet max output token limit Fixes #5850 * fix(azure_ai/): support content list for azure ai Fixes #4237 * fix(litellm_logging.py): always set saved_cache_cost Set to 0 by default * fix(fireworks_ai/cost_calculator.py): add fireworks ai default pricing handles calling 405b+ size models * fix(slack_alerting.py): fix error alerting for failed spend tracking Fixes regression with slack alerting error monitoring
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The Feature
in the model_list, having a variable (like clientside_configurable_parameters with a default value of [] that contains a list of which litellm parameters can be overwritten by a client. This preserves the current behavior of no values overwritten by a client, but allows for new behavior of configuring variables that a client can override; this also gives the level of control we would want
Motivation, pitch
Allows devs to switch between finetuned models easily (changing api base), and allows admin to define which models + which params can have this behaviour (prevents security issues like https://huntr.com/bounties/4001e1a2-7b7a-4776-a3ae-e6692ec3d997)
Twitter / LinkedIn details
cc: @nathanluskey
The text was updated successfully, but these errors were encountered: