-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: set upper bound on max_tokens parameter for LLMs #607
Conversation
or else, the API adds in the length of the input context and throws an error for max tokens
reminder to update this for all LLMs with defaults |
|
||
col1, col2 = gui.columns(2) | ||
with col1: | ||
gui.checkbox("Avoid Repetition", key="avoid_repetition") | ||
if not llm or llm.supports_json: | ||
|
||
if any(map(lambda llm: llm.supports_json, llms)): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we support json output artificially via prompting for models that don't support it natively, so we might wanna leave this like we had earlier?
@@ -87,7 +105,9 @@ def language_model_settings(selected_model: str = None): | |||
min_value=1, | |||
max_value=4, | |||
) | |||
if llm and not llm.is_chat_model and llm.llm_api == LLMApis.openai: | |||
if llms and any( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
would prefer any(<cond> for item in llms)
instead of the lambda version here!
Q/A checklist
You can visualize this using tuna:
To measure import time for a specific library:
To reduce import times, import libraries that take a long time inside the functions that use them instead of at the top of the file:
Legal Boilerplate
Look, I get it. The entity doing business as “Gooey.AI” and/or “Dara.network” was incorporated in the State of Delaware in 2020 as Dara Network Inc. and is gonna need some rights from me in order to utilize my contributions in this PR. So here's the deal: I retain all rights, title and interest in and to my contributions, and by keeping this boilerplate intact I confirm that Dara Network Inc can use, modify, copy, and redistribute my contributions, under its choice of terms.