Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

STORY: As a developer testing LLM prompt changes, I want an override option in the docker-compose, so that I can easily change the prompt without rebuilding containers. #931

Open
jeffbl opened this issue Dec 17, 2024 · 0 comments
Assignees

Comments

@jeffbl
Copy link
Member

jeffbl commented Dec 17, 2024

Right now, LLM prompts are hard coded in various containers, including text-followup, content-categoriser, and graphic-caption. This is good, and should be kept, since it means they are versioned in git. However, to change the prompt, the container must be rebuilt. This work item is to have an environment variable [PREPROCESSOR_NAME]_PROMPT_OVERRIDE that can be set to replace the hard-coded prompt.

Note this may also be of use to someone deploying an IMAGE server, and using another LLM that requires tweaks to the prompt. They would also be able to avoid rebuilding the container, and just make their change in the docker-compose.

@jeffbl jeffbl self-assigned this Dec 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant