-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add new text followup preprocessor #947
Open
jeffbl
wants to merge
20
commits into
main
Choose a base branch
from
text-followup
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
9 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Please ensure you've followed the checklist and provide all the required information before requesting a review.
If you do not have everything applicable to your PR, it will not be reviewed!
If you don't know what something is or if it applies to you, ask!
Resolves #918 and resolves #934.
This PR adds a new preprocessor
text-followup
designed to let a Monarch user speak a followup query for the current graphic, and receive an LLM-generated response. The new text-followup schema, which contains both a brief and a full response tag, was merged earlier.It also implements the prompt override as described in #931, but that issue is being left open since this needs to also be implemented in the other LLM-based preprocessors before it is closed. The format for this override is in the README.md file.
Testing includes running Venissa's script (see below) repeatedly with different prompts, which found many ways llama3.2-vision resisted producing good JSON that followed the prompt. Venissa also tested it with the text-followup-handler she is implementing, and it works end-to-end. It still fails sometimes, but these failures seem to be infrequent with the current prompt.
Also note that I did
git merge
from main into this branch, which means there are some spurious commits listed in this PR, but it should merge cleanly to main. Please let me know if there is any trouble, and I'll do rebase instead in the future, which I think would have avoided this problem.@VenissaCarolQuadros not making you a formal reviewer, but since you are the main consumer of this preprocessor, figured you might want to take a look at how the sausage is made, and any comments or suggestions are welcome.
Don't delete below this line.
Required Information
Coding/Commit Requirements
New Component Checklist (mandatory for new microservices)
docker-compose.yml
andbuild.yml
..github/workflows
.README.md
file that describes what the component does and what it depends on (other microservices, ML models, etc.).OR