Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Fixing tool prompt format #196

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
+0 −2
Open

fix: Fixing tool prompt format #196

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
9851449
fixing tool prompt format
dmartinol Feb 28, 2025
625df1d
removed setting after recent commits
dmartinol Mar 4, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion examples/agents/rag_as_attachments.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ def run_main(host: str, port: int, disable_safety: bool = False):
},
toolgroups=["builtin::rag"],
tool_choice="auto",
tool_prompt_format="json",
tool_prompt_format="python_list",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should be able to just remove this since we added meta-llama/llama-stack#1214

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, thank you! I will update it asap
BTW: is there any way to be informed of the ongoing/planned changes before pushing PRs that are already out-of-date? 😬

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know if there's a better way but I just look at open PRs periodically and check github notifications.

BTW I do plan to do a full clean up of tool_prompt_format as well.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW: I tried to remove the prompt format but resolve_model fails to resolve the model so the default prompt is still json 😞

These are the models I'm using:

% llama-stack-client models list

Available Models

┏━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┓
┃ model_type ┃ identifier       ┃ provider_resource_id ┃ metadata                 ┃ provider_id ┃
┡━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━┩
│ embedding  │ all-MiniLM-L6-v2 │ all-minilm:latest    │ {'embedding_dimension':  │ ollama      │
│            │                  │                      │ 384.0}                   │             │
├────────────┼──────────────────┼──────────────────────┼──────────────────────────┼─────────────┤
│ llm        │ llama3.2:1b      │ llama3.2:1b          │                          │ ollama      │
└────────────┴──────────────────┴──────────────────────┴──────────────────────────┴─────────────┘

Total models: 2

Changing the check to compare lower-cased strings seems to work, but I'm not sure it may break something else:

    for m in all_registered_models():
        if descriptor.lower in (m.descriptor().lower(), m.huggingface_repo):
            return m
    return None

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok let's get this in first

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ollama run llama3.2:1b, is that wrong?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

anyhow meta-llama/llama-stack#1360 should fix this

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will try again and clear the setting.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

confirm that it works w/ the default format

input_shields=available_shields if available_shields else [],
output_shields=available_shields if available_shields else [],
enable_session_persistence=False,
Expand Down
2 changes: 1 addition & 1 deletion examples/agents/rag_with_vector_db.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ def run_main(host: str, port: int, disable_safety: bool = False):
}
],
tool_choice="auto",
tool_prompt_format="json",
tool_prompt_format="python_list",
input_shields=available_shields if available_shields else [],
output_shields=available_shields if available_shields else [],
enable_session_persistence=False,
Expand Down