-
Notifications
You must be signed in to change notification settings - Fork 980
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
update llama 3.3 tool-calling syntax #274
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -72,14 +72,135 @@ Here's my response | |
|
||
## Tool Calling Formats | ||
|
||
Here we describe how to invoke the Llama 3.3 instruction tuned model for tool-calling (also called function-calling). We recommend zero-short function calling over built-in tools. | ||
|
||
### Zero shot function calling | ||
|
||
|
||
For Llama3.3 70B instruct models, we are continuing the format for zero shot function calling, introduced in llama 3.2. | ||
This format is designed to be more flexible and powerful than the format in 3.1. | ||
All available functions can be provided in the system message. | ||
|
||
Here is an example for the same, | ||
|
||
|
||
##### Input Prompt Format | ||
``` | ||
<|begin_of_text|><|start_header_id|>system<|end_header_id|> | ||
|
||
You are an expert in composing functions. You are given a question and a set of possible functions. | ||
Based on the question, you will need to make one or more function/tool calls to achieve the purpose. | ||
If none of the function can be used, point it out. If the given question lacks the parameters required by the function, | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Change "none of the function can" to "none of the function(s) can" (as we do further down in the prompt). I am thinking that this text is within the prompt, so we might not want to perturb it. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Since its within the prompt-we should not change it IMO |
||
also point it out. You should only return the function call in tools call sections. | ||
|
||
If you decide to invoke any of the function(s), you MUST put it in the format of [func_name1(params_name1=params_value1, params_name2=params_value2...), func_name2(params)] | ||
You SHOULD NOT include any other text in the response. | ||
|
||
Here is a list of functions in JSON format that you can invoke. | ||
|
||
[ | ||
{ | ||
"name": "get_weather", | ||
"description": "Get weather info for places", | ||
"parameters": { | ||
"type": "dict", | ||
"required": [ | ||
"city" | ||
], | ||
"properties": { | ||
"city": { | ||
"type": "string", | ||
"description": "The name of the city to get the weather for" | ||
}, | ||
"metric": { | ||
"type": "string", | ||
"description": "The metric for weather. Options are: celsius, fahrenheit", | ||
"default": "celsius" | ||
} | ||
} | ||
} | ||
} | ||
]<|eot_id|><|start_header_id|>user<|end_header_id|> | ||
|
||
What is the weather in SF and Seattle?<|eot_id|><|start_header_id|>assistant<|end_header_id|> | ||
|
||
|
||
``` | ||
|
||
##### Model Response Format | ||
``` | ||
[get_weather(city='San Francisco', metric='celsius'), get_weather(city='Seattle', metric='celsius')]<|eot_id|> | ||
``` | ||
|
||
|
||
##### Notes | ||
|
||
- The output supports multiple, and parallel tool calls natively | ||
- JSON format for defining the functions in the system prompt is similar to Llama3.1 | ||
|
||
|
||
### Zero shot function calling with user message | ||
|
||
|
||
While the default is to provide all function calls in a system message, in Llama3.3 model you can also provide information for all the available tools in a user message. | ||
|
||
|
||
##### Input Prompt Format | ||
``` | ||
<|begin_of_text|><|start_header_id|>user<|end_header_id|> | ||
|
||
Questions: Can you retrieve the details for the user with the ID 7890, who has black as their special request? | ||
Here is a list of functions in JSON format that you can invoke: | ||
[ | ||
{ | ||
"name": "get_user_info", | ||
"description": "Retrieve details for a specific user by their unique identifier. Note that the provided function is in Python 3 syntax.", | ||
"parameters": { | ||
"type": "dict", | ||
"required": [ | ||
"user_id" | ||
], | ||
"properties": { | ||
"user_id": { | ||
"type": "integer", | ||
"description": "The unique identifier of the user. It is used to fetch the specific user details from the database." | ||
}, | ||
"special": { | ||
"type": "string", | ||
"description": "Any special information or parameters that need to be considered while fetching user details.", | ||
"default": "none" | ||
} | ||
} | ||
} | ||
} | ||
] | ||
|
||
Should you decide to return the function call(s),Put it in the format of [func1(params_name=params_value, params_name2=params_value2...), func2(params)] | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Change ",Put it" to ", put them" (The changes above include a space after the comma.) |
||
|
||
NO other text MUST be included.<|eot_id|><|start_header_id|>assistant<|end_header_id|> | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. In the preceding example, we use "You SHOULD NOT include any other text in the response." In this case, we use "NO other text MUST be included." Is there a reason that we change the instructions for the second example? Should we keep the instructions parallel as much as possible. If there is no reason to change, but we change anyway, then a naive reader might infer that we did have a reason--even though we didn't. |
||
|
||
|
||
``` | ||
|
||
##### Model Response Format | ||
``` | ||
[get_user_info(user_id=7890, special='black')]<|eot_id|> | ||
``` | ||
|
||
|
||
##### Notes | ||
|
||
- The tool call format for the model is the same whether your function calls are provided in the system or user message. | ||
- While builtin tool calls end with a <|eom_id|>, notice the <|eot_id|> for zero shot tool calls. | ||
|
||
|
||
### Builtin Tool Calling | ||
|
||
The three built-in tools (brave_search, wolfram_alpha, and code interpreter) can be turned on using the system prompt: | ||
- Brave Search: Tool call to perform web searches. | ||
- Wolfram Alpha: Tool call to perform complex mathematical calculations. | ||
- Code Interpreter: Enables the model to output python code. | ||
|
||
## Builtin Tool Calling | ||
|
||
|
||
Here is an example of a conversation using brave search | ||
|
||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Change: "zero-short" to "zero-shot"