Support functions/tools in OpenAI API

[carsonwang]

Support functions/tools in the API to enable more use cases.
Refer to the OpenAI document below:
https://platform.openai.com/docs/api-reference/chat/create#chat-create-tools

tools:
A list of tools the model may call. Currently, only functions are supported as a tool. Use this to provide a list of functions the model may generate JSON inputs for.

tool_choice:
Controls which (if any) function is called by the model. none means the model will not call a function and instead generates a message. auto means the model can pick between generating a message or calling a function. Specifying a particular function via {"type": "function", "function": {"name": "my_function"}} forces the model to call that function.

none is the default when no functions are present. auto is the default if functions are present.

curl https://api.openai.com/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $OPENAI_API_KEY" \
-d '{
  "model": "gpt-3.5-turbo",
  "messages": [
    {
      "role": "user",
      "content": "What is the weather like in Boston?"
    }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_current_weather",
        "description": "Get the current weather in a given location",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "The city and state, e.g. San Francisco, CA"
            },
            "unit": {
              "type": "string",
              "enum": ["celsius", "fahrenheit"]
            }
          },
          "required": ["location"]
        }
      }
    }
  ],
  "tool_choice": "auto"
}'

[carsonwang]

We can use mistral 7b to test this. It will also be useful to add a Langchain example that leverage this API.

[carsonwang]

@xuechendi is working on it.

[xuechendi]

@carsonwang, I've been working on this issue but a little lost on How to enable.
Observations:

1. HuggingFace support for 'tools', 'tool_choice': no native parameter support in generate function for "tools", "tool_choice". Seems huggingface has their own way to support Tools: https://huggingface.co/docs/transformers/main/en/custom_tools

2. RayLLM tools/tool_choice support. RayLLM supports these two keywords in client API, but after digging into inference codes, can't find how these two keywords passing to model.inference function either vLLMEngine or TRTEngine.
   Even though AnyScale blog mentioned their fully support for function_calls, it is not come to RayLLM codes yet. Anyscale Endpoints: JSON Mode and Function calling Features

3. llama_cpp toos/tool_choice support. I find function_call implementation in llama_cpp_python for two models: functionary and chatML. But only functionary is accepting these two keywords natually, chatML is still convert it as plain text as Langchain.initialize_agent did. https://github.com/abetlen/llama-cpp-python/blob/main/llama_cpp/llama_chat_format.py#L2032

4. vLLM / triton tensor-RT / huggingface issues research on tools/tool_chain support. Not found aligned conclusion.

Below is my branch for duplicating RayLLM supports for tools/tool_choice:
xuechendi@42d9613

[xuechendi]

pr: #134