[Bug]: Can't have tool_calls in response from mistrail ai through bedrock

[Kaiohz]

What happened?

Hello,

when i try to use the proxy and call my mistral ai model through bedrock with a tools array filled it does not return response with tool_calls filled even though mistral is respondig. I was expecting tool_calls to be filled with get_weather.

My config.yml

model_list:
  - model_name: azure/gpt-4-turbo
    litellm_params:
      model: azure/gpt-4-turbo
      api_version: "2023-12-01-preview"
      api_base: "os.environ/AZURE_API_BASE"
      api_key: "os.environ/AZURE_API_KEY"
  - model_name: azure/gpt-4-32k
    litellm_params:
      model: azure/gpt-4-32k
      api_version: "2024-02-01"
      api_base: "os.environ/AZURE_API_BASE"
      api_key: "os.environ/AZURE_API_KEY"
  - model_name: azure/gpt-4
    litellm_params:
      model: azure/gpt-4
      api_version: "2024-02-01"
      api_base: "os.environ/AZURE_API_BASE"
      api_key: "os.environ/AZURE_API_KEY"
  - model_name: azure/gpt-4o
    litellm_params:
      model: azure/gpt-4o
      api_version: "2024-02-01"
      api_base: "os.environ/AZURE_API_BASE"
      api_key: "os.environ/AZURE_API_KEY"
  - model_name: azure/gpt-4o-mini
    litellm_params:
      model: azure/gpt-4o-mini
      api_version: "2024-02-01"
      api_base: "os.environ/AZURE_API_BASE"
      api_key: "os.environ/AZURE_API_KEY"
  - model_name: azure/gpt-35-turbo
    litellm_params:
      model: azure/gpt-35-turbo
      api_version: "2024-02-01"
      api_base: "os.environ/AZURE_API_BASE"
      api_key: "os.environ/AZURE_API_KEY"
  - model_name: azure/text-embedding-ada-002
    litellm_params:
      model: azure/text-embedding-ada-002
      api_version: "2024-02-01"
      api_base: "os.environ/AZURE_API_BASE"
      api_key: "os.environ/AZURE_API_KEY"
  - model_name: azure/text-embedding-3-large
    litellm_params:
      model: azure/text-embedding-3-large
      api_version: "2024-02-01"
      api_base: "os.environ/AZURE_API_BASE"
      api_key: "os.environ/AZURE_API_KEY"
  - model_name: bedrock/mistral-large
    litellm_params:
      model: "mistral.mistral-large-2402-v1:0"
      aws_access_key_id: "os.environ/AWS_ACCESS_KEY_ID"
      aws_secret_access_key: "os.environ/AWS_SECRET_ACCESS_KEY"
      aws_region_name: "os.environ/AWS_REGION_NAME"
general_settings:
  master_key: "os.environ/LLM_PROXY_API_KEY"
  database_connection_pool_limit: 20
litellm_settings:
  telemetry: False
  json_logs: True
  callbacks: ["custom/custom_callbacks.proxy_handler_instance"]
  success_callback: ["langfuse"]
  failure_callback: ["langfuse"]

Here is my entry json :

{
  "model": "bedrock/mistral-large",
  "messages": [
    {
      "role": "user",
      "content": "what is the weather in paris ?"
    }
  ],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_current_weather",
        "description": "Get the current weather in a given location",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "The city and state, e.g. San Francisco, CA"
            },
            "unit": {
              "type": "string",
              "enum": ["celsius", "fahrenheit"]
            }
          },
          "required": ["location"]
        }
      }
    }
  ]
}

Here is the answer i receive from the proxy :

{
	"id": "chatcmpl-65b8ffbd-4ee9-4c1f-94f6-00ac1038065b",
	"created": 1735915807,
	"model": "mistral.mistral-large-2402-v1:0",
	"object": "chat.completion",
	"system_fingerprint": null,
	"choices": [
		{
			"finish_reason": "stop",
			"index": 0,
			"message": {
				"content": "I don't have real-time data or browsing capabilities to provide current weather updates. However, you can check the weather in Paris by using a weather website or app, or by using a search engine and entering \"Paris weather\" as your query.",
				"role": "assistant",
				"tool_calls": null,
				"function_call": null
			}
		}
	],
	"usage": {
		"completion_tokens": 54,
		"prompt_tokens": 15,
		"total_tokens": 69,
		"completion_tokens_details": null,
		"prompt_tokens_details": null
	}
}

Thank you for your help.

Relevant log output

No response

Are you a ML Ops Team?

No

What LiteLLM version are you on ?

v1.56.8

Twitter / LinkedIn details

No response