chore(mlx-lm): support text type content in messages

[mzbac]

In the latest version of SmolAgents, it has started using message content fragments formatted as content = { type: 'text', content: 'hello world' } instead of using a string for the text content. This change breaks compatibility with the mlx-lm.server. To make the mlx-lm server compatible with this new format, added to convert content from text fragments to text strings in order to apply chat template.

After fixing, we can use mlx-lm.server with smolagents as shown below:

from smolagents import CodeAgent, OpenAIServerModel

model = OpenAIServerModel(
    api_base="http://<your-mlx-server>:8080",
    api_key="n/a",
    model_id="default_model",
    temperature=0,
    max_tokens=8000,
)

agent = CodeAgent(tools=[], model=model, add_base_tools=True)

agent.run(
    "How many seconds would it take for a leopard at full speed to run through Pont des Arts?"
)

[williamzebrowskI]

I think your fix could be optimized a bit further in a performance and efficiency standpoint.

Building flattened_text using += inside a loop works but creates a new string object on every iteration whereas join() can concatenate all objects in a single operation.

ie, in the process_message_content method:

message_copy["content"] = "".join(
    fragment["text"]
    for fragment in message_copy["content"]
    if fragment.get("type") == "text"
)

[awni]

Thanks for the fix for this! I'm a little suprised the chat template doesn't handle it directly given smolagents is also an HF thing.. but this seems like a good patch for now. I made a few simplifying edits and ran the formatter.. I don't think I messed anything up (tests still clear) and it was in-place to begin with since we update the body["messages"].

[mzbac]

Thanks for the fix for this! I'm a little suprised the chat template doesn't handle it directly given smolagents is also an HF thing.. but this seems like a good patch for now. I made a few simplifying edits and ran the formatter.. I don't think I messed anything up (tests still clear) and it was in-place to begin with since we update the body["messages"].

I guess the chat template is only meant for text content. OpenAI api provides broader support for images and tools.
Thank you for cleaning up my PR :)