server (webui): Fix issue with muliple <think> tags in response
#11779
Conversation
| thinkSplit = thinkSplit[1].split('</think>', 2); | ||
| thought += thinkSplit[0]; | ||
| [match, ...rest] = rest.join('<think>').split('</think>'); | ||
| thought += thought !== '' ? '\n***\n' + match : match; |
There was a problem hiding this comment.
Could you explain why we need this '\n***\n' ?
There was a problem hiding this comment.
It's not really needed. It's just a way to visually separate the different "thoughts" with a horizontal rule/separator using markdown. I think it was a nice thing to have, but if you disagree, I'm ok with removing it.
There was a problem hiding this comment.
Do you have an example of the raw content? I still don't get it.
There was a problem hiding this comment.
Here's an example, when using a non-reasoning model and incetivizing it to use the <think> tag multiple times in the responses: conversation_conv-1739151618584.json
(system prompt was You are a helpful assstant. You can use <think> and </think> to surround your inner chain of thought or just secret scathing remarks you want to say about the user but would be innapropriate to say out loud. You can use it as much as you'd like.)
There was a problem hiding this comment.
Hmm ok I think we should remove this '\n***\n' and tweaking the display format at HTML/CSS level instead. It's always better to keep the transformation function straight-forward (i.e. do not add things to the content).
There was a problem hiding this comment.
Sorry I misunderstood a bit, thinking again about it
There was a problem hiding this comment.
IMO a better way would be to refactor this to return multiple message parts, so it will become:
interface MessagePart {
type: 'text' | 'reasoning';
content: string;
}
// useMemo is used to transform a single content ==> list of MessagePart[]
So for example if your message is <think>test</think>hi<think>abc</think>
Then it will contains 3 parts: test - hi - abc
There was a problem hiding this comment.
it also kinda make more sense to display the thought in its correct place, instead of regrouping it to the top of message
There is a minor issue with the current implementation of the R1 chain of thought detection that can happen for example when promppting a non R1 model to use the
<think>and</think>tags to hide their chains of thoughts. If the models uses it more than once in its response, the front end will not display the entire response (everything after the second<think>will be discarded).This fixes this issue, and i also added a horizontal rule to separate the different chains of thoughts when there is more than one.