Add role splitting in markdown processor, remove old guidance regions

README.md

@@ -87,6 +87,12 @@ Note: as an alternative to explicitly passing the API keys in the constructor yo
 
 Request a non-streaming completion from the model.
 
+#### `requestChatCompletion(model: Model, options: { messages: Message[], generationOptions: CompletionOptions }, chunkCb: ChunkCb): Promise<CompletionResponse[]>`
+
+Request a completion from the model with a sequence of chat messages which have roles. A message should look like
+`{ role: 'user', content: 'Hello!' }` or `{ role: 'system', content: 'Hi!' }`. The `role` can be either `user`, `system` or `assistant`, no
+matter the model in use.
+
 ### ChatSession
 
 #### `constructor(completionService: CompletionService, model: string, systemPrompt: string)`
@@ -110,6 +116,13 @@ loadPrompt("Hello, may name is %%%(NAME)%%%", { NAME: "Omega" })
 * `importPromptSync(path: string, variables: Record<string, string>): string` - Load a prompt from a file with the given variables
 * `importPrompt(path: string, variables: Record<string, string>): Promise<string>` - Load a prompt from a file with the given variables, asynchronously returning a Promise
 
+### Flow
+
+For building complex multi-round conversations or agents, see the `Flow` class. It allows you to define a flow of messages
+and responses, and then run them in a sequence, with the ability to ask follow-up questions based on the previous responses.
+
+See the documentation [here](./docs/flow.md).
+
 ### More information
 
 For the full API, see the [TypeScript types](./src/index.d.ts). Not all methods are documented in README, the types are

docs/MarkdownProcessing.md

@@ -1,4 +1,4 @@
-LXL contains a simple templating system that allows you to conditionally insert data into the markdown at runtime.
+LXL contains a simple templating system ("MDP") that allows you to conditionally insert data into the markdown at runtime.
 
 To insert variables, you use `%%%(VARIABLE_NAME)%%%`. To insert a string based on a specific condition, you use ```%%%[...] if CONDITION%%%``` or with an else clause, ```%%%[...] if BOOLEAN_VAR else [...]%%%```. Note that the square brackets work like quotation mark strings in programming languages, so you can escape them with a backslash, like `\]`.
 
@@ -12,6 +12,7 @@ Would be loaded in JS like this:
 ```js
 const { importPromptSync } = require('langxlang')
 const prompt = importPromptSync('path-to-prompt.md', { NAME: 'Omega', HAS_PROMPT: true, IS_AI_STUDIO: false, LLM_NAME: 'Gemini 1.5 Pro' })
+console.log(prompt) // string
 ```
 
 And would result in:
@@ -38,50 +39,48 @@ You are running via API.
 
 Note: you can optionally put 2 spaces of tabulation in each line after an IF or ELSE block to make the code more readable. This is optional, and will be removed by the parser. If you actually want to put 2+ spaces of tabs, you can add an extra 2 spaces of tabulation (eg use 4 spaces to get 2, 6 to get 4, etc.).
 
-### Guidance Region
+#### Comments
 
-LLMs don't always give you the output you desire for your prompt. One approach to fixing
-this is by modifying your prompt to be more clear and adding more examples.
-Another standard approach is to provide some initial guidance to the model for what the response 
-should look like, to guarantee that the model will give you output similar to that you desire.
-For example, instead of just asking your model to output JSON or YAML and then
-ending the prompt with a question for the model to answer, you might end it with
-a markdown code block (like <code>```yml</code>), that the LLM would then complete the
-body for.
+You can specify a comment by using `<!---` and `-->` (note the additional dash in the starting token). This will be removed by the parser and not included in the output. We don't remove standard `<!--` comments to allow literal HTML comments in the markdown, which may be helpful when you expect markdown responses from the model.
 
-However, this can lead to messy code for you as you then have to prefix that guidance
-to the response you get from LXL. To make this easier, LXL provides a way to mark
-regions of your prompt as guidance. The guidance will then be automatically prepended
-to the model output you get from LXL. 
+### Roles
 
-As for how we send this data to the LLM, if the LLM is in chat mode, we split the user prompt
-by the base prompt and the guidance region, and mark the guidance region as being under
-the role `model` (Google Gemini API) or `assistant` (OpenAI GPT API) and the former as `system` or `user`.
-If the LLM is in completion mode, we simply append the guidance region to the prompt.
+LXL provides a way to take a markdown prompt template like above and then break up the message into a chat session, which includes several messages each with their own roles. This is opposed to getting a single prompt string from above. You can use all the above pre-processing features with this system. For example, given the following prompt template:
 
-To notate a region like this, you can use the following marker <code>%%%$GUIDANCE_START$%%%</code> in the prompt:
-
-User Prompt (prompt.md):
-<pre>
-Please convert this YAML to JSON:
-```yml
-hello: world
+```md
+<|SYSTEM|>
+Respond to the user like a pirate.
+<|USER|>
+How are you today?
+<|ASSISTANT|>
+Arrr, I be doin' well, matey! How can I help ye today?
+<|USER|>
+%%%(PROMPT)%%%
+<|ASSISTANT|>
 ```
-%%%$GUIDANCE_START$%%%
-```json
-</pre>
 
-This will result in:
-- role: system, message: "Please convert this YAML to JSON:\n```yml\nhello: world\n```\n"
-- role: model, message: "```json\n"
-
-And LXL's output will include the <code>```json</code> and the rest of the output as if they were both part of the model's output (this includes streaming).
+By using the following code:
+```js
+const { importPromptSync } = require('langxlang')
+const messages = importPromptSync('path-to-prompt.md', { PROMPT: 'What is the weather like?' }, {
+  roles: { // passing roles will return a messages array as opposed to a string
+    '<|SYSTEM|>': 'system',
+    '<|USER|>': 'user',
+    '<|ASSISTANT|>': 'assistant'
+  }
+})
+console.log(messages) // array
+```
 
-The usage in JS would look like:
+We can extract the following messages array that looks like this:
 ```js
-const { ChatSession, importPromptSync } = require('langxlang')
-const session = new ChatSession(service, 'gpt-3.5-turbo', '', {})
-// importPromptSync returns an object with the prompt and the guidance, that can be passed to sendMessage
-const prompt = importPromptSync('prompt.md', {})
-session.sendMessage(prompt).then(console.log)
-```
+[
+  { role: 'system', message: 'Respond to the user like a pirate.' },
+  { role: 'user', message: 'How are you today?' },
+  { role: 'assistant', message: 'Arrr, I be doin\' well, matey! How can I help ye today?' },
+  { role: 'user', message: 'What is the weather like?' },
+  // LXL will automatically remove empty messages
+]
+```
+
+This can be helpful for building dialogues beyond what a basic ChatMessage back and forth can give you. For more advanced use-cases (such as multi-round agents), see the [Flow](./flow.md) documentation.

docs/extras.md

@@ -0,0 +1,34 @@
+### Guidance Region
+
+LLMs don't always give you the output you desire for your prompt. One approach to fixing
+this is by modifying your prompt to be more clear and adding more examples.
+Another standard approach is to provide some initial guidance to the model for what the response 
+should look like, to guarantee that the model will give you output similar to that you desire.
+For example, instead of just asking your model to output JSON or YAML and then
+ending the prompt with a question for the model to answer, you might end it with
+a markdown code block (like <code>```yml</code>), that the LLM would then complete the
+body for.
+
+However, this can lead to messy code for you as you then have to prefix that guidance
+to the response you get from LXL. To make this easier, LXL provides a way to mark
+regions of your prompt as guidance. The guidance will then be automatically prepended
+to the model output you get from LXL. 
+
+By setting a message role as `guidance`, that message will be sent as a `model` or `assistant` (depending on platform) message to the LLM and then be prepended to the response you get from LXL.
+
+Here is an example:
+```js
+const { CompletionService } = require('langxlang')
+const service = new CompletionService()
+const [response] = await service.requestChatCompletion('gemini-1.0-pro', {
+  messages: [
+    { role: 'user', message: 'Please convert this YAML to JSON:\n```yml\nhello: world\n```\n' },
+    { role: 'guidance', message: '```json\n' }
+  ]
+})
+console.log(response) // { text: '```json\n{"hello": "world"}\n' }
+```
+
+Note: there can only be one guidance message and it must be the last one. You should remove
+it from the messages array the next call you do to requestChatCompletion. This feature works
+best when used with the role parsing system above.

docs/flow.md

@@ -38,12 +38,21 @@ Hello, how are you doing today on this %%%(DAY_OF_WEEK)%%%?
 %%%endif
 ````
 
-We can construct the following chain to ask the user how they are doing, 
-then ask them what day of the week tomorrow is, support a follow-up question 
-after the first response to ask the model to turn the response into YAML format:
+We can construct the following chain to:
+* ask the user how they are doing, 
+* then ask them what day of the week tomorrow is
+* then support a follow-up question after the first response (where they answered if they were ok), where we ask the model to turn its response into YAML format
+
 ```js
 const chain = (params) => ({
-  prompt: importRawSync('./prompt.md'),
+  prompt: {
+    // if we're using a prompt file with roles, we use specify `text` and `roles`. Otherwise, we can specify `user` and `system` as below.
+    text: importRawSync('./prompt.md'),
+    roles: {
+      '<USER>': 'user',
+      '<ASSISTANT>': 'assistant'
+    }
+  }
   with: {
     DAY_OF_WEEK: params.dayOfWeek
   },
@@ -93,7 +102,12 @@ A chain looks like this:
 ```js
 function chain (initialArguments) {
   return {
-    prompt: 'the prompt here',
+    prompt: {
+      // the system and user prompt will be propagated down the chain but you can override them at any point
+      // by specifying them in another `prompt` object
+      system: 'the system prompt here',
+      user: 'the user prompt here'
+    }
     // the with section contains the parameters to pass, like tools.loadPrompt(prompt, with)
     with: {
       SOME_PARAM: 'value'
@@ -116,7 +130,9 @@ and should return a string. The string will be used to find the next function to
 
 ```js
 const chain = (params) => ({
-  prompt: 'the prompt here',
+  prompt: {
+    user: 'the prompt here',
+  }
   with: {
     SOME_PARAM: 'value'
   },

src/ChatSession.js

@@ -177,7 +177,7 @@ class ChatSession {
       }
       response.content = message.guidanceText + response.content
     }
-    return { text: response.content, calledFunctions: this._calledFunctionsForRound }
+    return { content: response.content, text: response.content, calledFunctions: this._calledFunctionsForRound }
   }
 }