From 0f078d03a3e3ede1b577bfbef8b0b985827cd0bb Mon Sep 17 00:00:00 2001 From: Quarto GHA Workflow Runner Date: Fri, 13 Dec 2024 22:36:06 +0000 Subject: [PATCH] Built site for gh-pages --- .nojekyll | 2 +- index.html | 112 ++++++++++++++------------- prompt-design.html | 178 ++++++++++++++++++++----------------------- search.json | 20 ++--- sitemap.xml | 88 ++++++++++----------- structured-data.html | 105 +++++++++++-------------- tool-calling.html | 28 +++---- 7 files changed, 256 insertions(+), 277 deletions(-) diff --git a/.nojekyll b/.nojekyll index 85a7dc5..ea3835d 100644 --- a/.nojekyll +++ b/.nojekyll @@ -1 +1 @@ -c0e351eb \ No newline at end of file +dd8cf27c \ No newline at end of file diff --git a/index.html b/index.html index a921aaa..497e26b 100644 --- a/index.html +++ b/index.html @@ -209,12 +209,18 @@

On this page

chatlas

+

+ +PyPI MIT License Python Tests +

chatlas provides a simple and unified interface across large language model (llm) providers in Python. It abstracts away complexity from common tasks like streaming chat interfaces, tool calling, structured output, and much more. chatlas helps you prototype faster without painting you into a corner; for example, switching providers is as easy as changing one line of code, but provider specific features are still accessible when needed. Developer experience is also a key focus of chatlas: typing support, rich console output, and built-in tooling are all included.

(Looking for something similar to chatlas, but in R? Check out elmer!)

Install

-

chatlas isn’t yet on pypi, but you can install from Github:

-
pip install git+https://github.com/posit-dev/chatlas
+

Install the latest stable release from PyPI:

+
pip install -U chatlas
+

Or, install the latest development version from GitHub:

+
pip install -U git+https://github.com/posit-dev/chatlas

Model providers

@@ -251,21 +257,21 @@

Model choice

Using chatlas

You can chat via chatlas in several different ways, depending on whether you are working interactively or programmatically. They all start with creating a new chat object:

-
from chatlas import ChatOpenAI
-
-chat = ChatOpenAI(
-  model = "gpt-4o",
-  system_prompt = "You are a friendly but terse assistant.",
-)
+
from chatlas import ChatOpenAI
+
+chat = ChatOpenAI(
+  model = "gpt-4o",
+  system_prompt = "You are a friendly but terse assistant.",
+)

Interactive console

From a chat instance, it’s simple to start a web-based or terminal-based chat console, which is great for testing the capabilities of the model. In either case, responses stream in real-time, and context is preserved across turns.

-
chat.app()
-
+
chat.app()
+

A web app for chatting with an LLM via chatlas

Or, if you prefer to work from the terminal:

-
chat.console()
+
chat.console()
Entering chat console. Press Ctrl+C to quit.
 
 ?> Who created Python?
@@ -279,94 +285,94 @@ 
Interactive console
 
The .chat() method

 
For a more programmatic approach, you can use the .chat() method to ask a question and get a response. By default, the response prints to a rich console as it streams in:

-
chat.chat("What preceding languages most influenced Python?")

+
chat.chat("What preceding languages most influenced Python?")

 
Python was primarily influenced by ABC, with additional inspiration from C,
 Modula-3, and various other languages.

 
To ask a question about an image, pass one or more additional input arguments using content_image_file() and/or content_image_url():

-
from chatlas import content_image_url
-
-chat.chat(
-    content_image_url("https://www.python.org/static/img/python-logo.png"),
-    "Can you explain this logo?"
-)

+
from chatlas import content_image_url
+
+chat.chat(
+    content_image_url("https://www.python.org/static/img/python-logo.png"),
+    "Can you explain this logo?"
+)

 
The Python logo features two intertwined snakes in yellow and blue,
 representing the Python programming language. The design symbolizes...

 
To get the full response as a string, use the built-in str() function. Optionally, you can also suppress the rich console output by setting echo="none":

-
response = chat.chat("Who is Posit?", echo="none")
-print(str(response))

+
response = chat.chat("Who is Posit?", echo="none")
+print(str(response))

 
As we’ll see in later articles, echo="all" can also be useful for debugging, as it shows additional information, such as tool calls.

The .stream() method

If you want to do something with the response in real-time (i.e., as it arrives in chunks), use the .stream() method. This method returns an iterator that yields each chunk of the response as it arrives:

-
response = chat.stream("Who is Posit?")
-for chunk in response:
-    print(chunk, end="")
+
response = chat.stream("Who is Posit?")
+for chunk in response:
+    print(chunk, end="")

The .stream() method can also be useful if you’re building a chatbot or other programs that needs to display responses as they arrive.

Tool calling

Tool calling is as simple as passing a function with type hints and docstring to .register_tool().

-
import sys
-
-def get_current_python_version() -> str:
-    """Get the current version of Python."""
-    return sys.version
-
-chat.register_tool(get_current_python_version)
-chat.chat("What's the current version of Python?")
+
import sys
+
+def get_current_python_version() -> str:
+    """Get the current version of Python."""
+    return sys.version
+
+chat.register_tool(get_current_python_version)
+chat.chat("What's the current version of Python?")
The current version of Python is 3.13.

Learn more in the tool calling article

Structured data

Structured data (i.e., structured output) is as simple as passing a pydantic model to .extract_data().

-
from pydantic import BaseModel
-
-class Person(BaseModel):
-    name: str
-    age: int
-
-chat.extract_data(
-    "My name is Susan and I'm 13 years old", 
-    data_model=Person,
-)
+
from pydantic import BaseModel
+
+class Person(BaseModel):
+    name: str
+    age: int
+
+chat.extract_data(
+    "My name is Susan and I'm 13 years old", 
+    data_model=Person,
+)
{'name': 'Susan', 'age': 13}

Learn more in the structured data article

Export chat

Easily get a full markdown or HTML export of a conversation:

-
chat.export("index.html", title="Python Q&A")
+
chat.export("index.html", title="Python Q&A")

If the export doesn’t have all the information you need, you can also access the full conversation history via the .get_turns() method:

-
chat.get_turns()
+
chat.get_turns()

And, if the conversation is too long, you can specify which turns to include:

-
chat.export("index.html", turns=chat.get_turns()[-5:])
+
chat.export("index.html", turns=chat.get_turns()[-5:])

Async

chat methods tend to be synchronous by default, but you can use the async flavor by appending _async to the method name:

-
import asyncio
-
-async def main():
-    await chat.chat_async("What is the capital of France?")
-
-asyncio.run(main())
+
import asyncio
+
+async def main():
+    await chat.chat_async("What is the capital of France?")
+
+asyncio.run(main())

Typing support

chatlas has full typing support, meaning that, among other things, autocompletion just works in your favorite editor:

-
+

Autocompleting model options in ChatOpenAI

Troubleshooting

Sometimes things like token limits, tool errors, or other issues can cause problems that are hard to diagnose. In these cases, the echo="all" option is helpful for getting more information about what’s going on under the hood.

-
chat.chat("What is the capital of France?", echo="all")
+
chat.chat("What is the capital of France?", echo="all")

This shows important information like tool call results, finish reasons, and more.

If the problem isn’t self-evident, you can also reach into the .get_last_turn(), which contains the full response object, with full details about the completion.

-
+

Turn completion details with typing support

For monitoring issues in a production (or otherwise non-interactive) environment, you may want to enabling logging. Also, since chatlas builds on top of packages like anthropic and openai, you can also enable their debug logging to get lower-level information, like HTTP requests and response codes.

diff --git a/prompt-design.html b/prompt-design.html index 883e4b0..d342936 100644 --- a/prompt-design.html +++ b/prompt-design.html @@ -251,7 +251,7 @@

Best practices

Code generation

Let’s explore prompt design for a simple code generation task:

-
+
from chatlas import ChatAnthropic, ChatOpenAI
 
 question = """
@@ -262,11 +262,11 @@ 
Code generation

 

 
Basic flavour

 
When I don’t provide a system prompt, I sometimes get answers in a different language (like R):

-

+

 
chat = ChatAnthropic()
 _ = chat.chat(question)

 

-

+

 

 

 

@@ -284,14 +284,14 @@ 
Basic flavour

   )))
 
 # Alternative base R approach
-aggregate(. ~ age + sex, data = df[,c("age", "sex", letters)], 
+aggregate(. ~ age + sex, data = df[,c("age", "sex", letters[1:26])], 
          FUN = function(x) c(mean = mean(x), median = median(x)))

This will: 1. Group the data by age and sex 2. Calculate both mean and median for each variable a through z 3. Handle missing values with na.rm = TRUE 4. Return a dataframe with results for each age-sex combination

The output will have columns for age, sex, and mean/median values for each variable.

I can ensure that I always get Python code by providing a system prompt:

-
+
chat.system_prompt = "You are a helpful Python (not R) programming assistant."
 _ = chat.chat(question)
@@ -299,35 +299,34 @@

Basic flavour


-

Here’s how to compute mean and median for variables a through z using Python, particularly with pandas:

+

Here’s how to compute mean and median for variables a through z in Python using pandas:

import pandas as pd
 
 # Assuming your data is in a DataFrame called df
-result = df.groupby(['age', 'sex'])[list('abcdefghijklmnopqrstuvwxyz')].agg(['mean', 'median'])
-
-# If you want to reset the index to make age and sex regular columns
-result = result.reset_index()
-

Alternative approach with more explicit control:

-
import pandas as pd
-
-# Define the variables you want to analyze
-variables = list('abcdefghijklmnopqrstuvwxyz')
-
-# Group by age and sex, then calculate both statistics for each variable
-result = df.groupby(['age', 'sex'])[variables].agg({
-    var: ['mean', 'median'] for var in variables
-})
-
-# Optional: flatten column names
-result.columns = [f'{col[0]}_{col[1]}' for col in result.columns]
-result = result.reset_index()
-

This will: 1. Group your data by age and sex 2. Calculate both mean and median for each variable from ‘a’ to ‘z’ 3. Return a DataFrame with results for each age-sex combination

-

The resulting DataFrame will have a multi-level column structure where: - First level: variable names (a through z) - Second level: statistics (mean and median) - Rows will be unique age-sex combinations

+# Group by age and sex, then calculate mean and median for all variables a through z +result = df.groupby(['age', 'sex'])[list('abcdefghijklmnopqrstuvwxyz')].agg(['mean', 'median']) + +# If you want to handle missing values: +result = df.groupby(['age', 'sex'])[list('abcdefghijklmnopqrstuvwxyz')].agg({ + col: ['mean', 'median'] for col in list('abcdefghijklmnopqrstuvwxyz') +}).fillna(0) # or use .dropna() instead of fillna() if you prefer to remove NaN values + +# To make the output more readable, you can flatten the column names: +result.columns = [f'{col}_{stat}' for col, stat in result.columns]
+

This will: 1. Group your data by age and sex 2. Calculate mean and median for each variable from ‘a’ to ‘z’ 3. Return a DataFrame where: - The index will be the age and sex combinations - The columns will be each variable’s mean and median - Column names will be in the format ‘variable_statistic’ (e.g., ‘a_mean’, ‘a_median’, etc.)

+

Example output might look like:

+
                a_mean  a_median  b_mean  b_median  ...  z_mean  z_median
+age sex                                            ...
+20  M          23.5    24.0      45.2    46.0      ...  12.3    12.0
+    F          22.1    21.0      44.8    45.0      ...  11.9    12.0
+21  M          24.2    25.0      46.1    46.0      ...  12.5    13.0
+    F          23.8    24.0      45.9    46.0      ...  12.2    12.0
+...

Note that I’m using both a system prompt (which defines the general behaviour) and a user prompt (which asks the specific question). You could put all of the content in the user prompt and get similar results, but I think it’s helpful to use both to cleanly divide the general framing of the response from the specific questions that you want to ask.

Since I’m mostly interested in the code, I ask it to drop the explanation:

-
+
chat.system_prompt = """
   You are a helpful Python (not R) programming assistant.
   Just give me the code without any text explanation.
@@ -340,16 +339,12 @@ 
Basic flavour

 


 
import pandas as pd
 
-# Define variables
-vars = list('abcdefghijklmnopqrstuvwxyz')
-
-# Compute mean and median 
-result = df.groupby(['age', 'sex'])[vars].agg(['mean', 'median'])
-result = result.reset_index()

+result = df.groupby(['age', 'sex'])[list('abcdefghijklmnopqrstuvwxyz')].agg(['mean', 'median'])
+result.columns = [f'{col}_{stat}' for col, stat in result.columns]

In this case, I seem to mostly get pandas code. But if you want a different style, you can ask for it:

-
+
chat.system_prompt = """
   You are a helpful Python (not R) programming assistant who prefers polars to pandas.
   Just give me the code without any text explanation.
@@ -362,22 +357,18 @@ 
Basic flavour

 


 
import polars as pl
 
-vars = list('abcdefghijklmnopqrstuvwxyz')
-
-result = (df
-    .groupby(['age', 'sex'])
-    .agg([
-        pl.col(vars).mean(),
-        pl.col(vars).median()
-    ])
-)

+result = df.groupby(['age', 'sex']).agg([
+    pl.col(c).mean().alias(f'{c}_mean') for c in 'abcdefghijklmnopqrstuvwxyz'
+] + [
+    pl.col(c).median().alias(f'{c}_median') for c in 'abcdefghijklmnopqrstuvwxyz'
+])

Be explicit

If there’s something about the output that you don’t like, you can try being more explicit about it. For example, the code isn’t styled quite how I like, so I provide more details about what I do want:

-
+
chat.system_prompt = """
   You are a helpful Python (not R) programming assistant who prefers siuba to pandas.
   Just give me the code. I don't want any explanation or sample data.
@@ -394,20 +385,18 @@ 
Be explicit

 
from siuba import *
 from siuba.dply.vector import *
 
-vars = list('abcdefghijklmnopqrstuvwxyz')
-
-result = (df
-  >> group_by(_["age"], _["sex"])
-  >> summarize(
-    **{f"{var}_mean": _[var].mean() for var in vars},
-    **{f"{var}_median": _[var].median() for var in vars}
-  )
-)

+result = (df 
+  >> group_by(_.age, _.sex)
+  >> summarize(**{
+    f"{col}_{stat}": call(stat, _[col])
+    for col in "abcdefghijklmnopqrstuvwxyz"
+    for stat in ["mean", "median"]
+  }))

This still doesn’t yield exactly the code that I’d write, but it’s prety close.

You could provide a different prompt if you were looking for more explanation of the code:

-
+
chat.system_prompt = """
   You are an an expert Python (not R) programmer and a warm and supportive teacher.
   Help me understand the code you produce by explaining each function call with
@@ -421,28 +410,25 @@ 
Be explicit

 

 


 
import pandas as pd
-import numpy as np
-
-# Create list of letter variables
-vars = list('abcdefghijklmnopqrstuvwxyz')
-
-# Define both aggregation functions at once
-aggs = ['mean', 'median'] 
-
-# Group by age and sex, compute both stats for all letter variables
-result = (df
-    .groupby(['age', 'sex'])  # Group by these two variables
-    [vars]                    # Select only the letter columns
-    .agg(aggs)               # Calculate both statistics
-    .reset_index()           # Convert groups back to columns
-)

+
+# Create list of letters a-z to use as column names
+cols = list('abcdefghijklmnopqrstuvwxyz')
+
+# Group by age & sex, compute mean and median for all letter columns
+result = df.groupby(['age', 'sex'])[cols].agg([
+    'mean',    # Calculate mean for each column
+    'median'   # Calculate median for each column
+])
+
+# Flatten the multi-level column names for readability 
+result.columns = [f'{col}_{stat}' for col, stat in result.columns]

Teach it about new features

-

You can imagine LLMs as being a sort of an average of the internet at a given point in time. That means they will provide popular answers, which will tend to reflect older coding styles (either because the new features aren’t in their index, or the older features are so much more popular). So if you want your code to use specific features that are relatively recent, you might need to provide the examples yourself:

-
+

You can imagine LLMs as being a sort of an average of the internet at a given point in time. That means they will provide popular answers, which will tend to reflect older coding styles (either because the new features aren’t in their index, or the older features are so much more popular). So if you want your code to use specific newer language features, you might need to provide the examples yourself:

+
chat.system_prompt = """
   You are an expert R programmer.
   Just give me the code; no explanation in text.
@@ -481,11 +467,11 @@ 
Teach it about

Structured data

-

Providing a rich set of examples is a great way to encourage the output to produce exactly what you want. This is also known as multi-shot prompting. Here we’ll work through a prompt that I designed to extract structured data from recipes, but the same ideas apply in many other situations.

+

Providing a rich set of examples is a great way to encourage the output to produce exactly what you want. This is known as multi-shot prompting. Below we’ll work through a prompt that I designed to extract structured data from recipes, but the same ideas apply in many other situations.

Getting started

-

My overall goal is to turn a list of ingredients, like the following, into a nicely structured JSON that I can then analyse in Python (e.g. to compute the total weight, scale the recipe up or down, or to convert the units from volumes to weights).

-
+

My overall goal is to turn a list of ingredients, like the following, into a nicely structured JSON that I can then analyse in Python (e.g. compute the total weight, scale the recipe up or down, or convert the units from volumes to weights).

+
ingredients = """
   ¾ cup (150g) dark brown sugar
   2 large eggs
@@ -500,8 +486,8 @@ 
Getting started

 chat = ChatOpenAI(model="gpt-4o-mini")

(This isn’t the ingredient list for a real recipe but it includes a sampling of styles that I encountered in my project.)

-

If you don’t have strong feelings about what the data structure should look like, you can start with a very loose prompt and see what you get back. I find this a useful pattern for underspecified problems where a big part of the problem is just defining precisely what problem you want to solve. Seeing the LLM’s attempt at a data structure gives me something to immediately react to, rather than having to start from a blank page.

-
+

If you don’t have strong feelings about what the data structure should look like, you can start with a very loose prompt and see what you get back. I find this a useful pattern for underspecified problems where the heavy lifting lies with precisely defining the problem you want to solve. Seeing the LLM’s attempt to create a data structure gives me something to react to, rather than having to start from a blank page.

+
instruct_json = """
   You're an expert baker who also loves JSON. I am going to give you a list of
   ingredients and your job is to return nicely structured JSON. Just return the
@@ -514,15 +500,15 @@ 
Getting started


-

{ “ingredients”: [ { “name”: “dark brown sugar”, “amount”: “¾ cup”, “grams”: 150 }, { “name”: “large eggs”, “amount”: “2” }, { “name”: “sour cream”, “amount”: “¾ cup”, “grams”: 165 }, { “name”: “unsalted butter”, “amount”: “½ cup”, “grams”: 113, “state”: “melted” }, { “name”: “vanilla extract”, “amount”: “1 teaspoon” }, { “name”: “kosher salt”, “amount”: “¾ teaspoon” }, { “name”: “neutral oil”, “amount”: “⅓ cup”, “milliliters”: 80 }, { “name”: “all-purpose flour”, “amount”: “1½ cups”, “grams”: 190 }, { “name”: “sugar”, “amount”: “150g plus 1½ teaspoons” } ] }

+

{ “ingredients”: [ { “name”: “dark brown sugar”, “quantity”: “¾ cup”, “weight”: “150g” }, { “name”: “large eggs”, “quantity”: “2” }, { “name”: “sour cream”, “quantity”: “¾ cup”, “weight”: “165g” }, { “name”: “unsalted butter”, “quantity”: “½ cup”, “weight”: “113g”, “state”: “melted” }, { “name”: “vanilla extract”, “quantity”: “1 teaspoon” }, { “name”: “kosher salt”, “quantity”: “¾ teaspoon” }, { “name”: “neutral oil”, “quantity”: “⅓ cup”, “volume”: “80ml” }, { “name”: “all-purpose flour”, “quantity”: “1½ cups”, “weight”: “190g” }, { “name”: “sugar”, “quantity”: “150g plus 1½ teaspoons” } ] }

-

(I don’t know if the colour text, “You’re an expert baker who also loves JSON”, does anything, but I like to think this helps the LLM get into the right mindset of a very nerdy baker.)

+

(I don’t know if the additional colour, “You’re an expert baker who also loves JSON”, does anything, but I like to think this helps the LLM get into the right mindset of a very nerdy baker.)

Provide examples

This isn’t a bad start, but I prefer to cook with weight and I only want to see volumes if weight isn’t available so I provide a couple of examples of what I’m looking for. I was pleasantly suprised that I can provide the input and output examples in such a loose format.

-
+
instruct_weight = """
   Here are some examples of the sort of output I'm looking for:
 
@@ -543,11 +529,11 @@ 
Provide examples


-

{ “ingredients”: [ { “name”: “dark brown sugar”, “quantity”: 150, “unit”: “g” }, { “name”: “large eggs”, “quantity”: 2, “unit”: “count” }, { “name”: “sour cream”, “quantity”: 165, “unit”: “g” }, { “name”: “unsalted butter”, “quantity”: 113, “unit”: “g”, “state”: “melted” }, { “name”: “vanilla extract”, “quantity”: 1, “unit”: “teaspoon” }, { “name”: “kosher salt”, “quantity”: 0.75, “unit”: “teaspoon” }, { “name”: “neutral oil”, “quantity”: 80, “unit”: “ml” }, { “name”: “all-purpose flour”, “quantity”: 190, “unit”: “g” }, { “name”: “sugar”, “quantity”: 150, “unit”: “g plus 1½ teaspoons” } ] }

+

{ “ingredients”: [ { “name”: “dark brown sugar”, “quantity”: 150, “unit”: “g” }, { “name”: “large eggs”, “quantity”: 2, “unit”: “count” }, { “name”: “sour cream”, “quantity”: 165, “unit”: “g” }, { “name”: “unsalted butter”, “quantity”: 113, “unit”: “g”, “state”: “melted” }, { “name”: “vanilla extract”, “quantity”: 1, “unit”: “teaspoon” }, { “name”: “kosher salt”, “quantity”: ¾, “unit”: “teaspoon” }, { “name”: “neutral oil”, “quantity”: 80, “unit”: “ml” }, { “name”: “all-purpose flour”, “quantity”: 190, “unit”: “g” }, { “name”: “sugar”, “quantity”: “150g plus 1½ teaspoons”, “unit”: “g” } ] }

-

Just providing the examples seems to work remarkably well. But I found it useful to also include description of what the examples are trying to accomplish. I’m not sure if this helps the LLM or not, but it certainly makes it easier for me to understand the organisation and check that I’ve covered the key pieces that I’m interested in.

-
+

Just providing the examples seems to work remarkably well. But I found it useful to also include a description of what the examples are trying to accomplish. I’m not sure if this helps the LLM or not, but it certainly makes it easier for me to understand the organisation and check that I’ve covered the key pieces I’m interested in.

+
instruct_weight = """
   * If an ingredient has both weight and volume, extract only the weight:
 
@@ -567,8 +553,8 @@ 
Provide examples

 """

This structure also allows me to give the LLMs a hint about how I want multiple ingredients to be stored, i.e. as an JSON array.

-

I then iterated on the prompt, looking at the results from different recipes to get a sense of what the LLM was getting wrong. Much of this felt like I was iterating on my understanding of the problem as I didn’t start by knowing exactly how I wanted the data. For example, when I started out I didn’t really think about all the various ways that ingredients are specified. For later analysis, I always want quantities to be number, even if they were originally fractions, or the if the units aren’t precise (like a pinch). It made me to realise that some ingredients are unitless.

-
+

I then iterated on the prompt, looking at the results from different recipes to get a sense of what the LLM was getting wrong. Much of this felt like I was iterating on my own understanding of the problem as I didn’t start by knowing exactly how I wanted the data. For example, when I started out I didn’t really think about all the various ways that ingredients are specified. For later analysis, I always want quantities to be number, even if they were originally fractions, or the if the units aren’t precise (like a pinch). It made me to realise that some ingredients are unitless.

+
instruct_unit = """
 * If the unit uses a fraction, convert it to a decimal.
 
@@ -601,8 +587,8 @@ 
Provide examples

Structured data

-

Now that I’ve iterated to get a data structure that I like, it seems useful to formalise it and tell the LLM exactly what I’m looking for using structured data. This guarantees that the LLM will only return JSON, the JSON will have the fields that you expect, and that chatlas will convert it into an Python data structure for you.

-
+

Now that I’ve iterated to get a data structure I like, it seems useful to formalise it and tell the LLM exactly what I’m looking for when dealing with structured data. This guarantees that the LLM will only return JSON, that the JSON will have the fields that you expect, and that chatlas will convert it into an Python data structure for you.

+
from pydantic import BaseModel, Field
 
 class Ingredient(BaseModel):
@@ -625,14 +611,14 @@ 
Structured data

   {'name': 'kosher salt', 'quantity': 0.75, 'unit': 'teaspoon'},
   {'name': 'neutral oil', 'quantity': 80, 'unit': 'ml'},
   {'name': 'all-purpose flour', 'quantity': 190, 'unit': 'g'},
-  {'name': 'sugar', 'quantity': 150, 'unit': 'g plus 1½ teaspoons'}]}
+ {'name': 'sugar', 'quantity': 150, 'unit': 'g'}]}

Capturing raw input

-

One thing that I’d do next time would also be to include the raw ingredient name in the output. This doesn’t make much difference here, in this simple example, but it makes it much easier to align the input and the output and start to develop automated measures of how well my prompt is doing.

-
+

One thing that I’d do next time would also be to include the raw ingredient names in the output. This doesn’t make much difference in this simple example but it makes it much easier to align the input with the output and to start developing automated measures of how well my prompt is doing.

+
instruct_weight_input = """
   * If an ingredient has both weight and volume, extract only the weight:
 
@@ -652,7 +638,7 @@ 
Capturing raw input"""

I think this is particularly important if you’re working with even less structured text. For example, imagine you had this text:

-
+
recipe = """
   In a large bowl, cream together one cup of softened unsalted butter and a
   quarter cup of white sugar until smooth. Beat in an egg and 1 teaspoon of
@@ -665,7 +651,7 @@ 
Capturing raw input"""

Including the input text in the output makes it easier to see if it’s doing a good job:

-
+
chat.system_prompt = instruct_json + "\n" + instruct_weight_input
 _ = chat.chat(ingredients)
@@ -673,20 +659,20 @@

Capturing raw input


-

{ “ingredients”: [ { “name”: “dark brown sugar”, “quantity”: 150, “unit”: “g” }, { “name”: “large eggs”, “quantity”: 2, “unit”: “count” }, { “name”: “sour cream”, “quantity”: 165, “unit”: “g” }, { “name”: “unsalted butter”, “quantity”: 113, “unit”: “g”, “state”: “melted” }, { “name”: “vanilla extract”, “quantity”: 1, “unit”: “teaspoon” }, { “name”: “kosher salt”, “quantity”: 0.75, “unit”: “teaspoon” }, { “name”: “neutral oil”, “quantity”: 80, “unit”: “ml” }, { “name”: “all-purpose flour”, “quantity”: 190, “unit”: “g” }, { “name”: “sugar”, “quantity”: “150g plus 1½ teaspoons” } ] }

+

{ “ingredients”: [ { “name”: “dark brown sugar”, “quantity”: 150, “unit”: “g” }, { “name”: “large eggs”, “quantity”: 2, “unit”: “count” }, { “name”: “sour cream”, “quantity”: 165, “unit”: “g” }, { “name”: “unsalted butter”, “quantity”: 113, “unit”: “g”, “state”: “melted” }, { “name”: “vanilla extract”, “quantity”: 1, “unit”: “teaspoon” }, { “name”: “kosher salt”, “quantity”: 0.75, “unit”: “teaspoon” }, { “name”: “neutral oil”, “quantity”: 80, “unit”: “ml” }, { “name”: “all-purpose flour”, “quantity”: 190, “unit”: “g” }, { “name”: “sugar”, “quantity”: “150g plus 1½ teaspoons”, “unit”: “g” } ] }

-

When I ran it while writing this vignette, it seems to be working out the weight of the ingredients specified in volume, even though the prompt specifically asks it not to do that. This may suggest I need to broaden my examples.

+

When I ran it while writing this vignette, it seemed to be working out the weight of the ingredients specified in volume, even though the prompt specifically asks it not to. This may suggest I need to broaden my examples.

Token usage

-
+
from chatlas import token_usage
 token_usage()
-
[{'name': 'Anthropic', 'input': 5583, 'output': 1083},
- {'name': 'OpenAI', 'input': 2778, 'output': 918}]
+
[{'name': 'Anthropic', 'input': 6504, 'output': 1270},
+ {'name': 'OpenAI', 'input': 2909, 'output': 1043}]
diff --git a/search.json b/search.json index f09b24a..851c364 100644 --- a/search.json +++ b/search.json @@ -823,7 +823,7 @@ "href": "structured-data.html#structured-data-basics", "title": "Structured data", "section": "Structured data basics", - "text": "Structured data basics\nTo extract structured data you call the .extract_data() method instead of the .chat() method. You’ll also need to define a type specification that describes the structure of the data that you want (more on that shortly). Here’s a simple example that extracts two specific values from a string:\n\nclass Person(BaseModel):\n name: str\n age: int\n\n\nchat = ChatOpenAI()\nchat.extract_data(\n \"My name is Susan and I'm 13 years old\", \n data_model=Person,\n)\n\n{'name': 'Susan', 'age': 13}\n\n\nThe same basic idea works with images too:\n\nfrom chatlas import content_image_url\n\nclass Image(BaseModel):\n primary_shape: str\n primary_colour: str\n\nchat.extract_data(\n content_image_url(\"https://www.r-project.org/Rlogo.png\"),\n data_model=Image,\n)\n\n{'primary_shape': 'letter and oval', 'primary_colour': 'blue and gray'}" + "text": "Structured data basics\nTo extract structured data you call the .extract_data() method instead of the .chat() method. You’ll also need to define a type specification that describes the structure of the data that you want (more on that shortly). Here’s a simple example that extracts two specific values from a string:\n\nclass Person(BaseModel):\n name: str\n age: int\n\n\nchat = ChatOpenAI()\nchat.extract_data(\n \"My name is Susan and I'm 13 years old\", \n data_model=Person,\n)\n\n{'name': 'Susan', 'age': 13}\n\n\nThe same basic idea works with images too:\n\nfrom chatlas import content_image_url\n\nclass Image(BaseModel):\n primary_shape: str\n primary_colour: str\n\nchat.extract_data(\n content_image_url(\"https://www.r-project.org/Rlogo.png\"),\n data_model=Image,\n)\n\n{'primary_shape': 'oval and letter', 'primary_colour': 'gray and blue'}" }, { "objectID": "structured-data.html#data-types-basics", @@ -837,21 +837,21 @@ "href": "structured-data.html#examples", "title": "Structured data", "section": "Examples", - "text": "Examples\nThe following examples are closely inspired by the Claude documentation and hint at some of the ways you can use structured data extraction.\n\nExample 1: Article summarisation\n\nwith open(\"examples/third-party-testing.txt\") as f:\n text = f.read()\n\n\nclass ArticleSummary(BaseModel):\n \"\"\"Summary of the article.\"\"\"\n\n author: str = Field(description=\"Name of the article author\")\n\n topics: list[str] = Field(\n description=\"Array of topics, e.g. ['tech', 'politics']. Should be as specific as possible, and can overlap.\"\n )\n\n summary: str = Field(description=\"Summary of the article. One or two paragraphs max\")\n\n coherence: int = Field(\n description=\"Coherence of the article's key points, 0-100 (inclusive)\"\n )\n\n persuasion: float = Field(\n description=\"Article's persuasion score, 0.0-1.0 (inclusive)\"\n )\n\n\nchat = ChatOpenAI()\ndata = chat.extract_data(text, data_model=ArticleSummary)\nprint(json.dumps(data, indent=2))\n\n{\n \"author\": \"Anthropic\",\n \"topics\": [\n \"AI Policy\",\n \"Third-party Testing\",\n \"AI Safety\",\n \"AI Regulation\",\n \"Emerging Technologies\",\n \"Security and Ethics\"\n ],\n \"summary\": \"**Summary:** \\nThe article discusses the importance of implementing a robust third-party testing regime for advanced AI systems to ensure their safe deployment and mitigate potential societal risks. As AI models become more powerful and integrated into various sectors, there is a growing need for regulation and oversight to prevent misuse and accidental harm. The proposed testing regime would facilitate collaboration between industry, academia, and government to set safety standards, address national security concerns, and ensure equitable participation by companies of all sizes. The article also highlights the risks of regulatory capture and emphasizes the need for transparency and inclusivity in AI policy development. Anthropic, a developer of AI systems, advocates for these measures and outlines specific actions they will take to support effective regulatory frameworks.\",\n \"coherence\": 85,\n \"persuasion\": 0.8\n}\n\n\n\n\nExample 2: Named entity recognition\n\ntext = \"John works at Google in New York. He met with Sarah, the CEO of Acme Inc., last week in San Francisco.\"\n\n\nclass NamedEntity(BaseModel):\n \"\"\"Named entity in the text.\"\"\"\n\n name: str = Field(description=\"The extracted entity name\")\n\n type_: str = Field(description=\"The entity type, e.g. 'person', 'location', 'organization'\")\n\n context: str = Field(description=\"The context in which the entity appears in the text.\")\n\n\nclass NamedEntities(BaseModel):\n \"\"\"Named entities in the text.\"\"\"\n\n entities: list[NamedEntity] = Field(description=\"Array of named entities\")\n\n\nchat = ChatOpenAI()\ndata = chat.extract_data(text, data_model=NamedEntities)\npd.DataFrame(data[\"entities\"])\n\n\n\n\n\n\n\n\nname\ntype_\ncontext\n\n\n\n\n0\nJohn\nperson\nWorks at Google in New York\n\n\n1\nGoogle\norganization\nJohn works at Google in New York\n\n\n2\nNew York\nlocation\nJohn works there\n\n\n3\nSarah\nperson\nMet John in San Francisco\n\n\n4\nCEO\ntitle\nSarah's position at Acme Inc.\n\n\n5\nAcme Inc.\norganization\nSarah is the CEO\n\n\n6\nSan Francisco\nlocation\nJohn met with Sarah there last week\n\n\n7\nlast week\ntemporal\nWhen John met with Sarah\n\n\n\n\n\n\n\n\n\nExample 3: Sentiment analysis\n\ntext = \"The product was okay, but the customer service was terrible. I probably won't buy from them again.\"\n\nclass Sentiment(BaseModel):\n \"\"\"Extract the sentiment scores of a given text. Sentiment scores should sum to 1.\"\"\"\n\n positive_score: float = Field(\n description=\"Positive sentiment score, ranging from 0.0 to 1.0\"\n )\n\n negative_score: float = Field(\n description=\"Negative sentiment score, ranging from 0.0 to 1.0\"\n )\n\n neutral_score: float = Field(\n description=\"Neutral sentiment score, ranging from 0.0 to 1.0\"\n )\n\n\nchat = ChatOpenAI()\nchat.extract_data(text, data_model=Sentiment)\n\n{'positive_score': 0.1, 'negative_score': 0.7, 'neutral_score': 0.2}\n\n\nNote that we’ve asked nicely for the scores to sum 1, and they do in this example (at least when I ran the code), but it’s not guaranteed.\n\n\nExample 4: Text classification\n\nfrom typing import Literal\n\ntext = \"The new quantum computing breakthrough could revolutionize the tech industry.\"\n\n\nclass Classification(BaseModel):\n\n name: Literal[\n \"Politics\", \"Sports\", \"Technology\", \"Entertainment\", \"Business\", \"Other\"\n ] = Field(description=\"The category name\")\n\n score: float = Field(description=\"The classification score for the category, ranging from 0.0 to 1.0.\")\n\n\nclass Classifications(BaseModel):\n \"\"\"Array of classification results. The scores should sum to 1.\"\"\"\n\n classifications: list[Classification]\n\n\nchat = ChatOpenAI()\ndata = chat.extract_data(text, data_model=Classifications)\npd.DataFrame(data[\"classifications\"])\n\n\n\n\n\n\n\n\nname\nscore\n\n\n\n\n0\nTechnology\n0.85\n\n\n1\nBusiness\n0.10\n\n\n2\nOther\n0.05\n\n\n\n\n\n\n\n\n\nExample 5: Working with unknown keys\n\nfrom chatlas import ChatAnthropic\n\n\nclass Characteristics(BaseModel, extra=\"allow\"):\n \"\"\"All characteristics\"\"\"\n\n pass\n\n\nprompt = \"\"\"\n Given a description of a character, your task is to extract all the characteristics of that character.\n\n <description>\n The man is tall, with a beard and a scar on his left cheek. He has a deep voice and wears a black leather jacket.\n </description>\n\"\"\"\n\nchat = ChatAnthropic()\ndata = chat.extract_data(prompt, data_model=Characteristics)\nprint(json.dumps(data, indent=2))\n\n{\n \"physical_characteristics\": {\n \"height\": \"tall\",\n \"facial_features\": {\n \"beard\": true,\n \"scar\": {\n \"location\": \"left cheek\"\n }\n },\n \"voice\": \"deep\"\n },\n \"clothing\": {\n \"outerwear\": {\n \"type\": \"leather jacket\",\n \"color\": \"black\"\n }\n }\n}\n\n\nThis example only works with Claude, not GPT or Gemini, because only Claude supports adding arbitrary additional properties.\n\n\nExample 6: Extracting data from an image\nThis example comes from Dan Nguyen and you can see other interesting applications at that link. The goal is to extract structured data from this screenshot:\nThe goal is to extract structured data from this screenshot:\n\n\n\nA screenshot of schedule A: a table showing assets and “unearned” income\n\n\nEven without any descriptions, ChatGPT does pretty well:\n\nfrom chatlas import content_image_file\n\n\nclass Asset(BaseModel):\n assert_name: str\n owner: str\n location: str\n asset_value_low: int\n asset_value_high: int\n income_type: str\n income_low: int\n income_high: int\n tx_gt_1000: bool\n\n\nclass DisclosureReport(BaseModel):\n assets: list[Asset]\n\n\nchat = ChatOpenAI()\ndata = chat.extract_data(\n content_image_file(\"images/congressional-assets.png\"), data_model=DisclosureReport\n)\npd.DataFrame(data[\"assets\"])\n\n\n\n\n\n\n\n\nassert_name\nowner\nlocation\nasset_value_low\nasset_value_high\nincome_type\nincome_low\nincome_high\ntx_gt_1000\n\n\n\n\n0\n11 Zinfandel Lane - Home & Vineyard\nJT\nSt. Helena/Napa, CA, US\n5000001\n25000000\nGrape Sales\n100001\n1000000\nTrue\n\n\n1\n25 Point Lobos - Commercial Property\nSP\nSan Francisco/San Francisco, CA, US\n5000001\n25000000\nRent\n100001\n1000000\nFalse" + "text": "Examples\nThe following examples are closely inspired by the Claude documentation and hint at some of the ways you can use structured data extraction.\n\nExample 1: Article summarisation\n\nwith open(\"examples/third-party-testing.txt\") as f:\n text = f.read()\n\n\nclass ArticleSummary(BaseModel):\n \"\"\"Summary of the article.\"\"\"\n\n author: str = Field(description=\"Name of the article author\")\n\n topics: list[str] = Field(\n description=\"Array of topics, e.g. ['tech', 'politics']. Should be as specific as possible, and can overlap.\"\n )\n\n summary: str = Field(description=\"Summary of the article. One or two paragraphs max\")\n\n coherence: int = Field(\n description=\"Coherence of the article's key points, 0-100 (inclusive)\"\n )\n\n persuasion: float = Field(\n description=\"Article's persuasion score, 0.0-1.0 (inclusive)\"\n )\n\n\nchat = ChatOpenAI()\ndata = chat.extract_data(text, data_model=ArticleSummary)\nprint(json.dumps(data, indent=2))\n\n{\n \"author\": \"N/A\",\n \"topics\": [\n \"AI policy\",\n \"third-party testing\",\n \"technology safety\",\n \"regulatory capture\",\n \"AI regulation\"\n ],\n \"summary\": \"The article discusses the importance of implementing a robust third-party testing regime for frontier AI systems to ensure their safety and mitigate the risks of societal harm. It emphasizes the need for involvement from industry, government, and academia to develop standards and procedures for testing. Third-party testing is considered critical for addressing potential misuse or accidents associated with large-scale generative AI models. The article outlines the goals of developing an effective testing regime, which includes building trust, ensuring safety measures for powerful systems, and promoting coordination between countries. There is also discussion about balancing transparency and preventing harmful uses, as well as the potential for regulatory capture. Ultimately, this testing regime aims to complement sector-specific regulations and enable effective oversight of AI development.\",\n \"coherence\": 85,\n \"persuasion\": 0.89\n}\n\n\n\n\nExample 2: Named entity recognition\n\ntext = \"John works at Google in New York. He met with Sarah, the CEO of Acme Inc., last week in San Francisco.\"\n\n\nclass NamedEntity(BaseModel):\n \"\"\"Named entity in the text.\"\"\"\n\n name: str = Field(description=\"The extracted entity name\")\n\n type_: str = Field(description=\"The entity type, e.g. 'person', 'location', 'organization'\")\n\n context: str = Field(description=\"The context in which the entity appears in the text.\")\n\n\nclass NamedEntities(BaseModel):\n \"\"\"Named entities in the text.\"\"\"\n\n entities: list[NamedEntity] = Field(description=\"Array of named entities\")\n\n\nchat = ChatOpenAI()\ndata = chat.extract_data(text, data_model=NamedEntities)\npd.DataFrame(data[\"entities\"])\n\n\n\n\n\n\n\n\nname\ntype_\ncontext\n\n\n\n\n0\nJohn\nperson\nJohn works at Google in New York.\n\n\n1\nGoogle\norganization\nJohn works at Google in New York.\n\n\n2\nNew York\nlocation\nJohn works at Google in New York.\n\n\n3\nSarah\nperson\nHe met with Sarah, the CEO of Acme Inc., last ...\n\n\n4\nAcme Inc.\norganization\nHe met with Sarah, the CEO of Acme Inc., last ...\n\n\n5\nSan Francisco\nlocation\nHe met with Sarah, the CEO of Acme Inc., last ...\n\n\n\n\n\n\n\n\n\nExample 3: Sentiment analysis\n\ntext = \"The product was okay, but the customer service was terrible. I probably won't buy from them again.\"\n\nclass Sentiment(BaseModel):\n \"\"\"Extract the sentiment scores of a given text. Sentiment scores should sum to 1.\"\"\"\n\n positive_score: float = Field(\n description=\"Positive sentiment score, ranging from 0.0 to 1.0\"\n )\n\n negative_score: float = Field(\n description=\"Negative sentiment score, ranging from 0.0 to 1.0\"\n )\n\n neutral_score: float = Field(\n description=\"Neutral sentiment score, ranging from 0.0 to 1.0\"\n )\n\n\nchat = ChatOpenAI()\nchat.extract_data(text, data_model=Sentiment)\n\n{'positive_score': 0.1, 'negative_score': 0.7, 'neutral_score': 0.2}\n\n\nNote that we’ve asked nicely for the scores to sum 1, and they do in this example (at least when I ran the code), but it’s not guaranteed.\n\n\nExample 4: Text classification\n\nfrom typing import Literal\n\ntext = \"The new quantum computing breakthrough could revolutionize the tech industry.\"\n\n\nclass Classification(BaseModel):\n\n name: Literal[\n \"Politics\", \"Sports\", \"Technology\", \"Entertainment\", \"Business\", \"Other\"\n ] = Field(description=\"The category name\")\n\n score: float = Field(description=\"The classification score for the category, ranging from 0.0 to 1.0.\")\n\n\nclass Classifications(BaseModel):\n \"\"\"Array of classification results. The scores should sum to 1.\"\"\"\n\n classifications: list[Classification]\n\n\nchat = ChatOpenAI()\ndata = chat.extract_data(text, data_model=Classifications)\npd.DataFrame(data[\"classifications\"])\n\n\n\n\n\n\n\n\nname\nscore\n\n\n\n\n0\nTechnology\n0.7\n\n\n1\nBusiness\n0.2\n\n\n2\nOther\n0.1\n\n\n\n\n\n\n\n\n\nExample 5: Working with unknown keys\n\nfrom chatlas import ChatAnthropic\n\n\nclass Characteristics(BaseModel, extra=\"allow\"):\n \"\"\"All characteristics\"\"\"\n\n pass\n\n\nprompt = \"\"\"\n Given a description of a character, your task is to extract all the characteristics of that character.\n\n <description>\n The man is tall, with a beard and a scar on his left cheek. He has a deep voice and wears a black leather jacket.\n </description>\n\"\"\"\n\nchat = ChatAnthropic()\ndata = chat.extract_data(prompt, data_model=Characteristics)\nprint(json.dumps(data, indent=2))\n\n{\n \"appearance\": {\n \"height\": \"tall\",\n \"facial_features\": {\n \"beard\": true,\n \"scar\": {\n \"location\": \"left cheek\"\n }\n },\n \"voice\": \"deep\",\n \"clothing\": {\n \"outerwear\": {\n \"type\": \"leather jacket\",\n \"color\": \"black\"\n }\n }\n }\n}\n\n\nThis example only works with Claude, not GPT or Gemini, because only Claude supports adding arbitrary additional properties.\n\n\nExample 6: Extracting data from an image\nThis example comes from Dan Nguyen and you can see other interesting applications at that link. The goal is to extract structured data from this screenshot:\nThe goal is to extract structured data from this screenshot:\n\n\n\nA screenshot of schedule A: a table showing assets and “unearned” income\n\n\nEven without any descriptions, ChatGPT does pretty well:\n\nfrom chatlas import content_image_file\n\n\nclass Asset(BaseModel):\n assert_name: str\n owner: str\n location: str\n asset_value_low: int\n asset_value_high: int\n income_type: str\n income_low: int\n income_high: int\n tx_gt_1000: bool\n\n\nclass DisclosureReport(BaseModel):\n assets: list[Asset]\n\n\nchat = ChatOpenAI()\ndata = chat.extract_data(\n content_image_file(\"images/congressional-assets.png\"), data_model=DisclosureReport\n)\npd.DataFrame(data[\"assets\"])\n\n\n\n\n\n\n\n\nassert_name\nowner\nlocation\nasset_value_low\nasset_value_high\nincome_type\nincome_low\nincome_high\ntx_gt_1000\n\n\n\n\n0\n11 Zinfandel Lane - Home & Vineyard [RP]\nJT\nSt. Helena/Napa, CA, US\n5000001\n25000000\nGrape Sales\n100001\n1000000\nTrue\n\n\n1\n25 Point Lobos - Commercial Property [RP]\nSP\nSan Francisco/San Francisco, CA, US\n5000001\n25000000\nRent\n100001\n1000000\nFalse" }, { "objectID": "structured-data.html#advanced-data-types", "href": "structured-data.html#advanced-data-types", "title": "Structured data", "section": "Advanced data types", - "text": "Advanced data types\nNow that you’ve seen a few examples, it’s time to get into more specifics about data type declarations.\n\nRequired vs optional\nBy default, model fields are in a sense “required”, unless None is allowed in their type definition. Including None is a good idea if there’s any possibility of the input not containing the required fields as LLMs may hallucinate data in order to fulfill your spec.\nFor example, here the LLM hallucinates a date even though there isn’t one in the text:\n\nclass ArticleSpec(BaseModel):\n \"\"\"Information about an article written in markdown\"\"\"\n\n title: str = Field(description=\"Article title\")\n author: str = Field(description=\"Name of the author\")\n date: str = Field(description=\"Date written in YYYY-MM-DD format.\")\n\n\nprompt = \"\"\"\n Extract data from the following text:\n\n <text>\n # Structured Data\n By Hadley Wickham\n\n When using an LLM to extract data from text or images, you can ask the chatbot to nicely format it, in JSON or any other format that you like.\n </text>\n\"\"\"\n\nchat = ChatOpenAI()\ndata = chat.extract_data(prompt, data_model=ArticleSpec)\nprint(json.dumps(data, indent=2))\n\n{\n \"title\": \"Structured Data\",\n \"author\": \"Hadley Wickham\",\n \"date\": \"2023-10-16\"\n}\n\n\nNote that I’ve used more of an explict prompt here. For this example, I found that this generated better results, and it’s a useful place to put additional instructions.\nIf let the LLM know that the fields are all optional, it’ll instead return None for the missing fields:\n\nclass ArticleSpec(BaseModel):\n \"\"\"Information about an article written in markdown\"\"\"\n\n title: str = Field(description=\"Article title\")\n author: str = Field(description=\"Name of the author\")\n date: str | None = Field(description=\"Date written in YYYY-MM-DD format.\")\n\n\ndata = chat.extract_data(prompt, data_model=ArticleSpec)\nprint(json.dumps(data, indent=2))\n\n{\n \"title\": \"Structured Data\",\n \"author\": \"Hadley Wickham\",\n \"date\": null\n}\n\n\n\n\nData frames\nIf you want to define a data frame like data_model, you might be tempted to create a model like this, where each field is a list of scalar values:\nclass Persons(BaseModel):\n name: list[str]\n age: list[int]\nThis however, is not quite right because there’s no way to specify that each field should have the same length. Instead you need to turn the data structure “inside out”, and instead create an array of objects:\nclass Person(BaseModel):\n name: str\n age: int\n\nclass Persons(BaseModel):\n persons: list[Person]\nIf you’re familiar with the terms between row-oriented and column-oriented data frames, this is the same idea." + "text": "Advanced data types\nNow that you’ve seen a few examples, it’s time to get into more specifics about data type declarations.\n\nRequired vs optional\nBy default, model fields are in a sense “required”, unless None is allowed in their type definition. Including None is a good idea if there’s any possibility of the input not containing the required fields as LLMs may hallucinate data in order to fulfill your spec.\nFor example, here the LLM hallucinates a date even though there isn’t one in the text:\n\nclass ArticleSpec(BaseModel):\n \"\"\"Information about an article written in markdown\"\"\"\n\n title: str = Field(description=\"Article title\")\n author: str = Field(description=\"Name of the author\")\n date: str = Field(description=\"Date written in YYYY-MM-DD format.\")\n\n\nprompt = \"\"\"\n Extract data from the following text:\n\n <text>\n # Structured Data\n By Hadley Wickham\n\n When using an LLM to extract data from text or images, you can ask the chatbot to nicely format it, in JSON or any other format that you like.\n </text>\n\"\"\"\n\nchat = ChatOpenAI()\ndata = chat.extract_data(prompt, data_model=ArticleSpec)\nprint(json.dumps(data, indent=2))\n\n{\n \"title\": \"Structured Data\",\n \"author\": \"Hadley Wickham\",\n \"date\": \"2023-10-07\"\n}\n\n\nNote that I’ve used more of an explict prompt here. For this example, I found that this generated better results, and it’s a useful place to put additional instructions.\nIf let the LLM know that the fields are all optional, it’ll instead return None for the missing fields:\n\nclass ArticleSpec(BaseModel):\n \"\"\"Information about an article written in markdown\"\"\"\n\n title: str = Field(description=\"Article title\")\n author: str = Field(description=\"Name of the author\")\n date: str | None = Field(description=\"Date written in YYYY-MM-DD format.\")\n\n\ndata = chat.extract_data(prompt, data_model=ArticleSpec)\nprint(json.dumps(data, indent=2))\n\n{\n \"title\": \"Structured Data\",\n \"author\": \"Hadley Wickham\",\n \"date\": null\n}\n\n\n\n\nData frames\nIf you want to define a data frame like data_model, you might be tempted to create a model like this, where each field is a list of scalar values:\nclass Persons(BaseModel):\n name: list[str]\n age: list[int]\nThis however, is not quite right because there’s no way to specify that each field should have the same length. Instead you need to turn the data structure “inside out”, and instead create an array of objects:\nclass Person(BaseModel):\n name: str\n age: int\n\nclass Persons(BaseModel):\n persons: list[Person]\nIf you’re familiar with the terms between row-oriented and column-oriented data frames, this is the same idea." }, { "objectID": "structured-data.html#token-usage", "href": "structured-data.html#token-usage", "title": "Structured data", "section": "Token usage", - "text": "Token usage\nBelow is a summary of the tokens used to create the output in this example.\n\nfrom chatlas import token_usage\ntoken_usage()\n\n[{'name': 'OpenAI', 'input': 6081, 'output': 609},\n {'name': 'Anthropic', 'input': 463, 'output': 139}]" + "text": "Token usage\nBelow is a summary of the tokens used to create the output in this example.\n\nfrom chatlas import token_usage\ntoken_usage()\n\n[{'name': 'OpenAI', 'input': 6081, 'output': 615},\n {'name': 'Anthropic', 'input': 463, 'output': 137}]" }, { "objectID": "prompt-design.html", @@ -872,42 +872,42 @@ "href": "prompt-design.html#code-generation", "title": "Prompt design", "section": "Code generation", - "text": "Code generation\nLet’s explore prompt design for a simple code generation task:\n\nfrom chatlas import ChatAnthropic, ChatOpenAI\n\nquestion = \"\"\"\n How can I compute the mean and median of variables a, b, c, and so on,\n all the way up to z, grouped by age and sex.\n\"\"\"\n\n\nBasic flavour\nWhen I don’t provide a system prompt, I sometimes get answers in a different language (like R):\n\nchat = ChatAnthropic()\n_ = chat.chat(question)\n\n\n\n\n\n\n\nHere’s how to compute mean and median for variables a through z, grouped by age and sex:\n# Using dplyr\nlibrary(dplyr)\n\ndf %>%\n group_by(age, sex) %>%\n summarise(across(a:z, list(\n mean = ~mean(., na.rm = TRUE),\n median = ~median(., na.rm = TRUE)\n )))\n\n# Alternative base R approach\naggregate(. ~ age + sex, data = df[,c(\"age\", \"sex\", letters)], \n FUN = function(x) c(mean = mean(x), median = median(x)))\nThis will: 1. Group the data by age and sex 2. Calculate both mean and median for each variable a through z 3. Handle missing values with na.rm = TRUE 4. Return a dataframe with results for each age-sex combination\nThe output will have columns for age, sex, and mean/median values for each variable.\n\n\nI can ensure that I always get Python code by providing a system prompt:\n\nchat.system_prompt = \"You are a helpful Python (not R) programming assistant.\"\n_ = chat.chat(question)\n\n\n\n\n\nHere’s how to compute mean and median for variables a through z using Python, particularly with pandas:\nimport pandas as pd\n\n# Assuming your data is in a DataFrame called df\nresult = df.groupby(['age', 'sex'])[list('abcdefghijklmnopqrstuvwxyz')].agg(['mean', 'median'])\n\n# If you want to reset the index to make age and sex regular columns\nresult = result.reset_index()\nAlternative approach with more explicit control:\nimport pandas as pd\n\n# Define the variables you want to analyze\nvariables = list('abcdefghijklmnopqrstuvwxyz')\n\n# Group by age and sex, then calculate both statistics for each variable\nresult = df.groupby(['age', 'sex'])[variables].agg({\n var: ['mean', 'median'] for var in variables\n})\n\n# Optional: flatten column names\nresult.columns = [f'{col[0]}_{col[1]}' for col in result.columns]\nresult = result.reset_index()\nThis will: 1. Group your data by age and sex 2. Calculate both mean and median for each variable from ‘a’ to ‘z’ 3. Return a DataFrame with results for each age-sex combination\nThe resulting DataFrame will have a multi-level column structure where: - First level: variable names (a through z) - Second level: statistics (mean and median) - Rows will be unique age-sex combinations\n\n\nNote that I’m using both a system prompt (which defines the general behaviour) and a user prompt (which asks the specific question). You could put all of the content in the user prompt and get similar results, but I think it’s helpful to use both to cleanly divide the general framing of the response from the specific questions that you want to ask.\nSince I’m mostly interested in the code, I ask it to drop the explanation:\n\nchat.system_prompt = \"\"\"\n You are a helpful Python (not R) programming assistant.\n Just give me the code without any text explanation.\n\"\"\"\n_ = chat.chat(question)\n\n\n\n\n\nimport pandas as pd\n\n# Define variables\nvars = list('abcdefghijklmnopqrstuvwxyz')\n\n# Compute mean and median \nresult = df.groupby(['age', 'sex'])[vars].agg(['mean', 'median'])\nresult = result.reset_index()\n\n\nIn this case, I seem to mostly get pandas code. But if you want a different style, you can ask for it:\n\nchat.system_prompt = \"\"\"\n You are a helpful Python (not R) programming assistant who prefers polars to pandas.\n Just give me the code without any text explanation.\n\"\"\"\n_ = chat.chat(question)\n\n\n\n\n\nimport polars as pl\n\nvars = list('abcdefghijklmnopqrstuvwxyz')\n\nresult = (df\n .groupby(['age', 'sex'])\n .agg([\n pl.col(vars).mean(),\n pl.col(vars).median()\n ])\n)\n\n\n\n\nBe explicit\nIf there’s something about the output that you don’t like, you can try being more explicit about it. For example, the code isn’t styled quite how I like, so I provide more details about what I do want:\n\nchat.system_prompt = \"\"\"\n You are a helpful Python (not R) programming assistant who prefers siuba to pandas.\n Just give me the code. I don't want any explanation or sample data.\n * Spread long function calls across multiple lines.\n * Where needed, always indent function calls with two spaces.\n * Always use double quotes for strings.\n\"\"\"\n_ = chat.chat(question)\n\n\n\n\n\nfrom siuba import *\nfrom siuba.dply.vector import *\n\nvars = list('abcdefghijklmnopqrstuvwxyz')\n\nresult = (df\n >> group_by(_[\"age\"], _[\"sex\"])\n >> summarize(\n **{f\"{var}_mean\": _[var].mean() for var in vars},\n **{f\"{var}_median\": _[var].median() for var in vars}\n )\n)\n\n\nThis still doesn’t yield exactly the code that I’d write, but it’s prety close.\nYou could provide a different prompt if you were looking for more explanation of the code:\n\nchat.system_prompt = \"\"\"\n You are an an expert Python (not R) programmer and a warm and supportive teacher.\n Help me understand the code you produce by explaining each function call with\n a brief comment. For more complicated calls, add documentation to each\n argument. Just give me the code without any text explanation.\n\"\"\"\n_ = chat.chat(question)\n\n\n\n\n\nimport pandas as pd\nimport numpy as np\n\n# Create list of letter variables\nvars = list('abcdefghijklmnopqrstuvwxyz')\n\n# Define both aggregation functions at once\naggs = ['mean', 'median'] \n\n# Group by age and sex, compute both stats for all letter variables\nresult = (df\n .groupby(['age', 'sex']) # Group by these two variables\n [vars] # Select only the letter columns\n .agg(aggs) # Calculate both statistics\n .reset_index() # Convert groups back to columns\n)\n\n\n\n\nTeach it about new features\nYou can imagine LLMs as being a sort of an average of the internet at a given point in time. That means they will provide popular answers, which will tend to reflect older coding styles (either because the new features aren’t in their index, or the older features are so much more popular). So if you want your code to use specific features that are relatively recent, you might need to provide the examples yourself:\n\nchat.system_prompt = \"\"\"\n You are an expert R programmer.\n Just give me the code; no explanation in text.\n Use the `.by` argument rather than `group_by()`.\n dplyr 1.1.0 introduced per-operation grouping with the `.by` argument.\n e.g., instead of:\n\n transactions |>\n group_by(company, year) |>\n mutate(total = sum(revenue))\n\n write this:\n transactions |>\n mutate(\n total = sum(revenue),\n .by = c(company, year)\n )\n\"\"\"\n_ = chat.chat(question)\n\n\n\n\n\ndf |>\n summarise(\n across(a:z, list(\n mean = \\(x) mean(x, na.rm = TRUE),\n median = \\(x) median(x, na.rm = TRUE)\n )),\n .by = c(age, sex)\n )" + "text": "Code generation\nLet’s explore prompt design for a simple code generation task:\n\nfrom chatlas import ChatAnthropic, ChatOpenAI\n\nquestion = \"\"\"\n How can I compute the mean and median of variables a, b, c, and so on,\n all the way up to z, grouped by age and sex.\n\"\"\"\n\n\nBasic flavour\nWhen I don’t provide a system prompt, I sometimes get answers in a different language (like R):\n\nchat = ChatAnthropic()\n_ = chat.chat(question)\n\n\n\n\n\n\n\nHere’s how to compute mean and median for variables a through z, grouped by age and sex:\n# Using dplyr\nlibrary(dplyr)\n\ndf %>%\n group_by(age, sex) %>%\n summarise(across(a:z, list(\n mean = ~mean(., na.rm = TRUE),\n median = ~median(., na.rm = TRUE)\n )))\n\n# Alternative base R approach\naggregate(. ~ age + sex, data = df[,c(\"age\", \"sex\", letters[1:26])], \n FUN = function(x) c(mean = mean(x), median = median(x)))\nThis will: 1. Group the data by age and sex 2. Calculate both mean and median for each variable a through z 3. Handle missing values with na.rm = TRUE 4. Return a dataframe with results for each age-sex combination\nThe output will have columns for age, sex, and mean/median values for each variable.\n\n\nI can ensure that I always get Python code by providing a system prompt:\n\nchat.system_prompt = \"You are a helpful Python (not R) programming assistant.\"\n_ = chat.chat(question)\n\n\n\n\n\nHere’s how to compute mean and median for variables a through z in Python using pandas:\nimport pandas as pd\n\n# Assuming your data is in a DataFrame called df\n# Group by age and sex, then calculate mean and median for all variables a through z\nresult = df.groupby(['age', 'sex'])[list('abcdefghijklmnopqrstuvwxyz')].agg(['mean', 'median'])\n\n# If you want to handle missing values:\nresult = df.groupby(['age', 'sex'])[list('abcdefghijklmnopqrstuvwxyz')].agg({\n col: ['mean', 'median'] for col in list('abcdefghijklmnopqrstuvwxyz')\n}).fillna(0) # or use .dropna() instead of fillna() if you prefer to remove NaN values\n\n# To make the output more readable, you can flatten the column names:\nresult.columns = [f'{col}_{stat}' for col, stat in result.columns]\nThis will: 1. Group your data by age and sex 2. Calculate mean and median for each variable from ‘a’ to ‘z’ 3. Return a DataFrame where: - The index will be the age and sex combinations - The columns will be each variable’s mean and median - Column names will be in the format ‘variable_statistic’ (e.g., ‘a_mean’, ‘a_median’, etc.)\nExample output might look like:\n a_mean a_median b_mean b_median ... z_mean z_median\nage sex ...\n20 M 23.5 24.0 45.2 46.0 ... 12.3 12.0\n F 22.1 21.0 44.8 45.0 ... 11.9 12.0\n21 M 24.2 25.0 46.1 46.0 ... 12.5 13.0\n F 23.8 24.0 45.9 46.0 ... 12.2 12.0\n...\n\n\nNote that I’m using both a system prompt (which defines the general behaviour) and a user prompt (which asks the specific question). You could put all of the content in the user prompt and get similar results, but I think it’s helpful to use both to cleanly divide the general framing of the response from the specific questions that you want to ask.\nSince I’m mostly interested in the code, I ask it to drop the explanation:\n\nchat.system_prompt = \"\"\"\n You are a helpful Python (not R) programming assistant.\n Just give me the code without any text explanation.\n\"\"\"\n_ = chat.chat(question)\n\n\n\n\n\nimport pandas as pd\n\nresult = df.groupby(['age', 'sex'])[list('abcdefghijklmnopqrstuvwxyz')].agg(['mean', 'median'])\nresult.columns = [f'{col}_{stat}' for col, stat in result.columns]\n\n\nIn this case, I seem to mostly get pandas code. But if you want a different style, you can ask for it:\n\nchat.system_prompt = \"\"\"\n You are a helpful Python (not R) programming assistant who prefers polars to pandas.\n Just give me the code without any text explanation.\n\"\"\"\n_ = chat.chat(question)\n\n\n\n\n\nimport polars as pl\n\nresult = df.groupby(['age', 'sex']).agg([\n pl.col(c).mean().alias(f'{c}_mean') for c in 'abcdefghijklmnopqrstuvwxyz'\n] + [\n pl.col(c).median().alias(f'{c}_median') for c in 'abcdefghijklmnopqrstuvwxyz'\n])\n\n\n\n\nBe explicit\nIf there’s something about the output that you don’t like, you can try being more explicit about it. For example, the code isn’t styled quite how I like, so I provide more details about what I do want:\n\nchat.system_prompt = \"\"\"\n You are a helpful Python (not R) programming assistant who prefers siuba to pandas.\n Just give me the code. I don't want any explanation or sample data.\n * Spread long function calls across multiple lines.\n * Where needed, always indent function calls with two spaces.\n * Always use double quotes for strings.\n\"\"\"\n_ = chat.chat(question)\n\n\n\n\n\nfrom siuba import *\nfrom siuba.dply.vector import *\n\nresult = (df \n >> group_by(_.age, _.sex)\n >> summarize(**{\n f\"{col}_{stat}\": call(stat, _[col])\n for col in \"abcdefghijklmnopqrstuvwxyz\"\n for stat in [\"mean\", \"median\"]\n }))\n\n\nThis still doesn’t yield exactly the code that I’d write, but it’s prety close.\nYou could provide a different prompt if you were looking for more explanation of the code:\n\nchat.system_prompt = \"\"\"\n You are an an expert Python (not R) programmer and a warm and supportive teacher.\n Help me understand the code you produce by explaining each function call with\n a brief comment. For more complicated calls, add documentation to each\n argument. Just give me the code without any text explanation.\n\"\"\"\n_ = chat.chat(question)\n\n\n\n\n\nimport pandas as pd\n\n# Create list of letters a-z to use as column names\ncols = list('abcdefghijklmnopqrstuvwxyz')\n\n# Group by age & sex, compute mean and median for all letter columns\nresult = df.groupby(['age', 'sex'])[cols].agg([\n 'mean', # Calculate mean for each column\n 'median' # Calculate median for each column\n])\n\n# Flatten the multi-level column names for readability \nresult.columns = [f'{col}_{stat}' for col, stat in result.columns]\n\n\n\n\nTeach it about new features\nYou can imagine LLMs as being a sort of an average of the internet at a given point in time. That means they will provide popular answers, which will tend to reflect older coding styles (either because the new features aren’t in their index, or the older features are so much more popular). So if you want your code to use specific newer language features, you might need to provide the examples yourself:\n\nchat.system_prompt = \"\"\"\n You are an expert R programmer.\n Just give me the code; no explanation in text.\n Use the `.by` argument rather than `group_by()`.\n dplyr 1.1.0 introduced per-operation grouping with the `.by` argument.\n e.g., instead of:\n\n transactions |>\n group_by(company, year) |>\n mutate(total = sum(revenue))\n\n write this:\n transactions |>\n mutate(\n total = sum(revenue),\n .by = c(company, year)\n )\n\"\"\"\n_ = chat.chat(question)\n\n\n\n\n\ndf |>\n summarise(\n across(a:z, list(\n mean = \\(x) mean(x, na.rm = TRUE),\n median = \\(x) median(x, na.rm = TRUE)\n )),\n .by = c(age, sex)\n )" }, { "objectID": "prompt-design.html#structured-data", "href": "prompt-design.html#structured-data", "title": "Prompt design", "section": "Structured data", - "text": "Structured data\nProviding a rich set of examples is a great way to encourage the output to produce exactly what you want. This is also known as multi-shot prompting. Here we’ll work through a prompt that I designed to extract structured data from recipes, but the same ideas apply in many other situations.\n\nGetting started\nMy overall goal is to turn a list of ingredients, like the following, into a nicely structured JSON that I can then analyse in Python (e.g. to compute the total weight, scale the recipe up or down, or to convert the units from volumes to weights).\n\ningredients = \"\"\"\n ¾ cup (150g) dark brown sugar\n 2 large eggs\n ¾ cup (165g) sour cream\n ½ cup (113g) unsalted butter, melted\n 1 teaspoon vanilla extract\n ¾ teaspoon kosher salt\n ⅓ cup (80ml) neutral oil\n 1½ cups (190g) all-purpose flour\n 150g plus 1½ teaspoons sugar\n\"\"\"\nchat = ChatOpenAI(model=\"gpt-4o-mini\")\n\n(This isn’t the ingredient list for a real recipe but it includes a sampling of styles that I encountered in my project.)\nIf you don’t have strong feelings about what the data structure should look like, you can start with a very loose prompt and see what you get back. I find this a useful pattern for underspecified problems where a big part of the problem is just defining precisely what problem you want to solve. Seeing the LLM’s attempt at a data structure gives me something to immediately react to, rather than having to start from a blank page.\n\ninstruct_json = \"\"\"\n You're an expert baker who also loves JSON. I am going to give you a list of\n ingredients and your job is to return nicely structured JSON. Just return the\n JSON and no other commentary.\n\"\"\"\nchat.system_prompt = instruct_json\n_ = chat.chat(ingredients)\n\n\n\n\n\n{ “ingredients”: [ { “name”: “dark brown sugar”, “amount”: “¾ cup”, “grams”: 150 }, { “name”: “large eggs”, “amount”: “2” }, { “name”: “sour cream”, “amount”: “¾ cup”, “grams”: 165 }, { “name”: “unsalted butter”, “amount”: “½ cup”, “grams”: 113, “state”: “melted” }, { “name”: “vanilla extract”, “amount”: “1 teaspoon” }, { “name”: “kosher salt”, “amount”: “¾ teaspoon” }, { “name”: “neutral oil”, “amount”: “⅓ cup”, “milliliters”: 80 }, { “name”: “all-purpose flour”, “amount”: “1½ cups”, “grams”: 190 }, { “name”: “sugar”, “amount”: “150g plus 1½ teaspoons” } ] }\n\n\n(I don’t know if the colour text, “You’re an expert baker who also loves JSON”, does anything, but I like to think this helps the LLM get into the right mindset of a very nerdy baker.)\n\n\nProvide examples\nThis isn’t a bad start, but I prefer to cook with weight and I only want to see volumes if weight isn’t available so I provide a couple of examples of what I’m looking for. I was pleasantly suprised that I can provide the input and output examples in such a loose format.\n\ninstruct_weight = \"\"\"\n Here are some examples of the sort of output I'm looking for:\n\n ¾ cup (150g) dark brown sugar\n {\"name\": \"dark brown sugar\", \"quantity\": 150, \"unit\": \"g\"}\n\n ⅓ cup (80ml) neutral oil\n {\"name\": \"neutral oil\", \"quantity\": 80, \"unit\": \"ml\"}\n\n 2 t ground cinnamon\n {\"name\": \"ground cinnamon\", \"quantity\": 2, \"unit\": \"teaspoon\"}\n\"\"\"\n\nchat.system_prompt = instruct_json + \"\\n\" + instruct_weight\n_ = chat.chat(ingredients)\n\n\n\n\n\n{ “ingredients”: [ { “name”: “dark brown sugar”, “quantity”: 150, “unit”: “g” }, { “name”: “large eggs”, “quantity”: 2, “unit”: “count” }, { “name”: “sour cream”, “quantity”: 165, “unit”: “g” }, { “name”: “unsalted butter”, “quantity”: 113, “unit”: “g”, “state”: “melted” }, { “name”: “vanilla extract”, “quantity”: 1, “unit”: “teaspoon” }, { “name”: “kosher salt”, “quantity”: 0.75, “unit”: “teaspoon” }, { “name”: “neutral oil”, “quantity”: 80, “unit”: “ml” }, { “name”: “all-purpose flour”, “quantity”: 190, “unit”: “g” }, { “name”: “sugar”, “quantity”: 150, “unit”: “g plus 1½ teaspoons” } ] }\n\n\nJust providing the examples seems to work remarkably well. But I found it useful to also include description of what the examples are trying to accomplish. I’m not sure if this helps the LLM or not, but it certainly makes it easier for me to understand the organisation and check that I’ve covered the key pieces that I’m interested in.\n\ninstruct_weight = \"\"\"\n * If an ingredient has both weight and volume, extract only the weight:\n\n ¾ cup (150g) dark brown sugar\n [\n {\"name\": \"dark brown sugar\", \"quantity\": 150, \"unit\": \"g\"}\n ]\n\n* If an ingredient only lists a volume, extract that.\n\n 2 t ground cinnamon\n ⅓ cup (80ml) neutral oil\n [\n {\"name\": \"ground cinnamon\", \"quantity\": 2, \"unit\": \"teaspoon\"},\n {\"name\": \"neutral oil\", \"quantity\": 80, \"unit\": \"ml\"}\n ]\n\"\"\"\n\nThis structure also allows me to give the LLMs a hint about how I want multiple ingredients to be stored, i.e. as an JSON array.\nI then iterated on the prompt, looking at the results from different recipes to get a sense of what the LLM was getting wrong. Much of this felt like I was iterating on my understanding of the problem as I didn’t start by knowing exactly how I wanted the data. For example, when I started out I didn’t really think about all the various ways that ingredients are specified. For later analysis, I always want quantities to be number, even if they were originally fractions, or the if the units aren’t precise (like a pinch). It made me to realise that some ingredients are unitless.\n\ninstruct_unit = \"\"\"\n* If the unit uses a fraction, convert it to a decimal.\n\n ⅓ cup sugar\n ½ teaspoon salt\n [\n {\"name\": \"dark brown sugar\", \"quantity\": 0.33, \"unit\": \"cup\"},\n {\"name\": \"salt\", \"quantity\": 0.5, \"unit\": \"teaspoon\"}\n ]\n\n* Quantities are always numbers\n\n pinch of kosher salt\n [\n {\"name\": \"kosher salt\", \"quantity\": 1, \"unit\": \"pinch\"}\n ]\n\n* Some ingredients don't have a unit.\n 2 eggs\n 1 lime\n 1 apple\n [\n {\"name\": \"egg\", \"quantity\": 2},\n {\"name\": \"lime\", \"quantity\": 1},\n {\"name\", \"apple\", \"quantity\": 1}\n ]\n\"\"\"\n\nYou might want to take a look at the full prompt to see what I ended up with.\n\n\nStructured data\nNow that I’ve iterated to get a data structure that I like, it seems useful to formalise it and tell the LLM exactly what I’m looking for using structured data. This guarantees that the LLM will only return JSON, the JSON will have the fields that you expect, and that chatlas will convert it into an Python data structure for you.\n\nfrom pydantic import BaseModel, Field\n\nclass Ingredient(BaseModel):\n \"Ingredient name\"\n name: str = Field(description=\"Ingredient name\")\n quantity: float\n unit: str | None = Field(description=\"Unit of measurement\")\n\nclass Ingredients(BaseModel):\n items: list[Ingredient]\n\nchat.system_prompt = instruct_json + \"\\n\" + instruct_weight\nchat.extract_data(ingredients, data_model=Ingredients)\n\n{'items': [{'name': 'dark brown sugar', 'quantity': 150, 'unit': 'g'},\n {'name': 'large eggs', 'quantity': 2, 'unit': 'count'},\n {'name': 'sour cream', 'quantity': 165, 'unit': 'g'},\n {'name': 'unsalted butter', 'quantity': 113, 'unit': 'g'},\n {'name': 'vanilla extract', 'quantity': 1, 'unit': 'teaspoon'},\n {'name': 'kosher salt', 'quantity': 0.75, 'unit': 'teaspoon'},\n {'name': 'neutral oil', 'quantity': 80, 'unit': 'ml'},\n {'name': 'all-purpose flour', 'quantity': 190, 'unit': 'g'},\n {'name': 'sugar', 'quantity': 150, 'unit': 'g plus 1½ teaspoons'}]}\n\n\n\n\nCapturing raw input\nOne thing that I’d do next time would also be to include the raw ingredient name in the output. This doesn’t make much difference here, in this simple example, but it makes it much easier to align the input and the output and start to develop automated measures of how well my prompt is doing.\n\ninstruct_weight_input = \"\"\"\n * If an ingredient has both weight and volume, extract only the weight:\n\n ¾ cup (150g) dark brown sugar\n [\n {\"name\": \"dark brown sugar\", \"quantity\": 150, \"unit\": \"g\", \"input\": \"¾ cup (150g) dark brown sugar\"}\n ]\n\n * If an ingredient only lists a volume, extract that.\n\n 2 t ground cinnamon\n ⅓ cup (80ml) neutral oil\n [\n {\"name\": \"ground cinnamon\", \"quantity\": 2, \"unit\": \"teaspoon\", \"input\": \"2 t ground cinnamon\"},\n {\"name\": \"neutral oil\", \"quantity\": 80, \"unit\": \"ml\", \"input\": \"⅓ cup (80ml) neutral oil\"}\n ]\n\"\"\"\n\nI think this is particularly important if you’re working with even less structured text. For example, imagine you had this text:\n\nrecipe = \"\"\"\n In a large bowl, cream together one cup of softened unsalted butter and a\n quarter cup of white sugar until smooth. Beat in an egg and 1 teaspoon of\n vanilla extract. Gradually stir in 2 cups of all-purpose flour until the\n dough forms. Finally, fold in 1 cup of semisweet chocolate chips. Drop\n spoonfuls of dough onto an ungreased baking sheet and bake at 350°F (175°C)\n for 10-12 minutes, or until the edges are lightly browned. Let the cookies\n cool on the baking sheet for a few minutes before transferring to a wire\n rack to cool completely. Enjoy!\n\"\"\"\n\nIncluding the input text in the output makes it easier to see if it’s doing a good job:\n\nchat.system_prompt = instruct_json + \"\\n\" + instruct_weight_input\n_ = chat.chat(ingredients)\n\n\n\n\n\n{ “ingredients”: [ { “name”: “dark brown sugar”, “quantity”: 150, “unit”: “g” }, { “name”: “large eggs”, “quantity”: 2, “unit”: “count” }, { “name”: “sour cream”, “quantity”: 165, “unit”: “g” }, { “name”: “unsalted butter”, “quantity”: 113, “unit”: “g”, “state”: “melted” }, { “name”: “vanilla extract”, “quantity”: 1, “unit”: “teaspoon” }, { “name”: “kosher salt”, “quantity”: 0.75, “unit”: “teaspoon” }, { “name”: “neutral oil”, “quantity”: 80, “unit”: “ml” }, { “name”: “all-purpose flour”, “quantity”: 190, “unit”: “g” }, { “name”: “sugar”, “quantity”: “150g plus 1½ teaspoons” } ] }\n\n\nWhen I ran it while writing this vignette, it seems to be working out the weight of the ingredients specified in volume, even though the prompt specifically asks it not to do that. This may suggest I need to broaden my examples." + "text": "Structured data\nProviding a rich set of examples is a great way to encourage the output to produce exactly what you want. This is known as multi-shot prompting. Below we’ll work through a prompt that I designed to extract structured data from recipes, but the same ideas apply in many other situations.\n\nGetting started\nMy overall goal is to turn a list of ingredients, like the following, into a nicely structured JSON that I can then analyse in Python (e.g. compute the total weight, scale the recipe up or down, or convert the units from volumes to weights).\n\ningredients = \"\"\"\n ¾ cup (150g) dark brown sugar\n 2 large eggs\n ¾ cup (165g) sour cream\n ½ cup (113g) unsalted butter, melted\n 1 teaspoon vanilla extract\n ¾ teaspoon kosher salt\n ⅓ cup (80ml) neutral oil\n 1½ cups (190g) all-purpose flour\n 150g plus 1½ teaspoons sugar\n\"\"\"\nchat = ChatOpenAI(model=\"gpt-4o-mini\")\n\n(This isn’t the ingredient list for a real recipe but it includes a sampling of styles that I encountered in my project.)\nIf you don’t have strong feelings about what the data structure should look like, you can start with a very loose prompt and see what you get back. I find this a useful pattern for underspecified problems where the heavy lifting lies with precisely defining the problem you want to solve. Seeing the LLM’s attempt to create a data structure gives me something to react to, rather than having to start from a blank page.\n\ninstruct_json = \"\"\"\n You're an expert baker who also loves JSON. I am going to give you a list of\n ingredients and your job is to return nicely structured JSON. Just return the\n JSON and no other commentary.\n\"\"\"\nchat.system_prompt = instruct_json\n_ = chat.chat(ingredients)\n\n\n\n\n\n{ “ingredients”: [ { “name”: “dark brown sugar”, “quantity”: “¾ cup”, “weight”: “150g” }, { “name”: “large eggs”, “quantity”: “2” }, { “name”: “sour cream”, “quantity”: “¾ cup”, “weight”: “165g” }, { “name”: “unsalted butter”, “quantity”: “½ cup”, “weight”: “113g”, “state”: “melted” }, { “name”: “vanilla extract”, “quantity”: “1 teaspoon” }, { “name”: “kosher salt”, “quantity”: “¾ teaspoon” }, { “name”: “neutral oil”, “quantity”: “⅓ cup”, “volume”: “80ml” }, { “name”: “all-purpose flour”, “quantity”: “1½ cups”, “weight”: “190g” }, { “name”: “sugar”, “quantity”: “150g plus 1½ teaspoons” } ] }\n\n\n(I don’t know if the additional colour, “You’re an expert baker who also loves JSON”, does anything, but I like to think this helps the LLM get into the right mindset of a very nerdy baker.)\n\n\nProvide examples\nThis isn’t a bad start, but I prefer to cook with weight and I only want to see volumes if weight isn’t available so I provide a couple of examples of what I’m looking for. I was pleasantly suprised that I can provide the input and output examples in such a loose format.\n\ninstruct_weight = \"\"\"\n Here are some examples of the sort of output I'm looking for:\n\n ¾ cup (150g) dark brown sugar\n {\"name\": \"dark brown sugar\", \"quantity\": 150, \"unit\": \"g\"}\n\n ⅓ cup (80ml) neutral oil\n {\"name\": \"neutral oil\", \"quantity\": 80, \"unit\": \"ml\"}\n\n 2 t ground cinnamon\n {\"name\": \"ground cinnamon\", \"quantity\": 2, \"unit\": \"teaspoon\"}\n\"\"\"\n\nchat.system_prompt = instruct_json + \"\\n\" + instruct_weight\n_ = chat.chat(ingredients)\n\n\n\n\n\n{ “ingredients”: [ { “name”: “dark brown sugar”, “quantity”: 150, “unit”: “g” }, { “name”: “large eggs”, “quantity”: 2, “unit”: “count” }, { “name”: “sour cream”, “quantity”: 165, “unit”: “g” }, { “name”: “unsalted butter”, “quantity”: 113, “unit”: “g”, “state”: “melted” }, { “name”: “vanilla extract”, “quantity”: 1, “unit”: “teaspoon” }, { “name”: “kosher salt”, “quantity”: ¾, “unit”: “teaspoon” }, { “name”: “neutral oil”, “quantity”: 80, “unit”: “ml” }, { “name”: “all-purpose flour”, “quantity”: 190, “unit”: “g” }, { “name”: “sugar”, “quantity”: “150g plus 1½ teaspoons”, “unit”: “g” } ] }\n\n\nJust providing the examples seems to work remarkably well. But I found it useful to also include a description of what the examples are trying to accomplish. I’m not sure if this helps the LLM or not, but it certainly makes it easier for me to understand the organisation and check that I’ve covered the key pieces I’m interested in.\n\ninstruct_weight = \"\"\"\n * If an ingredient has both weight and volume, extract only the weight:\n\n ¾ cup (150g) dark brown sugar\n [\n {\"name\": \"dark brown sugar\", \"quantity\": 150, \"unit\": \"g\"}\n ]\n\n* If an ingredient only lists a volume, extract that.\n\n 2 t ground cinnamon\n ⅓ cup (80ml) neutral oil\n [\n {\"name\": \"ground cinnamon\", \"quantity\": 2, \"unit\": \"teaspoon\"},\n {\"name\": \"neutral oil\", \"quantity\": 80, \"unit\": \"ml\"}\n ]\n\"\"\"\n\nThis structure also allows me to give the LLMs a hint about how I want multiple ingredients to be stored, i.e. as an JSON array.\nI then iterated on the prompt, looking at the results from different recipes to get a sense of what the LLM was getting wrong. Much of this felt like I was iterating on my own understanding of the problem as I didn’t start by knowing exactly how I wanted the data. For example, when I started out I didn’t really think about all the various ways that ingredients are specified. For later analysis, I always want quantities to be number, even if they were originally fractions, or the if the units aren’t precise (like a pinch). It made me to realise that some ingredients are unitless.\n\ninstruct_unit = \"\"\"\n* If the unit uses a fraction, convert it to a decimal.\n\n ⅓ cup sugar\n ½ teaspoon salt\n [\n {\"name\": \"dark brown sugar\", \"quantity\": 0.33, \"unit\": \"cup\"},\n {\"name\": \"salt\", \"quantity\": 0.5, \"unit\": \"teaspoon\"}\n ]\n\n* Quantities are always numbers\n\n pinch of kosher salt\n [\n {\"name\": \"kosher salt\", \"quantity\": 1, \"unit\": \"pinch\"}\n ]\n\n* Some ingredients don't have a unit.\n 2 eggs\n 1 lime\n 1 apple\n [\n {\"name\": \"egg\", \"quantity\": 2},\n {\"name\": \"lime\", \"quantity\": 1},\n {\"name\", \"apple\", \"quantity\": 1}\n ]\n\"\"\"\n\nYou might want to take a look at the full prompt to see what I ended up with.\n\n\nStructured data\nNow that I’ve iterated to get a data structure I like, it seems useful to formalise it and tell the LLM exactly what I’m looking for when dealing with structured data. This guarantees that the LLM will only return JSON, that the JSON will have the fields that you expect, and that chatlas will convert it into an Python data structure for you.\n\nfrom pydantic import BaseModel, Field\n\nclass Ingredient(BaseModel):\n \"Ingredient name\"\n name: str = Field(description=\"Ingredient name\")\n quantity: float\n unit: str | None = Field(description=\"Unit of measurement\")\n\nclass Ingredients(BaseModel):\n items: list[Ingredient]\n\nchat.system_prompt = instruct_json + \"\\n\" + instruct_weight\nchat.extract_data(ingredients, data_model=Ingredients)\n\n{'items': [{'name': 'dark brown sugar', 'quantity': 150, 'unit': 'g'},\n {'name': 'large eggs', 'quantity': 2, 'unit': 'count'},\n {'name': 'sour cream', 'quantity': 165, 'unit': 'g'},\n {'name': 'unsalted butter', 'quantity': 113, 'unit': 'g'},\n {'name': 'vanilla extract', 'quantity': 1, 'unit': 'teaspoon'},\n {'name': 'kosher salt', 'quantity': 0.75, 'unit': 'teaspoon'},\n {'name': 'neutral oil', 'quantity': 80, 'unit': 'ml'},\n {'name': 'all-purpose flour', 'quantity': 190, 'unit': 'g'},\n {'name': 'sugar', 'quantity': 150, 'unit': 'g'}]}\n\n\n\n\nCapturing raw input\nOne thing that I’d do next time would also be to include the raw ingredient names in the output. This doesn’t make much difference in this simple example but it makes it much easier to align the input with the output and to start developing automated measures of how well my prompt is doing.\n\ninstruct_weight_input = \"\"\"\n * If an ingredient has both weight and volume, extract only the weight:\n\n ¾ cup (150g) dark brown sugar\n [\n {\"name\": \"dark brown sugar\", \"quantity\": 150, \"unit\": \"g\", \"input\": \"¾ cup (150g) dark brown sugar\"}\n ]\n\n * If an ingredient only lists a volume, extract that.\n\n 2 t ground cinnamon\n ⅓ cup (80ml) neutral oil\n [\n {\"name\": \"ground cinnamon\", \"quantity\": 2, \"unit\": \"teaspoon\", \"input\": \"2 t ground cinnamon\"},\n {\"name\": \"neutral oil\", \"quantity\": 80, \"unit\": \"ml\", \"input\": \"⅓ cup (80ml) neutral oil\"}\n ]\n\"\"\"\n\nI think this is particularly important if you’re working with even less structured text. For example, imagine you had this text:\n\nrecipe = \"\"\"\n In a large bowl, cream together one cup of softened unsalted butter and a\n quarter cup of white sugar until smooth. Beat in an egg and 1 teaspoon of\n vanilla extract. Gradually stir in 2 cups of all-purpose flour until the\n dough forms. Finally, fold in 1 cup of semisweet chocolate chips. Drop\n spoonfuls of dough onto an ungreased baking sheet and bake at 350°F (175°C)\n for 10-12 minutes, or until the edges are lightly browned. Let the cookies\n cool on the baking sheet for a few minutes before transferring to a wire\n rack to cool completely. Enjoy!\n\"\"\"\n\nIncluding the input text in the output makes it easier to see if it’s doing a good job:\n\nchat.system_prompt = instruct_json + \"\\n\" + instruct_weight_input\n_ = chat.chat(ingredients)\n\n\n\n\n\n{ “ingredients”: [ { “name”: “dark brown sugar”, “quantity”: 150, “unit”: “g” }, { “name”: “large eggs”, “quantity”: 2, “unit”: “count” }, { “name”: “sour cream”, “quantity”: 165, “unit”: “g” }, { “name”: “unsalted butter”, “quantity”: 113, “unit”: “g”, “state”: “melted” }, { “name”: “vanilla extract”, “quantity”: 1, “unit”: “teaspoon” }, { “name”: “kosher salt”, “quantity”: 0.75, “unit”: “teaspoon” }, { “name”: “neutral oil”, “quantity”: 80, “unit”: “ml” }, { “name”: “all-purpose flour”, “quantity”: 190, “unit”: “g” }, { “name”: “sugar”, “quantity”: “150g plus 1½ teaspoons”, “unit”: “g” } ] }\n\n\nWhen I ran it while writing this vignette, it seemed to be working out the weight of the ingredients specified in volume, even though the prompt specifically asks it not to. This may suggest I need to broaden my examples." }, { "objectID": "prompt-design.html#token-usage", "href": "prompt-design.html#token-usage", "title": "Prompt design", "section": "Token usage", - "text": "Token usage\n\nfrom chatlas import token_usage\ntoken_usage()\n\n[{'name': 'Anthropic', 'input': 5583, 'output': 1083},\n {'name': 'OpenAI', 'input': 2778, 'output': 918}]" + "text": "Token usage\n\nfrom chatlas import token_usage\ntoken_usage()\n\n[{'name': 'Anthropic', 'input': 6504, 'output': 1270},\n {'name': 'OpenAI', 'input': 2909, 'output': 1043}]" }, { "objectID": "tool-calling.html", "href": "tool-calling.html", "title": "Introduction", "section": "", - "text": "One of the most interesting aspects of modern chat models is their ability to make use of external tools that are defined by the caller.\nWhen making a chat request to the chat model, the caller advertises one or more tools (defined by their function name, description, and a list of expected arguments), and the chat model can choose to respond with one or more “tool calls”. These tool calls are requests from the chat model to the caller to execute the function with the given arguments; the caller is expected to execute the functions and “return” the results by submitting another chat request with the conversation so far, plus the results. The chat model can then use those results in formulating its response, or, it may decide to make additional tool calls.\nNote that the chat model does not directly execute any external tools! It only makes requests for the caller to execute them. It’s easy to think that tool calling might work like this:\n\n\n\nDiagram showing showing the wrong mental model of tool calls: a user initiates a request that flows to the assistant, which then runs the code, and returns the result back to the user.”\n\n\nBut in fact it works like this:\n\n\n\nDiagram showing the correct mental model for tool calls: a user sends a request that needs a tool call, the assistant request that the user’s runs that tool, returns the result to the assistant, which uses it to generate the final answer.\n\n\nThe value that the chat model brings is not in helping with execution, but with knowing when it makes sense to call a tool, what values to pass as arguments, and how to use the results in formulating its response.\n\nfrom chatlas import ChatOpenAI\n\n\nMotivating example\nLet’s take a look at an example where we really need an external tool. Chat models generally do not have access to “real-time” information, such as current events, weather, etc. Let’s see what happens when we ask the chat model about the weather in a specific location:\n\nchat = ChatOpenAI(model=\"gpt-4o-mini\")\n_ = chat.chat(\"What's the weather like today in Duluth, MN?\")\n\n\n\n\n\nI’m sorry, but I can’t provide real-time weather updates. I recommend checking a reliable weather website or app for the latest information on the weather in Duluth, MN.\n\n\nFortunately, the model is smart enough to know that it doesn’t have access to real-time information, and it doesn’t try to make up an answer. However, we can help it out by providing a tool that can fetch the weather for a given location.\n\n\nDefining a tool function\nAt it turns out, LLMs are pretty good at figuring out ‘structure’ like latitude and longitude from ‘unstructured’ things like a location name. So we can write a tool function that takes a latitude and longitude and returns the current temperature at that location. Here’s an example of how you might write such a function using the Open-Meteo API:\n\nimport requests\n\ndef get_current_temperature(latitude: float, longitude: float):\n \"\"\"\n Get the current weather given a latitude and longitude.\n\n Parameters\n ----------\n latitude\n The latitude of the location.\n longitude\n The longitude of the location.\n \"\"\"\n lat_lng = f\"latitude={latitude}&longitude={longitude}\"\n url = f\"https://api.open-meteo.com/v1/forecast?{lat_lng}¤t=temperature_2m,wind_speed_10m&hourly=temperature_2m,relative_humidity_2m,wind_speed_10m\"\n response = requests.get(url)\n json = response.json()\n return json[\"current\"]\n\nNote that we’ve gone through the trouble of adding the following to our function:\n\nType hints for function arguments\nA docstring that explains what the function does and what arguments it expects (as well as descriptions for the arguments themselves)\n\nProviding these hints and documentation is very important, as it helps the chat model understand how to use your tool correctly!\nLet’s test it:\n\nget_current_temperature(46.7867, -92.1005)\n\n{'time': '2024-12-11T21:15',\n 'interval': 900,\n 'temperature_2m': -18.7,\n 'wind_speed_10m': 15.5}\n\n\n\n\nUsing the tool\nIn order for the LLM to make use of our tool, we need to register it with the chat object. This is done by calling the register_tool method on the chat object.\n\nchat.register_tool(get_current_temperature)\n\nNow let’s retry our original question:\n\n_ = chat.chat(\"What's the weather like today in Duluth, MN?\")\n\n\n\n\n\n\nThe current weather in Duluth, MN, is -18.5°C with a wind speed of 15.5 km/h. Please dress warmly if you’re going outside!\n\n\nThat’s correct! Without any further guidance, the chat model decided to call our tool function and successfully used its result in formulating its response.\nThis tool example was extremely simple, but you can imagine doing much more interesting things from tool functions: calling APIs, reading from or writing to a database, kicking off a complex simulation, or even calling a complementary GenAI model (like an image generator). Or if you are using chatlas in a Shiny app, you could use tools to set reactive values, setting off a chain of reactive updates. This is precisely what the sidebot dashboard does to allow for an AI assisted “drill-down” into the data.\n\n\nTrouble-shooting\nWhen the execution of a tool function fails, chatlas sends the exception message back to the chat model. This can be useful for gracefully handling errors in the chat model. However, this can also lead to confusion as to why a response did not come back as expected. If you encounter such a situation, you can set echo=\"all\" in the chat.chat() method to see the full conversation, including tool calls and their results.\n\ndef get_current_temperature(latitude: float, longitude: float):\n \"Get the current weather given a latitude and longitude.\"\n raise ValueError(\"Failed to get current temperature\")\n\nchat.register_tool(get_current_temperature)\n\n_ = chat.chat(\"What's the weather like today in Duluth, MN?\")\n\n\n\n\n\n\nI’m currently unable to retrieve the weather information for Duluth, MN. I recommend checking a reliable weather website or app for the latest updates on the weather conditions there.\n\n\n\n\nTool limitations\nRemember that tool arguments come from the chat model, and tool results are returned to the chat model. That means that only simple, JSON-compatible data types can be used as inputs and outputs. It’s highly recommended that you stick to basic types for each function parameter (e.g. str, float/int, bool, None, list, tuple, dict). And you can forget about using functions, classes, external pointers, and other complex (i.e., non-serializable) Python objects as arguments or return values. Returning data frames seems to work OK (as long as you return the JSON representation – .to_json()), although be careful not to return too much data, as it all counts as tokens (i.e., they count against your context window limit and also cost you money)." + "text": "One of the most interesting aspects of modern chat models is their ability to make use of external tools that are defined by the caller.\nWhen making a chat request to the chat model, the caller advertises one or more tools (defined by their function name, description, and a list of expected arguments), and the chat model can choose to respond with one or more “tool calls”. These tool calls are requests from the chat model to the caller to execute the function with the given arguments; the caller is expected to execute the functions and “return” the results by submitting another chat request with the conversation so far, plus the results. The chat model can then use those results in formulating its response, or, it may decide to make additional tool calls.\nNote that the chat model does not directly execute any external tools! It only makes requests for the caller to execute them. It’s easy to think that tool calling might work like this:\n\n\n\nDiagram showing showing the wrong mental model of tool calls: a user initiates a request that flows to the assistant, which then runs the code, and returns the result back to the user.”\n\n\nBut in fact it works like this:\n\n\n\nDiagram showing the correct mental model for tool calls: a user sends a request that needs a tool call, the assistant request that the user’s runs that tool, returns the result to the assistant, which uses it to generate the final answer.\n\n\nThe value that the chat model brings is not in helping with execution, but with knowing when it makes sense to call a tool, what values to pass as arguments, and how to use the results in formulating its response.\n\nfrom chatlas import ChatOpenAI\n\n\nMotivating example\nLet’s take a look at an example where we really need an external tool. Chat models generally do not have access to “real-time” information, such as current events, weather, etc. Let’s see what happens when we ask the chat model about the weather in a specific location:\n\nchat = ChatOpenAI(model=\"gpt-4o-mini\")\n_ = chat.chat(\"What's the weather like today in Duluth, MN?\")\n\n\n\n\n\nI’m unable to provide real-time weather updates or current local conditions. I recommend checking a weather website or app for the latest information on the weather in Duluth, MN.\n\n\nFortunately, the model is smart enough to know that it doesn’t have access to real-time information, and it doesn’t try to make up an answer. However, we can help it out by providing a tool that can fetch the weather for a given location.\n\n\nDefining a tool function\nAt it turns out, LLMs are pretty good at figuring out ‘structure’ like latitude and longitude from ‘unstructured’ things like a location name. So we can write a tool function that takes a latitude and longitude and returns the current temperature at that location. Here’s an example of how you might write such a function using the Open-Meteo API:\n\nimport requests\n\ndef get_current_temperature(latitude: float, longitude: float):\n \"\"\"\n Get the current weather given a latitude and longitude.\n\n Parameters\n ----------\n latitude\n The latitude of the location.\n longitude\n The longitude of the location.\n \"\"\"\n lat_lng = f\"latitude={latitude}&longitude={longitude}\"\n url = f\"https://api.open-meteo.com/v1/forecast?{lat_lng}¤t=temperature_2m,wind_speed_10m&hourly=temperature_2m,relative_humidity_2m,wind_speed_10m\"\n response = requests.get(url)\n json = response.json()\n return json[\"current\"]\n\nNote that we’ve gone through the trouble of adding the following to our function:\n\nType hints for function arguments\nA docstring that explains what the function does and what arguments it expects (as well as descriptions for the arguments themselves)\n\nProviding these hints and documentation is very important, as it helps the chat model understand how to use your tool correctly!\nLet’s test it:\n\nget_current_temperature(46.7867, -92.1005)\n\n{'time': '2024-12-13T22:30',\n 'interval': 900,\n 'temperature_2m': -11.3,\n 'wind_speed_10m': 8.7}\n\n\n\n\nUsing the tool\nIn order for the LLM to make use of our tool, we need to register it with the chat object. This is done by calling the register_tool method on the chat object.\n\nchat.register_tool(get_current_temperature)\n\nNow let’s retry our original question:\n\n_ = chat.chat(\"What's the weather like today in Duluth, MN?\")\n\n\n\n\n\n\nToday’s weather in Duluth, MN, shows a temperature of -11.2°C. The wind speed is around 8.7 km/h. Make sure to dress warmly if you’re heading outside!\n\n\nThat’s correct! Without any further guidance, the chat model decided to call our tool function and successfully used its result in formulating its response.\nThis tool example was extremely simple, but you can imagine doing much more interesting things from tool functions: calling APIs, reading from or writing to a database, kicking off a complex simulation, or even calling a complementary GenAI model (like an image generator). Or if you are using chatlas in a Shiny app, you could use tools to set reactive values, setting off a chain of reactive updates. This is precisely what the sidebot dashboard does to allow for an AI assisted “drill-down” into the data.\n\n\nTrouble-shooting\nWhen the execution of a tool function fails, chatlas sends the exception message back to the chat model. This can be useful for gracefully handling errors in the chat model. However, this can also lead to confusion as to why a response did not come back as expected. If you encounter such a situation, you can set echo=\"all\" in the chat.chat() method to see the full conversation, including tool calls and their results.\n\ndef get_current_temperature(latitude: float, longitude: float):\n \"Get the current weather given a latitude and longitude.\"\n raise ValueError(\"Failed to get current temperature\")\n\nchat.register_tool(get_current_temperature)\n\n_ = chat.chat(\"What's the weather like today in Duluth, MN?\")\n\n\n\n\n\n\nI am currently unable to retrieve the weather information. However, you can check a weather website or app for the latest updates on the weather in Duluth, MN.\n\n\n\n\nTool limitations\nRemember that tool arguments come from the chat model, and tool results are returned to the chat model. That means that only simple, JSON-compatible data types can be used as inputs and outputs. It’s highly recommended that you stick to basic types for each function parameter (e.g. str, float/int, bool, None, list, tuple, dict). And you can forget about using functions, classes, external pointers, and other complex (i.e., non-serializable) Python objects as arguments or return values. Returning data frames seems to work OK (as long as you return the JSON representation – .to_json()), although be careful not to return too much data, as it all counts as tokens (i.e., they count against your context window limit and also cost you money)." }, { "objectID": "index.html", "href": "index.html", "title": "chatlas", "section": "", - "text": "chatlas provides a simple and unified interface across large language model (llm) providers in Python. It abstracts away complexity from common tasks like streaming chat interfaces, tool calling, structured output, and much more. chatlas helps you prototype faster without painting you into a corner; for example, switching providers is as easy as changing one line of code, but provider specific features are still accessible when needed. Developer experience is also a key focus of chatlas: typing support, rich console output, and built-in tooling are all included.\n(Looking for something similar to chatlas, but in R? Check out elmer!)\n\n\nchatlas isn’t yet on pypi, but you can install from Github:\npip install git+https://github.com/posit-dev/chatlas\n\n\n\nchatlas supports a variety of model providers. See the API reference for more details (like managing credentials) on each provider.\n\nAnthropic (Claude): ChatAnthropic().\nGitHub model marketplace: ChatGithub().\nGoogle (Gemini): ChatGoogle().\nGroq: ChatGroq().\nOllama local models: ChatOllama().\nOpenAI: ChatOpenAI().\nperplexity.ai: ChatPerplexity().\n\nIt also supports the following enterprise cloud providers:\n\nAWS Bedrock: ChatBedrockAnthropic().\nAzure OpenAI: ChatAzureOpenAI().\n\nTo use a model provider that isn’t listed here, you have two options:\n\nIf the model is OpenAI compatible, use ChatOpenAI() with the appropriate base_url and api_key (see ChatGithub for a reference).\nIf you’re motivated, implement a new provider by subclassing Provider and implementing the required methods.\n\n\n\n\nIf you’re using chatlas inside your organisation, you’ll be limited to what your org allows, which is likely to be one provided by a big cloud provider (e.g. ChatAzureOpenAI() and ChatBedrockAnthropic()). If you’re using chatlas for your own personal exploration, you have a lot more freedom so we have a few recommendations to help you get started:\n\nChatOpenAI() or ChatAnthropic() are both good places to start. ChatOpenAI() defaults to GPT-4o, but you can use model = \"gpt-4o-mini\" for a cheaper lower-quality model, or model = \"o1-mini\" for more complex reasoning. ChatAnthropic() is similarly good; it defaults to Claude 3.5 Sonnet which we have found to be particularly good at writing code.\nChatGoogle() is great for large prompts, because it has a much larger context window than other models. It allows up to 1 million tokens, compared to Claude 3.5 Sonnet’s 200k and GPT-4o’s 128k.\nChatOllama(), which uses Ollama, allows you to run models on your own computer. The biggest models you can run locally aren’t as good as the state of the art hosted models, but they also don’t share your data and and are effectively free.\n\n\n\n\nYou can chat via chatlas in several different ways, depending on whether you are working interactively or programmatically. They all start with creating a new chat object:\nfrom chatlas import ChatOpenAI\n\nchat = ChatOpenAI(\n model = \"gpt-4o\",\n system_prompt = \"You are a friendly but terse assistant.\",\n)\n\n\nFrom a chat instance, it’s simple to start a web-based or terminal-based chat console, which is great for testing the capabilities of the model. In either case, responses stream in real-time, and context is preserved across turns.\nchat.app()\n\n\n\nOr, if you prefer to work from the terminal:\nchat.console()\nEntering chat console. Press Ctrl+C to quit.\n\n?> Who created Python?\n\nPython was created by Guido van Rossum. He began development in the late 1980s and released the first version in 1991. \n\n?> Where did he develop it?\n\nGuido van Rossum developed Python while working at Centrum Wiskunde & Informatica (CWI) in the Netherlands. \n\n\n\nFor a more programmatic approach, you can use the .chat() method to ask a question and get a response. By default, the response prints to a rich console as it streams in:\nchat.chat(\"What preceding languages most influenced Python?\")\nPython was primarily influenced by ABC, with additional inspiration from C,\nModula-3, and various other languages.\nTo ask a question about an image, pass one or more additional input arguments using content_image_file() and/or content_image_url():\nfrom chatlas import content_image_url\n\nchat.chat(\n content_image_url(\"https://www.python.org/static/img/python-logo.png\"),\n \"Can you explain this logo?\"\n)\nThe Python logo features two intertwined snakes in yellow and blue,\nrepresenting the Python programming language. The design symbolizes...\nTo get the full response as a string, use the built-in str() function. Optionally, you can also suppress the rich console output by setting echo=\"none\":\nresponse = chat.chat(\"Who is Posit?\", echo=\"none\")\nprint(str(response))\nAs we’ll see in later articles, echo=\"all\" can also be useful for debugging, as it shows additional information, such as tool calls.\n\n\n\nIf you want to do something with the response in real-time (i.e., as it arrives in chunks), use the .stream() method. This method returns an iterator that yields each chunk of the response as it arrives:\nresponse = chat.stream(\"Who is Posit?\")\nfor chunk in response:\n print(chunk, end=\"\")\nThe .stream() method can also be useful if you’re building a chatbot or other programs that needs to display responses as they arrive.\n\n\n\nTool calling is as simple as passing a function with type hints and docstring to .register_tool().\nimport sys\n\ndef get_current_python_version() -> str:\n \"\"\"Get the current version of Python.\"\"\"\n return sys.version\n\nchat.register_tool(get_current_python_version)\nchat.chat(\"What's the current version of Python?\")\nThe current version of Python is 3.13.\nLearn more in the tool calling article\n\n\n\nStructured data (i.e., structured output) is as simple as passing a pydantic model to .extract_data().\nfrom pydantic import BaseModel\n\nclass Person(BaseModel):\n name: str\n age: int\n\nchat.extract_data(\n \"My name is Susan and I'm 13 years old\", \n data_model=Person,\n)\n{'name': 'Susan', 'age': 13}\nLearn more in the structured data article\n\n\n\nEasily get a full markdown or HTML export of a conversation:\nchat.export(\"index.html\", title=\"Python Q&A\")\nIf the export doesn’t have all the information you need, you can also access the full conversation history via the .get_turns() method:\nchat.get_turns()\nAnd, if the conversation is too long, you can specify which turns to include:\nchat.export(\"index.html\", turns=chat.get_turns()[-5:])\n\n\n\nchat methods tend to be synchronous by default, but you can use the async flavor by appending _async to the method name:\nimport asyncio\n\nasync def main():\n await chat.chat_async(\"What is the capital of France?\")\n\nasyncio.run(main())\n\n\n\nchatlas has full typing support, meaning that, among other things, autocompletion just works in your favorite editor:\n\n\n\n\n\n\nSometimes things like token limits, tool errors, or other issues can cause problems that are hard to diagnose. In these cases, the echo=\"all\" option is helpful for getting more information about what’s going on under the hood.\nchat.chat(\"What is the capital of France?\", echo=\"all\")\nThis shows important information like tool call results, finish reasons, and more.\nIf the problem isn’t self-evident, you can also reach into the .get_last_turn(), which contains the full response object, with full details about the completion.\n\n\n\nFor monitoring issues in a production (or otherwise non-interactive) environment, you may want to enabling logging. Also, since chatlas builds on top of packages like anthropic and openai, you can also enable their debug logging to get lower-level information, like HTTP requests and response codes.\n$ export CHATLAS_LOG=info\n$ export OPENAI_LOG=info\n$ export ANTHROPIC_LOG=info\n\n\n\nIf you’re new to world LLMs, you might want to read the Get Started guide, which covers some basic concepts and terminology.\nOnce you’re comfortable with the basics, you can explore more in-depth topics like prompt design or the API reference." + "text": "chatlas provides a simple and unified interface across large language model (llm) providers in Python. It abstracts away complexity from common tasks like streaming chat interfaces, tool calling, structured output, and much more. chatlas helps you prototype faster without painting you into a corner; for example, switching providers is as easy as changing one line of code, but provider specific features are still accessible when needed. Developer experience is also a key focus of chatlas: typing support, rich console output, and built-in tooling are all included.\n(Looking for something similar to chatlas, but in R? Check out elmer!)\n\n\nInstall the latest stable release from PyPI:\npip install -U chatlas\nOr, install the latest development version from GitHub:\npip install -U git+https://github.com/posit-dev/chatlas\n\n\n\nchatlas supports a variety of model providers. See the API reference for more details (like managing credentials) on each provider.\n\nAnthropic (Claude): ChatAnthropic().\nGitHub model marketplace: ChatGithub().\nGoogle (Gemini): ChatGoogle().\nGroq: ChatGroq().\nOllama local models: ChatOllama().\nOpenAI: ChatOpenAI().\nperplexity.ai: ChatPerplexity().\n\nIt also supports the following enterprise cloud providers:\n\nAWS Bedrock: ChatBedrockAnthropic().\nAzure OpenAI: ChatAzureOpenAI().\n\nTo use a model provider that isn’t listed here, you have two options:\n\nIf the model is OpenAI compatible, use ChatOpenAI() with the appropriate base_url and api_key (see ChatGithub for a reference).\nIf you’re motivated, implement a new provider by subclassing Provider and implementing the required methods.\n\n\n\n\nIf you’re using chatlas inside your organisation, you’ll be limited to what your org allows, which is likely to be one provided by a big cloud provider (e.g. ChatAzureOpenAI() and ChatBedrockAnthropic()). If you’re using chatlas for your own personal exploration, you have a lot more freedom so we have a few recommendations to help you get started:\n\nChatOpenAI() or ChatAnthropic() are both good places to start. ChatOpenAI() defaults to GPT-4o, but you can use model = \"gpt-4o-mini\" for a cheaper lower-quality model, or model = \"o1-mini\" for more complex reasoning. ChatAnthropic() is similarly good; it defaults to Claude 3.5 Sonnet which we have found to be particularly good at writing code.\nChatGoogle() is great for large prompts, because it has a much larger context window than other models. It allows up to 1 million tokens, compared to Claude 3.5 Sonnet’s 200k and GPT-4o’s 128k.\nChatOllama(), which uses Ollama, allows you to run models on your own computer. The biggest models you can run locally aren’t as good as the state of the art hosted models, but they also don’t share your data and and are effectively free.\n\n\n\n\nYou can chat via chatlas in several different ways, depending on whether you are working interactively or programmatically. They all start with creating a new chat object:\nfrom chatlas import ChatOpenAI\n\nchat = ChatOpenAI(\n model = \"gpt-4o\",\n system_prompt = \"You are a friendly but terse assistant.\",\n)\n\n\nFrom a chat instance, it’s simple to start a web-based or terminal-based chat console, which is great for testing the capabilities of the model. In either case, responses stream in real-time, and context is preserved across turns.\nchat.app()\n\n\n\nOr, if you prefer to work from the terminal:\nchat.console()\nEntering chat console. Press Ctrl+C to quit.\n\n?> Who created Python?\n\nPython was created by Guido van Rossum. He began development in the late 1980s and released the first version in 1991. \n\n?> Where did he develop it?\n\nGuido van Rossum developed Python while working at Centrum Wiskunde & Informatica (CWI) in the Netherlands. \n\n\n\nFor a more programmatic approach, you can use the .chat() method to ask a question and get a response. By default, the response prints to a rich console as it streams in:\nchat.chat(\"What preceding languages most influenced Python?\")\nPython was primarily influenced by ABC, with additional inspiration from C,\nModula-3, and various other languages.\nTo ask a question about an image, pass one or more additional input arguments using content_image_file() and/or content_image_url():\nfrom chatlas import content_image_url\n\nchat.chat(\n content_image_url(\"https://www.python.org/static/img/python-logo.png\"),\n \"Can you explain this logo?\"\n)\nThe Python logo features two intertwined snakes in yellow and blue,\nrepresenting the Python programming language. The design symbolizes...\nTo get the full response as a string, use the built-in str() function. Optionally, you can also suppress the rich console output by setting echo=\"none\":\nresponse = chat.chat(\"Who is Posit?\", echo=\"none\")\nprint(str(response))\nAs we’ll see in later articles, echo=\"all\" can also be useful for debugging, as it shows additional information, such as tool calls.\n\n\n\nIf you want to do something with the response in real-time (i.e., as it arrives in chunks), use the .stream() method. This method returns an iterator that yields each chunk of the response as it arrives:\nresponse = chat.stream(\"Who is Posit?\")\nfor chunk in response:\n print(chunk, end=\"\")\nThe .stream() method can also be useful if you’re building a chatbot or other programs that needs to display responses as they arrive.\n\n\n\nTool calling is as simple as passing a function with type hints and docstring to .register_tool().\nimport sys\n\ndef get_current_python_version() -> str:\n \"\"\"Get the current version of Python.\"\"\"\n return sys.version\n\nchat.register_tool(get_current_python_version)\nchat.chat(\"What's the current version of Python?\")\nThe current version of Python is 3.13.\nLearn more in the tool calling article\n\n\n\nStructured data (i.e., structured output) is as simple as passing a pydantic model to .extract_data().\nfrom pydantic import BaseModel\n\nclass Person(BaseModel):\n name: str\n age: int\n\nchat.extract_data(\n \"My name is Susan and I'm 13 years old\", \n data_model=Person,\n)\n{'name': 'Susan', 'age': 13}\nLearn more in the structured data article\n\n\n\nEasily get a full markdown or HTML export of a conversation:\nchat.export(\"index.html\", title=\"Python Q&A\")\nIf the export doesn’t have all the information you need, you can also access the full conversation history via the .get_turns() method:\nchat.get_turns()\nAnd, if the conversation is too long, you can specify which turns to include:\nchat.export(\"index.html\", turns=chat.get_turns()[-5:])\n\n\n\nchat methods tend to be synchronous by default, but you can use the async flavor by appending _async to the method name:\nimport asyncio\n\nasync def main():\n await chat.chat_async(\"What is the capital of France?\")\n\nasyncio.run(main())\n\n\n\nchatlas has full typing support, meaning that, among other things, autocompletion just works in your favorite editor:\n\n\n\n\n\n\nSometimes things like token limits, tool errors, or other issues can cause problems that are hard to diagnose. In these cases, the echo=\"all\" option is helpful for getting more information about what’s going on under the hood.\nchat.chat(\"What is the capital of France?\", echo=\"all\")\nThis shows important information like tool call results, finish reasons, and more.\nIf the problem isn’t self-evident, you can also reach into the .get_last_turn(), which contains the full response object, with full details about the completion.\n\n\n\nFor monitoring issues in a production (or otherwise non-interactive) environment, you may want to enabling logging. Also, since chatlas builds on top of packages like anthropic and openai, you can also enable their debug logging to get lower-level information, like HTTP requests and response codes.\n$ export CHATLAS_LOG=info\n$ export OPENAI_LOG=info\n$ export ANTHROPIC_LOG=info\n\n\n\nIf you’re new to world LLMs, you might want to read the Get Started guide, which covers some basic concepts and terminology.\nOnce you’re comfortable with the basics, you can explore more in-depth topics like prompt design or the API reference." }, { "objectID": "index.html#install", "href": "index.html#install", "title": "chatlas", "section": "", - "text": "chatlas isn’t yet on pypi, but you can install from Github:\npip install git+https://github.com/posit-dev/chatlas" + "text": "Install the latest stable release from PyPI:\npip install -U chatlas\nOr, install the latest development version from GitHub:\npip install -U git+https://github.com/posit-dev/chatlas" }, { "objectID": "index.html#model-providers", diff --git a/sitemap.xml b/sitemap.xml index b743ee7..43f2735 100644 --- a/sitemap.xml +++ b/sitemap.xml @@ -2,178 +2,178 @@ https://posit-dev.github.io/chatlas/reference/types.ChatResponseAsync.html - 2024-12-11T21:18:23.194Z + 2024-12-13T22:34:07.126Z https://posit-dev.github.io/chatlas/reference/Provider.html - 2024-12-11T21:18:02.686Z + 2024-12-13T22:33:46.486Z https://posit-dev.github.io/chatlas/reference/types.SubmitInputArgsT.html - 2024-12-11T21:18:02.686Z + 2024-12-13T22:33:46.486Z https://posit-dev.github.io/chatlas/reference/Tool.html - 2024-12-11T21:18:02.686Z + 2024-12-13T22:33:46.486Z https://posit-dev.github.io/chatlas/reference/types.ChatResponse.html - 2024-12-11T21:18:23.190Z + 2024-12-13T22:34:07.122Z https://posit-dev.github.io/chatlas/reference/interpolate.html - 2024-12-11T21:18:23.154Z + 2024-12-13T22:34:07.086Z https://posit-dev.github.io/chatlas/reference/types.TokenUsage.html - 2024-12-11T21:18:02.686Z + 2024-12-13T22:33:46.486Z https://posit-dev.github.io/chatlas/reference/types.ContentImage.html - 2024-12-11T21:18:23.170Z + 2024-12-13T22:34:07.102Z https://posit-dev.github.io/chatlas/reference/types.ImageContentTypes.html - 2024-12-11T21:18:02.686Z + 2024-12-13T22:33:46.486Z https://posit-dev.github.io/chatlas/reference/ChatPerplexity.html - 2024-12-11T21:18:23.090Z + 2024-12-13T22:34:07.022Z https://posit-dev.github.io/chatlas/reference/types.MISSING_TYPE.html - 2024-12-11T21:18:02.686Z + 2024-12-13T22:33:46.486Z https://posit-dev.github.io/chatlas/reference/image_file.html - 2024-12-11T21:18:02.686Z + 2024-12-13T22:33:46.486Z https://posit-dev.github.io/chatlas/reference/ChatBedrockAnthropic.html - 2024-12-11T21:18:23.058Z + 2024-12-13T22:34:06.986Z https://posit-dev.github.io/chatlas/reference/ChatOpenAI.html - 2024-12-11T21:18:23.086Z + 2024-12-13T22:34:07.014Z https://posit-dev.github.io/chatlas/reference/content_image_file.html - 2024-12-11T21:18:23.142Z + 2024-12-13T22:34:07.074Z https://posit-dev.github.io/chatlas/reference/ChatAnthropic.html - 2024-12-11T21:18:23.042Z + 2024-12-13T22:34:06.970Z https://posit-dev.github.io/chatlas/reference/interpolate_file.html - 2024-12-11T21:18:23.158Z + 2024-12-13T22:34:07.090Z https://posit-dev.github.io/chatlas/reference/Chat.html - 2024-12-11T21:18:23.138Z + 2024-12-13T22:34:07.070Z https://posit-dev.github.io/chatlas/reference/ChatOllama.html - 2024-12-11T21:18:23.078Z + 2024-12-13T22:34:07.010Z https://posit-dev.github.io/chatlas/web-apps.html - 2024-12-11T21:18:02.686Z + 2024-12-13T22:33:46.486Z https://posit-dev.github.io/chatlas/structured-data.html - 2024-12-11T21:18:02.686Z + 2024-12-13T22:33:46.486Z https://posit-dev.github.io/chatlas/prompt-design.html - 2024-12-11T21:18:02.682Z + 2024-12-13T22:33:46.486Z https://posit-dev.github.io/chatlas/tool-calling.html - 2024-12-11T21:18:02.686Z + 2024-12-13T22:33:46.486Z https://posit-dev.github.io/chatlas/index.html - 2024-12-11T21:18:25.738Z + 2024-12-13T22:34:09.610Z https://posit-dev.github.io/chatlas/get-started.html - 2024-12-11T21:18:02.678Z + 2024-12-13T22:33:46.478Z https://posit-dev.github.io/chatlas/reference/ChatGithub.html - 2024-12-11T21:18:23.062Z + 2024-12-13T22:34:06.994Z https://posit-dev.github.io/chatlas/reference/ChatAzureOpenAI.html - 2024-12-11T21:18:23.050Z + 2024-12-13T22:34:06.978Z https://posit-dev.github.io/chatlas/reference/Turn.html - 2024-12-11T21:18:23.166Z + 2024-12-13T22:34:07.098Z https://posit-dev.github.io/chatlas/reference/content_image_plot.html - 2024-12-11T21:18:02.686Z + 2024-12-13T22:33:46.486Z https://posit-dev.github.io/chatlas/reference/types.ContentToolRequest.html - 2024-12-11T21:18:23.182Z + 2024-12-13T22:34:07.114Z https://posit-dev.github.io/chatlas/reference/types.ContentText.html - 2024-12-11T21:18:23.182Z + 2024-12-13T22:34:07.110Z https://posit-dev.github.io/chatlas/reference/types.MISSING.html - 2024-12-11T21:18:02.686Z + 2024-12-13T22:33:46.486Z https://posit-dev.github.io/chatlas/reference/types.ContentImageRemote.html - 2024-12-11T21:18:23.178Z + 2024-12-13T22:34:07.106Z https://posit-dev.github.io/chatlas/reference/ChatGroq.html - 2024-12-11T21:18:23.074Z + 2024-12-13T22:34:07.006Z https://posit-dev.github.io/chatlas/reference/types.ContentToolResult.html - 2024-12-11T21:18:23.186Z + 2024-12-13T22:34:07.118Z https://posit-dev.github.io/chatlas/reference/index.html - 2024-12-11T21:18:23.026Z + 2024-12-13T22:34:06.954Z https://posit-dev.github.io/chatlas/reference/image_url.html - 2024-12-11T21:18:02.686Z + 2024-12-13T22:33:46.486Z https://posit-dev.github.io/chatlas/reference/types.Content.html - 2024-12-11T21:18:02.686Z + 2024-12-13T22:33:46.486Z https://posit-dev.github.io/chatlas/reference/token_usage.html - 2024-12-11T21:18:02.686Z + 2024-12-13T22:33:46.486Z https://posit-dev.github.io/chatlas/reference/types.ContentJson.html - 2024-12-11T21:18:23.178Z + 2024-12-13T22:34:07.110Z https://posit-dev.github.io/chatlas/reference/image_plot.html - 2024-12-11T21:18:02.686Z + 2024-12-13T22:33:46.486Z https://posit-dev.github.io/chatlas/reference/types.ContentImageInline.html - 2024-12-11T21:18:23.174Z + 2024-12-13T22:34:07.106Z https://posit-dev.github.io/chatlas/reference/content_image_url.html - 2024-12-11T21:18:02.686Z + 2024-12-13T22:33:46.486Z https://posit-dev.github.io/chatlas/reference/ChatGoogle.html - 2024-12-11T21:18:23.066Z + 2024-12-13T22:34:06.998Z diff --git a/structured-data.html b/structured-data.html index fa15585..78c5b89 100644 --- a/structured-data.html +++ b/structured-data.html @@ -228,7 +228,7 @@

Structured data

When using an LLM to extract data from text or images, you can ask the chatbot to nicely format it, in JSON or any other format that you like. This will generally work well most of the time, but there’s no guarantee that you’ll actually get the exact format that you want. In particular, if you’re trying to get JSON, find that it’s typically surrounded in ```json, and you’ll occassionally get text that isn’t actually valid JSON. To avoid these challenges you can use a recent LLM feature: structured data (aka structured output). With structured data, you supply a type specification that exactly defines the object structure that you want and the LLM will guarantee that’s what you get back.

-
+
import json
 import pandas as pd
 from chatlas import ChatOpenAI
@@ -237,7 +237,7 @@ 
Structured data

 

 
Structured data basics

 
To extract structured data you call the .extract_data() method instead of the .chat() method. You’ll also need to define a type specification that describes the structure of the data that you want (more on that shortly). Here’s a simple example that extracts two specific values from a string:

-

+

 
class Person(BaseModel):
     name: str
     age: int
@@ -253,7 +253,7 @@ 
Structured data bas
 

 

 
The same basic idea works with images too:

-

+

 
from chatlas import content_image_url
 
 class Image(BaseModel):
@@ -265,7 +265,7 @@ 
Structured data bas
   data_model=Image,
 )

 

-
{'primary_shape': 'letter and oval', 'primary_colour': 'blue and gray'}

+
{'primary_shape': 'oval and letter', 'primary_colour': 'gray and blue'}

 

 

 

@@ -273,7 +273,7 @@ 
Structured data bas
 
Data types basics

 
To define your desired type specification (also known as a schema), you use a pydantic model.

 
In addition to the model definition with field names and types, you may also want to provide the LLM with an additional context about what each field/model represents. In this case, include a Field(description="...") for each field, and a docstring for each model. This is a good place to ask nicely for other attributes you’ll like the value to possess (e.g. minimum or maximum values, date formats, …). You aren’t guaranteed that these requests will be honoured, but the LLM will usually make a best effort to do so.

-

+

 
class Person(BaseModel):
     """A person"""
 
@@ -292,7 +292,7 @@ 
Examples

 
The following examples are closely inspired by the Claude documentation and hint at some of the ways you can use structured data extraction.

 

 
Example 1: Article summarisation

-

+

 
with open("examples/third-party-testing.txt") as f:
     text = f.read()
 
@@ -322,25 +322,24 @@ 
Example 1:
 print(json.dumps(data, indent=2))

 

 
{
-  "author": "Anthropic",
+  "author": "N/A",
   "topics": [
-    "AI Policy",
-    "Third-party Testing",
-    "AI Safety",
-    "AI Regulation",
-    "Emerging Technologies",
-    "Security and Ethics"
+    "AI policy",
+    "third-party testing",
+    "technology safety",
+    "regulatory capture",
+    "AI regulation"
   ],
-  "summary": "**Summary:** \nThe article discusses the importance of implementing a robust third-party testing regime for advanced AI systems to ensure their safe deployment and mitigate potential societal risks. As AI models become more powerful and integrated into various sectors, there is a growing need for regulation and oversight to prevent misuse and accidental harm. The proposed testing regime would facilitate collaboration between industry, academia, and government to set safety standards, address national security concerns, and ensure equitable participation by companies of all sizes. The article also highlights the risks of regulatory capture and emphasizes the need for transparency and inclusivity in AI policy development. Anthropic, a developer of AI systems, advocates for these measures and outlines specific actions they will take to support effective regulatory frameworks.",
+  "summary": "The article discusses the importance of implementing a robust third-party testing regime for frontier AI systems to ensure their safety and mitigate the risks of societal harm. It emphasizes the need for involvement from industry, government, and academia to develop standards and procedures for testing. Third-party testing is considered critical for addressing potential misuse or accidents associated with large-scale generative AI models. The article outlines the goals of developing an effective testing regime, which includes building trust, ensuring safety measures for powerful systems, and promoting coordination between countries. There is also discussion about balancing transparency and preventing harmful uses, as well as the potential for regulatory capture. Ultimately, this testing regime aims to complement sector-specific regulations and enable effective oversight of AI development.",
   "coherence": 85,
-  "persuasion": 0.8
+  "persuasion": 0.89
 }

 

 

 

 

 
Example 2: Named entity recognition

-

+

 
text = "John works at Google in New York. He met with Sarah, the CEO of Acme Inc., last week in San Francisco."
 
 
@@ -381,49 +380,37 @@ 
Example
 0
 John
 person
-Works at Google in New York
+John works at Google in New York.
 
 
 1
 Google
 organization
-John works at Google in New York
+John works at Google in New York.
 
 
 2
 New York
 location
-John works there
+John works at Google in New York.
 
 
 3
 Sarah
 person
-Met John in San Francisco
+He met with Sarah, the CEO of Acme Inc., last ...
 
 
 4
-CEO
-title
-Sarah's position at Acme Inc.
-
-
-5
 Acme Inc.
 organization
-Sarah is the CEO
+He met with Sarah, the CEO of Acme Inc., last ...
 
-
-6
+
+5
 San Francisco
 location
-John met with Sarah there last week
-
-
-7
-last week
-temporal
-When John met with Sarah
+He met with Sarah, the CEO of Acme Inc., last ...
 
 
 
@@ -434,7 +421,7 @@ 
Example
 

 

 
Example 3: Sentiment analysis

-

+

 
text = "The product was okay, but the customer service was terrible. I probably won't buy from them again."
 
 class Sentiment(BaseModel):
@@ -463,7 +450,7 @@ 
Example 3: Se
 

 

 
Example 4: Text classification

-

+

 
from typing import Literal
 
 text = "The new quantum computing breakthrough could revolutionize the tech industry."
@@ -503,17 +490,17 @@ 
Example 4: T
 
 0
 Technology
-0.85
+0.7
 
 
 1
 Business
-0.10
+0.2
 
 
 2
 Other
-0.05
+0.1
 
 
 
@@ -524,7 +511,7 @@ 
Example 4: T
 

 

 
Example 5: Working with unknown keys

-

+

 
from chatlas import ChatAnthropic
 
 
@@ -547,7 +534,7 @@ 
Exampl
 print(json.dumps(data, indent=2))

 

 
{
-  "physical_characteristics": {
+  "appearance": {
     "height": "tall",
     "facial_features": {
       "beard": true,
@@ -555,12 +542,12 @@ 
Exampl
         "location": "left cheek"
       }
     },
-    "voice": "deep"
-  },
-  "clothing": {
-    "outerwear": {
-      "type": "leather jacket",
-      "color": "black"
+    "voice": "deep",
+    "clothing": {
+      "outerwear": {
+        "type": "leather jacket",
+        "color": "black"
+      }
     }
   }
 }

@@ -579,7 +566,7 @@ 
Ex
 
 

 
Even without any descriptions, ChatGPT does pretty well:

-

+

 
from chatlas import content_image_file
 
 
@@ -626,7 +613,7 @@ 
Ex
 
 
 0
-11 Zinfandel Lane - Home & Vineyard
+11 Zinfandel Lane - Home & Vineyard [RP]
 JT
 St. Helena/Napa, CA, US
 5000001
@@ -638,7 +625,7 @@ 
Ex
 
 
 1
-25 Point Lobos - Commercial Property
+25 Point Lobos - Commercial Property [RP]
 SP
 San Francisco/San Francisco, CA, US
 5000001
@@ -663,7 +650,7 @@ 
Advanced data typesRequired vs optional

 
By default, model fields are in a sense “required”, unless None is allowed in their type definition. Including None is a good idea if there’s any possibility of the input not containing the required fields as LLMs may hallucinate data in order to fulfill your spec.

 
For example, here the LLM hallucinates a date even though there isn’t one in the text:

-

+

 
class ArticleSpec(BaseModel):
     """Information about an article written in markdown"""
 
@@ -690,13 +677,13 @@ 
Required vs optional<
 
{
   "title": "Structured Data",
   "author": "Hadley Wickham",
-  "date": "2023-10-16"
+  "date": "2023-10-07"
 }

 

 

 
Note that I’ve used more of an explict prompt here. For this example, I found that this generated better results, and it’s a useful place to put additional instructions.

 
If let the LLM know that the fields are all optional, it’ll instead return None for the missing fields:

-

+

 
class ArticleSpec(BaseModel):
     """Information about an article written in markdown"""
 
@@ -735,12 +722,12 @@ 
Data frames

 

 
Token usage

 
Below is a summary of the tokens used to create the output in this example.

-

+

 
from chatlas import token_usage
 token_usage()

 

-
[{'name': 'OpenAI', 'input': 6081, 'output': 609},
- {'name': 'Anthropic', 'input': 463, 'output': 139}]

+
[{'name': 'OpenAI', 'input': 6081, 'output': 615},
+ {'name': 'Anthropic', 'input': 463, 'output': 137}]

 

 

 
@@ -1169,7 +1156,7 @@ 
Token usage

 });
 
 
 
-
 
 
-