02_in_context_learning.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## In-Context Learning"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With the increasing size and complexity of model architectures, [large language models (LLMs) have demonstrated in-context learning (ICL) ability](https://splab.sdu.edu.cn/GPT3.pdf). This enables LLMs to perform tasks and generate responses based on the context provided in the input prompt, without requiring explicit fine-tuning or retraining. In practice, this context includes one or a few demonstration examples that guide (condition) the model in performing downstream tasks such as classification, question/answering, information extraction, reasoning, and data analysis. [In 2022, researchers at Anthropic investigated the hypothesis that *'induction [attention] heads'* were the primary mechanism driving ICL](https://transformer-circuits.pub/2022/in-context-learning-and-induction-heads/index.html). These specialized units attend earlier parts of the input to copy and complete sequences, which would allow models to adapt to patterns and generate responses aligned to the provided context.\n",
    "\n",
    "This notebook explores the concept of ICL, demonstrating its practical application in Named Entity Recognition (NER). NER is a `Natural Language Processing` task that identifies and classifies named entities (NE) into predefined semantic categories (such as persons, organizations, locations, events, time expressions, and quantities). By converting raw text into structured information, NER makes data more actionable, facilitating tasks like information extraction, data aggregation, analytics, and social media monitoring."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p align=\"center\">\n",
    "  <img src=\"./static/in_context_learning.png\">\n",
    "</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. Setup"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 1.1 Install/Upgrade Python packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install openai tenacity --quiet | tail -n 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 1.2 Load packages and OPENAI_API_KEY"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can generate an API key in the OpenAI web interface. See https://platform.openai.com/account/api-keys for details."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook works with the latest OpeanAI models `gpt-4o` and `gpt-4o-mini`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "import logging\n",
    "import os\n",
    "import openai\n",
    "\n",
    "from tenacity import retry, wait_random_exponential, stop_after_attempt\n",
    "\n",
    "logging.basicConfig(level=logging.INFO, format=' %(asctime)s - %(levelname)s - %(message)s')\n",
    "\n",
    "OPENAI_MODEL = 'gpt-4o-mini'\n",
    "client = openai.OpenAI(api_key=os.environ.get(\"OPENAI_API_KEY\", \"<your OpenAI API key if not set as env var>\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Define the NER labels to be Identified"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We define a standard set of NER labels to showcase a wide range of use cases. However, for our specific task of enriching text with knowledge base links, only a subset is practically required."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "labels = [\n",
    "    \"person\",      # people, including fictional characters\n",
    "    \"fac\",         # buildings, airports, highways, bridges\n",
    "    \"org\",         # organizations, companies, agencies, institutions\n",
    "    \"gpe\",         # geopolitical entities like countries, cities, states\n",
    "    \"loc\",         # non-gpe locations\n",
    "    \"product\",     # vehicles, foods, appareal, appliances, software, toys \n",
    "    \"event\",       # named sports, scientific milestones, historical events\n",
    "    \"work_of_art\", # titles of books, songs, movies\n",
    "    \"law\",         # named laws, acts, or legislations\n",
    "    \"language\",    # any named language\n",
    "    \"date\",        # absolute or relative dates or periods\n",
    "    \"time\",        # time units smaller than a day\n",
    "    \"percent\",     # percentage (e.g., \"twenty percent\", \"18%\")\n",
    "    \"money\",       # monetary values, including unit\n",
    "    \"quantity\",    # measurements, e.g., weight or distance\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. Prepare messages"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The [chat completions API](https://platform.openai.com/docs/guides/gpt/chat-completions-api) takes a list of messages as input and delivers a model-generated message as an output. While the chat format is primarily designed for facilitating multi-turn conversations, it is equally efficient for single-turn tasks without any preceding conversation. For our purposes, we will specify a message for the system, assistant, and user roles."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.1 System Message"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `system message` (prompt) sets the assistant's behavior by defining its desired persona and task. We also delineate the specific set of entity labels we aim to identify."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Although one can instruct the model to format its response, it has to be noted that both `gpt-4o` and `gpt-4o-mini` have been fine-tuned to discern when a function should be invoked, and to reply with `JSON` formatted according to the function's signature. This capability streamlines our prompt and enables us to receive structured data directly from the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "def system_message(labels):\n",
    "    return f\"\"\"\n",
    "You are an expert in Natural Language Processing. Your task is to identify common Named Entities (NER) in a given text.\n",
    "The possible common Named Entities (NER) types are exclusively: ({\", \".join(labels)}).\"\"\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.2 Assistant Message"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`Assistant messages` usually store previous assistant responses. However, as in our scenario, they can also be crafted to provide examples of the desired behavior. While OpenAI is able to execute `zero-shot` Named Entity Recognition, we have found that a `one-shot` approach produces more precise results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "def assisstant_message():\n",
    "    return f\"\"\"\n",
    "EXAMPLE:\n",
    "    Text: 'In Germany, in 1440, goldsmith Johannes Gutenberg invented the movable-type printing press. His work led to an information revolution and the unprecedented mass-spread / \n",
    "    of literature throughout Europe. Modelled on the design of the existing screw presses, a single Renaissance movable-type printing press could produce up to 3,600 pages per workday.'\n",
    "    {{\n",
    "        \"gpe\": [\"Germany\", \"Europe\"],\n",
    "        \"date\": [\"1440\"],\n",
    "        \"person\": [\"Johannes Gutenberg\"],\n",
    "        \"product\": [\"movable-type printing press\"],\n",
    "        \"event\": [\"Renaissance\"],\n",
    "        \"quantity\": [\"3,600 pages\"],\n",
    "        \"time\": [\"workday\"]\n",
    "    }}\n",
    "--\"\"\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.3 User Message"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `user message` provides the specific text for the assistant task:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "def user_message(text):\n",
    "    return f\"\"\"\n",
    "TASK:\n",
    "    Text: {text}\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4. Inference"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 4.1 Chat Completion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The Chat Completions API accepts inputs via the messages parameter, which is an array of message objects:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "@retry(wait=wait_random_exponential(min=1, max=10), stop=stop_after_attempt(3))\n",
    "def run_openai_task(labels, text):\n",
    "    messages = [\n",
    "          {\"role\": \"system\", \"content\": system_message(labels=labels)},\n",
    "          {\"role\": \"assistant\", \"content\": assisstant_message()},\n",
    "          {\"role\": \"user\", \"content\": user_message(text=text)}\n",
    "      ]\n",
    "\n",
    "    response = openai.chat.completions.create(\n",
    "        model=\"gpt-4o-mini\",\n",
    "        messages=messages,\n",
    "        temperature=0,\n",
    "        frequency_penalty=0,\n",
    "        presence_penalty=0,\n",
    "\n",
    "    )\n",
    "\n",
    "    response_message = response.choices[0].message\n",
    "    \n",
    "    return {\"response\": response, \n",
    "            \"response_message\": response_message.content}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 4.2 Run OpenAI Task"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 2024-08-15 21:11:43,558 - INFO - HTTP Request: POST https://api.openai.com/v1/chat/completions \"HTTP/1.1 200 OK\"\n"
     ]
    }
   ],
   "source": [
    "text = \"\"\"The Beatles were an English rock band formed in Liverpool in 1960, comprising John Lennon, Paul McCartney, George Harrison, and Ringo Starr.\n",
    "          Rooted in skiffle, beat and 1950s rock 'n' roll, their sound incorporated elements of classical music and traditional pop in innovative ways.\n",
    "          The Beatles are the best-selling music act of all time, with estimated sales of 600 million units worldwide.\n",
    "       \"\"\"\n",
    "result = run_openai_task(labels, text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      " 2024-08-15 21:11:44,872 - INFO - Response: {\n",
      "    \"org\": [\"The Beatles\"],\n",
      "    \"gpe\": [\"England\", \"Liverpool\"],\n",
      "    \"date\": [\"1960\"],\n",
      "    \"person\": [\"John Lennon\", \"Paul McCartney\", \"George Harrison\", \"Ringo Starr\"],\n",
      "    \"product\": [\"rock band\"],\n",
      "    \"quantity\": [\"600 million units\"],\n",
      "    \"event\": [\"1950s rock 'n' roll\"]\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "logging.info(f\"Response: {result['response_message']}\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}