Instruction-fine-tuning.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "# Fine-Tune a Generative AI Model for Dialogue Summarization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook focuses on fine-tuning the FLAN-T5 model from Hugging Face for improved dialogue summarization, exploring full fine-tuning and evaluating results with ROUGE metrics, followed by Parameter Efficient Fine-Tuning (PEFT) to demonstrate its advantages despite slightly lower performance metrics."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name='1'></a>\n",
    "## 1 - Import the necessary components."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import warnings\n",
    "from datasets import load_dataset\n",
    "from transformers import AutoModelForSeq2SeqLM, AutoTokenizer, GenerationConfig, TrainingArguments, Trainer\n",
    "import torch\n",
    "import time\n",
    "import evaluate\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "warnings.filterwarnings('ignore')\n",
    "# tqdm library makes the loops show a smart progress meter.\n",
    "import pandas as pd\n",
    "from tqdm.notebook import tqdm\n",
    "tqdm.pandas()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "<a name='1.2'></a>\n",
    "### 1.2 - Load Dataset and LLM "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DatasetDict({\n",
       "    train: Dataset({\n",
       "        features: ['id', 'dialogue', 'summary', 'topic'],\n",
       "        num_rows: 12460\n",
       "    })\n",
       "    validation: Dataset({\n",
       "        features: ['id', 'dialogue', 'summary', 'topic'],\n",
       "        num_rows: 500\n",
       "    })\n",
       "    test: Dataset({\n",
       "        features: ['id', 'dialogue', 'summary', 'topic'],\n",
       "        num_rows: 1500\n",
       "    })\n",
       "})"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "huggingface_dataset_name = \"knkarthick/dialogsum\"\n",
    "dataset = load_dataset(huggingface_dataset_name)\n",
    "dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "Load the pre-trained FLAN-T5 model and its tokenizer from Hugging Face, specifically using the small version. Setting `torch_dtype=torch.bfloat16` specifies the memory type for the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "model_name='google/flan-t5-base'\n",
    "original_model = AutoModelForSeq2SeqLM.from_pretrained(model_name, torch_dtype=torch.bfloat16)\n",
    "tokenizer = AutoTokenizer.from_pretrained(model_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "The number of model parameters and the count of trainable parameters can be extracted using below function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "trainable model parameters: 247577856\n",
      "all model parameters: 247577856\n",
      "percentage of trainable model parameters: 100.00%\n"
     ]
    }
   ],
   "source": [
    "def print_number_of_trainable_model_parameters(model):\n",
    "    trainable_model_params = 0\n",
    "    all_model_params = 0\n",
    "    for _, param in model.named_parameters():\n",
    "        all_model_params += param.numel()\n",
    "        if param.requires_grad:\n",
    "            trainable_model_params += param.numel()\n",
    "    return f\"trainable model parameters: {trainable_model_params}\\nall model parameters: {all_model_params}\\npercentage of trainable model parameters: {100 * trainable_model_params / all_model_params:.2f}%\"\n",
    "\n",
    "print(print_number_of_trainable_model_parameters(original_model))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "<a name='1.3'></a>\n",
    "### 1.3 - Test the Model with Zero Shot Inferencing\n",
    "Testing the model with zero-shot inference reveals that it struggles to summarize the dialogue compared to the baseline summary; however, it extracts key information, indicating potential for fine-tuning to improve task performance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "---------------------------------------------------------------------------------------------------\n",
      "INPUT PROMPT:\n",
      "\n",
      "Summarize the following conversation.\n",
      "\n",
      "#Person1#: Have you considered upgrading your system?\n",
      "#Person2#: Yes, but I'm not sure what exactly I would need.\n",
      "#Person1#: You could consider adding a painting program to your software. It would allow you to make up your own flyers and banners for advertising.\n",
      "#Person2#: That would be a definite bonus.\n",
      "#Person1#: You might also want to upgrade your hardware because it is pretty outdated now.\n",
      "#Person2#: How can we do that?\n",
      "#Person1#: You'd probably need a faster processor, to begin with. And you also need a more powerful hard disc, more memory and a faster modem. Do you have a CD-ROM drive?\n",
      "#Person2#: No.\n",
      "#Person1#: Then you might want to add a CD-ROM drive too, because most new software programs are coming out on Cds.\n",
      "#Person2#: That sounds great. Thanks.\n",
      "\n",
      "Summary:\n",
      "\n",
      "---------------------------------------------------------------------------------------------------\n",
      "BASELINE HUMAN SUMMARY:\n",
      "#Person1# teaches #Person2# how to upgrade software and hardware in #Person2#'s system.\n",
      "\n",
      "---------------------------------------------------------------------------------------------------\n",
      "MODEL GENERATION - ZERO SHOT:\n",
      "#Person1#: I'm thinking of upgrading my computer.\n"
     ]
    }
   ],
   "source": [
    "index = 200\n",
    "\n",
    "dialogue = dataset['test'][index]['dialogue']\n",
    "summary = dataset['test'][index]['summary']\n",
    "\n",
    "prompt = f\"\"\"\n",
    "Summarize the following conversation.\n",
    "\n",
    "{dialogue}\n",
    "\n",
    "Summary:\n",
    "\"\"\"\n",
    "\n",
    "inputs = tokenizer(prompt, return_tensors='pt')\n",
    "output = tokenizer.decode(\n",
    "    original_model.generate(\n",
    "        inputs[\"input_ids\"], \n",
    "        max_new_tokens=200,\n",
    "    )[0], \n",
    "    skip_special_tokens=True\n",
    ")\n",
    "\n",
    "dash_line = '-'.join('' for x in range(100))\n",
    "print(dash_line)\n",
    "print(f'INPUT PROMPT:\\n{prompt}')\n",
    "print(dash_line)\n",
    "print(f'BASELINE HUMAN SUMMARY:\\n{summary}\\n')\n",
    "print(dash_line)\n",
    "print(f'MODEL GENERATION - ZERO SHOT:\\n{output}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "<a name='2'></a>\n",
    "## 2 - Perform Full Fine-Tuning"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "<a name='2.1'></a>\n",
    "### 2.1 - Preprocess the Dialog-Summary Dataset\n",
    "Convert the dialog-summary pairs into explicit instructions for the LLM by adding `Summarize the following conversation` at the beginning of the dialogue and `Summary` at the start of the summary. The format should be:\n",
    "\n",
    "Training prompt (dialogue):\n",
    "```\n",
    "Summarize the following conversation.\n",
    "\n",
    "    Chris: This is his part of the conversation.\n",
    "    Antje: This is her part of the conversation.\n",
    "    \n",
    "Summary: \n",
    "```\n",
    "\n",
    "Training response (summary):\n",
    "```\n",
    "Both Chris and Antje participated in the conversation.\n",
    "```\n",
    "\n",
    "Then preprocess the prompt-response dataset into tokens and pull out their `input_ids` (1 per token)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def tokenize_function(example):\n",
    "    start_prompt = 'Summarize the following conversation.\\n\\n'\n",
    "    end_prompt = '\\n\\nSummary: '\n",
    "    prompt = [start_prompt + dialogue + end_prompt for dialogue in example[\"dialogue\"]]\n",
    "    example['input_ids'] = tokenizer(prompt, padding=\"max_length\", truncation=True, return_tensors=\"pt\").input_ids\n",
    "    example['labels'] = tokenizer(example[\"summary\"], padding=\"max_length\", truncation=True, return_tensors=\"pt\").input_ids\n",
    "    \n",
    "    return example\n",
    "\n",
    "# The dataset actually contains 3 diff splits: train, validation, test.\n",
    "# The tokenize_function code is handling all data across all splits in batches.\n",
    "tokenized_datasets = dataset.map(tokenize_function, batched=True)\n",
    "tokenized_datasets = tokenized_datasets.remove_columns(['id', 'topic', 'dialogue', 'summary',])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "To save time, subsampling the dataset will be performed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "tokenized_datasets = tokenized_datasets.filter(lambda example, index: index % 100 == 0, with_indices=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "Check the shapes of all three parts of the dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shapes of the datasets:\n",
      "Training: (125, 2)\n",
      "Validation: (5, 2)\n",
      "Test: (15, 2)\n",
      "DatasetDict({\n",
      "    train: Dataset({\n",
      "        features: ['input_ids', 'labels'],\n",
      "        num_rows: 125\n",
      "    })\n",
      "    validation: Dataset({\n",
      "        features: ['input_ids', 'labels'],\n",
      "        num_rows: 5\n",
      "    })\n",
      "    test: Dataset({\n",
      "        features: ['input_ids', 'labels'],\n",
      "        num_rows: 15\n",
      "    })\n",
      "})\n"
     ]
    }
   ],
   "source": [
    "print(f\"Shapes of the datasets:\")\n",
    "print(f\"Training: {tokenized_datasets['train'].shape}\")\n",
    "print(f\"Validation: {tokenized_datasets['validation'].shape}\")\n",
    "print(f\"Test: {tokenized_datasets['test'].shape}\")\n",
    "print(tokenized_datasets)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The output dataset is ready for fine-tuning."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name='2.2'></a>\n",
    "### 2.2 - Fine-Tune the Model with the Preprocessed Dataset\n",
    "Utilize the built-in Hugging Face `Trainer` class to train the model, passing the preprocessed dataset along with the original model. Other training parameters can be determined experimentally, with no need for detailed exploration at this time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# output_dir = f'./dialogue-summary-training-{str(int(time.time()))}'\n",
    "# training_args = TrainingArguments(\n",
    "#     output_dir=output_dir,\n",
    "#     learning_rate=1e-5,\n",
    "#     num_train_epochs=1,\n",
    "#     weight_decay=0.01,\n",
    "#     logging_steps=1,\n",
    "#     max_steps=1\n",
    "# )\n",
    "# trainer = Trainer(\n",
    "#     model=original_model,\n",
    "#     args=training_args,\n",
    "#     train_dataset=tokenized_datasets['train'],\n",
    "#     eval_dataset=tokenized_datasets['validation']\n",
    "# )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# trainer.train()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Training a fully fine-tuned model takes hours on a GPU, so a checkpoint of the fine-tuned model is used throughout this notebook, referred to as the **instruct model**. The size of the instruct model is approximately 1GB."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "Create an instance of the `AutoModelForSeq2SeqLM` class for the instruct model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "instruct_model = AutoModelForSeq2SeqLM.from_pretrained(\"./flan-dialogue-summary-checkpoint\", torch_dtype=torch.bfloat16)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name='2.3'></a>\n",
    "### 2.3 - Evaluate the Model Qualitatively (Human Evaluation)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "---------------------------------------------------------------------------------------------------\n",
      "BASELINE HUMAN SUMMARY:\n",
      "#Person1# teaches #Person2# how to upgrade software and hardware in #Person2#'s system.\n",
      "---------------------------------------------------------------------------------------------------\n",
      "ORIGINAL MODEL:\n",
      "#Person1#: I'm thinking of upgrading my computer.\n",
      "---------------------------------------------------------------------------------------------------\n",
      "INSTRUCT MODEL:\n",
      "#Person1# suggests #Person2# adding a painting program to #Person2#'s software and upgrading the hardware. #Person2# also wants to add a CD-ROM drive.\n"
     ]
    }
   ],
   "source": [
    "index = 200\n",
    "dialogue = dataset['test'][index]['dialogue']\n",
    "human_baseline_summary = dataset['test'][index]['summary']\n",
    "\n",
    "prompt = f\"\"\"\n",
    "Summarize the following conversation.\n",
    "\n",
    "{dialogue}\n",
    "\n",
    "Summary:\n",
    "\"\"\"\n",
    "\n",
    "input_ids = tokenizer(prompt, return_tensors=\"pt\").input_ids\n",
    "\n",
    "original_model_outputs = original_model.generate(input_ids=input_ids, generation_config=GenerationConfig(max_new_tokens=200, num_beams=1))\n",
    "original_model_text_output = tokenizer.decode(original_model_outputs[0], skip_special_tokens=True)\n",
    "\n",
    "instruct_model_outputs = instruct_model.generate(input_ids=input_ids, generation_config=GenerationConfig(max_new_tokens=200, num_beams=1))\n",
    "instruct_model_text_output = tokenizer.decode(instruct_model_outputs[0], skip_special_tokens=True)\n",
    "\n",
    "print(dash_line)\n",
    "print(f'BASELINE HUMAN SUMMARY:\\n{human_baseline_summary}')\n",
    "print(dash_line)\n",
    "print(f'ORIGINAL MODEL:\\n{original_model_text_output}')\n",
    "print(dash_line)\n",
    "print(f'INSTRUCT MODEL:\\n{instruct_model_text_output}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name='2.4'></a>\n",
    "### 2.4 - Evaluate the Model Quantitatively (with ROUGE Metric)\n",
    "The ROUGE metric quantifies the validity of model-generated summaries by comparing them to a human-created baseline summary, indicating the effectiveness gained through fine-tuning."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "rouge = evaluate.load('rouge')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Generate the outputs for the sample of the test dataset (only 10 dialogues and summaries to save time), and save the results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>human_baseline_summaries</th>\n",
       "      <th>original_model_summaries</th>\n",
       "      <th>instruct_model_summaries</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Ms. Dawson helps #Person1# to write a memo to ...</td>\n",
       "      <td>#Person1#: I need to take a dictation for you.</td>\n",
       "      <td>#Person1# asks Ms. Dawson to take a dictation ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>In order to prevent employees from wasting tim...</td>\n",
       "      <td>#Person1#: I need to take a dictation for you.</td>\n",
       "      <td>#Person1# asks Ms. Dawson to take a dictation ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Ms. Dawson takes a dictation for #Person1# abo...</td>\n",
       "      <td>#Person1#: I need to take a dictation for you.</td>\n",
       "      <td>#Person1# asks Ms. Dawson to take a dictation ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>#Person2# arrives late because of traffic jam....</td>\n",
       "      <td>The traffic jam at the Carrefour intersection ...</td>\n",
       "      <td>#Person2# got stuck in traffic again. #Person1...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>#Person2# decides to follow #Person1#'s sugges...</td>\n",
       "      <td>The traffic jam at the Carrefour intersection ...</td>\n",
       "      <td>#Person2# got stuck in traffic again. #Person1...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>#Person2# complains to #Person1# about the tra...</td>\n",
       "      <td>The traffic jam at the Carrefour intersection ...</td>\n",
       "      <td>#Person2# got stuck in traffic again. #Person1...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>#Person1# tells Kate that Masha and Hero get d...</td>\n",
       "      <td>Masha and Hero are getting divorced.</td>\n",
       "      <td>Masha and Hero are getting divorced. Kate can'...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>#Person1# tells Kate that Masha and Hero are g...</td>\n",
       "      <td>Masha and Hero are getting divorced.</td>\n",
       "      <td>Masha and Hero are getting divorced. Kate can'...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>#Person1# and Kate talk about the divorce betw...</td>\n",
       "      <td>Masha and Hero are getting divorced.</td>\n",
       "      <td>Masha and Hero are getting divorced. Kate can'...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>#Person1# and Brian are at the birthday party ...</td>\n",
       "      <td>#Person1#: Happy birthday, Brian. #Person2#: T...</td>\n",
       "      <td>Brian's birthday is coming. #Person1# invites ...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                            human_baseline_summaries  \\\n",
       "0  Ms. Dawson helps #Person1# to write a memo to ...   \n",
       "1  In order to prevent employees from wasting tim...   \n",
       "2  Ms. Dawson takes a dictation for #Person1# abo...   \n",
       "3  #Person2# arrives late because of traffic jam....   \n",
       "4  #Person2# decides to follow #Person1#'s sugges...   \n",
       "5  #Person2# complains to #Person1# about the tra...   \n",
       "6  #Person1# tells Kate that Masha and Hero get d...   \n",
       "7  #Person1# tells Kate that Masha and Hero are g...   \n",
       "8  #Person1# and Kate talk about the divorce betw...   \n",
       "9  #Person1# and Brian are at the birthday party ...   \n",
       "\n",
       "                            original_model_summaries  \\\n",
       "0     #Person1#: I need to take a dictation for you.   \n",
       "1     #Person1#: I need to take a dictation for you.   \n",
       "2     #Person1#: I need to take a dictation for you.   \n",
       "3  The traffic jam at the Carrefour intersection ...   \n",
       "4  The traffic jam at the Carrefour intersection ...   \n",
       "5  The traffic jam at the Carrefour intersection ...   \n",
       "6               Masha and Hero are getting divorced.   \n",
       "7               Masha and Hero are getting divorced.   \n",
       "8               Masha and Hero are getting divorced.   \n",
       "9  #Person1#: Happy birthday, Brian. #Person2#: T...   \n",
       "\n",
       "                            instruct_model_summaries  \n",
       "0  #Person1# asks Ms. Dawson to take a dictation ...  \n",
       "1  #Person1# asks Ms. Dawson to take a dictation ...  \n",
       "2  #Person1# asks Ms. Dawson to take a dictation ...  \n",
       "3  #Person2# got stuck in traffic again. #Person1...  \n",
       "4  #Person2# got stuck in traffic again. #Person1...  \n",
       "5  #Person2# got stuck in traffic again. #Person1...  \n",
       "6  Masha and Hero are getting divorced. Kate can'...  \n",
       "7  Masha and Hero are getting divorced. Kate can'...  \n",
       "8  Masha and Hero are getting divorced. Kate can'...  \n",
       "9  Brian's birthday is coming. #Person1# invites ...  "
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dialogues = dataset['test'][0:10]['dialogue']\n",
    "human_baseline_summaries = dataset['test'][0:10]['summary']\n",
    "\n",
    "original_model_summaries = []\n",
    "instruct_model_summaries = []\n",
    "\n",
    "for _, dialogue in enumerate(dialogues):\n",
    "    prompt = f\"\"\"\n",
    "Summarize the following conversation.\n",
    "\n",
    "{dialogue}\n",
    "\n",
    "Summary: \"\"\"\n",
    "    input_ids = tokenizer(prompt, return_tensors=\"pt\").input_ids\n",
    "\n",
    "    original_model_outputs = original_model.generate(input_ids=input_ids, generation_config=GenerationConfig(max_new_tokens=200))\n",
    "    original_model_text_output = tokenizer.decode(original_model_outputs[0], skip_special_tokens=True)\n",
    "    original_model_summaries.append(original_model_text_output)\n",
    "\n",
    "    instruct_model_outputs = instruct_model.generate(input_ids=input_ids, generation_config=GenerationConfig(max_new_tokens=200))\n",
    "    instruct_model_text_output = tokenizer.decode(instruct_model_outputs[0], skip_special_tokens=True)\n",
    "    instruct_model_summaries.append(instruct_model_text_output)\n",
    "    \n",
    "zipped_summaries = list(zip(human_baseline_summaries, original_model_summaries, instruct_model_summaries))\n",
    " \n",
    "df = pd.DataFrame(zipped_summaries, columns = ['human_baseline_summaries', 'original_model_summaries', 'instruct_model_summaries'])\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "Evaluate the models computing ROUGE metrics. Notice the improvement in the results!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ORIGINAL MODEL:\n",
      "{'rouge1': np.float64(0.25036039886039885), 'rouge2': np.float64(0.11573401534526855), 'rougeL': np.float64(0.2286324786324786), 'rougeLsum': np.float64(0.2303917378917379)}\n",
      "INSTRUCT MODEL:\n",
      "{'rouge1': np.float64(0.39543570996335975), 'rouge2': np.float64(0.17186878842221226), 'rougeL': np.float64(0.2806870881294845), 'rougeLsum': np.float64(0.281746348889206)}\n"
     ]
    }
   ],
   "source": [
    "original_model_results = rouge.compute(\n",
    "    predictions=original_model_summaries,\n",
    "    references=human_baseline_summaries[0:len(original_model_summaries)],\n",
    "    use_aggregator=True,\n",
    "    use_stemmer=True,\n",
    ")\n",
    "\n",
    "instruct_model_results = rouge.compute(\n",
    "    predictions=instruct_model_summaries,\n",
    "    references=human_baseline_summaries[0:len(instruct_model_summaries)],\n",
    "    use_aggregator=True,\n",
    "    use_stemmer=True,\n",
    ")\n",
    "\n",
    "print('ORIGINAL MODEL:')\n",
    "print(original_model_results)\n",
    "print('INSTRUCT MODEL:')\n",
    "print(instruct_model_results)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The file `data/dialogue-summary-training-results.csv` contains pre-populated model results for evaluating a larger data set across each model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ORIGINAL MODEL:\n",
      "{'rouge1': np.float64(0.23328464502586696), 'rouge2': np.float64(0.0760949386582638), 'rougeL': np.float64(0.20150430978201175), 'rougeLsum': np.float64(0.2014999133289781)}\n",
      "INSTRUCT MODEL:\n",
      "{'rouge1': np.float64(0.4214360700703818), 'rouge2': np.float64(0.18058992754713823), 'rougeL': np.float64(0.3384594986152175), 'rougeLsum': np.float64(0.33806946616372496)}\n"
     ]
    }
   ],
   "source": [
    "results = pd.read_csv(\"data/dialogue-summary-training-results.csv\")\n",
    "\n",
    "human_baseline_summaries = results['human_baseline_summaries'].values\n",
    "original_model_summaries = results['original_model_summaries'].values\n",
    "instruct_model_summaries = results['instruct_model_summaries'].values\n",
    "\n",
    "original_model_results = rouge.compute(\n",
    "    predictions=original_model_summaries,\n",
    "    references=human_baseline_summaries[0:len(original_model_summaries)],\n",
    "    use_aggregator=True,\n",
    "    use_stemmer=True,\n",
    ")\n",
    "\n",
    "instruct_model_results = rouge.compute(\n",
    "    predictions=instruct_model_summaries,\n",
    "    references=human_baseline_summaries[0:len(instruct_model_summaries)],\n",
    "    use_aggregator=True,\n",
    "    use_stemmer=True,\n",
    ")\n",
    "\n",
    "print('ORIGINAL MODEL:')\n",
    "print(original_model_results)\n",
    "print('INSTRUCT MODEL:')\n",
    "print(instruct_model_results)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "The results show substantial improvement in all ROUGE metrics:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Absolute percentage improvement of INSTRUCT MODEL over ORIGINAL MODEL\n",
      "rouge1: 18.82%\n",
      "rouge2: 10.45%\n",
      "rougeL: 13.70%\n",
      "rougeLsum: 13.66%\n"
     ]
    }
   ],
   "source": [
    "print(\"Absolute percentage improvement of INSTRUCT MODEL over ORIGINAL MODEL\")\n",
    "\n",
    "improvement = (np.array(list(instruct_model_results.values())) - np.array(list(original_model_results.values())))\n",
    "for key, value in zip(instruct_model_results.keys(), improvement):\n",
    "    print(f'{key}: {value*100:.2f}%')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name='3'></a>\n",
    "## 3 - Perform Parameter Efficient Fine-Tuning (PEFT)\n",
    "Parameter Efficient Fine-Tuning (PEFT) offers a more efficient alternative to full fine-tuning, producing comparable evaluation results. PEFT encompasses methods like Low-Rank Adaptation (LoRA) and prompt tuning, with LoRA specifically enabling fine-tuning using fewer resources. It creates a smaller \"LoRA adapter\" that, when combined with the original LLM at inference, reduces memory requirements while allowing multiple tasks to share the same base model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name='3.1'></a>\n",
    "### 3.1 - Setup the PEFT/LoRA model for Fine-Tuning\n",
    "Setting up the PEFT/LoRA model for fine-tuning involves freezing the underlying LLM and training only the adapter. The LoRA configuration includes a `rank` (r) hyper-parameter that determines the rank or dimension of the adapter being trained."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from peft import LoraConfig, get_peft_model, TaskType\n",
    "\n",
    "lora_config = LoraConfig(\n",
    "    r=32, # Rank\n",
    "    lora_alpha=32,\n",
    "    target_modules=[\"q\", \"v\"],\n",
    "    lora_dropout=0.05,\n",
    "    bias=\"none\",\n",
    "    task_type=TaskType.SEQ_2_SEQ_LM # FLAN-T5\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "Add LoRA adapter layers/parameters to the original LLM to be trained."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "trainable model parameters: 3538944\n",
      "all model parameters: 251116800\n",
      "percentage of trainable model parameters: 1.41%\n"
     ]
    }
   ],
   "source": [
    "peft_model = get_peft_model(original_model, \n",
    "                            lora_config)\n",
    "print(print_number_of_trainable_model_parameters(peft_model))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "<a name='3.2'></a>\n",
    "### 3.2 - Train PEFT Adapter\n",
    "\n",
    "Define training arguments and create `Trainer` instance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# output_dir = f'./peft-dialogue-summary-training-{str(int(time.time()))}'\n",
    "\n",
    "# peft_training_args = TrainingArguments(\n",
    "#     output_dir=output_dir,\n",
    "#     auto_find_batch_size=True,\n",
    "#     learning_rate=1e-3, # Higher learning rate than full fine-tuning.\n",
    "#     num_train_epochs=1,\n",
    "#     logging_steps=1,\n",
    "#     max_steps=1    \n",
    "# )\n",
    "    \n",
    "# peft_trainer = Trainer(\n",
    "#     model=peft_model,\n",
    "#     args=peft_training_args,\n",
    "#     train_dataset=tokenized_datasets[\"train\"],\n",
    "# )\n",
    "\n",
    "# peft_trainer.train()\n",
    "\n",
    "# peft_model_path=\"./peft-dialogue-summary-local-checkpoint\"\n",
    "\n",
    "# peft_trainer.model.save_pretrained(peft_model_path)\n",
    "# tokenizer.save_pretrained(peft_model_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "That training was performed on a subset of data. To load a fully trained PEFT model, read a checkpoint of a PEFT model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "Add an adapter to the original FLAN-T5 model by setting `is_trainable=False` for inference purposes; for further training, `is_trainable=True` would be used."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from peft import PeftModel, PeftConfig\n",
    "\n",
    "peft_model_base = AutoModelForSeq2SeqLM.from_pretrained(\"google/flan-t5-base\", torch_dtype=torch.bfloat16)\n",
    "tokenizer = AutoTokenizer.from_pretrained(\"google/flan-t5-base\")\n",
    "\n",
    "peft_model = PeftModel.from_pretrained(peft_model_base, \n",
    "                                       './peft-dialogue-summary-checkpoint/', \n",
    "                                       torch_dtype=torch.bfloat16,\n",
    "                                       is_trainable=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "The number of trainable parameters will be `0` due to `is_trainable=False` setting:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "trainable model parameters: 0\n",
      "all model parameters: 251116800\n",
      "percentage of trainable model parameters: 0.00%\n"
     ]
    }
   ],
   "source": [
    "print(print_number_of_trainable_model_parameters(peft_model))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name='3.3'></a>\n",
    "### 3.3 - Evaluate the Model Qualitatively (Human Evaluation)\n",
    "Make inferences for the same example using the original model, the fully fine-tuned model, and the PEFT model to compare their outputs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "---------------------------------------------------------------------------------------------------\n",
      "BASELINE HUMAN SUMMARY:\n",
      "#Person1# teaches #Person2# how to upgrade software and hardware in #Person2#'s system.\n",
      "---------------------------------------------------------------------------------------------------\n",
      "ORIGINAL MODEL:\n",
      "#Person1#: I'm thinking of upgrading my computer.\n",
      "---------------------------------------------------------------------------------------------------\n",
      "INSTRUCT MODEL:\n",
      "#Person1# suggests #Person2# adding a painting program to #Person2#'s software and upgrading the hardware. #Person2# also wants to add a CD-ROM drive.\n",
      "---------------------------------------------------------------------------------------------------\n",
      "PEFT MODEL: #Person1# recommends adding a painting program to #Person2#'s software and upgrading hardware. #Person2# also wants to upgrade the hardware because it's outdated now.\n"
     ]
    }
   ],
   "source": [
    "index = 200\n",
    "dialogue = dataset['test'][index]['dialogue']\n",
    "human_baseline_summary = dataset['test'][index]['summary']\n",
    "\n",
    "prompt = f\"\"\"\n",
    "Summarize the following conversation.\n",
    "\n",
    "{dialogue}\n",
    "\n",
    "Summary: \"\"\"\n",
    "\n",
    "input_ids = tokenizer(prompt, return_tensors=\"pt\").input_ids\n",
    "\n",
    "original_model_outputs = original_model.generate(input_ids=input_ids, generation_config=GenerationConfig(max_new_tokens=200, num_beams=1))\n",
    "original_model_text_output = tokenizer.decode(original_model_outputs[0], skip_special_tokens=True)\n",
    "\n",
    "instruct_model_outputs = instruct_model.generate(input_ids=input_ids, generation_config=GenerationConfig(max_new_tokens=200, num_beams=1))\n",
    "instruct_model_text_output = tokenizer.decode(instruct_model_outputs[0], skip_special_tokens=True)\n",
    "\n",
    "peft_model_outputs = peft_model.generate(input_ids=input_ids, generation_config=GenerationConfig(max_new_tokens=200, num_beams=1))\n",
    "peft_model_text_output = tokenizer.decode(peft_model_outputs[0], skip_special_tokens=True)\n",
    "\n",
    "print(dash_line)\n",
    "print(f'BASELINE HUMAN SUMMARY:\\n{human_baseline_summary}')\n",
    "print(dash_line)\n",
    "print(f'ORIGINAL MODEL:\\n{original_model_text_output}')\n",
    "print(dash_line)\n",
    "print(f'INSTRUCT MODEL:\\n{instruct_model_text_output}')\n",
    "print(dash_line)\n",
    "print(f'PEFT MODEL: {peft_model_text_output}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name='3.4'></a>\n",
    "### 3.4 - Evaluate the Model Quantitatively (with ROUGE Metric)\n",
    "Perform inferences for the sample of the test dataset (only 10 dialogues and summaries to save time). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>human_baseline_summaries</th>\n",
       "      <th>original_model_summaries</th>\n",
       "      <th>instruct_model_summaries</th>\n",
       "      <th>peft_model_summaries</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Ms. Dawson helps #Person1# to write a memo to ...</td>\n",
       "      <td>#Person1#: I need to take a dictation for you.</td>\n",
       "      <td>#Person1# asks Ms. Dawson to take a dictation ...</td>\n",
       "      <td>#Person1# asks Ms. Dawson to take a dictation ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>In order to prevent employees from wasting tim...</td>\n",
       "      <td>#Person1#: I need to take a dictation for you.</td>\n",
       "      <td>#Person1# asks Ms. Dawson to take a dictation ...</td>\n",
       "      <td>#Person1# asks Ms. Dawson to take a dictation ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Ms. Dawson takes a dictation for #Person1# abo...</td>\n",
       "      <td>#Person1#: I need to take a dictation for you.</td>\n",
       "      <td>#Person1# asks Ms. Dawson to take a dictation ...</td>\n",
       "      <td>#Person1# asks Ms. Dawson to take a dictation ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>#Person2# arrives late because of traffic jam....</td>\n",
       "      <td>The traffic jam at the Carrefour intersection ...</td>\n",
       "      <td>#Person2# got stuck in traffic again. #Person1...</td>\n",
       "      <td>#Person2# got stuck in traffic and #Person1# s...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>#Person2# decides to follow #Person1#'s sugges...</td>\n",
       "      <td>The traffic jam at the Carrefour intersection ...</td>\n",
       "      <td>#Person2# got stuck in traffic again. #Person1...</td>\n",
       "      <td>#Person2# got stuck in traffic and #Person1# s...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>#Person2# complains to #Person1# about the tra...</td>\n",
       "      <td>The traffic jam at the Carrefour intersection ...</td>\n",
       "      <td>#Person2# got stuck in traffic again. #Person1...</td>\n",
       "      <td>#Person2# got stuck in traffic and #Person1# s...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>#Person1# tells Kate that Masha and Hero get d...</td>\n",
       "      <td>Masha and Hero are getting divorced.</td>\n",
       "      <td>Masha and Hero are getting divorced. Kate can'...</td>\n",
       "      <td>Kate tells #Person2# Masha and Hero are gettin...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>#Person1# tells Kate that Masha and Hero are g...</td>\n",
       "      <td>Masha and Hero are getting divorced.</td>\n",
       "      <td>Masha and Hero are getting divorced. Kate can'...</td>\n",
       "      <td>Kate tells #Person2# Masha and Hero are gettin...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>#Person1# and Kate talk about the divorce betw...</td>\n",
       "      <td>Masha and Hero are getting divorced.</td>\n",
       "      <td>Masha and Hero are getting divorced. Kate can'...</td>\n",
       "      <td>Kate tells #Person2# Masha and Hero are gettin...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>#Person1# and Brian are at the birthday party ...</td>\n",
       "      <td>#Person1#: Happy birthday, Brian. #Person2#: T...</td>\n",
       "      <td>Brian's birthday is coming. #Person1# invites ...</td>\n",
       "      <td>Brian remembers his birthday and invites #Pers...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                            human_baseline_summaries  \\\n",
       "0  Ms. Dawson helps #Person1# to write a memo to ...   \n",
       "1  In order to prevent employees from wasting tim...   \n",
       "2  Ms. Dawson takes a dictation for #Person1# abo...   \n",
       "3  #Person2# arrives late because of traffic jam....   \n",
       "4  #Person2# decides to follow #Person1#'s sugges...   \n",
       "5  #Person2# complains to #Person1# about the tra...   \n",
       "6  #Person1# tells Kate that Masha and Hero get d...   \n",
       "7  #Person1# tells Kate that Masha and Hero are g...   \n",
       "8  #Person1# and Kate talk about the divorce betw...   \n",
       "9  #Person1# and Brian are at the birthday party ...   \n",
       "\n",
       "                            original_model_summaries  \\\n",
       "0     #Person1#: I need to take a dictation for you.   \n",
       "1     #Person1#: I need to take a dictation for you.   \n",
       "2     #Person1#: I need to take a dictation for you.   \n",
       "3  The traffic jam at the Carrefour intersection ...   \n",
       "4  The traffic jam at the Carrefour intersection ...   \n",
       "5  The traffic jam at the Carrefour intersection ...   \n",
       "6               Masha and Hero are getting divorced.   \n",
       "7               Masha and Hero are getting divorced.   \n",
       "8               Masha and Hero are getting divorced.   \n",
       "9  #Person1#: Happy birthday, Brian. #Person2#: T...   \n",
       "\n",
       "                            instruct_model_summaries  \\\n",
       "0  #Person1# asks Ms. Dawson to take a dictation ...   \n",
       "1  #Person1# asks Ms. Dawson to take a dictation ...   \n",
       "2  #Person1# asks Ms. Dawson to take a dictation ...   \n",
       "3  #Person2# got stuck in traffic again. #Person1...   \n",
       "4  #Person2# got stuck in traffic again. #Person1...   \n",
       "5  #Person2# got stuck in traffic again. #Person1...   \n",
       "6  Masha and Hero are getting divorced. Kate can'...   \n",
       "7  Masha and Hero are getting divorced. Kate can'...   \n",
       "8  Masha and Hero are getting divorced. Kate can'...   \n",
       "9  Brian's birthday is coming. #Person1# invites ...   \n",
       "\n",
       "                                peft_model_summaries  \n",
       "0  #Person1# asks Ms. Dawson to take a dictation ...  \n",
       "1  #Person1# asks Ms. Dawson to take a dictation ...  \n",
       "2  #Person1# asks Ms. Dawson to take a dictation ...  \n",
       "3  #Person2# got stuck in traffic and #Person1# s...  \n",
       "4  #Person2# got stuck in traffic and #Person1# s...  \n",
       "5  #Person2# got stuck in traffic and #Person1# s...  \n",
       "6  Kate tells #Person2# Masha and Hero are gettin...  \n",
       "7  Kate tells #Person2# Masha and Hero are gettin...  \n",
       "8  Kate tells #Person2# Masha and Hero are gettin...  \n",
       "9  Brian remembers his birthday and invites #Pers...  "
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dialogues = dataset['test'][0:10]['dialogue']\n",
    "human_baseline_summaries = dataset['test'][0:10]['summary']\n",
    "\n",
    "original_model_summaries = []\n",
    "instruct_model_summaries = []\n",
    "peft_model_summaries = []\n",
    "\n",
    "for idx, dialogue in enumerate(dialogues):\n",
    "    prompt = f\"\"\"\n",
    "Summarize the following conversation.\n",
    "\n",
    "{dialogue}\n",
    "\n",
    "Summary: \"\"\"\n",
    "    \n",
    "    input_ids = tokenizer(prompt, return_tensors=\"pt\").input_ids\n",
    "\n",
    "    human_baseline_text_output = human_baseline_summaries[idx]\n",
    "    \n",
    "    original_model_outputs = original_model.generate(input_ids=input_ids, generation_config=GenerationConfig(max_new_tokens=200))\n",
    "    original_model_text_output = tokenizer.decode(original_model_outputs[0], skip_special_tokens=True)\n",
    "\n",
    "    instruct_model_outputs = instruct_model.generate(input_ids=input_ids, generation_config=GenerationConfig(max_new_tokens=200))\n",
    "    instruct_model_text_output = tokenizer.decode(instruct_model_outputs[0], skip_special_tokens=True)\n",
    "\n",
    "    peft_model_outputs = peft_model.generate(input_ids=input_ids, generation_config=GenerationConfig(max_new_tokens=200))\n",
    "    peft_model_text_output = tokenizer.decode(peft_model_outputs[0], skip_special_tokens=True)\n",
    "\n",
    "    original_model_summaries.append(original_model_text_output)\n",
    "    instruct_model_summaries.append(instruct_model_text_output)\n",
    "    peft_model_summaries.append(peft_model_text_output)\n",
    "\n",
    "zipped_summaries = list(zip(human_baseline_summaries, original_model_summaries, instruct_model_summaries, peft_model_summaries))\n",
    " \n",
    "df = pd.DataFrame(zipped_summaries, columns = ['human_baseline_summaries', 'original_model_summaries', 'instruct_model_summaries', 'peft_model_summaries'])\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Compute ROUGE score for this subset of the data. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ORIGINAL MODEL:\n",
      "{'rouge1': np.float64(0.25036039886039885), 'rouge2': np.float64(0.11573401534526855), 'rougeL': np.float64(0.2286324786324786), 'rougeLsum': np.float64(0.2303917378917379)}\n",
      "INSTRUCT MODEL:\n",
      "{'rouge1': np.float64(0.39543570996335975), 'rouge2': np.float64(0.17186878842221226), 'rougeL': np.float64(0.2806870881294845), 'rougeLsum': np.float64(0.281746348889206)}\n",
      "PEFT MODEL:\n",
      "{'rouge1': np.float64(0.37172277715264146), 'rouge2': np.float64(0.12054554371970103), 'rougeL': np.float64(0.276540165983151), 'rougeLsum': np.float64(0.2757318532221029)}\n"
     ]
    }
   ],
   "source": [
    "rouge = evaluate.load('rouge')\n",
    "\n",
    "original_model_results = rouge.compute(\n",
    "    predictions=original_model_summaries,\n",
    "    references=human_baseline_summaries[0:len(original_model_summaries)],\n",
    "    use_aggregator=True,\n",
    "    use_stemmer=True,\n",
    ")\n",
    "\n",
    "instruct_model_results = rouge.compute(\n",
    "    predictions=instruct_model_summaries,\n",
    "    references=human_baseline_summaries[0:len(instruct_model_summaries)],\n",
    "    use_aggregator=True,\n",
    "    use_stemmer=True,\n",
    ")\n",
    "\n",
    "peft_model_results = rouge.compute(\n",
    "    predictions=peft_model_summaries,\n",
    "    references=human_baseline_summaries[0:len(peft_model_summaries)],\n",
    "    use_aggregator=True,\n",
    "    use_stemmer=True,\n",
    ")\n",
    "\n",
    "print('ORIGINAL MODEL:')\n",
    "print(original_model_results)\n",
    "print('INSTRUCT MODEL:')\n",
    "print(instruct_model_results)\n",
    "print('PEFT MODEL:')\n",
    "print(peft_model_results)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice, that PEFT model results are not too bad, while the training process was much easier!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Load the ROUGE scores for the PEFT model from the `data/dialogue-summary-training-results.csv` file and compare its performance to the other models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ORIGINAL MODEL:\n",
      "{'rouge1': np.float64(0.23328464502586696), 'rouge2': np.float64(0.0760949386582638), 'rougeL': np.float64(0.20150430978201175), 'rougeLsum': np.float64(0.2014999133289781)}\n",
      "INSTRUCT MODEL:\n",
      "{'rouge1': np.float64(0.4214360700703818), 'rouge2': np.float64(0.18058992754713823), 'rougeL': np.float64(0.3384594986152175), 'rougeLsum': np.float64(0.33806946616372496)}\n",
      "PEFT MODEL:\n",
      "{'rouge1': np.float64(0.4082136846059069), 'rouge2': np.float64(0.1634455011123068), 'rougeL': np.float64(0.3252854603732126), 'rougeLsum': np.float64(0.3249835254522996)}\n"
     ]
    }
   ],
   "source": [
    "human_baseline_summaries = results['human_baseline_summaries'].values\n",
    "original_model_summaries = results['original_model_summaries'].values\n",
    "instruct_model_summaries = results['instruct_model_summaries'].values\n",
    "peft_model_summaries     = results['peft_model_summaries'].values\n",
    "\n",
    "original_model_results = rouge.compute(\n",
    "    predictions=original_model_summaries,\n",
    "    references=human_baseline_summaries[0:len(original_model_summaries)],\n",
    "    use_aggregator=True,\n",
    "    use_stemmer=True,\n",
    ")\n",
    "\n",
    "instruct_model_results = rouge.compute(\n",
    "    predictions=instruct_model_summaries,\n",
    "    references=human_baseline_summaries[0:len(instruct_model_summaries)],\n",
    "    use_aggregator=True,\n",
    "    use_stemmer=True,\n",
    ")\n",
    "\n",
    "peft_model_results = rouge.compute(\n",
    "    predictions=peft_model_summaries,\n",
    "    references=human_baseline_summaries[0:len(peft_model_summaries)],\n",
    "    use_aggregator=True,\n",
    "    use_stemmer=True,\n",
    ")\n",
    "\n",
    "print('ORIGINAL MODEL:')\n",
    "print(original_model_results)\n",
    "print('INSTRUCT MODEL:')\n",
    "print(instruct_model_results)\n",
    "print('PEFT MODEL:')\n",
    "print(peft_model_results)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The results show less of an improvement over full fine-tuning, but the benefits of PEFT typically outweigh the slightly-lower performance metrics.\n",
    "\n",
    "Calculate the improvement of PEFT over the original model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Absolute percentage improvement of PEFT MODEL over ORIGINAL MODEL\n",
      "rouge1: 17.49%\n",
      "rouge2: 8.74%\n",
      "rougeL: 12.38%\n",
      "rougeLsum: 12.35%\n"
     ]
    }
   ],
   "source": [
    "print(\"Absolute percentage improvement of PEFT MODEL over ORIGINAL MODEL\")\n",
    "\n",
    "improvement = (np.array(list(peft_model_results.values())) - np.array(list(original_model_results.values())))\n",
    "for key, value in zip(peft_model_results.keys(), improvement):\n",
    "    print(f'{key}: {value*100:.2f}%')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now calculate the improvement of PEFT over a full fine-tuned model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Absolute percentage improvement of PEFT MODEL over INSTRUCT MODEL\n",
      "rouge1: -1.32%\n",
      "rouge2: -1.71%\n",
      "rougeL: -1.32%\n",
      "rougeLsum: -1.31%\n"
     ]
    }
   ],
   "source": [
    "print(\"Absolute percentage improvement of PEFT MODEL over INSTRUCT MODEL\")\n",
    "\n",
    "improvement = (np.array(list(peft_model_results.values())) - np.array(list(instruct_model_results.values())))\n",
    "for key, value in zip(peft_model_results.keys(), improvement):\n",
    "    print(f'{key}: {value*100:.2f}%')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There is a small percentage decrease in ROUGE metrics for the PEFT model compared to the fully fine-tuned model, but training requires significantly fewer computing and memory resources, often utilizing just a single GPU."
   ]
  }
 ],
 "metadata": {
  "availableInstances": [
   {
    "_defaultOrder": 0,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.t3.medium",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 1,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.t3.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 2,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.t3.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 3,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.t3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 4,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 5,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 6,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 7,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 8,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 9,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 10,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 11,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 12,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5d.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 13,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5d.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 14,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5d.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 15,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5d.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 16,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5d.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 17,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5d.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 18,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5d.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 19,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 20,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": true,
    "memoryGiB": 0,
    "name": "ml.geospatial.interactive",
    "supportedImageNames": [
     "sagemaker-geospatial-v1-0"
    ],
    "vcpuNum": 0
   },
   {
    "_defaultOrder": 21,
    "_isFastLaunch": true,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.c5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 22,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.c5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 23,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.c5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 24,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.c5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 25,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 72,
    "name": "ml.c5.9xlarge",
    "vcpuNum": 36
   },
   {
    "_defaultOrder": 26,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 96,
    "name": "ml.c5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 27,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 144,
    "name": "ml.c5.18xlarge",
    "vcpuNum": 72
   },
   {
    "_defaultOrder": 28,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.c5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 29,
    "_isFastLaunch": true,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g4dn.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 30,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g4dn.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 31,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g4dn.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 32,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g4dn.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 33,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g4dn.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 34,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g4dn.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 35,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 61,
    "name": "ml.p3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 36,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 244,
    "name": "ml.p3.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 37,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 488,
    "name": "ml.p3.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 38,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.p3dn.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 39,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.r5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 40,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.r5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 41,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.r5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 42,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.r5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 43,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.r5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 44,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.r5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 45,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.r5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 46,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.r5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 47,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 48,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 49,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 50,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 51,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 52,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 53,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.g5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 54,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.g5.48xlarge",
    "vcpuNum": 192
   },
   {
    "_defaultOrder": 55,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 56,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4de.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 57,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.trn1.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 58,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.trn1.32xlarge",
    "vcpuNum": 128
   },
   {
    "_defaultOrder": 59,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.trn1n.32xlarge",
    "vcpuNum": 128
   }
  ],
  "colab": {
   "name": "Fine-tune a language model",
   "provenance": []
  },
  "instance_type": "ml.m5.2xlarge",
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.4"
  },
  "vscode": {
   "interpreter": {
    "hash": "aee8b7b246df8f9039afb4144a1f6fd8d2ca17a180786b69acc140d282b71a49"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}