HW3-.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "**Task Description:**\n",
        "You have learned about transformers and their applications in natural language processing. In this assignment, you will apply your knowledge by implementing a transformer-based model to solve a text classification task.\n",
        "\n",
        "\n",
        "**Dataset:**\n",
        "You will be using the IMDB movie review dataset, which contains movie reviews labeled as positive or negative sentiment. The dataset will be downloaded and loaded using Python's file handling capabilities.\n",
        "\n",
        "**Task:**\n",
        "\n",
        "Your task is to build a transformer-based model using the torch.nn.Transformer module to classify movie reviews as positive or negative sentiment. You can use the provided dataset for training and evaluation.\n",
        "\n",
        "**Instructions:**\n",
        "\n",
        "(1) Download and Extract the IMDB Dataset:Run the following script to download and extract the IMDB dataset:"
      ],
      "metadata": {
        "id": "ldfrKsqM1zml"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "FZnKJoXqu31t"
      },
      "outputs": [],
      "source": [
        "import os\n",
        "import tarfile\n",
        "import urllib.request\n",
        "\n",
        "# Function to download and extract IMDB dataset\n",
        "def download_extract_imdb(root=\"./imdb_data\"):\n",
        "    if not os.path.exists(root):\n",
        "        os.makedirs(root)\n",
        "\n",
        "    url = \"http://ai.stanford.edu/~amaas/data/sentiment/aclImdb_v1.tar.gz\"\n",
        "    filename = os.path.join(root, \"aclImdb_v1.tar.gz\")\n",
        "    urllib.request.urlretrieve(url, filename)\n",
        "\n",
        "    # Extract the tar.gz file\n",
        "    with tarfile.open(filename, \"r:gz\") as tar:\n",
        "        tar.extractall(root)\n",
        "\n",
        "# Download and extract IMDB dataset\n",
        "download_extract_imdb()\n"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "(2) Load and Preprocess the Dataset:Use the following script to load the IMDB dataset, preprocess it, and tokenize the reviews:"
      ],
      "metadata": {
        "id": "OlZts1t91u0m"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import os\n",
        "from torchtext.data.utils import get_tokenizer\n",
        "\n",
        "# Set up tokenizer\n",
        "tokenizer = get_tokenizer(\"basic_english\")\n",
        "\n",
        "# Load training data\n",
        "def load_imdb_data(root=\"./imdb_data/aclImdb\"):\n",
        "    train_data = []\n",
        "    for label in [\"pos\", \"neg\"]:\n",
        "        label_dir = os.path.join(root, \"train\", label)\n",
        "        for filename in os.listdir(label_dir):\n",
        "            with open(os.path.join(label_dir, filename), \"r\", encoding=\"utf-8\") as file:\n",
        "                review = file.read()\n",
        "                # Tokenize review\n",
        "                tokenized_review = tokenizer(review)\n",
        "                train_data.append((tokenized_review, 1 if label == \"pos\" else 0))\n",
        "    return train_data\n",
        "\n",
        "# Load training data\n",
        "train_data = load_imdb_data()\n",
        "\n",
        "# Load testing data\n",
        "def load_test_data(root=\"./imdb_data/aclImdb\"):\n",
        "    test_data = []\n",
        "    for label in [\"pos\", \"neg\"]:\n",
        "        label_dir = os.path.join(root, \"test\", label)\n",
        "        for filename in os.listdir(label_dir):\n",
        "            with open(os.path.join(label_dir, filename), \"r\", encoding=\"utf-8\") as file:\n",
        "                review = file.read()\n",
        "                # Tokenize review\n",
        "                tokenized_review = tokenizer(review)\n",
        "                test_data.append((tokenized_review, 1 if label == \"pos\" else 0))\n",
        "    return test_data\n",
        "\n",
        "# Load testing data\n",
        "test_data = load_test_data()\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "p577rCQPu67M"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Display tokenized positive and negative examples\n",
        "print(\"Tokenized Positive Example:\")\n",
        "print(train_data[0][0])\n",
        "print(\"Tokenized Negative Example:\")\n",
        "print(train_data[len(train_data)//2][0])"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1Fic4RUKvg2_",
        "outputId": "b18fe37a-84bb-4643-c51c-36bbb6966959"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Tokenized Positive Example:\n",
            "['.', '.', '.', 'for', 'paris', 'is', 'a', 'moveable', 'feast', '.', 'ernest', 'hemingway', 'it', 'is', 'impossible', 'to', 'count', 'how', 'many', 'great', 'talents', 'have', 'immortalized', 'paris', 'in', 'paintings', ',', 'novels', ',', 'songs', ',', 'poems', ',', 'short', 'but', 'unforgettable', 'quotes', ',', 'and', 'yes', '-', 'movies', '.', 'the', 'celebrated', 'film', 'director', 'max', 'ophüls', 'said', 'about', 'paris', ',', 'it', 'offered', 'the', 'shining', 'wet', 'boulevards', 'under', 'the', 'street', 'lights', ',', 'breakfast', 'in', 'montmartre', 'with', 'cognac', 'in', 'your', 'glass', ',', 'coffee', 'and', 'lukewarm', 'brioche', ',', 'gigolos', 'and', 'prostitutes', 'at', 'night', '.', 'everyone', 'in', 'the', 'world', 'has', 'two', 'fatherlands', 'his', 'own', 'and', 'paris', '.', 'paris', 'is', 'always', 'associated', 'with', 'love', 'and', 'romance', ',', 'and', 'paris', ',', 'je', 't', \"'\", 'aime', 'which', 'is', 'subtitled', 'petite', 'romances', ',', 'is', 'a', 'collection', 'of', 'short', 'films', ',', 'often', 'sketches', 'from', '18', 'talented', 'directors', 'from', 'all', 'over', 'the', 'world', '.', 'in', 'each', ',', 'we', 'become', 'familiar', 'with', 'one', 'of', 'the', 'city', 'of', 'light', '20', 'arrondissements', 'and', 'with', 'the', 'parisians', 'of', 'all', 'ages', ',', 'genders', ',', 'colors', ',', 'and', 'backgrounds', 'who', 'all', 'deal', 'in', 'love', 'in', 'its', 'many', 'variations', 'and', 'stages', '.', 'in', 'some', 'of', 'the', 'petite', 'romances', 'we', 'are', 'the', 'witnesses', 'of', 'the', 'unexpected', 'encounters', 'of', 'the', 'strangers', 'that', 'lead', 'to', 'instant', 'interest', ',', 'closeness', ',', 'and', 'perhaps', 'relationship', 'like', 'for', 'podalydès', 'and', 'florence', 'muller', 'in', 'the', 'street', 'of', 'montmartre', 'in', 'the', 'opening', 'film', 'or', 'for', 'cyril', 'descours', 'and', 'leïla', 'bekhti', 'as', 'a', 'white', 'boy', 'and', 'a', 'muslim', 'girl', 'whose', 'cross-cultural', 'romance', 'directed', 'by', 'gurinder', 'chadha', 'begins', 'on', 'quais', 'de', 'seine', '.', 'i', 'would', 'include', 'into', 'this', 'category', 'the', 'humorous', 'short', 'film', 'by', 'gus', 'van', 'sant', '.', 'in', 'le', 'marais', 'one', 'boy', 'pours', 'his', 'heart', 'out', 'to', 'another', 'boy', 'confessing', 'of', 'sudden', 'unexpected', 'closeness', ',', 'asking', 'permission', 'to', 'call', '-', 'never', 'realizing', 'that', 'the', 'object', 'of', 'his', 'interest', 'does', 'not', 'understand', 'french', '.', 'some', 'of', 'the', 'vignettes', 'are', 'poignant', 'and', 'even', 'dark', '.', 'in', 'walter', 'salles', 'and', 'daniela', 'thomas', \"'\", 'loin', 'du', '16ème', ',', 'catalina', 'sandino', 'mareno', '(', 'amazing', 'oscar', 'nominated', 'debut', 'for', 'maria', 'full', 'of', 'grace', ')', 'is', 'single', ',', 'working-class', 'mother', 'who', 'has', 'to', 'work', 'as', 'a', 'nanny', 'in', 'a', 'wealthy', 'neighborhood', 'to', 'pay', 'for', 'daycare', 'where', 'she', 'drops', 'her', 'baby', 'every', 'morning', 'before', 'she', 'goes', 'to', 'work', '.', 'one', 'of', 'most', 'memorable', 'and', 'truly', 'heartbreaking', 'films', 'is', 'place', 'des', 'fêtes', 'by', 'oliver', 'schmitz', '.', 'aïssa', 'maïga', 'and', 'seydou', 'boro', 'co-star', 'as', 'two', 'young', 'people', 'for', 'who', 'love', 'could', 'have', 'happened', '.', 'there', 'were', 'the', 'promises', 'of', 'it', 'but', 'it', 'was', 'cut', 'short', 'due', 'to', 'hatred', 'and', 'intolerance', 'that', 'are', 'present', 'everywhere', ',', 'and', 'the', 'city', 'of', 'love', 'and', 'light', 'is', 'no', 'exception', '.', 'another', 'one', 'that', 'really', 'got', 'to', 'me', 'was', 'bastille', ',', 'written', 'and', 'directed', 'by', 'isabel', 'coixet', ',', 'starring', 'sergio', 'castellitto', ',', 'miranda', 'richardson', ',', 'and', 'leonor', 'watling', '.', 'castellitto', 'has', 'fallen', 'out', 'of', 'love', 'with', 'his', 'wife', ',', 'richardson', 'but', 'when', 'he', 'is', 'ready', 'to', 'leave', 'with', 'the', 'beautiful', 'mistress', ',', 'the', 'devastating', 'news', 'from', 'his', 'wife', \"'\", 's', 'doctor', 'arrives', '.', '.', '.', 'i', 'can', 'go', 'on', 'reflecting', 'on', 'all', '18', 'small', 'gems', '.', 'i', 'like', 'some', 'of', 'them', 'very', 'much', '.', 'the', 'others', 'felt', 'weak', 'and', 'perhaps', 'will', 'be', 'forgotten', 'soon', 'but', 'overall', ',', 'i', 'am', 'very', 'glad', 'that', 'i', 'bought', 'the', 'dvd', 'and', 'i', 'know', 'that', 'i', 'will', 'return', 'to', 'my', 'favorite', 'films', 'again', 'and', 'again', '.', 'they', 'are', 'place', 'des', 'fêtes', 'that', 'i', \"'\", 've', 'mentioned', 'already', ',', 'père-lachaise', 'directed', 'by', 'wes', 'craven', 'that', 'involves', 'the', 'ghost', 'of', 'one', 'of', 'the', 'wittiest', 'and', 'cleverest', 'men', 'ever', ',', 'oscar', 'wilde', '(', 'alexander', 'payne', ',', 'the', 'director', 'of', 'sideways', ')', 'who', 'would', 'save', 'one', 'troubled', 'relationship', '.', 'payne', 'also', 'directed', '14th', 'arrondissement', 'in', 'which', 'a', 'lonely', 'middle-aged', 'post-worker', 'from', 'denver', ',', 'co', 'explores', 'the', 'city', 'on', 'her', 'own', 'providing', 'the', 'voice', 'over', 'in', 'french', 'with', 'the', 'heavy', 'accent', '.', 'payne', \"'\", 's', 'entry', 'is', 'one', 'of', 'the', 'most', 'moving', 'and', 'along', 'with', 'hilarious', 'tuileries', 'by', 'joel', 'and', 'ethan', 'coen', 'with', '(', 'who', 'else', '?', ')', ')', 'steve', 'buschemi', 'is', 'my', 'absolute', 'favorite', '.', 'in', 'both', 'shorts', ',', 'american', 'tourists', 'sit', 'on', 'the', 'benches', '(', 'margo', 'in', 'the', 'park', ',', 'and', 'steve', 'in', 'paris', 'metro', 'after', 'visiting', 'louvers', ')', 'observing', 'the', 'life', 'around', 'them', 'with', 'the', 'different', 'results', '.', 'while', 'margo', 'may', 'say', ',', 'my', 'feeling', \"'\", 's', 'sad', 'and', 'light', 'my', 'sorrow', 'is', 'bright', '.', '.', '.', 'steve', \"'\", 's', 'character', 'will', 'find', 'out', 'that', 'sometimes', ',', 'even', 'the', 'most', 'comprehensive', 'and', 'useful', 'tourist', 'guide', 'would', 'not', 'help', 'a', 'tourist', 'avoiding', 'doing', 'the', 'wrong', 'things', 'in', 'a', 'foreign', 'country', '.']\n",
            "Tokenized Negative Example:\n",
            "['what', 'ever', 'you', 'do', 'do', 'not', 'waste', 'your', 'time', 'on', 'this', 'pointless', '.', 'movie', '.', 'a', 'remake', 'that', 'did', 'not', 'need', 'to', 'be', 'retold', '.', 'everyone', 'coming', 'out', 'of', 'the', 'theater', 'had', 'the', 'same', 'comments', '.', 'worst', 'movie', 'i', 'ever', 'saw', '.', 'save', 'your', 'time', 'and', 'money', '!', '!', '!', 'nicgolas', 'cage', 'was', 'biking', 'down', 'hills', ',', 'swimming', 'in', 'murky', 'water', 'and', 'rolling', 'down', 'hills', 'while', 'being', 'attacked', 'by', 'bees', 'but', 'yet', 'his', 'suit', 'was', 'still', 'perfectly', 'pressed', 'and', 'shirt', 'crisp', 'white', 'until', 'the', 'very', 'last', 'scene', '.', 'although', 'a', 'good', 'cast', 'with', 'ellen', 'bernstein', 'and', 'cage', 'the', 'acting', 'was', 'just', 'as', 'unbelievable', 'as', 'the', 'movie', 'itself', '.', 'it', 'is', 'amazing', 'how', 'good', 'actors', 'can', 'do', 'such', 'bad', 'movies', '.', 'don', \"'\", 't', 'they', 'get', 'a', 'copy', 'of', 'the', 'script', 'first', '.', 'if', 'you', 'still', 'have', 'any', 'interest', 'at', 'all', 'in', 'seeing', 'the', 'movie', 'at', 'the', 'very', 'least', 'wait', 'for', 'it', 'to', 'come', 'out', 'on', 'dvd', '.']\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Display tokenized examples with labels for training dataset\n",
        "print(\"Training Dataset:\")\n",
        "for review, label in train_data[:3]:\n",
        "    print(\"Label:\", \"Positive\" if label == 1 else \"Negative\")\n",
        "    print(\"Tokenized Review:\", review)\n",
        "    print()\n",
        "\n",
        "# Display tokenized examples with labels for testing dataset\n",
        "print(\"Testing Dataset:\")\n",
        "for review, label in test_data[:3]:\n",
        "    print(\"Label:\", \"Positive\" if label == 1 else \"Negative\")\n",
        "    print(\"Tokenized Review:\", review)\n",
        "    print()\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "wgp2OksU2KfU",
        "outputId": "2968d124-e86c-495c-d1ab-7dc5591dd1ca"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Training Dataset:\n",
            "Label: Positive\n",
            "Tokenized Review: ['.', '.', '.', 'for', 'paris', 'is', 'a', 'moveable', 'feast', '.', 'ernest', 'hemingway', 'it', 'is', 'impossible', 'to', 'count', 'how', 'many', 'great', 'talents', 'have', 'immortalized', 'paris', 'in', 'paintings', ',', 'novels', ',', 'songs', ',', 'poems', ',', 'short', 'but', 'unforgettable', 'quotes', ',', 'and', 'yes', '-', 'movies', '.', 'the', 'celebrated', 'film', 'director', 'max', 'ophüls', 'said', 'about', 'paris', ',', 'it', 'offered', 'the', 'shining', 'wet', 'boulevards', 'under', 'the', 'street', 'lights', ',', 'breakfast', 'in', 'montmartre', 'with', 'cognac', 'in', 'your', 'glass', ',', 'coffee', 'and', 'lukewarm', 'brioche', ',', 'gigolos', 'and', 'prostitutes', 'at', 'night', '.', 'everyone', 'in', 'the', 'world', 'has', 'two', 'fatherlands', 'his', 'own', 'and', 'paris', '.', 'paris', 'is', 'always', 'associated', 'with', 'love', 'and', 'romance', ',', 'and', 'paris', ',', 'je', 't', \"'\", 'aime', 'which', 'is', 'subtitled', 'petite', 'romances', ',', 'is', 'a', 'collection', 'of', 'short', 'films', ',', 'often', 'sketches', 'from', '18', 'talented', 'directors', 'from', 'all', 'over', 'the', 'world', '.', 'in', 'each', ',', 'we', 'become', 'familiar', 'with', 'one', 'of', 'the', 'city', 'of', 'light', '20', 'arrondissements', 'and', 'with', 'the', 'parisians', 'of', 'all', 'ages', ',', 'genders', ',', 'colors', ',', 'and', 'backgrounds', 'who', 'all', 'deal', 'in', 'love', 'in', 'its', 'many', 'variations', 'and', 'stages', '.', 'in', 'some', 'of', 'the', 'petite', 'romances', 'we', 'are', 'the', 'witnesses', 'of', 'the', 'unexpected', 'encounters', 'of', 'the', 'strangers', 'that', 'lead', 'to', 'instant', 'interest', ',', 'closeness', ',', 'and', 'perhaps', 'relationship', 'like', 'for', 'podalydès', 'and', 'florence', 'muller', 'in', 'the', 'street', 'of', 'montmartre', 'in', 'the', 'opening', 'film', 'or', 'for', 'cyril', 'descours', 'and', 'leïla', 'bekhti', 'as', 'a', 'white', 'boy', 'and', 'a', 'muslim', 'girl', 'whose', 'cross-cultural', 'romance', 'directed', 'by', 'gurinder', 'chadha', 'begins', 'on', 'quais', 'de', 'seine', '.', 'i', 'would', 'include', 'into', 'this', 'category', 'the', 'humorous', 'short', 'film', 'by', 'gus', 'van', 'sant', '.', 'in', 'le', 'marais', 'one', 'boy', 'pours', 'his', 'heart', 'out', 'to', 'another', 'boy', 'confessing', 'of', 'sudden', 'unexpected', 'closeness', ',', 'asking', 'permission', 'to', 'call', '-', 'never', 'realizing', 'that', 'the', 'object', 'of', 'his', 'interest', 'does', 'not', 'understand', 'french', '.', 'some', 'of', 'the', 'vignettes', 'are', 'poignant', 'and', 'even', 'dark', '.', 'in', 'walter', 'salles', 'and', 'daniela', 'thomas', \"'\", 'loin', 'du', '16ème', ',', 'catalina', 'sandino', 'mareno', '(', 'amazing', 'oscar', 'nominated', 'debut', 'for', 'maria', 'full', 'of', 'grace', ')', 'is', 'single', ',', 'working-class', 'mother', 'who', 'has', 'to', 'work', 'as', 'a', 'nanny', 'in', 'a', 'wealthy', 'neighborhood', 'to', 'pay', 'for', 'daycare', 'where', 'she', 'drops', 'her', 'baby', 'every', 'morning', 'before', 'she', 'goes', 'to', 'work', '.', 'one', 'of', 'most', 'memorable', 'and', 'truly', 'heartbreaking', 'films', 'is', 'place', 'des', 'fêtes', 'by', 'oliver', 'schmitz', '.', 'aïssa', 'maïga', 'and', 'seydou', 'boro', 'co-star', 'as', 'two', 'young', 'people', 'for', 'who', 'love', 'could', 'have', 'happened', '.', 'there', 'were', 'the', 'promises', 'of', 'it', 'but', 'it', 'was', 'cut', 'short', 'due', 'to', 'hatred', 'and', 'intolerance', 'that', 'are', 'present', 'everywhere', ',', 'and', 'the', 'city', 'of', 'love', 'and', 'light', 'is', 'no', 'exception', '.', 'another', 'one', 'that', 'really', 'got', 'to', 'me', 'was', 'bastille', ',', 'written', 'and', 'directed', 'by', 'isabel', 'coixet', ',', 'starring', 'sergio', 'castellitto', ',', 'miranda', 'richardson', ',', 'and', 'leonor', 'watling', '.', 'castellitto', 'has', 'fallen', 'out', 'of', 'love', 'with', 'his', 'wife', ',', 'richardson', 'but', 'when', 'he', 'is', 'ready', 'to', 'leave', 'with', 'the', 'beautiful', 'mistress', ',', 'the', 'devastating', 'news', 'from', 'his', 'wife', \"'\", 's', 'doctor', 'arrives', '.', '.', '.', 'i', 'can', 'go', 'on', 'reflecting', 'on', 'all', '18', 'small', 'gems', '.', 'i', 'like', 'some', 'of', 'them', 'very', 'much', '.', 'the', 'others', 'felt', 'weak', 'and', 'perhaps', 'will', 'be', 'forgotten', 'soon', 'but', 'overall', ',', 'i', 'am', 'very', 'glad', 'that', 'i', 'bought', 'the', 'dvd', 'and', 'i', 'know', 'that', 'i', 'will', 'return', 'to', 'my', 'favorite', 'films', 'again', 'and', 'again', '.', 'they', 'are', 'place', 'des', 'fêtes', 'that', 'i', \"'\", 've', 'mentioned', 'already', ',', 'père-lachaise', 'directed', 'by', 'wes', 'craven', 'that', 'involves', 'the', 'ghost', 'of', 'one', 'of', 'the', 'wittiest', 'and', 'cleverest', 'men', 'ever', ',', 'oscar', 'wilde', '(', 'alexander', 'payne', ',', 'the', 'director', 'of', 'sideways', ')', 'who', 'would', 'save', 'one', 'troubled', 'relationship', '.', 'payne', 'also', 'directed', '14th', 'arrondissement', 'in', 'which', 'a', 'lonely', 'middle-aged', 'post-worker', 'from', 'denver', ',', 'co', 'explores', 'the', 'city', 'on', 'her', 'own', 'providing', 'the', 'voice', 'over', 'in', 'french', 'with', 'the', 'heavy', 'accent', '.', 'payne', \"'\", 's', 'entry', 'is', 'one', 'of', 'the', 'most', 'moving', 'and', 'along', 'with', 'hilarious', 'tuileries', 'by', 'joel', 'and', 'ethan', 'coen', 'with', '(', 'who', 'else', '?', ')', ')', 'steve', 'buschemi', 'is', 'my', 'absolute', 'favorite', '.', 'in', 'both', 'shorts', ',', 'american', 'tourists', 'sit', 'on', 'the', 'benches', '(', 'margo', 'in', 'the', 'park', ',', 'and', 'steve', 'in', 'paris', 'metro', 'after', 'visiting', 'louvers', ')', 'observing', 'the', 'life', 'around', 'them', 'with', 'the', 'different', 'results', '.', 'while', 'margo', 'may', 'say', ',', 'my', 'feeling', \"'\", 's', 'sad', 'and', 'light', 'my', 'sorrow', 'is', 'bright', '.', '.', '.', 'steve', \"'\", 's', 'character', 'will', 'find', 'out', 'that', 'sometimes', ',', 'even', 'the', 'most', 'comprehensive', 'and', 'useful', 'tourist', 'guide', 'would', 'not', 'help', 'a', 'tourist', 'avoiding', 'doing', 'the', 'wrong', 'things', 'in', 'a', 'foreign', 'country', '.']\n",
            "\n",
            "Label: Positive\n",
            "Tokenized Review: ['i', 'love', 'the', 'movie', ',', 'it', 'was', 'a', 'very', 'interesting', 'fantasy', 'movie', 'b/c', 'of', 'the', 'real', 'meaning', 'of', 'family', 'in', 'it', ',', 'the', 'history', 'of', 'our', 'country', ',', 'the', 'fun-filled', 'action', 'displayed', 'in', 'the', 'movie', '.', 'i', 'watch', 'time', '@', 'the', 'top', 'about', '4', 'x', \"'\", 's', 'a', 'week', 'and', 'i', 'just', 'love', 'it', '!', 'i', 'wish', 'that', 'a', 'sequel', 'had', 'of', 'been', 'made', 'to', 'see', 'more', 'of', 'susan', \"'\", 's', 'dad', 'in', 'the', 'past', 'and', 'watching', 'how', 'susan', 'delt', 'with', 'her', 'new', 'baby', 'sister', 'and', 'having', 'no', 'telephone', ',', 'computers', ',', 'gameboys', 'or', 'anything', 'of', 'the', '21st', 'century', '.', 'i', 'hope', 'everyone', 'else', 'enjoyed', 'the', 'movie', 'as', 'much', 'as', 'i', 'did', 'i', 'guess', 'you', 'could', 'say', 'i', \"'\", 'm', 'a', 'time', 'at', 'the', 'top', 'fanatic', 'and', 'i', 'don', \"'\", 't', 'mind', '.', 'the', 'lil', 'boy', 'in', 'the', 'movie', 'robert', 'lincoln', 'walker', 'was', 'simply', 'adorible', 'i', 'wonder', 'who', 'he', 'is', 'and', 'how', 'old', 'he', 'is', 'today', '.', 'does', 'anyone', 'know', 'if', 'he', \"'\", 's', 'played', 'in', 'over', 'movies', 'or', 'tv', 'shows', '?']\n",
            "\n",
            "Label: Positive\n",
            "Tokenized Review: ['full', 'moon', 'high', '(', '1981', ')', '3', 'of', '5', 'dir', 'larry', 'cohen', 'stars', 'adam', 'arkin', ',', 'ed', 'mcmahon', ',', 'roz', 'kelly', 'tony', '(', 'arkin', ')', 'is', 'your', 'average', 'ordinary', 'high', 'school', 'guy', '.', 'prepping', 'for', 'the', 'big', 'homecoming', 'game', ',', 'girlfriend', 'trouble', 'and', 'growing', 'hair', 'in', 'strange', 'places', 'you', 'know', 'the', 'usual', '.', 'but', 'the', 'hair', 'in', 'strange', 'places', 'part', 'gets', 'a', 'wee', 'bit', 'out', 'of', 'control', 'when', 'a', 'trip', 'to', 'transylvania', 'with', 'his', 'father', '(', 'yes', '!', 'mcmahon', ')', 'leaves', 'him', 'with', 'a', 'wicked', 'case', 'of', 'the', 'furballs', '.', 'now', 'doomed', 'to', 'walk', 'the', 'world', 'forever', 'young', 'as', 'a', 'werewolf', 'how', 'will', 'adam', 'get', 'any', 'girls', '?', \"'\", 'full', 'moon', 'high', \"'\", 'is', 'not', 'often', 'talked', 'about', 'but', 'it', 'is', 'a', 'silly', 'and', 'entertaining', 'horror', 'spoof', '.', 'larry', 'cohen', '(', 'q', 'the', 'winged', 'serpent', ',', 'the', 'stuff', ')', 'incorporates', 'as', 'many', 'gags', 'as', 'he', 'can', 'possibly', 'come', 'up', 'with', 'as', 'writer', '/', 'director', '.', 'arkin', '(', 'h20', 'halloween', ',', '20', 'years', 'later', ')', 'shows', 'nice', 'timing', 'in', 'the', 'lead', 'role', '.', 'if', 'you', 'happen', 'to', 'be', 'a', 'fan', 'of', 'spoofs', 'like', \"'\", 'airplane', \"'\", 'and', \"'\", 'student', 'bodies', \"'\", 'i', 'think', 'you', \"'\", 'll', 'have', 'fun', 'with', 'this', 'chuckle-fest', '.']\n",
            "\n",
            "Testing Dataset:\n",
            "Label: Positive\n",
            "Tokenized Review: ['i', 'think', 'that', 'the', 'basic', 'idea', 'of', 'any', 'movie', 'is', 'to', 'entertain', 'or', 'to', 'inform', '.', 'if', 'you', 'want', 'information', 'you', 'are', 'looking', 'at', 'true', 'life', 'movies', 'and', 'historical', 'movies', '.', 'sometimes', 'these', 'are', 'one', 'of', 'the', 'same', '.', 'the', 'other', 'side', 'of', 'the', 'coin', 'is', 'to', 'entertain', '.', 'did', 'hitch', 'entertain', 'me', '?', 'yes', 'it', 'did', '.', 'okay', 'the', 'formula', 'is', 'standard', '.', 'boy', 'meets', 'girl', 'or', 'in', 'this', 'case', 'boys', 'met', 'girls', '.', 'they', 'get', 'together', 'have', 'a', 'falling', 'out', 'then', 'get', 'back', 'together', '.', 'however', 'the', 'way', 'it', 'happened', 'in', 'this', 'movie', 'was', 'refreshing', '.', 'i', 'particularly', 'liked', 'the', 'bar', 'scene', 'with', 'hitch', 'and', 'sara', '.', 'the', 'allegra', 'albert', 'romance', 'was', 'a', 'delight', 'to', 'watch', 'unfold', ',', 'most', 'real', 'men', 'are', 'shy', 'when', 'it', 'comes', 'to', 'wooing', 'the', 'woman', 'of', 'their', 'dreams', 'and', 'had', 'i', 'had', 'hitch', \"'\", 's', 'advice', 'i', 'would', 'probably', 'have', 'got', 'my', 'wife', 'up', 'the', 'altar', 'in', 'half', 'the', 'time', '.', 'i', 'read', 'the', 'first', 'comment', 'on', 'this', 'film', 'that', 'appeared', 'to', 'suggest', 'that', 'this', 'movie', 'was', 'played', 'safely', 'and', 'good', 'have', 'had', 'a', 'few', 'more', 'laughs', '.', 'i', 'tend', 'to', 'disagree', 'there', 'are', 'so', 'many', 'laughs', 'you', 'can', 'pack', 'into', 'a', 'romantic', 'comedy', 'without', 'turning', 'it', 'into', 'a', 'farce', '.', 'besides', 'relationships', 'have', 'there', 'serious', 'moments', '.', 'all', 'in', 'all', 'i', 'found', 'hitch', 'quite', 'entertaining', ',', 'the', 'actors', 'did', 'a', 'good', 'job', '(', 'i', 'will', 'be', 'looking', 'out', 'for', 'them', 'in', 'other', 'movies', ')', 'and', 'hitch', 'is', 'a', 'film', 'that', 'i', 'am', 'very', 'happy', 'to', 'have', 'in', 'my', 'dvd', 'collection', '.']\n",
            "\n",
            "Label: Positive\n",
            "Tokenized Review: ['i', 'was', 'fortunate', 'to', 'see', 'a', 'screening', 'of', 'this', 'remarkable', 'short', 'film', 'by', 'joshua', 'leonard', 'before', 'its', 'premiere', 'at', 'the', '2005', 'sundance', 'festival', '.', 'in', 'twelve', 'brief', 'but', 'exquisite', 'minutes', ',', 'leonard', 'takes', 'us', 'on', 'a', 'life-changing', 'journey', 'as', 'he', 'probes', 'one', 'of', 'the', 'most', 'controversial', 'contemporary', 'social', 'and', 'ethical', 'issues', 'facing', 'our', 'society', '.', 'the', 'film', 'embodies', 'elegant', 'direction', ',', 'moving', 'performances', 'and', 'a', 'heart-', 'rending', 'story', '.', 'kelli', 'garner', 'and', 'lucas', 'haas', 'radiate', 'as', 'the', 'two', 'lovers', '.', 'and', ',', 'in', 'his', 'first', 'venture', 'into', 'dramatic', 'narrative', ',', 'leonard', 'proves', 'to', 'be', 'a', 'director', 'with', 'a', 'propitious', 'future', '.', 'i', 'feel', 'this', 'short', 'should', 'be', 'expanded', 'into', 'a', 'feature', 'film', '.', 'it', \"'\", 's', 'difficult', 'to', 'describe', 'talent', ',', 'but', 'as', 'this', 'debut', 'film', 'demonstrates', ',', 'you', 'know', 'it', 'when', 'you', 'see', 'it', '!']\n",
            "\n",
            "Label: Positive\n",
            "Tokenized Review: ['this', 'movie', 'is', 'like', 'happiness', 'meets', 'lost', 'in', 'translation', 'with', 'a', 'sixth', 'sense', 'ending', '(', 'or', 'maybe', 'a', 'crying', 'game', 'surprise', ')', ',', 'and', 'the', 'best', 'soundtrack', 'i', \"'\", 've', 'probably', 'ever', 'heard', '.', '.', '.', 'if', 'that', 'all', 'make', 'sense', '.', 'the', 'first', '30', 'seconds', 'pretty', 'much', 'tells', 'you', 'you', \"'\", 're', 'in', 'for', 'a', 'twisted', 'ride', '.', '(', 'i', 'was', 'surprised', 'no', 'one', 'walked', 'out', 'right', 'away', 'during', 'the', 'brooklyn', 'premiere', '.', ')', 'but', 'from', 'there', ',', 'the', 'film', 'settles', 'down', 'into', 'a', 'talk-fest', 'between', 'two', 'really', 'damaged', 'people', ',', 'daphne', 'and', 'buddy', '.', 'they', \"'\", 're', 'lonely', ',', 'mess-up', ',', 'and', 'boy', 'do', 'they', 'talk', 'about', 'sex', '.', 'daphne', 'brings', 'to', 'life', 'her', 'most', 'interesting', 'tales', 'of', 'escorting', ',', 'some', 'are', 'quite', 'funny', '(', 'mr', '.', 'chang', ')', 'some', 'disturbing', '(', 'the', 'harlan', 'scenes', 'with', 'music', 'that', 'tells', 'us', 'what', 'we', 'see', 'might', 'now', 'be', 'what', \"'\", 's', 'going', 'on', ',', 'or', 'what', 'daphne', 'is', 'really', 'feeling', ')', ',', 'and', 'because', 'i', 'have', 'a', 'friend', 'who', 'used', 'to', 'escort', ',', 'i', 'might', 'add', ',', 'most', 'seem', 'quite', 'real', '.', 'you', 'are', 'alone', 'is', 'multi-layered', 'and', 'mostly', 'brilliant', '.', 'okay', ',', 'maybe', 'a', 'couple', 'minutes', 'less', 'of', 'the', 'talking', ',', 'and', 'i', 'don', \"'\", 't', 'know', 'that', 'we', \"'\", 'd', 'have', 'missed', 'anything', '.', 'then', 'again', ',', 'i', 'need', 'to', 'see', 'it', 'again', 'knowing', 'the', 'ending', '.', 'i', 'like', 'this', 'movie', '.', '(', 'the', 'director', 'asked', 'people', 'in', 'the', 'brooklyn', 'audience', 'to', 'write', 'a', 'review', 'on', 'imdb', 'because', 'a', 'lot', 'of', 'people', 'read', 'them', '.', 'request', 'granted', '.', ')']\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "This script loads the IMDB dataset, tokenizes the reviews using the basic_english tokenizer, and displays tokenized examples for both positive and negative sentiment reviews.\n",
        "\n",
        "(3) Implement the Transformer Model:Implement the Transformer model using the torch.nn.Transformer module.\n",
        "\n",
        "(4)Train the Model:Define loss function and optimizer, and train the model on the training dataset.\n",
        "\n",
        "(5) Evaluate the Model:Evaluate the trained model on the testing dataset.\n",
        "\n",
        "(6) Calculate accuracy and other relevant metrics.\n",
        "\n",
        "Submission:Submit your implementation along with a brief report describing your model architecture, training procedure, evaluation results, and any insights gained."
      ],
      "metadata": {
        "id": "iW1iFsYaClk4"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Your code here: implement the Transformer model\n",
        "\n",
        "# Your code here: define loss function and optimizer\n",
        "\n",
        "# Your code here: train the model\n",
        "\n",
        "# Your code here: evaluate the model\n"
      ],
      "metadata": {
        "id": "7yag80x_Ctc3"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "PNdl9F0dCxpQ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Submission Instructions:**\n",
        "\n",
        "Submit your Python code in a single notebook file, show your work in detail."
      ],
      "metadata": {
        "id": "XrEXBFw4C3Gw"
      }
    }
  ]
}