2-text-classification/one_hot_vector.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## One-hot Vector\n",
    "\n",
    "One-hot vectors are frequently used in machine learning to encode categorical data, which can be either ordinal or nominal. For example, suppose we wanted to encode the names of fruits. One way to do this would be to map each name to a unique ID, as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Imports\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Name</th>\n",
       "      <th>Label ID</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Melon</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Watermelon</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Orange</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Melon</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         Name  Label ID\n",
       "0       Melon         0\n",
       "1  Watermelon         1\n",
       "2      Orange         2\n",
       "3       Melon         0"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "categorical_df = pd.DataFrame(\n",
    " {\"Name\": [\"Melon\", \"Watermelon\", \"Orange\", \"Melon\"], \"Label ID\": [0,1,2,0]})\n",
    "categorical_df"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The problem with this approach is that it creates a fictitious ordering between the names, and neural networks are really good at learning these kinds of relationships. So instead, we can create a new column for each category and assign a 1 where the\n",
    "category is true, and a 0 otherwise. In **Pandas**, this can be implemented with the *get_dummies()* function as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Melon</th>\n",
       "      <th>Orange</th>\n",
       "      <th>Watermelon</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Melon  Orange  Watermelon\n",
       "0      1       0           0\n",
       "1      0       0           1\n",
       "2      0       1           0\n",
       "3      1       0           0"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.get_dummies(categorical_df[\"Name\"])"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The rows of this DataFrame are the one-hot vectors, which have a single “hot” entry with a 1 and 0s everywhere else.\n",
    "\n",
    "To create one-hot vectors we can use the **PyTorch** or **Tensorflow** libraries to generate these vectors using the *F.one_hot()* and *tf.one_hot* methods respectively\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "fruits_label_ids=[0,1,2,0,1,2,2,2,1,2,3]\n",
    "num_classes=len(list(set(fruits_label_ids)))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tensor index: 0\n",
      "One-hot: tensor([1, 0, 0, 0])\n"
     ]
    }
   ],
   "source": [
    "#PyTorch\n",
    "import torch\n",
    "import torch.nn.functional as F\n",
    "\n",
    "input_ids = torch.tensor(fruits_label_ids)\n",
    "one_hot_encodings = F.one_hot(input_ids, num_classes=num_classes)\n",
    "\n",
    "print(f\"Tensor index: {fruits_label_ids[0]}\")\n",
    "print(f\"One-hot: {one_hot_encodings[0]}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tensor index: 0\n",
      "One-hot: tensor([1, 0, 0, 0])\n"
     ]
    }
   ],
   "source": [
    "#TensorFlow\n",
    "import tensorflow as tf\n",
    "\n",
    "tf.one_hot(fruits_label_ids, num_classes)\n",
    "\n",
    "print(f\"Tensor index: {fruits_label_ids[0]}\")\n",
    "print(f\"One-hot: {one_hot_encodings[0]}\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  },
  "orig_nbformat": 4,
  "vscode": {
   "interpreter": {
    "hash": "fcd5e99b3f4d209fb77e3753520e7bb18e2dbb8cf950aa58c2b06da10909cacd"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}